Pattern recognition system for heavy machinery environment perception

Abstract

The present application belongs to the field of intelligent perception and pattern recognition technology of engineering machinery, and relates to a pattern recognition system for heavy machinery environment perception, which solves the problem that the prior art cannot meet the demand of heavy machinery operation on the safety, robustness and traceability of the perception system. The present application collects the sparse event stream output by the event camera and reconstructs to obtain a reconstructed image; calculates the radar confidence and IMU confidence based on the double IMU data, predicts the dust concentration by the dust diffusion equation and synthesizes the visual confidence; extracts the visual and radar target distribution and embeds into the symmetric positive manifold, fuses to obtain the fused target distribution; calculates the energy value of the observation vector by the pre-constructed energy base model and converts into an abnormal score, and outputs the abnormal flag; calculates the global confidence, converts the safety rules into differentiable operations through the differentiable logic reasoning submodule, outputs the safety decision conclusion after model checking verification, and generates the control instruction and sends it through the bus.

Description

Technical Field

[0001] This invention belongs to the field of intelligent sensing and pattern recognition technology for engineering machinery, and specifically relates to a pattern recognition system for environmental sensing of heavy machinery. Background Technology

[0002] Heavy machinery operates under harsh and complex conditions, typically characterized by strong time-varying vibrations of 2-50Hz with amplitudes exceeding ±3g, and high concentrations of dust (PM10 concentration greater than 500μg / m³). These specific operating conditions severely interfere with data acquisition by sensing devices, easily leading to issues such as motion blur and point cloud trailing in sensor data. Field tests have verified that traditional cameras retain less than 30% of effective information under 3g vibration conditions, and pure visual sensing solutions exhibit obstacle recall rates below 50% in dust environments with PM10 concentrations of 300μg / m³, resulting in a significant degradation in sensing performance.

[0003] Most current mainstream environmental perception technologies are developed for passenger vehicle autonomous driving scenarios and are not adapted to the extreme coupled working conditions of heavy machinery with strong vibrations and high dust levels. As a result, their practical applications have many inherent defects, as follows: First, the pure vision solution has extremely poor environmental adaptability. It is prone to sensing failure in harsh working conditions such as high dust, nighttime, and backlight. This directly leads to a serious lack of environmental perception capability when heavy machinery is operating at night or in high dust, making it impossible to guarantee the needs of operational observation.

[0004] Secondly, lidar sensing solutions are not only expensive in terms of hardware, but also have significant performance shortcomings in heavy machinery operation scenarios: high concentrations of dust will generate a lot of echo noise, strong time-varying vibrations will cause severe distortion of point clouds, and dual interference will greatly reduce the accuracy and reliability of point cloud data. Therefore, lidar cannot be used as the core data source for heavy machinery environmental perception on its own.

[0005] Third, traditional fixed-weight multi-sensor fusion strategies lack adaptability and cannot respond to sudden and severe vibrations or sudden changes in sensor noise during heavy machinery operation. When a single sensor fails due to interference, the fusion weights cannot be dynamically adjusted, resulting in a deterioration in the overall fusion performance and an inability to guarantee perception stability.

[0006] Fourth, classic sensor confidence assessment methods can only perform single-frame data quality detection and lack spatiotemporal prediction capabilities. When multiple sensors are simultaneously subjected to severe interference and are in a low-confidence state, they cannot accurately identify blind spots in perception safety. This can lead to the perception system continuing to operate under the risk of overall perception failure, posing a significant operational safety hazard.

[0007] In summary, the numerous shortcomings of existing technologies ultimately lead to a series of problems for heavy machinery operating under conditions of strong vibration and high dust coupling, such as low reliability of environmental perception, lack of physical interpretability in confidence assessment, reliance on massive labeled samples for anomaly detection, and lack of formal verification support for safety decisions. These issues fail to meet the requirements of heavy machinery operations for the safety, robustness, and traceability of the perception system. Summary of the Invention

[0008] To address the aforementioned problems in existing technologies—namely, their inability to meet the safety, robustness, and traceability requirements of sensing systems for heavy machinery operations—this invention proposes a pattern recognition system for heavy machinery environmental sensing. This system includes: The biomimetic non-uniform sampling sensing module is configured to: acquire the sparse event stream output by the event camera and reconstruct the reconstructed image; The confidence prediction module is configured to: calculate radar confidence based on vehicle body IMU and support IMU data; calculate IMU confidence based on vehicle body acceleration; predict dust concentration based on spatial coordinates, time, wind speed, vibration, and excavation markers using a dust diffusion equation; receive the predicted dust concentration and vibration, and synthesize visual confidence using exponential decay and neural network modulation. The target distribution extraction and fusion module is configured to: receive the reconstructed image and extract the visual target distribution; receive the radar point cloud and extract the radar target distribution; embed the visual target distribution and the radar target distribution into a symmetric positive definite manifold respectively; calculate the visual weight and the radar weight; and perform weighted fusion along the geodesic to obtain the fused target distribution. The anomaly detection module is configured to: calculate the energy value of the observation vector at the current moment using a pre-built energy basis model and convert it into an anomaly score, and then output an anomaly flag after comparing it with a threshold; the observation vector includes the visual confidence, radar confidence, IMU confidence, number of targets, dust concentration, vibration spectrum, vehicle status, and rule satisfaction. The safety decision module is configured to: receive the visual confidence level, radar confidence level, IMU confidence level, and anomaly flag; calculate the global confidence level; convert the safety rules into differentiable operations through a differentiable logic reasoning submodule according to preset safety rules; verify that all reachable states satisfy the temporal logic safety property through model checking; and output a safety decision conclusion. The preconditions for the safety rules include global confidence level, vehicle speed, obstacle distance, slope, and anomaly flag. The control output module is configured to receive the safety decision conclusion, generate control commands, and send them to the vehicle control unit via a bus.

[0009] Preferably, the method for acquiring the sparse event stream output by the event camera and reconstructing it to obtain the reconstructed image is as follows: Acquire a sparse event stream output by the event camera. The sparse event stream consists of several events, each of which contains pixel coordinates, a timestamp, and the polarity of brightness change. The event spatiotemporal density field of each spatial region is calculated based on the exponentially decaying time window. The event spatiotemporal density value of each spatial region is the sum of all events falling within that spatial region after exponential decay weighting according to their timestamps. The reconstruction time resolution of each spatial region is dynamically adjusted according to the event spatiotemporal density field to obtain the adjusted reconstruction time resolution; wherein, the reconstruction frequency of regions with high event spatiotemporal density is higher than that of regions with low event spatiotemporal density. Using the adjusted reconstruction temporal resolution and the event spatiotemporal density field of each spatial region as input, a dense image reconstruction is performed on the sparse event stream via a multilayer perceptron to obtain the reconstructed image.

[0010] Preferably, the visual target distribution and the radar target distribution are respectively embedded into a symmetric positive definite manifold, the visual weight and radar weight are calculated, and weighted fusion is performed along the geodesic to obtain the fused target distribution. The method is as follows: Receive the visual target distribution and the radar target distribution; The mean vector and covariance matrix of the visual target distribution are mapped into visual symmetric positive definite matrices according to the standard embedding method, and the mean vector and covariance matrix of the radar target distribution are mapped into radar symmetric positive definite matrices according to the standard embedding method. Calculate the visual weight and radar weight separately using the power formula; The visual tangent vector is obtained by calculating the logarithmic mapping of the visual symmetric positive definite matrix, and the radar tangent vector is obtained by calculating the logarithmic mapping of the radar symmetric positive definite matrix. The product of the visual tangent vector and the visual weight is added to the product of the radar tangent vector and the radar weight to obtain the fused tangent vector. Calculate the exponential mapping of the fused tangent vector to obtain the fused symmetric positive definite matrix. Then, analyze the fused mean vector and covariance matrix in the fused symmetric positive definite matrix to obtain the fused target distribution.

[0011] Preferably, the radar confidence level is calculated based on the data from the vehicle body IMU and the bracket IMU. Specifically, the radar confidence level is calculated as the product of the exponential decay of relative acceleration and the exponential decay of relative angular velocity, wherein the relative acceleration and the relative angular velocity are obtained by subtracting the measured values ​​from the vehicle body IMU and the bracket IMU, respectively.

[0012] Preferably, the IMU confidence level is calculated based on the vehicle body acceleration, specifically by calculating the one-sided exponential decay of the vehicle body acceleration. The confidence level is at its maximum when the vehicle body acceleration is below the upper limit of normal acceleration, and the confidence level decays exponentially with the excess amount when the vehicle body acceleration exceeds the upper limit of normal acceleration.

[0013] Preferably, in the confidence prediction module, dust concentration is predicted based on spatial coordinates, time, wind speed, vibration, and excavation markers using a dust diffusion equation. The method is as follows: A physical information neural network is constructed based on spatial coordinates, time, wind speed, vibration acceleration, and excavation markers; The physical information neural network is pre-trained using the dust diffusion equation as a physical prior. During training, the squared error between the labeled true dust concentration and the predicted dust concentration is used as the data fitting loss, the square of the equation residual is used as the physical constraint loss, and the sum of the squares of the parameters of the physical information neural network is used as the regularization loss. The data fitting loss, the physical constraint loss, and the regularization loss are weighted and summed to obtain the total loss. The physical information neural network is trained with the goal of minimizing the total loss. The trained physical information neural network is then propagated forward based on spatial coordinates, time, wind speed, vibration acceleration, and excavation markers to output a predicted dust concentration.

[0014] Preferably, the energy value of the observation vector at the current time is calculated using a pre-constructed energy basis model and converted into anomaly scores. The method is as follows: Using training data containing only normal operating conditions as input, an energy basis model containing single-point potential energy, paired potential energy, and observed potential energy is constructed. Unsupervised training is performed by comparing divergence to obtain the trained energy basis model. The observation vector at the current moment is taken as input. The observation vector is composed of visual confidence, radar confidence, IMU confidence, number of targets, dust concentration, vibration spectrum, vehicle status and rule satisfaction. Input the observation vector into the trained energy basis model to calculate the energy value at the current time. Using the mean and standard deviation of the training set energy as input, the anomaly score is obtained by subtracting the mean of the training set energy from the energy value at the current moment and then dividing by the standard deviation of the training set energy. The anomaly score is compared with a three-sigma threshold. If the anomaly score is greater than the three-sigma threshold, the output is true; otherwise, the output is false, thus obtaining an anomaly flag.

[0015] Preferably, based on preset security rules, the security rules are transformed into differentiable operations through a differentiable logic reasoning submodule. Model checks verify that all reachable states satisfy the temporal logic safety property, and a security decision conclusion is output. The method is as follows: The global confidence level, vehicle speed, obstacle distance, slope, and anomaly markers are used as inputs to serve as prerequisite variables for safety rules. Multiple security rules are preset, each rule includes a precondition and a conclusion, and the precondition is composed of comparison, conjunction or disjunction of the precondition variables; In the differentiable logic reasoning submodule: For comparison types, the satisfiability function is constructed as a Sigmoid function multiplied by the temperature parameter and then by the difference between the variable and the threshold. For conjunctive premises, the satisfiability function is constructed as the product of the satisfiability functions of each subpremises; For the premises of the disjunctive type, the satisfiability function is constructed as a product of the satisfiability functions of each subpresupposition negated; The satisfiability of all rules is weighted and summed, then mapped using the Sigmoid function to obtain the satisfiability of the reasoning conclusion. The model checks and verifies that all reachable states satisfy the temporal logic safety property, and this is output as the safety decision conclusion.

[0016] Preferably, all reachable states are verified to satisfy the time-series logic safety property through model checking, specifically as follows: The global confidence level, the vehicle speed, the obstacle distance, and the slope are each discretized into a finite number of discrete gears, which are then combined to form a finite state space. The safety properties are described using a linear time-series logic formula. These safety properties include: if the vehicle speed is greater than the speed threshold, the global confidence level must be greater than the confidence threshold at some future moment; if there is an obstacle and the distance is less than the distance threshold, braking must be performed in the next step; and once the safety mode is entered, the speed will never exceed the safety speed threshold. The safety decision module is modeled as a finite state automaton, with the finite state space as the state set and the safety decision conclusion as the state transition output; The symbolic model checking algorithm traverses all reachable states to verify whether the linear sequential logic formula is satisfied in each state.

[0017] Preferably, the visual weight and radar weight are calculated separately according to the power formula, and the method is as follows: The visual weight is obtained by dividing the power of the visual confidence by the sum of the power of the visual confidence and the power of the radar confidence. The radar weight is obtained by dividing the power of the radar confidence by the sum of the power of the visual confidence and the power of the radar confidence. The power exponent is greater than one, which is used to amplify the difference between the visual confidence level and the radar confidence level.

[0018] The beneficial effects of this invention are: 1) By introducing an event-driven perception mechanism to replace the traditional uniform frame sampling through a biomimetic non-uniform sampling perception mechanism, the reconstruction time resolution of each spatial region is dynamically adjusted based on the event spatiotemporal density field. This breaks the inherent trade-off between motion blur and data redundancy in traditional cameras, significantly improves the effective information retention rate and data acquisition efficiency of images under strong vibration conditions, and enhances the robustness of the perception system. 2) By using a non-Euclidean geometry adaptive geodesic fusion architecture, the visual target distribution and radar target distribution are embedded into a symmetrical positive definite manifold. Power-law weights are used for weighted fusion along the geodesic. Manifold geometry preservation replaces Euclidean space weighted averaging, which solves the problems of traditional fusion methods destroying the positive definiteness and anisotropy of covariance and the physical unrealizability of the target distribution after fusion, thus improving the reliability of the perception system. 3) By using a physical information neural network confidence prediction mechanism, a dust concentration prediction network is constructed with the dust convection-diffusion-settling equation as the physical prior. Combined with the vehicle body vibration and support vibration separated by dual IMUs, the radar confidence is calculated by multiplicative exponential decay of relative acceleration and relative angular velocity, and the visual confidence is synthesized by multiplicative combination of dust concentration exponential decay and vibration neural network modulation. This realizes the replacement of pure data-driven evaluation with physical information-driven confidence assessment, solves the problems of lack of physical interpretability and poor generalization ability of small samples in confidence assessment, and provides a physical basis for the traceability of the perception system. 4) An unsupervised anomaly detection mechanism based on energy basis model is constructed, which includes single-point potential energy, paired potential energy and observed potential energy. The model is trained unsupervised by contrast divergence using training data containing only normal working conditions. The model outputs Z-score anomaly score and anomaly label. Energy basis distribution learning replaces rule threshold anomaly detection, which solves the problems of difficulty in labeling anomaly samples and difficulty in exhaustively enumerating rules under complex working conditions, and further enhances the robustness of the perception system. 5) Through the neural symbolic differentiable logic safe decision-making architecture, the safety rules are transformed into differentiable operations through the differentiable logic reasoning submodule. After the model checks and verifies that all reachable states satisfy the temporal logic safety property, the safety decision conclusion is output. Differentiable symbolic reasoning and formal verification replace the hard decision-making of the rule engine, which solves the problems of lack of interpretability, inability to be formally verified, and difficulty in proving safety in safety decisions. It mathematically guarantees the safety of the perception system and provides a complete traceability chain for the decision-making process. In summary, through the collaborative design of biomimetic non-uniform sampling, non-Euclidean geometric manifold fusion, physical information neural network confidence prediction, energy-based unsupervised anomaly detection, and neural symbol differentiable logic safety decision-making, a complete technical closed loop from data acquisition, target fusion, confidence assessment, anomaly detection, and safety decision-making has been formed, effectively meeting the core stringent requirements of heavy machinery operations for the safety, robustness, and traceability of the sensing system. Attached Figure Description

[0019] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is a block diagram of the pattern recognition system for heavy machinery environment perception according to the present invention.

[0020] Figure 2 This is a flowchart of the differentiable logic reasoning submodule of the pattern recognition system for heavy machinery environment perception according to the present invention. Detailed Implementation

[0021] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the invention. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0022] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0023] To more clearly illustrate the pattern recognition system for heavy machinery environment perception of the present invention, the following description is in conjunction with... Figures 1 to 2 The steps in the embodiments of the present invention will be described in detail below.

[0024] The first embodiment of the present invention is a pattern recognition system for heavy machinery environmental perception, see [link to relevant documentation]. Figure 1 The system includes: The biomimetic non-uniform sampling sensing module is configured to: acquire the sparse event stream output by the event camera and reconstruct the reconstructed image; In this embodiment, the biological vision system (such as the compound eye of an insect) adopts a non-uniform spatiotemporal sampling strategy—high sampling rate in areas of intense motion and low sampling rate in static areas. This biomimetic mechanism is introduced into the environmental perception of heavy machinery to solve the problem of large amounts of motion blur and redundant data generated by traditional uniform sampling under severe vibration. The sparse event stream output by the event camera is collected and reconstructed to obtain the reconstructed image. The method is as follows: The sparse event stream output by the event camera is acquired. The sparse event stream consists of several events, each of which contains pixel coordinates, a timestamp, and brightness change polarity. Specifically, the sparse event stream is acquired by two event cameras (model: IniVation DAVIS346, resolution 346×260 pixels, time resolution 1 microsecond, dynamic range 120dB) installed on the top of the heavy machinery cab (such as an excavator) in the forward and backward directions. Each event contains pixel coordinates xi∈[0,345], yi∈[0,259], timestamp ti (unit: microsecond, precision 1μs), and brightness change polarity pi∈{-1,+1}, where +1 indicates brightness increase and -1 indicates brightness decrease. The sparse event stream is acquired in real time with a sampling period of 0.033 seconds (corresponding to a frame rate of 30Hz). The event spatiotemporal density field of each spatial region is calculated based on the exponentially decaying time window. The event spatiotemporal density value of each spatial region is the sum of all events falling into that spatial region after exponential decay weighting according to their timestamps. Specifically, at startup, the following preset parameters are first set: the time constant τ of the exponential decay time window is 50ms, the grid is divided into 40 rows × 30 columns, the minimum reconstruction frequency is 1Hz, and the maximum reconstruction frequency is 200Hz. The time constant τ = 50ms means that the half-life of the event influence decays exponentially with time is about 35ms. This value is set according to 1% to 2% of the typical operation cycle of the heavy machine (2~5 seconds). The first 10 frames of data are used to initialize the statistical baseline of the spatiotemporal density field. After entering the real-time reconstruction loop, the event spatiotemporal density field of each spatial region is calculated based on the exponentially decaying time window. Specifically, the image is divided into a grid of 40 rows by 30 columns, with each grid being 8.65 pixels by 8.67 pixels. The event spatiotemporal density value of each spatial region is the sum of all events falling within that spatial region after exponential decay weighting according to their timestamps. The reconstruction time resolution of each spatial region is dynamically adjusted according to the event spatiotemporal density field to obtain the adjusted reconstruction time resolution; wherein, the reconstruction frequency of regions with high event spatiotemporal density is higher than that of regions with low event spatiotemporal density; specifically, regions with high event spatiotemporal density are reconstructed once every 5 milliseconds (equivalent to 200 Hz), and regions with low event spatiotemporal density are reconstructed once every 1000 milliseconds (equivalent to 1 Hz). Using the adjusted reconstruction temporal resolution and the event spatiotemporal density field of each spatial region as input, a dense image reconstruction is performed on the sparse event stream via a multilayer perceptron to obtain a reconstructed image. Specifically, the multilayer perceptron contains 4 hidden layers, each with 256 neurons, and the activation function is ReLU (Modified Linear Unit, defined as ReLU(z)=max(0,z). The network input undergoes position encoding processing, mapping the 1-dimensional input to a 12-dimensional high-dimensional space. The number of encoding layers L=6, and the larger L is, the richer the high-frequency information. The output reconstructed image resolution is 40×30 grids, which is upsampled to 320×256 pixels by bilinear interpolation to obtain the reconstructed image. The confidence prediction module is configured to: calculate radar confidence based on vehicle body IMU and support IMU data; calculate IMU confidence based on vehicle body acceleration; predict dust concentration based on spatial coordinates, time, wind speed, vibration, and excavation markers using a dust diffusion equation; receive the predicted dust concentration and vibration, and synthesize visual confidence using exponential decay and neural network modulation. In this embodiment, the traditional confidence level is a data-driven black box model, which is difficult to generalize to unseen working conditions. The PINN is constructed using the dust diffusion physical equation as a priori, so that the confidence level has physical meaning and achieves high accuracy with small samples. The dual IMU calculates the relative vibration through the difference, which can identify the elastic torsional vibration of the sensor support. The confidence level formula adopts multiplicative exponential decay to reflect the independent deterioration effect of dust and vibration on the confidence level. The radar confidence level is calculated based on data from the vehicle-mounted IMU and the support IMU. Specifically, the radar confidence level is calculated as the product of the exponential decay of relative acceleration and the exponential decay of relative angular velocity, where the relative acceleration and relative angular velocity are obtained by subtracting the measured values ​​from the vehicle-mounted IMU and the support IMU, respectively. Specifically, this is achieved using two MEMS-IMUs (model: ADI ADIS16507, acceleration measurement range ±6g, angular velocity measurement range ±1000°) installed on the vehicle frame and at the root of the sensor support. Inertial data is collected at a sampling rate of 200Hz (per second), and the vehicle's IMU measures the vehicle's acceleration. (unit: g) and vehicle angular velocity (Unit: rad / s), support IMU measures support acceleration. (unit: g) and support angular velocity (Unit: rad / s) Subtract the acceleration measured by the vehicle body IMU from the acceleration measured by the support IMU to obtain the relative acceleration vector. Subtract the angular velocity measured by the vehicle body IMU from the angular velocity measured by the support IMU to obtain the relative angular velocity vector. ; Calculate the Euclidean norm of the relative acceleration vector Euclidean norm of relative angular velocity vector Radar confidence level Calculated by the following formula: ; Among them, radar confidence Dimensionless, with values ​​ranging from [0,1]. The acceleration threshold indicates that the confidence level decreases to 36.8% when the relative acceleration exceeds 2.5g. The angular velocity threshold indicates that the confidence level decreases to 36.8% when the relative angular velocity exceeds 2.0 rad / s. This formula uses a multiplicative exponential decay form to reflect the independent deterioration effect of acceleration and angular velocity on radar point cloud quality. The IMU confidence level is calculated based on vehicle acceleration. Specifically, it is calculated using a one-sided exponential decay method for vehicle acceleration. The confidence level is at its maximum when the vehicle acceleration is below the upper limit of normal acceleration, and decreases exponentially with the excess amount when the vehicle acceleration exceeds the upper limit of normal acceleration. Specifically, the IMU confidence level... Calculated by the following formula: ; Among them, IMU confidence Dimensionless, with values ​​ranging from [0,1]. The acceleration norm of the vehicle body, in g; This represents the upper limit of normal acceleration; values ​​below this do not affect the confidence level. To achieve saturation acceleration, the decay rate is controlled. When the vehicle's acceleration falls below the upper limit of normal acceleration, When the vehicle acceleration exceeds the upper limit of normal acceleration, the confidence level decays exponentially with the excess amount; the formula adopts a one-sided exponential decay form, only penalizing vibrations that exceed the normal threshold. In the confidence prediction module, dust concentration is predicted based on spatial coordinates, time, wind speed, vibration, and excavation markers using a dust diffusion equation. The method is as follows: Based on spatial coordinates x , y (unit: m), time t (unit: s), wind speed (u, v, w) (unit: m / s), vibration acceleration (a) x ,a y ,a z (unit: g) and excavation markers (Dimensionless, taking values ​​of 0 or 1, 1 indicating mining in progress) Construct a Physical Information Neural Network (PINN). The PINN structure consists of 6 hidden layers, each with 256 neurons. The activation function is tanh (hyperbolic tangent function, defined as: ; The physical information neural network is pre-trained using the dust diffusion equation as a physical prior. The dust diffusion equation is a convection-diffusion-sedimentation equation that describes dust concentration. (Unit: μg / m) 3 The training process used the actual values ​​of the labeled dust concentration to study the evolution over time and space. With the predicted dust concentration The squared error is used as the data fitting loss, and the residual of the equation is used as the data fitting loss. The square of the physical constraint loss is used as the physical constraint loss, and the sum of the squares of the parameters of the physical information neural network is used as the regularization loss. The data fitting loss, the physical constraint loss, and the regularization loss are weighted and summed to obtain the total loss, where the physical constraint loss weights are... Regularized loss weights ; The physical information neural network is trained with the goal of minimizing the total loss, and the trained physical information neural network is then applied to spatial coordinates ( x , y Time t, wind speed (u, v, w), vibration acceleration (a) x ,a y ,a z ) and excavation markers Perform forward propagation and output predicted dust concentration. ; The predicted dust concentration and vibration are received, and a visual confidence score is synthesized using exponential decay and neural network modulation; specifically, the predicted dust concentration... Divide by dust threshold Then, taking the exponential decay, we obtain the dust concentration decay term; the vibration acceleration (a) is then... x ,a y ,a z The input is a three-layer fully connected neural network (3→64→32→1). The network output is mapped using a sigmoid function to obtain the vibration modulation term. The dust concentration attenuation term is multiplied by the vibration modulation term to obtain the visual confidence level. The formula expression is: ; Among them, visual confidence Dimensionless, value range [0,1]; dust threshold correspond Visibility was approximately 100 meters. This maps the input to the (0,1) interval; It is a three-layer fully connected neural network (3→64→32→1), and the input is the triaxial vibration acceleration (a x ,a y ,a zThe formula outputs a scalar; it combines physical predictions (exponential decay of dust concentration) with data-driven (vibration modulation) multiplicatively, while maintaining physical interpretability. Upon initial startup, the parameters of the physical information neural network are initialized using a pre-trained model. The visual confidence Cv and radar confidence Cr are initialized to 0.5. The first 100 frames are used as a warm-up phase. During the warm-up, confidence-weighted fusion is not used, but equal-weighted fusion (wv=wr=0.5) is used. After the warm-up is completed, the accumulated deterministic knowledge is used to fine-tune PINN. After that, confidence-weighted fusion is enabled. The target distribution extraction and fusion module is configured to: receive the reconstructed image and extract the visual target distribution; receive the radar point cloud and extract the radar target distribution; embed the visual target distribution and the radar target distribution into a symmetric positive definite manifold respectively; calculate the visual weight and the radar weight; and perform weighted fusion along the geodesic to obtain the fused target distribution. In this embodiment, traditional fusion performs weighted averaging in Euclidean space, which disrupts the geometric structure of the statistical manifold where each sensor output resides. For example, the covariance matrix of radar point clouds is a point on a symmetric positive definite manifold, and Euclidean averaging would produce a non-positive definite matrix, which is physically unrealizable. Geodesic fusion is performed on a symmetric positive definite manifold to maintain the positive definiteness of the covariance and the geometric structure. At the same time, power-law weights (α=1.5) amplify the confidence differences, allowing high-confidence sensors to dominate the fusion. The visual target distribution and the radar target distribution are embedded into symmetric positive definite manifolds respectively, the visual weights and radar weights are calculated, and weighted fusion is performed along the geodesics to obtain the fused target distribution. The method is as follows: The system receives the visual target distribution and the radar target distribution; specifically, it extracts the visual target distribution based on the reconstructed image using a lightweight U-Net. The U-Net contains a 3-layer encoder and a 3-layer decoder. The number of encoder channels increases from 64 to 256, while the number of decoder channels decreases symmetrically. Skip connections preserve spatial details, and the system outputs a class probability map. k(x,y)∈[0,1], k∈{1,2,3} correspond to rigid obstacles (stones, concrete blocks), flexible obstacles (soil piles, construction nets) and passable terrain, respectively. For each connected region, i.e. each target, its Gaussian distribution parameters are calculated. The target center μv is the weighted average of all pixel coordinates with the category probability map as the weight. The covariance matrix Σv is the weighted average of the vector outer product of each pixel's deviation from the center with the category probability map as the weight. Two 4D millimeter-wave radars (model: Arbe Phoenix, field of view 120°) installed at the front and rear positions of the top of the cab are used. ×30 Radar point clouds are collected (maximum detection range 200 meters). Each point cloud contains its 3D coordinates, radial velocity, and signal-to-noise ratio. The typical number of point clouds is 50 to 500. Based on the radar point clouds, the DBSCAN algorithm is used for clustering, with a neighborhood radius of 1.5 meters and a minimum number of points of 3. For each cluster (i.e., each target), its Gaussian distribution parameters are calculated. The target center μr is the arithmetic mean of the coordinates of all points within the cluster, and the covariance matrix Σr is the arithmetic mean of the outer products of the vectors deviating from the center of each point within the cluster. The radar point cloud coordinates are projected onto the image plane using the rotation matrix R and translation vector T obtained through joint calibration: [u,v]T=proj(R [xr,yr,zr]T+T), after conversion, the mean and covariance of radar targets are unified in pixels, consistent with the distribution of visual targets; The mean vector μv and covariance matrix Σv of the visual target distribution are mapped to a visual symmetric positive definite matrix X using the standard embedding method. v The mean vector μr and covariance matrix Σ of the radar target distribution are... r Mapped to a radar symmetric positive definite matrix X according to the standard embedding method. r ; ; Where Xv and Xr are 3×3 symmetric positive definite matrices, constructed by multiplying the 2×2 covariance moment Σ by the 2×1 central vector μ. T The sum is used as the 2×2 block in the upper left corner; the center vector μ is used as the 2×1 block in the upper right corner; the transpose of the center vector μT is used as the 1×2 block in the lower left corner; and the constant 1 is used as the 1×1 block in the lower right corner. This embedding method encodes all the information of the Gaussian distribution N(μ,Σ) into a 3×3 symmetric positive definite matrix. The visual weight and radar weight are calculated separately using the power formula, and the method is as follows: The visual weight wv is obtained by dividing the power of visual confidence Cv raised to the power of α by the sum of Cv raised to the power of α and radar confidence Cr raised to the power of α; the visual weight wv is obtained by dividing the power of radar confidence Cr raised to the power of α. α Dividing the power of Cv by the sum of the powers of Cv and Cr, we obtain the radar weight wr, expressed by the formula: ; Where wv and wr are the visual weight and radar weight, respectively, dimensionless, with values ​​ranging from [0,1]; Cv and Cr are the visual confidence and radar confidence, respectively, dimensionless, with values ​​ranging from [0,1]. Specifically, the confidence prediction module outputs the visual confidence Cv and radar confidence Cr, representing the reliability of visual perception and radar perception, respectively, calculated by the confidence prediction module based on the dust diffusion equation and dual IMU vibration measurement; the power exponent α=1.5 is used to amplify the difference between visual confidence and radar confidence, so that the high-confidence sensor dominates the fusion result; Calculate the visual symmetric positive definite matrix X v The visual tangent vector Zv is obtained by the logarithmic mapping of the radar symmetric positive definite matrix Xr. The radar tangent vector Zr is obtained by calculating the logarithmic mapping of the symmetric positive definite matrix Xr. The logarithmic mapping first performs eigenvalue decomposition on the symmetric positive definite matrix X=UΛU. T Then Z = log(X) = Ulog(Λ)U T , where log(Λ) is the natural logarithm of each diagonal element in the eigenvalue diagonal matrix Λ; The visual tangent vector Z v The product of the visual weight wv and the radar tangent vector Z r The product of the fusion vector and the radar weight wr is obtained. ; ; Calculate the fusion tangent vector The exponential mapping yields a fused symmetric positive definite matrix. The exponent is mapped to ,in Let the natural exponent be taken for each diagonal element in the eigenvalue diagonal matrix Λ, and let the fused symmetric positive definite matrix be... The blocks are as follows: ; in, A Let b be a 2×2 matrix, b be a 2×1 column vector, and c be a scalar. Extract the fused mean vector from the fused symmetric positive definite matrix. Covariance Matrix , ; The mean vector and the covariance matrix As the target distribution after fusion; The anomaly detection module is configured to: calculate the energy value of the observation vector at the current moment using a pre-built energy basis model and convert it into an anomaly score, and then output an anomaly flag after comparing it with a threshold; the observation vector includes the visual confidence, radar confidence, IMU confidence, number of targets, dust concentration, vibration spectrum, vehicle status, and rule satisfaction. In this embodiment, traditional anomaly detection requires explicitly defining rules or thresholds for normal and abnormal conditions. Under the complex operating conditions of heavy machinery, these rules are difficult to exhaustively enumerate, and abnormal samples are difficult to label. An energy-based model is used to learn the distribution of normal operating conditions, and anomalies are automatically identified through high energy levels, eliminating the need to label abnormal samples. Z-score standardization makes threshold setting independent of absolute energy values. A pre-built energy-based model is used to calculate the energy value of the observation vector at the current moment and convert it into anomaly scores. The method is as follows: Using training data containing only normal operating conditions as input, an energy basis model is constructed, comprising single-point potential energy, paired potential energy, and observed potential energy. Unsupervised training is performed using comparative divergence to obtain the trained energy basis model. Specifically, the energy function E(x,y) of the energy basis model is defined as the sum of the negative three terms, where single-point potential energy... ,in, All are learnable parameters; paired potential energy , is the output of a small neural network; observed potential energy Let x be the squared error between the observed and predicted values, x ∈ R64, and y ∈ R32. Unsupervised training is performed using contrastive divergence, and Langevin dynamics is used to sample from the model distribution. The step size is... =0.01, sampling steps T=500 ， After training, the trained energy basis model is obtained. Taking the observation vector at the current moment as input, the observation vector is composed of visual confidence. Radar confidence IMU confidence level The data is composed of a 32-dimensional observation vector, which is formed by splicing together the number of targets, dust concentration, vibration spectrum (20-dimensional, first 20 frequencies of acceleration FFT), vehicle status (vehicle speed, engine speed, hydraulic pressure, 3-dimensional), and rule satisfaction (4-dimensional). ); The observed vector is input into the trained energy basis model to calculate the energy value at the current time. ,in, The latent variable corresponding to the minimum energy; The mean of the training set energy and standard deviation As input, the anomaly score is obtained by subtracting the mean of the training set's energy from the current energy value and then dividing by the standard deviation of the training set's energy. ; ; The anomaly score is compared with a three-sigma threshold of 3, which corresponds to the 99.7% confidence quantile of a normal distribution. If the anomaly score is greater than the three-sigma threshold, the output is true; otherwise, the output is false, thus obtaining an anomaly flag. The safety decision module is configured to: receive the visual confidence level, radar confidence level, IMU confidence level, and anomaly flag; calculate the global confidence level; convert the safety rules into differentiable operations through a differentiable logic reasoning submodule according to preset safety rules; verify that all reachable states satisfy the temporal logic safety property through model checking; and output a safety decision conclusion. The preconditions for the safety rules include global confidence level, vehicle speed, obstacle distance, slope, and anomaly flag. In this embodiment, purely data-driven security decisions lack interpretability and formal verification capabilities. Security rules are encoded into symbolic logic and jointly reasoned with neural networks to achieve provably secure decisions. The differentiable logic reasoning submodule transforms symbolic rules into differentiable operations, supporting end-to-end learning. The model check traverses all reachable states to verify security properties. Receive the visual confidence score, radar confidence score, IMU confidence score, and the anomaly flag, and calculate the global confidence score. Specifically, the global confidence score... The confidence level is calculated by adding the minimum sum of visual confidence and radar confidence at all spatial locations to the IMU confidence level, where the IMU confidence level is uniformly distributed in space. Based on preset security rules, the security rules are transformed into differentiable operations through a differentiable logic reasoning submodule. Model checks verify that all reachable states satisfy the temporal logic safety property, and a security decision conclusion is output. The method is as follows: With global confidence Speed v (unit: m / s), obstacle distance d (unit: m), slope ψ (Unit degree) and the anomaly flag (Boolean value) are inputs, serving as prerequisite variables for the safety rules; Multiple safety rules are preset, each rule including preconditions and conclusions. The preconditions are formed by comparison, conjunction, or disjunction of the precondition variables. Specifically, five safety rules are preset: Rule R1: If the vehicle speed is greater than 0.5 meters per second, the global confidence level must be greater than 0.7; Rule R2: If a rigid obstacle is detected and the distance is less than 2 meters, braking is performed; Rule R3: If the slope is greater than 25 degrees and the global confidence level is less than 0.5, the articulated mechanism is locked; Rule R4: If the sum of the visual confidence level and the radar confidence level at all spatial locations is less than 0.5, the speed is reduced to 0.5 meters per second; Rule R5: If the anomaly flag is true, the vehicle enters safety mode. In the differentiable logic reasoning submodule: For comparison-type premises (assuming the atomic premise is x>θ or x<θ), the satisfiability function is constructed as a Sigmoid function multiplied by the temperature parameter and then multiplied by the difference between the variable and the threshold. ; in, Let [function name] be a satisfiability function, with a value range of [0,1]. For the Sigmoid function; The temperature parameter controls the sharpness of the soft threshold; the larger the value, the closer the Sigmoid function is to a step function, and the smaller the value, the smoother the transition; x is the variable value; θ is the threshold. For the premise of conjunction type (let the composite premise P = P1∧P2∧) ∧Pn, where n represents the number of sub-premises constituting the composite premise; P1, P2, , P n Let be the sub-premise, and ∧ be the logical conjunction operator, indicating that all sub-premisees must be satisfied simultaneously. The satisfiability function is constructed as the product of the satisfiability functions of each sub-premise. In the formula, Let be a satisfiability function. The symbol for multiplication; For the premise of disjunction (let the composite premise be P = P1∨P2∨), ∨Pn), constructing the satisfiability function as a product of satisfiability functions minus the negations of each subpresupposition, i.e. In the formula, ∨ is the logical disjunction (OR) operator. For the first The satisfiability of the negation of a sub-premise; After weighted summing of the satisfiability of all rules and mapping it using the Sigmoid function, the satisfiability of the reasoning conclusion is obtained. The expression for the satisfiability of the reasoning conclusion is: ; Where wr is the rule weight, which defaults to 1 and can be fine-tuned through learning; Conclusion is the satisfaction level of the security decision conclusion, with a value range of [0,1]. The model checks and verifies that all reachable states satisfy the sequential logic safety property, and outputs the safety decision conclusion as the result. In this embodiment, model checks are used to verify that all reachable states satisfy the time-series logic safety property, specifically as follows: The global confidence level The speed of the vehicle v The obstacle distance d and the slope ψEach value is discretized into a finite number of discrete gears, which are combined to form a finite state space. The vehicle speed is discretized into 6 gears (0, 0.25, 0.5, 0.75, 1.0, 1.50 meters per second), the confidence level is discretized into 6 gears (0, 0.2, 0.4, 0.6, 0.8, 1.0), the obstacle distance is discretized into 6 gears (0, 1, 2, 3, 5, 10 meters), and the slope is discretized into 6 gears (0, 10, 15, 20, 25, 30 degrees). These are combined to form a finite state space with a size of 1296 states. The safety property is described using a linear time-series logic formula. This property includes the following: if the vehicle speed is always greater than a speed threshold, then at some future moment, the global confidence level must be greater than the confidence threshold. If an obstacle exists and the distance is less than the distance threshold, braking must be performed as the next step. Once in safe mode, the speed will never exceed the safe speed threshold. , where □ is the "always" operator, meaning that it is true at all future moments; X is the "future" operator, indicating that it will be true at some future moment; X is the "next step" operator, indicating that it will be true at the next moment. For implication operators; brake is the braking action; In safe mode status; The safety decision module is modeled as a finite state automaton, with the finite state space as the state set and the safety decision conclusion as the state transition output; The symbolic model checking algorithm traverses all reachable states and verifies whether the linear temporal logic formula is satisfied in each state. After verification, the satisfiability of the reasoning conclusion is output as the security decision conclusion. The control output module is configured to: receive the safety decision conclusion, generate control commands, and send them to the vehicle control unit via a bus; In this embodiment, the safety decision conclusion is a satisfaction value, which needs to be converted into a binary control command. A threshold is used to trigger braking, deceleration, locking, or alarm actions. The method is as follows: Receive the security decision conclusion; specifically, receive the security decision conclusion (Conclusion) output by the security decision module. Control commands are generated and sent to the vehicle control unit via the bus. Specifically, if the decision result is greater than 0.5, it is determined to trigger an action; if braking is triggered, a CAN braking signal is generated (ID: 0x18F00054, data byte 0x01); if deceleration is triggered, a CAN speed command with a deceleration target of 0.5 meters per second is generated (ID: 0x18F00051, data byte 0x05); if articulation locking is triggered, a locking signal is generated; if an alarm is triggered, an audible and visual alarm command is generated. All control commands are sent to the vehicle control unit via the CAN bus.

[0025] Therefore, through a biomimetic non-uniform sampling perception mechanism, sparse event streams output by event cameras are collected, and the spatiotemporal density field of events is calculated based on an exponentially decaying time window. The reconstruction temporal resolution of each spatial region is dynamically adjusted according to the density field, replacing traditional uniform frame sampling with event-driven perception, breaking the inherent trade-off between motion blur and data redundancy. Through a non-Euclidean geometric adaptive geodesic fusion architecture, the visual target distribution and radar target distribution are embedded in a symmetrical positive definite manifold. Power-law weights are used to calculate the visual weights and radar weights along the geodesic trailing edge for weighted fusion. Manifold geometry preservation replaces Euclidean space weighted averaging, avoiding the problem of traditional fusion destroying the positive definiteness of covariance and anisotropy. Through a physical information neural network confidence prediction mechanism, relying on dual IMUs to separate vehicle body vibration and support vibration, on the one hand, radar confidence is calculated based on the multiplicative exponential decay of relative acceleration and relative angular velocity, and on the other hand, the physical information neural network prediction is constructed using the dust convection-diffusion-settling equation as a physical prior. For dust concentration, a multiplicative combination of dust concentration exponential decay and vibration neural network modulation is used to synthesize visual confidence scores. This replaces pure data-driven confidence assessment with physical information-driven assessment, addressing the issues of lack of interpretability and poor generalization ability for small samples in confidence assessment. An unsupervised anomaly detection mechanism based on an energy-based model is constructed, incorporating single-point potential energy, paired potential energy, and observed potential energy. Using training data containing only normal operating conditions, unsupervised training is conducted via comparative divergence to output anomaly scores and anomaly markers. Energy-based distribution learning replaces rule-based threshold anomaly detection, avoiding the difficulties in labeling anomaly samples and exhaustively listing rules. A neural symbolic differentiable logic safety decision-making architecture is employed. Safety rules are transformed into differentiable operations through a differentiable logic reasoning submodule. After model verification that all reachable states satisfy the temporal logic safety property, a safety decision conclusion is output. Differentiable symbolic reasoning and formal verification replace the hard decision-making of the rule engine, addressing the issues of lack of interpretability and provable safety in safety decisions.

[0026] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process and related explanations of the methods described above can be found in the corresponding processes in the foregoing system embodiments, and will not be repeated here.

[0027] Those skilled in the art will recognize that the modules and method steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. The programs corresponding to the software modules and method steps can be placed in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs, or any other form of storage medium known in the art. To clearly illustrate the interchangeability of electronic hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in electronic hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the invention.

[0028] The terms “first”, “second”, etc., are used to distinguish similar objects, not to describe or indicate a specific order or sequence.

[0029] The term "comprising" or any other similar term is intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus / device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent in such process, method, article, or apparatus / device.

[0030] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after such changes or substitutions will all fall within the scope of protection of the present invention.

Claims

1. A pattern recognition system for heavy machinery environmental perception, characterized in that, The system includes: The biomimetic non-uniform sampling sensing module is configured to: acquire the sparse event stream output by the event camera and reconstruct the reconstructed image; The confidence prediction module is configured to: calculate radar confidence based on vehicle body IMU and support IMU data; calculate IMU confidence based on vehicle body acceleration; predict dust concentration based on spatial coordinates, time, wind speed, vibration, and excavation markers using a dust diffusion equation; receive the predicted dust concentration and vibration, and synthesize visual confidence using exponential decay and neural network modulation. The target distribution extraction and fusion module is configured to: receive the reconstructed image and extract the visual target distribution; receive the radar point cloud and extract the radar target distribution; embed the visual target distribution and the radar target distribution into a symmetric positive definite manifold respectively; calculate the visual weight and the radar weight; and perform weighted fusion along the geodesic to obtain the fused target distribution. The anomaly detection module is configured to: calculate the energy value of the observation vector at the current moment using a pre-built energy basis model and convert it into an anomaly score, and then output an anomaly flag after comparing it with a threshold; the observation vector includes the visual confidence, radar confidence, IMU confidence, number of targets, dust concentration, vibration spectrum, vehicle status, and rule satisfaction. The safety decision module is configured to: receive the visual confidence level, radar confidence level, IMU confidence level, and anomaly flag; calculate the global confidence level; convert the safety rules into differentiable operations through a differentiable logic reasoning submodule according to preset safety rules; verify that all reachable states satisfy the temporal logic safety property through model checking; and output a safety decision conclusion. The preconditions for the safety rules include global confidence level, vehicle speed, obstacle distance, slope, and anomaly flag. The control output module is configured to receive the safety decision conclusion, generate control commands, and send them to the vehicle control unit via a bus.

2. The pattern recognition system for heavy machinery environmental perception according to claim 1, characterized in that, The method for acquiring sparse event streams output by event cameras and reconstructing them to obtain reconstructed images is as follows: Acquire a sparse event stream output by the event camera. The sparse event stream consists of several events, each of which contains pixel coordinates, a timestamp, and the polarity of brightness change. The event spatiotemporal density field of each spatial region is calculated based on the exponentially decaying time window. The event spatiotemporal density value of each spatial region is the sum of all events falling within that spatial region after exponential decay weighting according to their timestamps. The reconstruction time resolution of each spatial region is dynamically adjusted according to the event spatiotemporal density field to obtain the adjusted reconstruction time resolution; wherein, the reconstruction frequency of regions with high event spatiotemporal density is higher than that of regions with low event spatiotemporal density. Using the adjusted reconstruction temporal resolution and the event spatiotemporal density field of each spatial region as input, a dense image reconstruction is performed on the sparse event stream via a multilayer perceptron to obtain the reconstructed image.

3. The pattern recognition system for heavy machinery environmental perception according to claim 1, characterized in that, The visual target distribution and the radar target distribution are respectively embedded into symmetric positive definite manifolds, the visual weights and radar weights are calculated, and then weighted and fused along geodesics to obtain the fused target distribution. The method is as follows: Receive the visual target distribution and the radar target distribution; The mean vector and covariance matrix of the visual target distribution are mapped into visual symmetric positive definite matrices according to the standard embedding method, and the mean vector and covariance matrix of the radar target distribution are mapped into radar symmetric positive definite matrices according to the standard embedding method. Calculate the visual weight and radar weight separately using the power formula; The visual tangent vector is obtained by calculating the logarithmic mapping of the visual symmetric positive definite matrix, and the radar tangent vector is obtained by calculating the logarithmic mapping of the radar symmetric positive definite matrix. The product of the visual tangent vector and the visual weight is added to the product of the radar tangent vector and the radar weight to obtain the fused tangent vector. Calculate the exponential mapping of the fused tangent vector to obtain the fused symmetric positive definite matrix. Then, analyze the fused mean vector and covariance matrix in the fused symmetric positive definite matrix to obtain the fused target distribution.

4. The pattern recognition system for heavy machinery environmental perception according to claim 1, characterized in that, The radar confidence level is calculated based on data from the vehicle-mounted IMU and the bracket IMU. Specifically, the radar confidence level is calculated as the product of the exponential decay of relative acceleration and the exponential decay of relative angular velocity, wherein the relative acceleration and the relative angular velocity are obtained by subtracting the measured values ​​from the vehicle-mounted IMU and the bracket IMU, respectively.

5. The pattern recognition system for heavy machinery environmental perception according to claim 1, characterized in that, The confidence level of the IMU is calculated based on the vehicle body acceleration. Specifically, it is calculated according to the one-sided exponential decay of the vehicle body acceleration. The confidence level is at its maximum when the vehicle body acceleration is lower than the upper limit of normal acceleration, and the confidence level decays exponentially with the excess amount when the vehicle body acceleration exceeds the upper limit of normal acceleration.

6. The pattern recognition system for heavy machinery environmental perception according to claim 1, characterized in that, In the confidence prediction module, dust concentration is predicted based on spatial coordinates, time, wind speed, vibration, and excavation markers using a dust diffusion equation. The method is as follows: A physical information neural network is constructed based on spatial coordinates, time, wind speed, vibration acceleration, and excavation markers; The physical information neural network is pre-trained using the dust diffusion equation as a physical prior. During training, the squared error between the labeled true dust concentration and the predicted dust concentration is used as the data fitting loss, the square of the equation residual is used as the physical constraint loss, and the sum of the squares of the parameters of the physical information neural network is used as the regularization loss. The data fitting loss, the physical constraint loss, and the regularization loss are weighted and summed to obtain the total loss. The physical information neural network is trained with the goal of minimizing the total loss. The trained physical information neural network is then propagated forward based on spatial coordinates, time, wind speed, vibration acceleration, and excavation markers to output a predicted dust concentration.

7. The pattern recognition system for heavy machinery environmental perception according to claim 1, characterized in that, The energy value of the observation vector at the current time is calculated using a pre-built energy basis model and converted into anomaly scores. The method is as follows: Using training data containing only normal operating conditions as input, an energy basis model containing single-point potential energy, paired potential energy, and observed potential energy is constructed. Unsupervised training is performed by comparing divergence to obtain the trained energy basis model. The observation vector at the current moment is taken as input. The observation vector is composed of visual confidence, radar confidence, IMU confidence, number of targets, dust concentration, vibration spectrum, vehicle status and rule satisfaction. Input the observation vector into the trained energy basis model to calculate the energy value at the current time. Using the mean and standard deviation of the training set energy as input, the anomaly score is obtained by subtracting the mean of the training set energy from the energy value at the current moment and then dividing by the standard deviation of the training set energy. The anomaly score is compared with a three-sigma threshold. If the anomaly score is greater than the three-sigma threshold, the output is true; otherwise, the output is false, thus obtaining an anomaly flag.

8. The pattern recognition system for heavy machinery environmental perception according to claim 1, characterized in that, Based on preset security rules, the security rules are transformed into differentiable operations through a differentiable logic reasoning submodule. Model checks verify that all reachable states satisfy the temporal logic safety property, and a security decision conclusion is output. The method is as follows: The global confidence level, vehicle speed, obstacle distance, slope, and anomaly markers are used as inputs to serve as prerequisite variables for safety rules. Multiple security rules are preset, each rule includes a precondition and a conclusion, and the precondition is composed of comparison, conjunction or disjunction of the precondition variables; In the differentiable logic reasoning submodule: For comparison types, the satisfiability function is constructed as a Sigmoid function multiplied by the temperature parameter and then by the difference between the variable and the threshold. For conjunctive premises, the satisfiability function is constructed as the product of the satisfiability functions of each subpremises; For the premises of the disjunctive type, the satisfiability function is constructed as a product of the satisfiability functions of each subpresupposition negated; The satisfiability of all rules is weighted and summed, then mapped using the Sigmoid function to obtain the satisfiability of the reasoning conclusion. The model checks and verifies that all reachable states satisfy the temporal logic safety property, and this is output as the safety decision conclusion.

9. The pattern recognition system for heavy machinery environmental perception according to claim 8, characterized in that, The model checks and verifications confirm that all reachable states satisfy the time-series logic safety property, specifically: The global confidence level, the vehicle speed, the obstacle distance, and the slope are each discretized into a finite number of discrete gears, which are then combined to form a finite state space. The safety properties are described using a linear time-series logic formula. These safety properties include: if the vehicle speed is greater than the speed threshold, the global confidence level must be greater than the confidence threshold at some future moment; if there is an obstacle and the distance is less than the distance threshold, braking must be performed in the next step; and once the safety mode is entered, the speed will never exceed the safety speed threshold. The safety decision module is modeled as a finite state automaton, with the finite state space as the state set and the safety decision conclusion as the state transition output; The symbolic model checking algorithm traverses all reachable states to verify whether the linear sequential logic formula is satisfied in each state.

10. The pattern recognition system for heavy machinery environmental perception according to claim 3, characterized in that, The visual weight and radar weight are calculated separately using the power formula, and the method is as follows: The visual weight is obtained by dividing the power of the visual confidence by the sum of the power of the visual confidence and the power of the radar confidence. The radar weight is obtained by dividing the power of the radar confidence by the sum of the power of the visual confidence and the power of the radar confidence. The power exponent is greater than one, which is used to amplify the difference between the visual confidence level and the radar confidence level.

Images