Environment sensing method and device, vehicle automatic driving control method and medium

Abstract

The invention discloses an environment sensing method and device, a vehicle automatic driving control method and a medium. The method comprises the steps that an environment image is acquired; the environment image is input into a target detection model for processing to obtain detection information of at least one target object in the environment image, a histogram converter module is fused in the target detection model, and the histogram converter module is used for dynamically adjusting histogram distribution of the input image; tracking the target object based on the detection information to generate a historical motion track of the target object; performing trajectory prediction on the target object based on a preset state estimation algorithm and the historical motion trajectory to obtain a future motion trajectory of the target object; and generating an environment perception result based on the historical motion trail and the future motion trail. According to the invention, the histogram converter module is integrated into the target detection network, the whole process optimization from image feature enhancement to target behavior prediction is realized, and the accuracy of environment perception in a complex scene is improved.

Description

Technical Field

[0001] This application relates to the field of environmental perception technology, specifically to an environmental perception method, device, control method and medium for autonomous driving of vehicles. Background Technology

[0002] With the continuous development of deep learning algorithms and in-vehicle computing platforms, environmental perception technology based on visual sensors has become the core foundation for intelligent decision-making in autonomous vehicles. Currently, mainstream environmental perception solutions primarily rely on high-precision cameras and other visual sensors to identify and locate targets such as vehicles and pedestrians through target detection models, thereby providing information support for subsequent decision-making and control.

[0003] However, in real-world autonomous driving applications, especially in environments such as mining areas and complex urban roads, the visual environment in which vehicles operate presents significant challenges. First, existing object detection solutions exhibit a significant decline in perception performance under adverse weather conditions such as rain, fog, snow, and strong sunlight. Weather factors cause image degradation, including decreased contrast, color distortion, and blurred details, making it difficult for traditional object detection models to extract stable and robust features. This ultimately leads to missed detections, false detections, and a loss of positioning accuracy. Second, the unstable quality of the sensor data and the lack of adaptive enhancement processing for the raw image data fundamentally limit the accuracy of subsequent advanced cognitive functions such as trajectory tracking and prediction.

[0004] Therefore, existing environmental sensing technologies suffer from low accuracy. Summary of the Invention

[0005] The purpose of this application is to provide an environmental perception method, device, vehicle autonomous driving control method, and machine-readable storage medium to solve the problem of low accuracy in the environmental perception technology used in the prior art.

[0006] To achieve the above objectives, the first aspect of this application provides an environmental sensing method, which includes: Acquire environmental images; An environmental image is input into a target detection model for processing to obtain detection information of at least one target object in the environmental image. The target detection model incorporates a histogram converter module, which is used to dynamically adjust the histogram distribution of the input image. The target object is tracked based on the detection information to generate the target object's historical movement trajectory; Based on a preset state estimation algorithm and historical motion trajectories, the trajectory of the target object is predicted to obtain the future motion trajectory of the target object. Based on historical and future motion trajectories, environmental perception results are generated.

[0007] In this embodiment, the target detection model further includes a backbone network and a detection head; the environmental image is input into the target detection model for processing to obtain detection information of at least one target object in the environmental image, including: The initial feature map is obtained by extracting features from the environmental image through the backbone network; The initial feature map is adaptively adjusted in histogram distribution and enhanced in feature enhancement by the histogram converter module to obtain the enhanced feature map. The enhanced feature map is decoded by the detection head to generate detection information for at least one target object.

[0008] In this embodiment, the histogram converter module includes a dynamic grouped histogram attention submodule and a gated feedforward submodule; the histogram converter module performs adaptive histogram distribution adjustment and feature enhancement processing on the initial feature map to obtain an enhanced feature map, including: The initial feature map is calibrated by a dynamic grouping histogram attention submodule to obtain an attention-enhanced feature map. The attention-enhanced feature map is obtained by performing nonlinear transformation and feature fusion processing on the gated feedforward submodule.

[0009] In this embodiment of the application, the initial feature map is calibrated using a dynamic grouped histogram attention submodule to obtain an attention-enhanced feature map, including: The initial feature map is reorganized along the channel dimension to generate an ordered feature representation; Based on ordered feature representation, global feature associations are established through convolutional mapping, and valuable features are generated through depthwise separable convolutional operations. Value features are grouped and restructured to calculate attention weights; The attention weights are fused with the initial feature map to obtain the attention-enhanced feature map.

[0010] In this embodiment of the application, the attention-enhanced feature map is subjected to nonlinear transformation and feature fusion processing through a gated feedforward submodule to obtain the enhanced feature map, including: The resolution of the attention-enhanced feature map is adjusted to obtain a low-resolution feature map. Local feature extraction and nonlinear transformation are performed on the low-resolution feature map to obtain the processed low-resolution feature map. The processed low-resolution feature map is then gated and fused with the attention-enhanced feature map to obtain the fused feature map. The fused feature map is then subjected to resolution restoration processing to obtain the enhanced feature map.

[0011] In this embodiment of the application, during the process of trajectory prediction of the target object, the method further includes: dynamically adjusting the parameters of the preset state estimation algorithm based on a preset reward and punishment mechanism.

[0012] In this embodiment of the application, the parameters of the preset state estimation algorithm are dynamically adjusted based on a preset reward and punishment mechanism, including: Determine the first motion trend index of adjacent trajectory points in the historical motion trajectory, and the second motion trend index of adjacent trajectory points in the prediction. Compare whether the absolute difference between the second motion trend indicator and the first motion trend indicator exceeds a preset threshold; If the absolute difference exceeds a preset threshold, adjust the noise matrix of the preset state estimation algorithm.

[0013] In this embodiment, the noise matrix includes a process noise matrix and an observation noise matrix. The noise matrix for adjusting the preset state estimation algorithm includes: In the case of the initial adjustment trigger, increase the process noise matrix; If the adjustment is triggered again within a preset time and / or a preset number of times after the initial trigger, the observation noise matrix is ​​increased.

[0014] In this embodiment of the application, tracking the target object based on detection information to generate the historical motion trajectory of the target object includes: Based on the detection information, the target object is tracked by the tracker to generate an initial historical motion trajectory; After the sequence length of the initial historical motion trajectory reaches the first preset threshold, the coordinate jump points in the initial historical motion trajectory are smoothed by a fitting algorithm containing preset constraints in order to obtain the historical motion trajectory.

[0015] A second aspect of this application provides a control method for autonomous driving of a vehicle, the control method comprising: Determine the environmental perception results of the vehicle's surroundings, which are determined based on the environmental perception methods described above. Control vehicle movement based on environmental perception results.

[0016] A third aspect of this application provides an environmental sensing device, comprising: The memory is configured to store instructions; The processor is configured to retrieve instructions from memory and to implement the aforementioned environment-aware methods when executing instructions.

[0017] A fourth aspect of this application provides a machine-readable storage medium on which a program or instructions are stored, and when the program or instructions are executed by a processor, the above-described environment-aware method is implemented.

[0018] The above technical solution first acquires an environmental image, then inputs it into a target detection model for processing to obtain detection information of at least one target object in the environmental image. The target detection model incorporates a histogram converter module, which dynamically adjusts the histogram distribution of the input image. Next, based on the detection information, the target object is tracked to generate its historical motion trajectory. Further, based on a preset state estimation algorithm and the historical motion trajectory, the target object's trajectory is predicted to obtain its future motion trajectory. Finally, based on the historical and future motion trajectories, an environmental perception result is generated. This application, by integrating the histogram converter module into the target detection network and fusing trajectory prediction technology, achieves end-to-end optimization from image feature enhancement to target behavior prediction, significantly improving the accuracy of environmental perception in complex scenes.

[0019] Other features and advantages of the embodiments of this application will be described in detail in the following detailed description section. Attached Figure Description

[0020] The accompanying drawings are provided to further illustrate the embodiments of this application and form part of the specification. They are used together with the following detailed description to explain the embodiments of this application, but do not constitute a limitation on the embodiments of this application. In the drawings: Figure 1 A flowchart illustrating an environmental perception method provided in an embodiment of this application; Figure 2 This is a diagram of the fused module network structure provided in a specific embodiment of this application; Figure 3 A trajectory prediction route map based on a reward and punishment mechanism is provided as a specific embodiment of this application; Figure 4 A flowchart illustrating trajectory tracking and smoothing processing is provided for a specific embodiment of this application; Figure 5 A flowchart illustrating a vehicle autonomous driving control method provided in an embodiment of this application; Figure 6 This is a structural block diagram of an environmental sensing device provided in an embodiment of this application. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only for illustration and explanation of the embodiments of this application and are not intended to limit the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0022] It should be noted that if the embodiments of this application involve directional indicators (such as up, down, left, right, front, back, etc.), the directional indicators are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indicators will also change accordingly.

[0023] Furthermore, if the embodiments of this application involve descriptions such as "first" or "second," these descriptions are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, features defined with "first" or "second" may explicitly or implicitly include at least one of those features. Additionally, the technical solutions of various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. If the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed in this application.

[0024] Figure 1 This is a flowchart illustrating an environmental perception method provided in an embodiment of this application. Figure 1 As shown in the figure, this application provides an environmental perception method, which may include the following steps.

[0025] Step S101: Obtain an environmental image.

[0026] This application addresses the problem of insufficient environmental perception capabilities in complex environments (such as mining areas and rainy, snowy, or foggy weather) by proposing an integrated perception method that combines image enhancement, target tracking, and trajectory prediction. This method can be applied to intelligent fields requiring real-time perception and prediction of dynamic object behavior, such as autonomous driving, security monitoring, robotics, intelligent transportation, and industrial manufacturing. Environmental images refer to real-time or offline image data acquired through image acquisition devices, typically containing scene information such as roads, vehicles, pedestrians, and obstacles. Image acquisition devices can include vehicle-mounted cameras and LiDAR, among others.

[0027] Step S102: The environmental image is input into the target detection model for processing to obtain the detection information of at least one target object in the environmental image. The target detection model incorporates a histogram converter module, which is used to dynamically adjust the histogram distribution of the input image.

[0028] In this embodiment, the object detection model is a deep learning-based computer vision model used to identify target objects in images and output their location, category, and other detection information. To improve the feature extraction capability of the object detection model under adverse conditions, this embodiment integrates a histogram converter module into the object detection model. The histogram converter module is an image enhancement module that can be embedded in a neural network. It can dynamically adjust the histogram distribution of an image or feature map, dynamically adjusting the image contrast and brightness distribution during the feature extraction stage, thereby mitigating image degradation caused by weather conditions such as rain, fog, and snow. In one example, integrating the histogram converter module into the object detection model can be achieved by replacing one or more bottleneck structures in the original object detection network with the histogram converter module, thus deeply integrating the histogram converter module as a core feature extraction component into the network.

[0029] Specifically, after inputting the environmental image into the target detection model, the target detection model can output the bounding box and category information of the detected target objects, that is, obtain the detection information of at least one target object in the environmental image.

[0030] Step S103: Track the target object based on the detection information to generate the historical motion trajectory of the target object.

[0031] Specifically, after the target detection model outputs the bounding box and category information of the target object, the system can use tracking algorithms such as SORT to perform cross-frame association of the target and generate the historical motion trajectory of the target object.

[0032] Step S104: Based on the preset state estimation algorithm and historical motion trajectory, the trajectory of the target object is predicted to obtain the future motion trajectory of the target object.

[0033] In this embodiment, a preset state estimation algorithm is used to predict the future motion state of a target based on its historical motion trajectory. This algorithm can be Kalman filtering, particle filtering, etc. After generating the historical motion trajectory of the target object, the preset state estimation algorithm can be used to predict the motion path of the target object for several future frames, thereby obtaining the future motion trajectory of the target object.

[0034] Step S105: Generate environmental perception results based on historical and future motion trajectories.

[0035] In this embodiment, the environmental perception result is structured information output by the system, which may include the target object's location, trajectory, and behavioral intent. After obtaining the target object's historical and future movement trajectories, the historical and future movement trajectories are integrated to output the environmental perception result.

[0036] The above technical solution first acquires an environmental image, then inputs it into a target detection model for processing to obtain detection information of at least one target object in the environmental image. The target detection model incorporates a histogram converter module, which dynamically adjusts the histogram distribution of the input image. Next, based on the detection information, the target object is tracked to generate its historical motion trajectory. Further, based on a preset state estimation algorithm and the historical motion trajectory, the target object's trajectory is predicted to obtain its future motion trajectory. Finally, based on the historical and future motion trajectories, an environmental perception result is generated. This application, by integrating the histogram converter module into the target detection network and fusing trajectory prediction technology, achieves end-to-end optimization from image feature enhancement to target behavior prediction, significantly improving the accuracy of environmental perception in complex scenes.

[0037] In this embodiment, the target detection model further includes a backbone network and a detection head; the environmental image is input into the target detection model for processing to obtain detection information of at least one target object in the environmental image, including: The initial feature map is obtained by extracting features from the environmental image through the backbone network; The initial feature map is adaptively adjusted in histogram distribution and enhanced in feature enhancement by the histogram converter module to obtain the enhanced feature map. The enhanced feature map is decoded by the detection head to generate detection information for at least one target object.

[0038] In this embodiment, the backbone network is the front-end feature extraction network in the object detection model, such as ResNet or CSPDarkNet, which is responsible for extracting multi-level features from the original image. The detection head is the back-end structure of the object detection model, used to decode the feature map and output the bounding box, category, and confidence score of the target object.

[0039] Specifically, during the model training phase, the backbone network first performs convolution and pooling operations on the input image to generate an initial feature map. The histogram converter module adaptively processes the initial input feature map through a multi-level histogram reconstruction and feature recalibration mechanism. First, it analyzes the statistical distribution of each channel in the feature map and generates a set of adaptive transformation parameters accordingly. Then, through learnable histogram equalization, it dynamically stretches the contrast of high-information regions in the feature space while compressing the dynamic range of low-response regions, thereby optimizing the overall feature distribution. During processing, the histogram converter module can also introduce a lightweight channel-space co-weighting mechanism to softly select and fuse features at different locations based on contextual information. The final output is an enhanced feature map with significantly improved contrast and semantic discriminability, providing a more robust feature representation for the subsequent detection head. Finally, the detection head decodes the enhanced feature map and outputs the detection information of the target object.

[0040] In this embodiment, the histogram converter module includes a dynamic grouped histogram attention submodule and a gated feedforward submodule; the histogram converter module performs adaptive histogram distribution adjustment and feature enhancement processing on the initial feature map to obtain an enhanced feature map, including: The initial feature map is calibrated by a dynamic grouping histogram attention submodule to obtain an attention-enhanced feature map. The attention-enhanced feature map is obtained by performing nonlinear transformation and feature fusion processing on the gated feedforward submodule.

[0041] In this embodiment, the dynamically grouped histogram attention submodule is a variant of the attention mechanism. It enhances the model's ability to perceive key features by grouping, sorting, and weighting the feature map channels. The gated feedforward submodule is a forward propagation structure that controls information flow through a gating mechanism, enhancing the model's nonlinear expressive power.

[0042] Subsequently, the dynamic grouped histogram attention submodule performs channel reorganization, sorting, and attention weighting on the feature maps to enhance the model's ability to perceive target regions. Next, the gated feedforward submodule uses techniques such as depthwise separable convolution and the Mish activation function to perform nonlinear transformations and fusion on the feature maps, further improving the expressive power of the features.

[0043] In the above technical solution, the dynamic grouping attention mechanism enhances the ability to extract key features, while the gated feedforward structure improves the nonlinear expression and generalization ability of the model. The combination of the two enables the target detection model to maintain high detection accuracy and robustness in complex scenarios such as rain, fog, and snow, providing a reliable data foundation for the subsequent trajectory tracking and prediction modules of target objects.

[0044] In this embodiment of the application, the attention weight calibration process of the initial feature map is performed by the dynamic grouping histogram attention submodule to obtain the attention-enhanced feature map, which may include: The initial feature map is reorganized along the channel dimension to generate an ordered feature representation; Based on ordered feature representation, global feature associations are established through convolutional mapping, and valuable features are generated through depthwise separable convolutional operations. Value features are grouped and restructured to calculate attention weights; The attention weights are fused with the initial feature map to obtain the attention-enhanced feature map.

[0045] In this embodiment, the dynamic grouped histogram attention submodule can achieve refined attention calibration of the feature map through processing steps such as channel dimension reorganization, global feature association establishment, value feature generation and attention weight calculation, thereby improving the model's ability to perceive the target region.

[0046] Specifically, in the attention submodule, the initial input feature map is first split and sorted according to its channel dimension to generate ordered feature representations. Then, query, key, and value vectors are generated through convolution operations to establish global associations between the feature maps. Next, depthwise separable convolutions are used to enhance local details of the value features, generating valuable features. Afterward, the valuable features are grouped and their structure reconstructed, and attention weights are calculated. Finally, the attention weights are multiplied element-wise with the original feature map to obtain the attention-enhanced feature map. The entire process supports gradient backpropagation and can be jointly trained with the backbone network.

[0047] Thus, this embodiment of the application significantly improves the model's ability to perceive key target objects in complex scenes through a structured attention mechanism. Channel reorganization and global correlation establishment enhance the model's ability to model long-range dependencies, while depthwise separable convolution preserves local details without increasing computational burden. The overall structure improves detection accuracy while maintaining high inference efficiency, making it suitable for deployment on edge computing platforms.

[0048] In this embodiment, the attention-enhanced feature map is obtained by performing nonlinear transformation and feature fusion processing on the attention-enhanced feature map through a gated feedforward submodule, which may include: The resolution of the attention-enhanced feature map is adjusted to obtain a low-resolution feature map. Local feature extraction and nonlinear transformation are performed on the low-resolution feature map to obtain the processed low-resolution feature map. The processed low-resolution feature map is then gated and fused with the attention-enhanced feature map to obtain the fused feature map. The fused feature map is then subjected to resolution restoration processing to obtain the enhanced feature map.

[0049] In this embodiment, the gated feedforward submodule is used for deep processing of the feature map. This can be achieved by sequentially performing downsampling, nonlinear transformation, gated fusion, and upsampling to enhance the features. Specifically, firstly, the high-resolution attention-enhanced feature map is downsampled to obtain a low-resolution feature map. Downsampling expands the receptive field of each feature point, capturing more macroscopic semantic information while reducing computational complexity. Next, operations such as convolution are used to extract local features and perform nonlinear transformations on the low-resolution feature map to enhance local details and improve the expressive power of the features, forming refined and condensed features rich in global context. Further, the low-resolution feature map is upsampled to its original size and then gated-fused with the original attention-enhanced feature map to obtain a fused feature map. The gating mechanism generates a weight map between 0 and 1, automatically and precisely controlling the mixing ratio of the two information sets, achieving intelligent complementarity between macroscopic context and microscopic details. Finally, the fused feature map is upsampled back to its original spatial resolution, outputting the final enhanced feature map for seamless integration with the backbone network.

[0050] In this way, by combining multi-scale processing with gating adaptive fusion, the global context and local details of features are effectively unified, which can significantly improve the model's feature representation ability and robustness in complex scenarios, thereby improving the accuracy of subsequent object detection and trajectory prediction.

[0051] In a specific embodiment of this application, to improve the target detection performance of the target detection network under adverse weather conditions, this embodiment proposes a method to enhance the network's adaptability to weather-degraded image features by fusing a histogram converter module with the c2psa module in the YOLO model. This histogram converter module can also be fused with bottleneck structures in other detection models. This module can dynamically adjust the image histogram distribution, effectively mitigating the contrast reduction and detail blurring caused by weather factors such as rain, fog, and snow. The feature enhancement network structure based on the histogram converter, through a combination of hierarchical histogram equalization and attention mechanisms, significantly improves the model's feature discrimination ability in complex weather scenarios. The specific network structure is as follows: Figure 2 As shown, Figure 2 This is a diagram of the fused module network structure provided in a specific embodiment of this application. Specifically, the c2psa module is located between the YOLO11 backbone network and the detection head, serving as a key component for multi-scale feature fusion and playing a crucial role in the entire detection network. Replacing the bottleneck structure in the c2psa module with a histogram transformer can significantly improve the model's feature robustness and semantic information perception capabilities under adverse weather conditions. Figure 2As shown, replacing the standard convolutional plus residual connection structure with a histogram transformer allows for dynamic adjustment of the illumination distribution and local contrast of the feature map through differentiable histogram equalization, directly alleviating the feature blurring problem caused by weather degradation. Compared with the original standard structure, this structure improves the network's global feature extraction capability and enhances its ability to process texture details.

[0052] The histogram transformer mainly consists of a dynamically grouped histogram attention module and a gated feedforward module, both of which undergo layer normalization before being executed. The structure of the dynamically grouped histogram attention module is as follows: Figure 2 As shown in the Attention Histogram, the input data is first split into image channels and then sorted to better organize the data and improve the efficiency of subsequent processing. Then, a self-attention technique is used to establish global feature associations through q, k, and v convolution operations, and local details are preserved through depthwise separable convolution. Next, the data is further split and sorted, and reorganized through indexing operations. A reshaping operation is then used to adjust the shape to meet the needs of the attention mechanism. Subsequently, attention outputs under two different spatial partitioning methods are calculated to enhance the expression of key features. Finally, the attention weights are multiplied element-wise with the original feature map to enhance features in important regions and suppress unimportant regions. The result is then fed into the convolution module for feature dimension normalization.

[0053] Gated feedforward structure such as Figure 2 As shown in the feedforward diagram, the input feature map is first rearranged and segmented to reduce its resolution, facilitating depthwise separable convolution. Next, depthwise separable convolutions with kernel sizes of 3 and 5 are used to efficiently extract features, reducing the number of parameters while maintaining the receptive field. Then, a non-linear transformation using the Mish activation function enhances the model's expressive power, and a multiplication operation combines the original input and the processed output. Finally, a pixel inversion operation restores the low-resolution feature map to a high-resolution version for use in subsequent layers or the output.

[0054] In this embodiment of the application, during the process of trajectory prediction of the target object, the method further includes: dynamically adjusting the parameters of the preset state estimation algorithm based on a preset reward and punishment mechanism.

[0055] In this embodiment, the preset reward and punishment mechanism is a dynamic parameter adjustment strategy based on prediction error. It optimizes model performance by rewarding reasonable predictions and penalizing unreasonable predictions. In the trajectory prediction process for a target object, this embodiment introduces a preset reward and punishment mechanism to optimize the trajectory prediction process. By evaluating the reasonableness of the prediction results in real time, the parameters of the state estimation algorithm are dynamically adjusted to better adapt to the movement patterns of the target object and environmental changes, thereby improving prediction accuracy and robustness.

[0056] Specifically, in the trajectory prediction stage, the future trajectory of the target object is first predicted using a preset state estimation algorithm based on historical motion trajectories. Subsequently, a preset reward and penalty mechanism evaluates the deviation between the predicted trajectory and the historical trajectory. If the deviation is within an acceptable range, the current parameter settings are rewarded; if the deviation exceeds a threshold, a penalty is imposed, and the noise matrix of the filter or other key parameters are dynamically adjusted. These steps are repeated within each prediction cycle to ensure that the prediction model always adapts to the current environment.

[0057] Thus, by introducing a reward and punishment mechanism, this application achieves adaptive optimization of the trajectory prediction model, enabling the system to quickly respond to changes in the motion of the target object, avoiding prediction bias caused by model rigidity, and improving the accuracy and reliability of subsequent prediction results.

[0058] In this embodiment of the application, dynamically adjusting the parameters of the preset state estimation algorithm based on a preset reward and punishment mechanism may include: Determine the first motion trend index of adjacent trajectory points in the historical motion trajectory, and the second motion trend index of adjacent trajectory points in the prediction. Compare whether the absolute difference between the second motion trend indicator and the first motion trend indicator exceeds a preset threshold; If the absolute difference exceeds a preset threshold, adjust the noise matrix of the preset state estimation algorithm.

[0059] In this embodiment, the motion trend index is a quantitative indicator used to describe the direction of target motion, such as slope, velocity, and acceleration. The absolute difference is the absolute difference between two index values, used to measure the magnitude of the prediction deviation. The preset threshold is a pre-defined error tolerance range used to determine whether the prediction is reasonable. The noise matrix is ​​the covariance matrix used in the state estimation algorithm to describe process noise and observation noise.

[0060] This application embodiment can determine the reasonableness of the prediction by comparing the motion trend index of the historical motion trajectory and the predicted trajectory, and adjust the noise matrix of the state estimation algorithm to optimize the prediction performance if the prediction is unreasonable. Taking the motion trend index as the slope and the preset state estimation algorithm as a Kalman filter as an example, the following steps are taken: First, the average slope of adjacent points in the historical motion trajectory is calculated as a reference motion trend, and the slope of the predicted adjacent points is calculated as the prediction trend. Then, the absolute difference between the two is compared to see if it exceeds a preset threshold. If it does, the prediction is deemed unreasonable, triggering a penalty mechanism to adjust the process noise matrix or observation noise matrix of the Kalman filter to adapt to changes in the current motion mode. These steps are performed in each prediction frame to ensure that the prediction model always remains consistent with the actual motion of the target object.

[0061] Thus, this embodiment of the application achieves refined control over the trajectory prediction process through quantitative comparison of motion trends and dynamic adjustment of the noise matrix. Especially in scenarios with sudden changes in target motion patterns or strong environmental interference, this embodiment of the application can effectively suppress prediction deviations and improve the system's adaptability and robustness.

[0062] In this embodiment, the noise matrix includes a process noise matrix and an observation noise matrix. Adjusting the noise matrix of the preset state estimation algorithm may include: In the case of the initial adjustment trigger, increase the process noise matrix; If the adjustment is triggered again within a preset time and / or a preset number of times after the initial trigger, the observation noise matrix is ​​increased.

[0063] In this embodiment, the process noise matrix is ​​a covariance matrix describing the uncertainty during system state transition; the observation noise matrix is ​​a covariance matrix describing the uncertainty during sensor observation. This embodiment can differentiate the process noise matrix and observation noise matrix according to the frequency and timing of adjustment triggers to more accurately address different types of prediction biases.

[0064] Specifically, when the adjustment is triggered for the first time, it indicates that the motion pattern of the target object may change abruptly. In this case, the process noise matrix is ​​increased to allow the filter to adapt to the change more quickly. If the adjustment is triggered again within a preset time and / or a preset number of times after the initial trigger, i.e., a continuous triggering situation occurs, it indicates that there may be noise or anomalies in the observation data. In this case, the observation noise matrix is ​​increased to reduce the confidence in the current observation and avoid interference from abnormal data on the state estimation.

[0065] Thus, this embodiment of the application achieves fine-grained control of the noise matrix by differentiating between initial triggering and continuous triggering adjustment strategies. Initial triggering adjustment enhances the system's adaptability to sudden changes in motion, while continuous triggering adjustment improves the system's robustness to observation noise. The combination of the two significantly improves the stability and accuracy of trajectory prediction in complex environments.

[0066] In a specific embodiment of this application, it can be understood that accurate prediction and analysis of the trajectory of a target object can assess unknown risks and facilitate early intervention to minimize losses. After receiving the optimized historical motion trajectory, the Kalman filter will use these coordinate points as a reference to predict the target's movement path for approximately 30 frames at the next 6 coordinate points. A reward and penalty mechanism is introduced to judge the accuracy of these prediction points. The difference between the average slope of adjacent points of the historical motion trajectory and the slope of the predicted point is used as the judgment criterion. When the difference is within a set threshold range, it is judged as a good prediction; when the difference is outside the threshold range, a penalty is applied, and the parameters are readjusted for prediction. The specific technical implementation route is as follows: Figure 3 As shown, Figure 3 A trajectory prediction route map based on a reward and punishment mechanism is provided for a specific embodiment of this application.

[0067] Specifically, the optimized historical trajectory is first used to predict the subsequent six trajectory points through a Kalman filter. The core process can be divided into three stages: state modeling, prediction-update iteration, and parameter optimization. In the mining environment, target objects mostly move at a constant speed between loading and unloading points, rarely experiencing sudden stops or rapid starts. Therefore, a uniform speed model is chosen to predict moving targets in the mining area. Continuous prediction points are generated through iterative iteration, and penalties are applied to poor prediction points to adjust the filter parameters in real time.

[0068] By employing a reward and punishment mechanism in reinforcement learning, unreasonable slope changes in trajectory prediction are penalized, making the prediction results more consistent with physical laws. The main optimization direction is to suppress abrupt changes in motion direction between adjacent frames and encourage maintaining reasonable motion continuity. The average slope of adjacent points in the historical motion trajectory is defined as... The slope of adjacent prediction points is And set the allowable deviation threshold ε.

[0069] ; The penalty term is an exponentially growing function, and both the penalty and reward terms are incorporated into the observation update of the Kalman filter to minimize its original loss function.

[0070] When an unreasonable slope in the predicted point triggers a penalty term, this application employs a dynamic adjustment strategy in its specific embodiments, divided into initial trigger penalty and continuous triggering, corresponding to different adjustment directions. During the initial trigger, the model uncertainty is increased to allow the filter to adapt to abrupt changes more quickly, i.e., adjusting the process noise matrix. During continuous triggering, the confidence in the current observation is reduced to lessen the impact of abnormal observations on state estimation, i.e., adjusting the observation noise matrix. When the change in the predicted point slope is reasonable, the original historical trajectory is merged to form a new trajectory for continued prediction through the filter. Thus, the reward and penalty mechanism can pull the predicted point back towards the historical slope direction, suppressing physically unreasonable abrupt changes and making the predicted trajectory more closely match the target's movement path.

[0071] In this embodiment of the application, tracking the target object based on detection information to generate the historical motion trajectory of the target object may include: Based on the detection information, the target object is tracked by the tracker to generate an initial historical motion trajectory; After the sequence length of the initial historical motion trajectory reaches the first preset threshold, the coordinate jump points in the initial historical motion trajectory are smoothed by a fitting algorithm containing preset constraints in order to obtain the historical motion trajectory.

[0072] In this embodiment, the tracker can be SORT, DeepSORT, etc., used for cross-frame association and trajectory generation of the target object. The initial historical motion trajectory is the original trajectory sequence of the target object output by the tracker based on detection information, which may contain coordinate jumps and noise. This embodiment can smooth the initial historical motion trajectory by introducing a constrained fitting algorithm after tracking, eliminating coordinate jumps caused by factors such as detection jitter and road bumps, thereby improving trajectory quality and the accuracy of subsequent predictions.

[0073] Specifically, the tracker first performs cross-frame correlation on the detection information of the target object to generate an initial historical motion trajectory. Once the trajectory sequence length reaches a preset threshold (e.g., 20 frames), a constrained fitting algorithm is used to smooth the ordinates in the trajectory, limiting the coordinate changes between adjacent frames to within a physically reasonable range, thus obtaining an optimized historical motion trajectory. The fitting algorithm can be least squares, spline interpolation, etc.; the preset constraints are physical constraints introduced into the fitting algorithm, such as velocity limits and acceleration limits.

[0074] Thus, the embodiments of this application significantly improve the quality of historical motion trajectories through trajectory smoothing processing, reduce trajectory jitter caused by detection instability or environmental interference, and, by using a constrained fitting algorithm, retain the real maneuver characteristics of the target object while smoothing noise, ensuring the physical rationality and usability of the trajectory data.

[0075] In a specific embodiment of this application, in an autonomous driving scenario, considering that in a mining area autonomous driving system, although the improved YOLO11 target detection network can achieve accurate real-time coordinate box output, it suffers from weak coordinate stability and uncontrollable factors such as unstructured gravel road surfaces in mining areas. These factors can lead to unsmooth target trajectory curves obtained by the SORT tracker, which is detrimental to subsequent trajectory prediction. Therefore, this specific embodiment of the application can employ a least squares algorithm with constraints to process transition points in the trajectory, making the trajectory smoother and improving the accuracy of subsequent prediction results. The specific implementation process is as follows: Figure 4 As shown, Figure 4 This is a flowchart illustrating trajectory tracking and smoothing processing according to a specific embodiment of this application.

[0076] Specifically, the autonomous driving system first receives real-time data from the vehicle's onboard camera and uses an improved YOLO11 network to detect and identify typical targets in the mining area. Then, the SORT tracker processes the bounding box information obtained from the detection. The tracker maintains the motion state of each target object and uses the Hungarian algorithm to perform frame association matching of the targets. Finally, the historical motion trajectory of the same target is stored in every 10 frames as a node. If the length of the target sequence meets the preset length requirement, the system proceeds to the next step of trajectory fitting optimization. If it does not meet the requirement, the system directly outputs the original trajectory of the target object.

[0077] Next, constrained least-squares fitting is performed. Since SORT relies on detection results for tracking, unstable detection can lead to trajectory jitter or ID switching. In mining scenarios, because the camera-equipped platform and the target are at the same level, jitter in the X direction is negligible. However, the Y direction is prone to sudden changes due to road bumps, obstructions, or detection errors, causing trajectory jumps. To optimize this issue, specific embodiments of this application use a number of stored sequence points as... The vertical coordinate of the tracked target is Modeling is performed by fitting a function using the least squares method at given trajectory points to minimize the sum of squared residuals. Furthermore, the change in the ordinate of adjacent frames is limited to a threshold. A velocity constraint is specifically added to the Y-direction in the least squares method to directly restrict coordinate changes, reducing abrupt changes and making vertical motion smoother.

[0078] Finally, the y-coordinates optimized by the constrained least squares algorithm replace the original y-coordinates of the target detection boxes in the stored sequence, while the x-coordinates remain unchanged and are stored as a new coordinate sequence. Based on the sequence point length threshold (e.g., 20) set in the flowchart, targets with a sequence length less than 20 are considered to have low detection confidence or are misidentified, causing the SORT tracker to be unable to track them stably; these targets are not used for trajectory drawing. Finally, using the OpenCV library, the new coordinate point sequences with a sequence length greater than 20 are concatenated in the video detection frame as the basis for subsequent trajectory prediction.

[0079] Figure 5 This is a flowchart illustrating a vehicle autonomous driving control method provided in an embodiment of this application. Figure 5 As shown in the figure, this application embodiment also provides a control method for autonomous driving of a vehicle, which may include the following steps: Step S501: Determine the environmental perception result of the vehicle's environment. The environmental perception result is determined according to the environmental perception method described in the above embodiments. Step S502: Control the vehicle's movement based on the environmental perception results.

[0080] This application provides a specific application of the environmental perception method described above in the field of autonomous driving. During vehicle operation, surrounding environmental data is first collected using onboard sensors (such as cameras and radar). Subsequently, the environmental perception method described above is used to process the data in real time, obtaining accurate environmental perception results, such as identifying vehicles, pedestrians, lane lines, and drivable areas. Based on these perception results, the system further integrates a high-precision map with real-time vehicle status (such as position and speed) to perform dynamic driving decisions and trajectory planning. For example, when a decelerating vehicle is detected ahead, the system can decide to follow or overtake, and plan a corresponding safe and smooth driving trajectory. Finally, the control system converts this trajectory into specific steering, throttle, and braking commands, which are executed through the vehicle's drive-by-wire system, thereby achieving automated and safe vehicle operation.

[0081] Figure 6 This is a structural block diagram of an environmental sensing device provided in an embodiment of this application. Figure 6 As shown, this application provides an environmental sensing device, which may include: Memory 610 is configured to store instructions; The processor 620 is configured to retrieve instructions from the memory 610 and, when executing the instructions, to implement the environment awareness method described above.

[0082] This application also provides a machine-readable storage medium on which a program or instruction is stored, and when the program or instruction is executed by a processor, it implements the environment awareness method described above.

[0083] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0084] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0085] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0086] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0087] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0088] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0089] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0090] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0091] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. An environmental perception method, characterized in that, The environmental perception method includes: Acquire environmental images; The environmental image is input into a target detection model for processing to obtain detection information of at least one target object in the environmental image. The target detection model incorporates a histogram converter module, which is used to dynamically adjust the histogram distribution of the input image. The target object is tracked based on the detection information to generate the historical motion trajectory of the target object; Based on a preset state estimation algorithm and the historical motion trajectory, the trajectory of the target object is predicted to obtain the future motion trajectory of the target object; Based on the historical motion trajectory and the future motion trajectory, an environmental perception result is generated.

2. The environmental perception method according to claim 1, characterized in that, The target detection model also includes a backbone network and a detection head; The step of inputting the environmental image into a target detection model for processing to obtain detection information of at least one target object in the environmental image includes: The environmental image is used to extract features through the backbone network to obtain an initial feature map. The initial feature map is adaptively adjusted in histogram distribution and enhanced in the histogram converter module to obtain an enhanced feature map. The enhanced feature map is decoded by the detection head to generate detection information for the at least one target object.

3. The environmental perception method according to claim 2, characterized in that, The histogram converter module includes a dynamic grouped histogram attention submodule and a gated feedforward submodule; The step of performing adaptive histogram distribution adjustment and feature enhancement processing on the initial feature map through the histogram converter module to obtain an enhanced feature map includes: The initial feature map is calibrated by the dynamic grouping histogram attention submodule to obtain an attention-enhanced feature map. The attention-enhanced feature map is obtained by performing nonlinear transformation and feature fusion processing on the gated feedforward submodule.

4. The environmental perception method according to claim 3, characterized in that, The step of performing attention weight calibration on the initial feature map through the dynamic grouping histogram attention submodule to obtain an attention-enhanced feature map includes: The initial feature map is reorganized along the channel dimension to generate an ordered feature representation; Based on the ordered feature representation, global feature associations are established through convolutional mapping, and valuable features are generated through depthwise separable convolutional operations. The value features are grouped and their structures reconstructed to calculate attention weights; The attention weights are fused with the initial feature map to obtain the attention-enhanced feature map.

5. The environmental sensing method according to claim 3, characterized in that, The process of performing nonlinear transformation and feature fusion on the attention-enhanced feature map through the gated feedforward submodule to obtain the enhanced feature map includes: The attention-enhanced feature map is adjusted in resolution to obtain a low-resolution feature map; The low-resolution feature map is subjected to local feature extraction and nonlinear transformation to obtain the processed low-resolution feature map. The processed low-resolution feature map is then gated and fused with the attention-enhanced feature map to obtain the fused feature map. The fused feature map is subjected to resolution restoration processing to obtain the enhanced feature map.

6. The environmental perception method according to claim 1, characterized in that, In the process of predicting the trajectory of the target object, the method further includes: The parameters of the preset state estimation algorithm are dynamically adjusted based on a preset reward and punishment mechanism.

7. The environmental perception method according to claim 6, characterized in that, The parameters of the preset state estimation algorithm, which are dynamically adjusted based on a preset reward and punishment mechanism, include: Determine the first motion trend index of adjacent trajectory points in the historical motion trajectory, and the second motion trend index of the predicted adjacent trajectory points; Compare whether the absolute difference between the second motion trend index and the first motion trend index exceeds a preset threshold; If the absolute difference exceeds the preset threshold, the noise matrix of the preset state estimation algorithm is adjusted.

8. The environmental perception method according to claim 7, characterized in that, The noise matrix includes a process noise matrix and an observation noise matrix, and the noise matrix for adjusting the preset state estimation algorithm includes: In the case of the first triggered adjustment, the process noise matrix is ​​increased; If the adjustment is triggered again within a preset time and / or a preset number of times after the initial adjustment, the observation noise matrix is ​​increased.

9. The environmental perception method according to claim 1, characterized in that, The step of tracking the target object based on the detection information to generate the historical motion trajectory of the target object includes: Based on the detection information, the target object is tracked by a tracker to generate an initial historical motion trajectory; After the sequence length of the initial historical motion trajectory reaches a first preset threshold, the coordinate jump points in the initial historical motion trajectory are smoothed by a fitting algorithm containing preset constraints to obtain the historical motion trajectory.

10. A control method for autonomous driving of a vehicle, characterized in that, The control method includes: The environmental perception result of the vehicle's environment is determined, and the environmental perception result is determined by the environmental perception method according to any one of claims 1 to 9; The vehicle is controlled to move based on the environmental perception results.

11. An environmental sensing device, characterized in that, include: The memory is configured to store instructions; The processor is configured to retrieve instructions from memory and, when executing the instructions, to implement the environment-aware method according to any one of claims 1 to 9.

12. A machine-readable storage medium on which a program or instructions are stored, characterized in that, When the program or instructions are executed by the processor, they implement the environment-aware method according to any one of claims 1 to 9.

Images