Perception-decision-making-control method, device and equipment based on self-intelligent unmanned vehicle

Abstract

The invention discloses an intelligent unmanned vehicle perception-decision-control method, device and equipment, and relates to the field of unmanned vehicle control, and the method comprises the steps: carrying out the driving region semantic segmentation of each historical environment image in a historical environment image time sequence; generating a reference path in the drivable area; constructing a state space of the unmanned vehicle according to a reference path in the historical environment image, and performing reinforcement learning training on the target network based on the state space to obtain a control strategy model for path tracking of the unmanned vehicle; acquiring a reference path in the environment image acquired in real time, constructing a current state space of the unmanned vehicle according to the reference path in the environment image, and inputting the current state space into the control strategy model to obtain a control strategy for controlling the unmanned vehicle in real time; and performing online learning optimization on the control strategy model in the application process. The method can adapt to the GNSS unreliable environment, and meanwhile real-time high-precision unmanned vehicle autonomous operation is achieved.

Description

Technical Field

[0001] This application relates to the field of unmanned vehicle control technology, and in particular to an unmanned vehicle perception-decision-control method, device and equipment based on embodied intelligence. Background Technology

[0002] In the field of smart agriculture, automated operations of unmanned agricultural equipment in orchard environments have become an important development direction, relying on real-time and accurate autonomous navigation technology. However, the typical semi-structured orchard environment poses a severe challenge to existing navigation technologies: on the one hand, the dense canopy of fruit trees causes significant attenuation or loss of Global Navigation Satellite System (GNSS) signals, making positioning unreliable; on the other hand, complex lighting, foliage shading, and weed interference between rows make traditional vision-based navigation methods less robust and difficult to reliably identify the geometric features of fruit tree rows. Although deep learning-based semantic segmentation models (such as DeepLabV3+ and Fast-SCNN) have made progress in environmental perception, they generally face the dilemma of balancing accuracy and real-time performance. High-precision models are computationally complex and difficult to implement real-time inference on mobile embedded devices; while lightweight models lack sufficient segmentation accuracy and stability in complex scenarios. Furthermore, at the path planning and control level, existing methods (such as fuzzy control) rely on precise mathematical models and extensive parameter tuning. They exhibit weak adaptive capabilities when dealing with unknown operating conditions, resulting in significant path tracking deviations, limited control accuracy, and an inability to self-learn and adjust. Another prominent problem with current systems is that the perception, decision-making, and control modules are often isolated, lacking overall collaborative design. This prevents environmental perception information from serving motion control in the most efficient way, thus limiting the overall system performance. Therefore, there is an urgent need in this field for an autonomous operation system for unmanned vehicles in orchards that can adapt to unreliable GNSS environments and possesses real-time performance and high precision. Summary of the Invention

[0003] The purpose of this application is to provide a perception-decision-control method, device and equipment for unmanned vehicles based on embodied intelligence, which can adapt to unreliable GNSS environments and realize real-time high-precision autonomous operation of unmanned vehicles.

[0004] To achieve the above objectives, this application provides the following solution: Firstly, this application provides a perception-decision-control method for autonomous vehicles based on embodied intelligence, including: Collect historical environmental images of the target spatial region under different seasons, weather conditions, and shooting angles over a time series. A drivable region segmentation mask is obtained by performing drivable region semantic segmentation on each historical environment image in the historical environment image time series. Generate a reference path within the drivable area based on the drivable area segmentation mask; The state space of the autonomous vehicle is constructed based on the reference path in the historical environmental image. The target network is trained by reinforcement learning based on the state space to obtain the control strategy model of the autonomous vehicle for path tracking of the reference path. A reference path is obtained from the environmental image of the target spatial region in real time. The current state space of the unmanned vehicle is constructed based on the reference path in the environmental image. The current state space is input into the control strategy model to obtain the control strategy for real-time control of the unmanned vehicle. The control strategy model is optimized through online learning during application.

[0005] Optionally, a drivable region segmentation mask is obtained by performing drivable region semantic segmentation on each historical environment image in the historical environment image time series, specifically including: A lightweight semantic segmentation model is used to perform drivable region semantic segmentation on each historical environment image in the historical environment image time series to obtain a drivable region segmentation mask; The lightweight semantic segmentation model is an improved Fast-SCNN. The improvements of the improved Fast-SCNN include the introduction of a dynamic weight coordinate attention mechanism in the detail branch of Fast-SCNN, the introduction of an improved convolutional block attention module in the global branch, and the replacement of the original segmentation head with an ASPP structure. The dynamic weighted coordinate attention mechanism is based on the coordinate attention mechanism, and uses a lightweight multilayer perceptron to adaptively generate fusion weights for horizontal and vertical attention maps. The improved convolutional block attention module is based on the convolutional block attention module, which integrates the channel attention sub-module and the spatial attention sub-module in parallel and performs multi-scale pooling.

[0006] Optionally, a reference path within the drivable area is generated based on the drivable area segmentation mask, specifically including: Perform mathematical morphological elliptic kernel closure operation on the drivable region segmentation mask to obtain the processed mask region. The dynamic RANSAC algorithm is used to perform dynamic path planning on the processed mask region to obtain an initial reference path. The initial reference path is balanced using a spatiotemporal domain joint filtering mechanism to obtain a reference path within the drivable area.

[0007] Optionally, reinforcement learning is performed on the target network based on the state space to obtain the control strategy model for the autonomous vehicle to track the reference path, specifically including: The RN_DDQN algorithm is used to train the target network through reinforcement learning based on the state space, thereby obtaining the control strategy model for the autonomous vehicle to track the reference path. The RN_DDQN algorithm is an algorithm that integrates priority experience playback and N-step time-series reward into the DDQN framework.

[0008] Optionally, when training the target network using reinforcement learning, the state in the state space includes the position and heading angle of the autonomous vehicle; the actions in the dynamic space include control commands for going straight, turning left, and turning right, and the control commands for turning left and right each include preset small, medium, and large amplitude control commands; the reward function includes core rewards and penalties and superimposed rewards and penalties. The reward function is expressed as follows: ; ; ; in, For the total reward, With rewards and punishments as the core, To achieve cumulative rewards and punishments, For driverless cars in status s The lateral deviation value under ', Δ θ 1 indicates that the driverless car is in a certain state. s The heading deviation value below; R 1 is a reverse action penalty, when R core =-1, used to suppress reverse error redirection, with a value of -0.5; R 2 is for amplitude-matched rewards and penalties, when R core =+1, and the value is 0.2 when the lateral deviation matches the steering amplitude. R core =+1, and the value is -0.2 when the lateral deviation does not match the steering amplitude; The matching condition between lateral deviation and steering amplitude is that the lateral deviation is within a large deviation range and the steering amplitude is within a large steering amplitude range, or the lateral deviation is within a small deviation range and the steering amplitude is within a small steering amplitude range. The mismatch between lateral deviation and steering amplitude is defined as either a lateral deviation within a large deviation range and a steering amplitude within a small steering amplitude range, or a lateral deviation within a small deviation range and a steering amplitude within a large steering amplitude range. The large deviation range, large steering amplitude range, small deviation range, and small steering amplitude range are all preset values. R 3 represents the steady-state reward and punishment for micro-regions, used when | Δd 0|≤20 and| ΔθWhen 0|≤5, the value is 0.3 if the autonomous vehicle moves straight, and -0.3 if the autonomous vehicle turns. Δd 0 indicates that the driverless car is in a certain state. s The lateral deviation value, Δ θ 0 indicates that the driverless car is in a certain state. s The heading deviation value under the condition, state s 'for state' s The state at the next moment; R 4 is a reward for continuous improvement, when R core =+1, based on the record R core The number of times +1 is added, with values ​​of 0, 0.2, and 0.3; R 5 is a reward for collaborative improvement, when R core =+1, used to encourage the optimal maneuver that improves both yaw and roll simultaneously, with a value of +0.2.

[0009] Optionally, the state space of the autonomous vehicle is constructed based on the reference path described in the historical environmental images, specifically including: Using the tangent equation of the reference path as a reference, the two sides of the reference path are obtained by translating along the normal vector direction by d / 2; d is the width of the virtual path boundary. Based on the pose of the autonomous vehicle, multiple rays are simulated with the current position of the autonomous vehicle as the starting point. The intersection of each ray with the two side boundaries of the reference path is simulated to obtain the detection distance of the intersection of each ray with the parallel band formed by the two side boundaries. Each detection distance constitutes the state space of the autonomous vehicle.

[0010] Optionally, the two sides of the reference path are: ; in, and These are the x-coordinate and y-coordinate of the first side boundary, respectively. and Let A, B, and C be the x-coordinate and y-coordinate of the second side boundary, respectively, where A, B, and C are constants. The detection distances are represented as follows: ; in, The detection distance is the distance obtained by detecting the intersection of the i-th ray and the parallel band. , ( ) represents the location of the driverless car. For intermediate parameters, , θ The steering angle of the driverless car. jThe value can be 1 or 2. , and All are constants.

[0011] Optionally, the unmanned vehicle is an orchard unmanned vehicle, the target spatial area is a target orchard area, and the historical environmental image time series is a sequence of fruit tree row images.

[0012] Secondly, this application provides an autonomous vehicle perception-decision-control device based on embodied intelligence, the autonomous vehicle perception-decision-control device based on embodied intelligence comprising: The real-time perception layer is used to acquire environmental images of the target space region in real time, and to perform drivable area semantic segmentation on the environmental images to obtain a drivable area segmentation mask. The decision layer is used to generate a reference path within the drivable area based on the drivable area segmentation mask, construct the current state space of the unmanned vehicle based on the reference path, and obtain a control strategy for real-time control of the unmanned vehicle based on the control strategy model described in claim 1 based on the current state space of the unmanned vehicle; the control strategy model is optimized through online learning during application.

[0013] Thirdly, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the embodied intelligence-based autonomous vehicle perception-decision-control method described in any one of the above applications.

[0014] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the embodied intelligence-based autonomous vehicle perception-decision-control method described above.

[0015] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the embodied intelligence-based autonomous vehicle perception-decision-control method described above.

[0016] According to the specific embodiments provided in this application, the following technical effects are disclosed: This application provides a perception-decision-control method, device, and equipment for unmanned vehicles based on embodied intelligence. It involves real-time acquisition of reference paths from environmental images of a target spatial region, constructing the current state space of the unmanned vehicle based on these reference paths, and inputting the current state space into the control strategy model to obtain a control strategy for real-time control of the unmanned vehicle. This application constructs the state space based on real-time acquired environmental images, unaffected by unreliable GNSS factors, thus improving the real-time performance of unmanned vehicle control. Furthermore, this application first extracts the reference path and then performs path tracking; the control strategy model undergoes online learning and optimization during application, improving the reliability of the control strategy and consequently enhancing the accuracy of unmanned vehicle control. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating an embodiment of an autonomous vehicle perception-decision-control method based on embodied intelligence provided in this application.

[0019] Figure 2 This is a detailed flowchart illustrating an embodiment of the present application of an autonomous vehicle perception-decision-control method based on embodied intelligence.

[0020] Figure 3 This is a schematic diagram illustrating the principle framework of an autonomous vehicle perception-decision-control method based on embodied intelligence, provided as an embodiment of this application.

[0021] Figure 4 This is a schematic diagram of a virtual perception model provided in an embodiment of this application.

[0022] Figure 5 This is a schematic diagram of the structure of an improved lightweight semantic segmentation model provided in an embodiment of this application.

[0023] Figure 6 The cumulative score graph for each training round based on the RN_DDQN algorithm provided in one embodiment of this application.

[0024] Figure 7 This is a schematic diagram showing the results of drivable area extraction and path generation in different scenarios provided in an embodiment of this application.

[0025] Figure 8 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0026] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0027] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0028] In one exemplary embodiment, such as Figures 1-3 As shown, this application provides a perception-decision-control method for autonomous vehicles based on embodied intelligence. The method includes: Step 101: Collect historical environmental image time series of the target spatial area under different seasons, different weather conditions and different shooting angles.

[0029] Step 102: Perform drivable region semantic segmentation on each historical environment image in the historical environment image time series to obtain a drivable region segmentation mask.

[0030] Step 103: Generate a reference path within the drivable area based on the drivable area segmentation mask.

[0031] Step 104: Construct the state space of the unmanned vehicle based on the reference path in the historical environment image, and perform reinforcement learning training on the target network based on the state space to obtain the control strategy model of the unmanned vehicle for path tracking of the reference path.

[0032] Step 104: Obtain the reference path in the environmental image of the target space region collected in real time, construct the current state space of the unmanned vehicle based on the reference path in the environmental image, input the current state space into the control strategy model to obtain the control strategy for real-time control of the unmanned vehicle; the control strategy model is optimized online during application.

[0033] This application constructs a state space based on real-time acquired environmental images, which is not affected by unreliable GNSS factors, thus improving the real-time performance of unmanned vehicle control. In addition, this application first extracts a reference path and then performs path tracking. The control strategy model is optimized online during application, which improves the reliability of the control strategy and thus improves the accuracy of unmanned vehicle control.

[0034] In one example embodiment, the unmanned vehicle is an orchard unmanned vehicle, the target spatial area is a target orchard area, and the historical environmental image time series is a sequence of fruit tree row images.

[0035] The target orchard area covers two typical types of orchards: pear orchards and cherry orchards.

[0036] In one example embodiment, step 101 specifically includes: collecting a sequence of fruit tree rows images that include the phenological periods of spring, summer, autumn and winter, and simultaneously recording multiple lighting conditions such as cloudy, shaded, bright, and weak light, as well as extreme shooting angles such as ±30° pitch angle and ±45° sway angle.

[0037] In one example embodiment, step 102 specifically includes: using a lightweight semantic segmentation model to perform drivable region semantic segmentation on each historical environment image in the historical environment image time series to obtain a drivable region segmentation mask.

[0038] The lightweight semantic segmentation model is an improved Fast Semantic Segmentation Convolutional Neural Network (Fast-SCNN). This network natively adopts a dual-branch collaborative architecture, including a detail branch and a global branch, and the detail branch and the global branch share a shallow learning downsampling module, possessing the core advantages of lightweight design and high real-time performance. Specifically, the detail branch is a lightweight feature extraction branch constructed based on shallow features from the shared shallow learning downsampling module; the global branch is a deep feature extraction branch that receives the output of the shallow learning downsampling module as input, realizing deep semantic abstraction of shallow features and global context modeling. Its overall structure is as follows: Figure 5As shown. The improvements of the improved Fast-SCNN include the introduction of Dynamic Weight Coordinate Attention (DWCA) in the detail branches, the introduction of an improved convolutional block attention module in the global branches, and the replacement of the original segmentation head with an Atrous Spatial Pyramid Pooling (ASPP) structure. The model involves input, 2D convolutional layers (Conv2D), depthwise convolution (DWConv), depthwise separable convolutional layers (DSConv), DWCA, a bottleneck layer, a pyramid pooling layer, an enhanced deformable convolutional block attention module (ED-CBAM), upsampling, and a softmax function.

[0039] The dynamic weighted coordinate attention mechanism is based on the coordinate attention mechanism (CA) and adaptively generates fusion weights for horizontal and vertical attention maps using a lightweight multilayer perceptron (MLP). This dynamic balancing of attention allocation in two-dimensional space enables the lightweight semantic segmentation model to more accurately capture the structural features of fruit tree rows in both horizontal and vertical directions, enhancing its ability to model complex spatial relationships. Simultaneously, due to the use of a lightweight multilayer perceptron structure, this mechanism maintains the model's lightweight characteristics.

[0040] The improved convolutional block attention module is based on the convolutional block attention module, which integrates the channel attention sub-module and the spatial attention sub-module in parallel and performs multi-scale pooling.

[0041] (1) In the channel attention submodule, the combination of global average pooling (GAP), local region pooling (2×2 adaptive average pooling) and global max pooling (GMP) can statistically analyze features from different scales and angles, and more comprehensively integrate feature information across receptive fields, thereby improving the ability of channel attention to express deep semantic features. (2) In the spatial attention submodule, deformable convolution technology is used to reconstruct the feature sampling method. Deformable convolution enables the 7×7 convolution kernel to adaptively deform and sample according to the input content through dynamically generated offset fields, which can significantly improve the model's feature response ability to edges and small targets, and better capture the spatial position and shape information of the target. (3) A parallel adaptive fusion architecture is adopted to build a collaborative optimization mechanism for channel and spatial attention. In the parallel structure, channel attention and spatial attention can process features simultaneously and complement each other through adaptive fusion, achieving more accurate calibration of features while maintaining the advantage of lightweight design.

[0042] The ASPP structure, through parallel sampling of convolutional kernels with different dilation rates, can extract features at different scales. A larger dilation rate can capture broader contextual information and is suitable for feature extraction of larger targets or scenes; a smaller dilation rate focuses on local detailed features and is beneficial for the identification of small targets.

[0043] Table 1 shows the comparison data of the segmentation performance of the lightweight semantic segmentation model of this application with other models.

[0044] Table 1 Comparison of optimal indicators under different modes

[0045] Table 1 compares the segmentation performance of our model with BiseNetV2, DeepLabV3+, U-shaped convolutional neural network (UNet), Short-Term Dense Concatenate (STDC), and Fast-SCNN. Considering average pixel accuracy (mPA), F1 score, average intersection-over-union ratio (mIoU), inference speed (frames per second, FPS), and total parameter size, our model performs better. BiseNetV2 is an upgraded version of BiseNetV and is a neural network architecture used for real-time semantic segmentation.

[0046] The functions of steps 101 and 102 are implemented through the real-time perception layer. The improved lightweight semantic segmentation model is used to collect data on the semi-structured environment of the orchard and complete the semantic segmentation of the drivable area.

[0047] In one example embodiment, step 103 specifically includes steps 201-202.

[0048] Step 201: Perform mathematical morphological elliptic kernel closure operation on the drivable region segmentation mask to obtain the processed mask region.

[0049] The formula for calculating the nucleus closure operation of an ellipse in mathematical morphology is expressed as follows: .

[0050] CLOSE() is the closing operation in mathematical morphology (dilation followed by erosion). M A binary mask for the drivable area. S Elliptic nucleus: . This is the expansion operator. This is the erosion operator.

[0051] This application improves the topological integrity of the segmentation mask by eliminating weed noise of ≤5×5 pixels while repairing path region breaks caused by shadows through mathematical morphology elliptic kernel closure operation.

[0052] Step 202: Use the Random Sample Consensus (RANSAC) algorithm to perform dynamic path planning on the processed mask region to obtain an initial reference path.

[0053] Step 203: The initial reference path is balanced using a spatiotemporal domain joint filtering mechanism to obtain the reference path (optimal reference path) within the drivable area.

[0054] Specifically, step 202 includes: given a set of feature points RANSAC optimizes by maximizing the set of interior points, and its calculation formula is as follows: .

[0055] in, For linear parameters, ε =10 is the residual threshold. The set of interior points that satisfy the residual condition. Let n be the coordinates of the feature points and n be the number of feature points.

[0056] Given a feature point set P, which is the geometric representation of path-related feature points extracted from the "pixel-level drivable region (path region)" identified by the improved semantic segmentation model (such as edge points, corner points, and other geometrically distinctive points in the path region), it is a set of local feature representations of the path region.

[0057] The dynamic minimum sample size formula is derived based on the RANSAC probability model, satisfying the statistical significance condition. Where r is the proportion of inliers, the sample dimension s=2, and the calculation formula is as follows: .

[0058] Where min_samples is the dynamic minimum sample size of the RANSAC algorithm; |P| is the size of the feature point set P (i.e., the number of points). This is for floor function.

[0059] Design a joint spatiotemporal filtering mechanism, using exponential smoothing (cutoff frequency (0.12)). f s The concatenation of the 15-frame moving average with the longitudinal sampling interval Δ reduces trajectory fluctuations. The calculation formula is shown below. y Nyquist constraint with a value of 3, and lateral mean estimation σ 2 Threshold detection and multi-level geometric verification ( y max - y min >50), construct a systematic anomaly suppression pipeline.

[0060] ; .

[0061] Where H(z) is the transfer function of exponential smoothing; α is the coefficient of exponential smoothing; For discrete systems, the unit delay operator, The variance of the input noise. Where L is the variance of the output noise, and L is the number of frames for the moving average.

[0062] In one example embodiment, step 104 constructs a virtual simulation environment for the autonomous vehicle. In the virtual simulation environment, the state space, action space, and reward function of the autonomous vehicle are constructed based on the virtual perception model, and a path tracking control strategy is generated by training a reinforcement learning algorithm.

[0063] like Figure 4 As shown, the virtual perception model outputs the final detection range (normalized) for each beam based on the vehicle's position and attitude angles and virtual boundary parameters. l (i) The virtual sensing distribution map, composed of the detection results of all beams, reflects the relative positional relationship between the vehicle and the path boundary.

[0064] Step 104 involves constructing the state space of the autonomous vehicle based on the reference path described in the historical environmental image, specifically including steps 301-302.

[0065] Step 301: Using the tangent equation of the reference path as a reference, translate along the normal vector direction by d / 2 to obtain the two side boundaries of the reference path. d is the width of the virtual path boundary.

[0066] Step 302: Based on the pose of the unmanned vehicle, simulate multiple rays with the current position of the unmanned vehicle as the starting point, simulate the intersection detection of each ray with the two side boundaries of the reference path, and obtain the detection distance of each ray intersecting with the parallel band formed by the two side boundaries. Each detection distance constitutes the state space of the unmanned vehicle.

[0067] The two sides of the reference path are as follows: .

[0068] in, and Indicates the two side boundaries. and These are the x-coordinate and y-coordinate of the first side boundary, respectively. and Let A, B, and C be the x-coordinates and y-coordinates on the second side boundary, respectively, where A, B, and C are all constants.

[0069] Step 302 specifically includes: unmanned vehicle positioning ( x , y , θ ), No. i The channel beam is from ( x , y Departure, direction angle is θ + ir The ray (which actually represents this beam) is detected by the detection logic distance formula shown. The back / over-range results are filtered out, and the minimum distance on both sides is taken to simulate the intersection detection of the ray and the parallel zone (which is the area formed by the two side boundaries).

[0070] The detection distances are represented as follows: .

[0071] in, Let be the detection distance obtained by detecting the intersection of the i-th ray and the parallel band. j It can take the value 1 or 2. j Corresponding to the virtual path boundaries on both sides; , ( ) represents the location of the driverless car. For intermediate parameters, , θ The steering angle of the driverless car. This refers to the angular resolution, specifically the angular resolution of the virtual perception model. , and All are constants.

[0072] Based on the above ray scanning mechanism, the output results are organized into an array of length i, and the state space S is shown below: .

[0073] In one example embodiment, in step 104, reinforcement learning training is performed on the target network based on the state space to obtain the control strategy model for the autonomous vehicle to track the reference path. Specifically, this includes: using the RN_DDQN algorithm, reinforcement learning training is performed on the target network based on the state space to obtain the control strategy model for the autonomous vehicle to track the reference path. The RN_DDQN algorithm is an algorithm that integrates Priority Experience Playback (PER) and N-step temporal reward into the Double Deep Q-Network (DDQN) framework. In addition to the RN_DDQN framework, the reinforcement learning network in this application can also use Deep Q-Network (DQN), Double Deep Q-Network (DDQN), and Advantage Actor Critic (A2C).

[0074] Priority experience replay is integrated into the experience storage and sampling stage of DDQN. Temporal difference (TD) error is used to assign priority to experience to achieve non-uniform sampling, and importance sampling weights are introduced to correct gradient update bias. N-step time series reward is integrated into the target Q value calculation stage of DDQN. The cumulative value of discounted reward in the next N steps is used to replace the original single-step instant reward to calculate the target Q value.

[0075] When training the target network using reinforcement learning, the state space includes the position and heading angle of the autonomous vehicle; the dynamic space includes control commands for going straight, turning left and turning right, and the control commands for turning left and right include preset small, medium and large amplitude control commands; the reward function includes core rewards and penalties and superimposed rewards and penalties.

[0076] In the dynamic space, actions are defined as a discrete set of values ​​[0, 1, 2, 3, 4, 5, 6], representing control commands for going straight, making a small left turn, making a medium left turn, making a large left turn, making a small right turn, making a medium right turn, and making a large right turn, respectively. Dynamic space A is specifically represented as follows: .

[0077] The reward function is expressed as follows: .

[0078] .

[0079] .

[0080] in, For the total reward, With rewards and punishments as the core, To achieve cumulative rewards and punishments, For driverless cars in status s The lateral deviation value under ', Δ θ 1 indicates that the driverless car is in a certain state. s The heading deviation value under ', Δ θ 0 indicates that the driverless car is in a certain state. s The heading deviation value under the given conditions. State s 'for state' s The state at the next moment.

[0081] R 1 is a reverse action penalty, when R core =-1, used to suppress reverse error redirection, with a value of -0.5.

[0082] R 2 is for amplitude-matched rewards and penalties, when R core =+1, used to match the magnitude of the deviation with the steering angle, with a value of ±0.2. Specifically, when R core =+1, and the value is 0.2 when the lateral deviation matches the steering amplitude. R core =+1, and the value is -0.2 when the lateral deviation does not match the steering amplitude; The matching condition between lateral deviation and steering amplitude is that the lateral deviation is within a large deviation range and the steering amplitude is within a large steering amplitude range, or the lateral deviation is within a small deviation range and the steering amplitude is within a small steering amplitude range. The mismatch between lateral deviation and steering amplitude is defined as follows: the lateral deviation is within a large deviation range and the steering amplitude is within a small steering amplitude range; or the lateral deviation is within a small deviation range and the steering amplitude is within a large steering amplitude range. The large deviation range, large steering amplitude range, small deviation range, and small steering amplitude range are all preset values. Simply put, within the large deviation range, the large steering amplitude is 0.2, and the small steering amplitude is -0.2; within the small deviation range, the large steering amplitude is -0.2, and the small steering amplitude is 0.2.

[0083] R 3 represents the steady-state reward and punishment for micro-regions, used when | Δd 0|≤20 and| Δθ When 0|≤5 (small deviation range), oscillation is suppressed. If the unmanned vehicle moves straight, the value is 0.3; if the unmanned vehicle turns, the value is -0.3. Δd 0 indicates that the driverless car is in a certain state.s The lateral deviation value, Δ θ 0 indicates that the driverless car is in a certain state. s The heading deviation value under the condition, state s 'for state' s The state at the next moment, state s 'This represents the current state.

[0084] R 4 is a reward for continuous improvement, when R core =+1, used to encourage a trend of continuous improvement, based on records. R core The number of times +1 is added, with values ​​of 0, 0.2, and 0.3. Recorded continuously. R core The bonus is 0 when the number of attempts is 0, and the record is continuous. R core When the number of attempts is ≥2, an additional bonus of 0.2 is awarded, with consecutive records being recorded. R core When the number of attempts is ≥3, an additional reward of 0.3 is given; this gradient reward mechanism incentivizes the model to learn a continuous and stable trajectory deviation correction strategy, avoiding meaningless repetitive fine-tuning behavior.

[0085] R 5 is a reward for collaborative improvement, when R core =+1, used to encourage the optimal maneuver that improves both yaw and roll simultaneously, with a value of +0.2.

[0086] This application is based on the RN_DDQN algorithm, with the cumulative score for each training round as follows: Figure 6 As shown.

[0087] This application obtains the real-time reference path by calling a control strategy model, taking real-time reference path geometric parameters as input, and mapping the spatial path relationship from the image straight line equation (based on the equation in the image coordinate system) to the vehicle coordinate system and the world coordinate system. Figure 7 As shown, Figure 7 (a) and (b) are reference paths obtained from environmental images from two angles. A state space representation is constructed based on a virtual perception model. Online learning enables simultaneous operation and learning, achieving precise path tracking with deep collaboration throughout the entire process.

[0088] Based on the same inventive concept, this application also provides a device for implementing the aforementioned autonomous vehicle perception-decision-control device based on embodied intelligence. The solution provided by this device is similar to the solution described in the above method. Therefore, the specific limitations of one or more embodiments of the autonomous vehicle perception-decision-control device based on embodied intelligence provided below can be found in the limitations of the autonomous vehicle perception-decision-control method based on embodied intelligence described above, and will not be repeated here.

[0089] In one exemplary embodiment, an autonomous vehicle perception-decision-control device based on embodied intelligence is provided, the device comprising a real-time perception layer and a decision layer.

[0090] The real-time perception layer is used to acquire environmental images of the target spatial region in real time, and to perform drivable region semantic segmentation on the environmental images to obtain a drivable region segmentation mask.

[0091] The decision layer is used to generate a reference path within the drivable area based on the drivable area segmentation mask, construct the current state space of the unmanned vehicle based on the reference path, and obtain a control strategy for real-time control of the unmanned vehicle based on the control strategy model described in claim 1 based on the current state space of the unmanned vehicle; the control strategy model is optimized through online learning during application.

[0092] The system in this application adopts a modular and hierarchical design, with each link being independently optimizable and highly collaborative. It can quickly adapt to different agricultural vehicles and complex orchard environments, providing key technical support for intelligent agricultural autonomous navigation and promoting the application of embodied intelligence in the agricultural field.

[0093] In one exemplary embodiment, a computer device is provided, which may be a server or a terminal, and its internal structure diagram may be as follows. Figure 8As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores data related to an embodied intelligence-based autonomous vehicle perception-decision-control method. The I / O interfaces are used for information exchange between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements an embodied intelligence-based autonomous vehicle perception-decision-control method.

[0094] Those skilled in the art will understand that Figure 8 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0095] In one exemplary embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0096] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0097] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0098] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0099] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0100] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0101] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0102] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A perception-decision-control method for unmanned vehicles based on embodied intelligence, characterized in that, The autonomous vehicle perception-decision-control method based on embodied intelligence includes: Collect historical environmental images of the target spatial region under different seasons, weather conditions, and shooting angles over a time series. A drivable region segmentation mask is obtained by performing drivable region semantic segmentation on each historical environment image in the historical environment image time series. Generate a reference path within the drivable area based on the drivable area segmentation mask; The state space of the autonomous vehicle is constructed based on the reference path in the historical environmental image. The target network is trained by reinforcement learning based on the state space to obtain the control strategy model of the autonomous vehicle for path tracking of the reference path. A reference path is obtained from the environmental image of the target spatial region in real time. The current state space of the unmanned vehicle is constructed based on the reference path in the environmental image. The current state space is input into the control strategy model to obtain the control strategy for real-time control of the unmanned vehicle. The control strategy model is optimized through online learning during application.

2. The autonomous vehicle perception-decision-control method based on embodied intelligence according to claim 1, characterized in that, For each historical environment image in the historical environment image time series, perform drivable region semantic segmentation to obtain a drivable region segmentation mask, specifically including: A lightweight semantic segmentation model is used to perform drivable region semantic segmentation on each historical environment image in the historical environment image time series to obtain a drivable region segmentation mask; The lightweight semantic segmentation model is an improved Fast-SCNN. The improvements of the improved Fast-SCNN include the introduction of a dynamic weight coordinate attention mechanism in the detail branch of Fast-SCNN, the introduction of an improved convolutional block attention module in the global branch, and the replacement of the original segmentation head with an ASPP structure. The dynamic weighted coordinate attention mechanism is based on the coordinate attention mechanism, and uses a lightweight multilayer perceptron to adaptively generate fusion weights for horizontal and vertical attention maps. The improved convolutional block attention module is based on the convolutional block attention module, which integrates the channel attention sub-module and the spatial attention sub-module in parallel and performs multi-scale pooling.

3. The autonomous vehicle perception-decision-control method based on embodied intelligence according to claim 1, characterized in that, Generate a reference path within the drivable area based on the drivable area segmentation mask, specifically including: Perform mathematical morphological elliptic kernel closure operation on the drivable region segmentation mask to obtain the processed mask region. The dynamic RANSAC algorithm is used to perform dynamic path planning on the processed mask region to obtain an initial reference path. The initial reference path is balanced using a spatiotemporal domain joint filtering mechanism to obtain a reference path within the drivable area.

4. The autonomous vehicle perception-decision-control method based on embodied intelligence according to claim 1, characterized in that, Based on the state space, the target network is trained using reinforcement learning to obtain the control strategy model for the autonomous vehicle to track the reference path, specifically including: The RN_DDQN algorithm is used to train the target network through reinforcement learning based on the state space, thereby obtaining the control strategy model for the autonomous vehicle to track the reference path. The RN_DDQN algorithm is an algorithm that integrates priority experience playback and N-step time-series reward into the DDQN framework.

5. The autonomous vehicle perception-decision-control method based on embodied intelligence according to claim 1, characterized in that, When training the target network using reinforcement learning, the state space includes the position and heading angle of the autonomous vehicle; the dynamic space includes the control commands for going straight, turning left and turning right, and the control commands for turning left and right include preset small, medium and large amplitude control commands; the reward function includes core rewards and penalties and superimposed rewards and penalties. The reward function is expressed as follows: ; ； ； in, For the total reward, With rewards and punishments as the core, To achieve cumulative rewards and punishments, For driverless cars in status s The lateral deviation value under ', Δ θ 1 indicates that the driverless car is in a certain state. s The heading deviation value below; R 1 is a reverse action penalty, when R core =-1, with a value of -0.5; R 2 is for amplitude-matched rewards and penalties, when R core =+1, and the value is 0.2 when the lateral deviation matches the steering amplitude. R core =+1, and the value is -0.2 when the lateral deviation does not match the steering amplitude; The matching condition between lateral deviation and steering amplitude is that the lateral deviation is within a large deviation range and the steering amplitude is within a large steering amplitude range, or the lateral deviation is within a small deviation range and the steering amplitude is within a small steering amplitude range. The mismatch between lateral deviation and steering amplitude is defined as either a lateral deviation within a large deviation range and a steering amplitude within a small steering amplitude range, or a lateral deviation within a small deviation range and a steering amplitude within a large steering amplitude range. The large deviation range, large steering amplitude range, small deviation range, and small steering amplitude range are all preset values. R 3 represents the steady-state reward and punishment for micro-regions, used when | Δd 0|≤20 and| Δθ When 0|≤5, the value is 0.3 if the autonomous vehicle moves straight, and -0.3 if the autonomous vehicle turns. Δd 0 indicates that the driverless car is in a certain state. s The lateral deviation value, Δ θ 0 indicates that the driverless car is in a certain state. s The heading deviation value under the condition, state s 'for state' s The state at the next moment; R 4 is a reward for continuous improvement, when R core =+1, based on the record R core The number of times +1 is added, with values ​​of 0, 0.2, and 0.3; R 5 is a reward for collaborative improvement, when R core =+1, used to encourage the optimal maneuver that improves both yaw and roll simultaneously, with a value of +0.

2.

6. The autonomous vehicle perception-decision-control method based on embodied intelligence according to claim 5, characterized in that, The state space of the autonomous vehicle is constructed based on the reference path described in historical environmental images, specifically including: Using the tangent equation of the reference path as a reference, the two sides of the reference path are obtained by translating along the normal vector direction by d / 2; d is the width of the virtual path boundary. Based on the pose of the autonomous vehicle, multiple rays are simulated with the current position of the autonomous vehicle as the starting point. The intersection of each ray with the two side boundaries of the reference path is simulated to obtain the detection distance of the intersection of each ray with the parallel band formed by the two side boundaries. Each detection distance constitutes the state space of the autonomous vehicle.

7. The autonomous vehicle perception-decision-control method based on embodied intelligence according to claim 6, characterized in that, The two sides of the reference path are as follows: ； in, and These are the x-coordinate and y-coordinate of the first side boundary, respectively. and Let A, B, and C be the x-coordinate and y-coordinate of the second side boundary, respectively, where A, B, and C are constants. The detection distances are represented as follows: ； in, The detection distance is the distance obtained by detecting the intersection of the i-th ray and the parallel band. , ( ) represents the location of the driverless car. For intermediate parameters, , θ The steering angle of the driverless car. j The value can be 1 or 2. , and All are constants.

8. The autonomous vehicle perception-decision-control method based on embodied intelligence according to claim 1, characterized in that, The unmanned vehicle is an orchard unmanned vehicle, the target spatial area is the target orchard area, and the historical environmental image time series is a sequence of fruit tree row images.

9. A perception-decision-control device for unmanned vehicles based on embodied intelligence, characterized in that, The autonomous vehicle perception-decision-control device based on embodied intelligence includes: The real-time perception layer is used to acquire environmental images of the target space region in real time, and to perform drivable area semantic segmentation on the environmental images to obtain a drivable area segmentation mask. The decision layer is used to generate a reference path within the drivable area based on the drivable area segmentation mask, construct the current state space of the unmanned vehicle based on the reference path, and obtain a control strategy for real-time control of the unmanned vehicle based on the control strategy model described in claim 1 based on the current state space of the unmanned vehicle; the control strategy model is optimized through online learning during application.

10. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the embodied intelligence-based autonomous vehicle perception-decision-control method according to any one of claims 1-8.

Images