Navigation obstacle avoidance method and system in low-confidence and feature similar environment

Abstract

The invention belongs to the technical field of intelligent navigation, and discloses a navigation obstacle avoidance method and system in a low-confidence and feature similar environment. The method comprises the following steps: acquiring multi-modal sensor data of obstacles in a detection area in a preset low-confidence environment by using a sensor; preprocessing the data of the multi-modal sensor; introducing a generative adversarial network to carry out image enhancement processing on the denoised image; performing cross-domain fusion on the data by using a converter architecture; obtaining an obstacle position, an obstacle type and a confidence degree distribution value based on a deep neural network, and constructing and obtaining an obstacle confidence degree map; dynamically planning an obstacle avoidance path by combining a deep reinforcement learning algorithm based on an obstacle confidence map obtained in real time, and performing auxiliary planning of the obstacle avoidance path based on an MPC model; the method can effectively cope with high challenge scenes such as tunnels and is widely applied to the fields of industrial automation, intelligent transportation and the like.

Description

Technical Field

[0001] The present invention relates to the field of intelligent navigation technology, and more specifically, to a navigation obstacle avoidance method and system in a low confidence and feature-similar environment. Background Art

[0002] In the navigation systems of mobile robots and autonomous vehicles, obstacle avoidance and recognition technology is an important component to ensure the safety and stability of the system. However, in low-confidence environments (such as complex scenarios such as insufficient light, sensor noise interference, and missing data), traditional obstacle avoidance methods often have difficulty accurately perceiving the location and dynamic information of obstacles, resulting in path planning errors and even collision risks.

[0003] The Chinese patent application with publication number CN119088030A discloses a path planning method and system for humanoid robot obstacle avoidance: including obtaining the humanoid robot's environmental image, processing the environmental image using an adaptive threshold segmentation method to obtain obstacle areas and obstacle-free areas; using an obstacle segmentation algorithm based on region growth to identify obstacle contours; using a dual-view monocular vision positioning method to calculate the position coordinates of obstacles in space; using a dynamic detection and branch extension path planning algorithm to obtain path planning results. This invention ensures the efficient obstacle avoidance and path planning capabilities of humanoid robots in complex environments.

[0004] Although the above method can meet most scenarios, research and practical application of the above method and existing technology have found that the above method and existing technology have at least the following defects:

[0005] In low-confidence environments such as insufficient light, high sensor noise or missing data, the feature information in the environment tends to become extremely similar, and the threshold cannot be accurately determined, resulting in incorrect or incomplete segmentation of the obstacle area, affecting subsequent recognition and path planning; the dual-view monocular vision positioning method is prone to depth perception errors when calculating the obstacle position coordinates. For obstacles that are far away or in complex spatial structures, the positioning accuracy is low, which makes the robot inaccurately judge the actual position of the obstacle, affecting the safety of obstacle avoidance path planning.

[0006] In view of this, the present invention proposes a navigation obstacle avoidance method and system in a low confidence and feature-similar environment to solve the above-mentioned problem. Summary of the invention

[0007] In order to overcome the above-mentioned defects of the prior art and to achieve the above-mentioned purpose, the present invention provides the following technical solution: a navigation obstacle avoidance method in a low confidence and feature-similar environment, comprising the following steps:

[0008] Using sensors to collect multimodal sensor data of obstacles in a detection area under a preset low confidence environment, the multimodal sensor data includes raw image data, raw lidar point cloud, and raw inertial measurement unit information;

[0009] Preprocessing the multimodal sensor data to obtain denoised multimodal sensor data; the denoised multimodal sensor data includes denoised images, lidar point clouds and inertial measurement unit information;

[0010] Generative adversarial networks are introduced to perform image enhancement processing on denoised images to obtain image data;

[0011] Use the transformer architecture to perform cross-domain fusion of image data, lidar point cloud, and inertial measurement unit information to obtain fused multimodal data;

[0012] The fused multimodal data is used as the input of the deep neural network to obtain the obstacle position, obstacle type and confidence distribution value, and construct an obstacle confidence map;

[0013] Based on the obstacle confidence map obtained in real time, the obstacle avoidance path is dynamically planned in combination with the deep reinforcement learning algorithm, and the obstacle avoidance path is assisted in planning based on the MPC model.

[0014] Furthermore, the method of dynamically planning the obstacle avoidance path in combination with the deep reinforcement learning algorithm includes:

[0015] Step 1: Define the state space , state space Including the robot's current position, speed, obstacle confidence map and global path planning information;

[0016] Step 2: Define the action space. It includes the union of basic actions and obstacle avoidance action sequences;

[0017] Step 3: Design reward function.

[0018] Step 4: Build a Q network based on DQN ,in, For status, For action, is the parameter of the network; it is used to learn the optimal Q function to maximize the value of the cumulative reward function; using the experience replay mechanism, the four-tuple in each decision-making process is stored in the preset experience replay buffer, the four-tuple includes the state, action and reward corresponding to the current moment, and the state corresponding to the next moment;

[0019] Define the loss function of the Q network; optimize the Q network with the minimum loss function value as the optimization goal to obtain the corresponding optimal Q network. When it is necessary to plan an obstacle avoidance path, use the optimal Q network to select and execute the optimal action according to the current state, and use the optimal action as the dynamically planned obstacle avoidance path.

[0020] Furthermore, the method of obtaining the reward function includes:

[0021] Step 3.1, design the distance reward for reaching the target point based on the Euclidean distance between the robot's current position and the target point coordinates;

[0022] Step 3.2, designing an obstacle avoidance safety reward based on the sum of confidences of the surrounding area of ​​the robot's current position obtained based on the preset robot safe obstacle avoidance distance;

[0023] Step 3.3: Design a path smoothness reward based on the difference between the current action and the previous action.

[0024] Step 3.4: Design time-effective rewards based on the decision time of the deep reinforcement learning algorithm;

[0025] Step 3.5: Calculate the reward function based on the distance reward, obstacle avoidance safety reward, path smoothness reward, and timeliness reward.

[0026] Furthermore, the method for assisting obstacle avoidance path planning based on the MPC model includes:

[0027] Based on the time Euler angles around the X-axis, Y-axis, and Z-axis, and the robot at time The robot's motion model is established by using the angular velocity components around the X-axis, Y-axis and Z-axis and the Euler angles of the robot around the X-axis, Y-axis and Z-axis at the next moment;

[0028] Predicting the future The state of the time step is obtained by iterating the motion model based on the current state and action sequence. The prediction time domain is set to , the predicted state sequence is ;in, For the moment Predicting the future moment The state vector of

[0029] The optimization objective function of the MPC model is to minimize the distance from the robot to the target point and the sum of the confidences;

[0030] The optimal action sequence is obtained by solving the optimization objective function through the nature-inspired optimization algorithm. In a low-confidence environment, if the deep reinforcement learning algorithm fails to output an obstacle avoidance path within the preset time, the optimal action sequence obtained by the MPC model is used as the obstacle avoidance path for auxiliary planning.

[0031] Furthermore, the state information and action information generated when the obstacle avoidance path is dynamically planned are fed back to the deep neural network to update the obstacle position, obstacle type and confidence distribution value output by the deep neural network;

[0032] Design a joint training loss function to jointly optimize the deep neural network and deep reinforcement learning algorithm with the goal of minimizing the joint training loss function;

[0033] Methods for obtaining the joint training loss function include:

[0034] Calculate obstacle recognition loss, distance loss, obstacle avoidance loss and path planning loss respectively;

[0035] The obstacle recognition loss, distance loss, obstacle avoidance loss and path planning loss are comprehensively calculated to obtain the joint training loss function.

[0036] Furthermore, the method for obtaining the obstacle confidence map includes:

[0037] Step 1: Discretize the spatial coordinates of the entire detection area according to the preset dimensions to form W spatial grids;

[0038] Step 2: Read the obtained obstacle positions, obstacle types and confidence distribution values ​​one by one, encode the obstacle types to obtain codes, and determine the spatial grid position corresponding to the obstacle according to each obstacle position;

[0039] Step 3: For the spatial grid position, if the same spatial grid position contains only one obstacle, the spatial grid position containing only one obstacle is assigned a value according to the obstacle type code and the corresponding confidence distribution value; if the same spatial grid position contains more than one obstacle, the maximum confidence distribution value of all the obstacles and the corresponding obstacle type code are taken to assign a value to the spatial grid position containing more than one obstacle;

[0040] Step 4: Obstacle confidence map is obtained after all spatial grids are assigned values.

[0041] Furthermore, the training method of the deep neural network includes:

[0042] Step 1: Divide the training set data containing the fused multimodal data into a preset batch size, and extract a batch of fused multimodal data for training each time;

[0043] Step 2: Select a deconvolutional neural network as a generator, the input of the generator is a random noise vector, and the output is synthetic multimodal data with a similar distribution to the fused multimodal data; select a convolutional neural network as a discriminator, the input of the discriminator is the fused multimodal data or the synthetic multimodal data, and the output is the discrimination probability of discriminating that the input data is the fused multimodal data;

[0044] Step 3: A batch of random noise vectors are used as input of the generator, and the generator generates a batch of synthetic multimodal data; the synthetic multimodal data is input into the discriminator, and the discriminator outputs the corresponding discrimination probability; according to the output of the discriminator, the probability of minimizing the synthetic multimodal data being judged as false is used as the loss function of the generator, and the parameters of the generator are updated through the back propagation algorithm;

[0045] Step 4: randomly extract a batch of fused multimodal data from the training set, and use the extracted fused multimodal data and the synthetic multimodal data generated by the generator after updating the parameters as the input of the discriminator, and the discriminator outputs the corresponding discrimination probability; the discriminator's loss function is to maximize the probability of correctly discriminating the fused multimodal data and the synthetic multimodal data, and the discriminator's parameters are updated through the back propagation algorithm;

[0046] Step 5: Randomly introduce pseudo noise as the input of the updated generator, fuse it with the training set, obtain a new training set, extract a batch of training data from the new training set and input it into the deep neural model. After calculations at each layer of the deep neural model, obtain the location of the obstacle, obstacle type and confidence distribution value;

[0047] Step 6: According to the total loss function, calculate the total loss function value corresponding to the prediction result of the current batch of training data; with the goal of minimizing the total loss function value, calculate the gradient of the total loss function value to all parameters in the deep neural model through the back propagation algorithm, and use the Adam optimizer to update the parameters according to the gradient;

[0048] Step 7: Repeat the above training cycle process until the deep neural model reaches the preset optimal performance condition, and use the deep neural model corresponding to the preset optimal performance condition as the deep neural network.

[0049] Furthermore, the method of obtaining the total loss function includes:

[0050] According to The training data belongs to The probability of class obstacles is calculated using weighted cross entropy loss;

[0051] Calculate the intersection-over-union loss based on the obstacle position bounding box;

[0052] Calculating a confidence correction term based on the confidence score;

[0053] The total loss function is calculated based on weighted cross entropy loss, intersection-over-union loss, and confidence correction term.

[0054] Furthermore, the method of obtaining fused multimodal data includes:

[0055] Step 1: Obtain an image feature input sequence: divide the image data into T blocks of size P×P; input the T blocks into the convolutional neural network in sequence to obtain T corresponding image vectors, concatenate the T image vectors to obtain image features, and use the image features as the first input sequence of the transformer;

[0056] Obtaining the input sequence of LiDAR point cloud features: Divide the three-dimensional space corresponding to the LiDAR point cloud into spatial grids, calculate the average coordinates of all LiDAR point clouds in each spatial grid, and the point density, where the point density is the number of LiDAR point clouds in the spatial grid, concatenate the average coordinates and the point density as spatial grid features, concatenate all spatial grid features to obtain LiDAR point cloud features, and use the LiDAR point cloud features as the second input sequence of the transformer;

[0057] Obtaining an inertial measurement unit feature input sequence: concatenating the robot's acceleration and angular velocity to obtain an inertial measurement unit feature, and using the inertial measurement unit feature as the third input sequence of the converter;

[0058] Step 2: Pass the first input sequence, the second input sequence, and the third input sequence through a linear embedding layer respectively to obtain a corresponding mapping sequence of a unified dimension;

[0059] Step 3: Add position codes to the vector features in each mapping sequence, add the vector features and position codes together to obtain the final embedding vector;

[0060] Step 4: Perform three different linear transformations on the embedding vector to obtain the query vector, key vector, and value vector respectively;

[0061] Step 5: Obtain the attention score by calculating the dot product of the query vector and the key vector;

[0062] Step 6: Normalize the attention scores to obtain the attention weight of the vector feature at each position relative to the vector features at other positions;

[0063] Step 7: Calculate the weighted sum according to the attention weight and the value vector to obtain the output of the self-attention mechanism, and use the output of the self-attention mechanism as the fused multimodal data.

[0064] Furthermore, the method of adding position coding includes:

[0065] When The dimension of the vector feature at each position When is an even number, the position of the vector feature is encoded based on the sine function; The dimension of the vector feature at each position When is an odd number, the position of the vector feature is encoded based on the cosine function.

[0066] Furthermore, the method for obtaining image data includes:

[0067] Step 1: Data division: Divide the collected denoised images into training set, validation set and test set according to the preset ratio;

[0068] Step 2: Model construction: Select the U-Net structure as the generator, use the random noise vector as the input of the generator, and obtain the generated image data with the same size and number of channels as the denoised image; select the convolutional neural network as the discriminator architecture, use the denoised image and the generated image data as the input of the discriminator, and obtain the probability that the input data is the denoised image;

[0069] Step 3, network training: randomly extract M denoised images from the training set, marked as 1, indicating a real image; the generator generates corresponding generated image data according to the input random noise vector, marked as 0, indicating a generated image; the denoised image and the generated image data are respectively input into the discriminator, and the discriminator outputs the discrimination result; according to the output of the discriminator and the labels corresponding to the denoised image and the generated image data, the parameters of the generator and the discriminator are optimized by the back propagation algorithm with the goal of minimizing the value of the loss function; wherein the loss function is the sum of the generator loss and the discriminator loss, and the cross entropy loss function is used as the generator loss to minimize the probability that the generated image data is identified as false by the discriminator; the cross entropy loss function is used as the discriminator loss to maximize the probability of correctly classifying the denoised image data and the generated image data; repeat the above steps until the generative adversarial network reaches a convergence state;

[0070] Step 4: Image enhancement: Input the denoised image into the generator of the generative adversarial network corresponding to the converged state to generate an enhanced image: use the enhanced image as image data.

[0071] A navigation and obstacle avoidance system in a low confidence and feature-similar environment, implementing the navigation and obstacle avoidance method in a low confidence and feature-similar environment, comprising:

[0072] Data acquisition module: uses sensors to collect multimodal sensor data of obstacles in the detection area under a preset low-confidence environment. The multimodal sensor data includes raw image data, raw lidar point cloud, and raw inertial measurement unit information.

[0073] Data processing module: pre-processes the multimodal sensor data to obtain denoised multimodal sensor data; the denoised multimodal sensor data includes denoised images, lidar point clouds and inertial measurement unit information;

[0074] The first processing module: introduce the generative adversarial network to perform image enhancement processing on the denoised image to obtain image data;

[0075] Data fusion module: uses the transformer architecture to perform cross-domain fusion of image data, lidar point cloud and inertial measurement unit information to obtain fused multi-modal data;

[0076] The second processing module: uses the fused multimodal data as the input of the deep neural network to obtain the obstacle position, obstacle type and confidence distribution value, and construct an obstacle confidence map;

[0077] Obstacle avoidance planning module: Based on the obstacle confidence map obtained in real time, the deep reinforcement learning algorithm is combined to dynamically plan the obstacle avoidance path, and the obstacle avoidance path is assisted in planning based on model predictive control;

[0078] Information feedback module: Feedback the state information and action information generated based on the dynamic planning of obstacle avoidance path to the deep neural network, so as to update the obstacle position, obstacle type and confidence distribution value output by the deep neural network;

[0079] Joint optimization module: Design a joint training loss function to jointly optimize the deep neural network and deep reinforcement learning algorithm with the goal of minimizing the joint training loss function.

[0080] The technical effects and advantages of the navigation obstacle avoidance method and system in low confidence and feature-similar environments of the present invention are as follows:

[0081] The present invention enhances the obstacle recognition capability in low-confidence and feature-similar environments by combining fused multimodal data with deep neural networks. By evaluating the confidence of different sensor data, the high-confidence data is fused and processed, so as to obtain the depth information of obstacles more accurately and reduce the depth perception error. By evaluating the influence of illumination change on feature extraction through confidence, the illumination invariant feature is used to accurately extract obstacle features in strong light, weak light or illumination mutation environments, maintain high-confidence recognition results, and avoid misjudgment caused by illumination. At the same time, the obstacle avoidance path is dynamically planned based on the deep reinforcement learning algorithm, and the obstacle avoidance path is assisted by the MPC model. The planning method improves the intelligence and safety of real-time navigation decision-making. The collaborative optimization and closed-loop feedback mechanism of the two ensure the robustness and real-time performance of the system, especially in environments with insufficient light, blurred or repeated features, providing a reliable obstacle avoidance navigation solution, which can effectively cope with high-challenge scenes such as tunnels, and is widely used in industrial automation, intelligent transportation and other fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0082] Figure 1 It is a flowchart of the navigation obstacle avoidance method in low confidence and feature-similar environments of the present invention;

[0083] Figure 2 is the obstacle confidence map of the present invention;

[0084] Figure 3 It is a block diagram of the navigation and obstacle avoidance system in low confidence and feature-similar environments of the present invention;

[0085] Figure 4 It is a schematic diagram of the system interface of the present invention. DETAILED DESCRIPTION

[0086] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0087] Example 1

[0088] See also Figure 1 As shown, the navigation obstacle avoidance method in the low confidence and feature-similar environment described in this embodiment includes the following steps:

[0089] Using sensors to collect multimodal sensor data of obstacles in a detection area under a preset low confidence environment, the multimodal sensor data includes raw image data, raw lidar point cloud, and raw inertial measurement unit information;

[0090] Collecting raw image data can intuitively present the scene of the surrounding environment, including the appearance, texture, color and other features of obstacles. Based on the raw image data, detection and identification can be performed quickly and accurately. Collecting raw LiDAR point cloud can accurately provide distance information between obstacles and the vehicle itself. In a low-confidence image environment, when image data is difficult to accurately judge the distance of obstacles due to similar features, LiDAR point cloud can provide reliable distance data; LiDAR's three-dimensional point cloud data can supplement the missing spatial information and help better build an environmental model, so as to more effectively identify and avoid obstacles. Raw inertial measurement unit information can provide real-time information on its own motion state, including acceleration and angular velocity. In a low-confidence image environment, when visual information is disturbed or difficult to accurately judge, inertial measurement unit information can help determine its own motion state and then infer the relative motion relationship with obstacles. Using the attitude information provided by the inertial measurement unit, attitude compensation can be performed on the measurement data of sensors (such as LiDAR and cameras), so that obstacle information obtained at different times can be fused and processed in a unified coordinate system, improving the stability and accuracy of obstacle recognition.

[0091] Preprocessing the multimodal sensor data to obtain denoised multimodal sensor data; the denoised multimodal sensor data includes denoised images, lidar point clouds and inertial measurement unit information;

[0092] Methods for obtaining denoised multimodal sensor data include:

[0093] Step 1. Use the original image data as the input of the convolutional neural network to obtain the output denoised image; the convolutional neural network selects perceptual loss as the loss function, and the perceptual loss is the feature difference between the denoised image and the original noise-free image. Through the back propagation algorithm, the gradient of the loss function to the network parameters is calculated, and the selected optimizer is used to update the network parameters according to the gradient; the training process is repeated until the network converges or achieves the predetermined training effect; among them, the convolutional neural network adopts the VGGNet network architecture, and the convolution step size of the VGGNet network architecture is obtained based on the optimization of the nature-inspired optimization algorithm.

[0094] Step 2: Remove high-frequency noise from the original LiDAR point cloud through frequency domain analysis to obtain the LiDAR point cloud. The method for obtaining the LiDAR point cloud includes:

[0095] Step 2.1, use fast Fourier transform to convert the original lidar point cloud from time domain to frequency domain to obtain the spectrum information corresponding to the original lidar point cloud;

[0096] Step 2.2, comparing the spectrum information with a preset threshold, and obtaining the frequency range and frequency domain interval to which the high-frequency noise exceeding the preset threshold belongs;

[0097] Step 2.3, setting the bandwidth of the band-stop filter according to the frequency range to which the high-frequency noise belongs, and using the band-stop filter to filter the frequency domain interval to obtain filtered spectrum information;

[0098] Step 2.4: The frequency domain data after high-frequency noise removal is inversely transformed through inverse fast Fourier transform, and converted from the frequency domain back to the time domain to obtain the lidar point cloud after high-frequency noise removal.

[0099] Step 3: Filter the original inertial measurement unit information to obtain inertial measurement unit information;

[0100] Methods for obtaining inertial measurement unit information include:

[0101] The data is collected using a filter window with a window size of N, and the average value of the collected N consecutive data points is calculated as the filtered inertial measurement unit information.

[0102] Generative adversarial networks are introduced to perform image enhancement processing on denoised images to obtain image data;

[0103] Methods for obtaining image data include:

[0104] Step 1: Data division: Divide the collected denoised images into training set, validation set and test set according to the preset ratio;

[0105] Step 2: Model construction: Select the U-Net structure as the generator, use the random noise vector as the input of the generator, and obtain the generated image data with the same size and number of channels as the denoised image; select the convolutional neural network as the discriminator architecture, use the denoised image and the generated image data as the input of the discriminator, and obtain the probability that the input data is the denoised image;

[0106] Step 3, network training: randomly extract M denoised images from the training set, marked as 1, indicating a real image; the generator generates corresponding generated image data according to the input random noise vector, marked as 0, indicating a generated image; the denoised image and generated image data are respectively input into the discriminator, and the discriminator outputs the discrimination result; according to the output of the discriminator and the labels corresponding to the denoised image and generated image data, the parameters of the generator and the discriminator are optimized through the back propagation algorithm with the goal of minimizing the value of the loss function; wherein the loss function is the sum of the generator loss and the discriminator loss, and the cross entropy loss function is used as the generator loss to minimize the probability that the generated image data is identified as false by the discriminator; the cross entropy loss function is used as the discriminator loss to maximize the probability of correctly classifying the denoised image data and the generated image data; repeat the above steps for multiple rounds of iterative training until the generative adversarial network reaches a convergence state.

[0107] Step 4: Image enhancement: Input the denoised image into the generator of the generative adversarial network corresponding to the converged state to generate an enhanced image: use the enhanced image as image data.

[0108] Use the transformer architecture to perform cross-domain fusion of image data, lidar point cloud, and inertial measurement unit information to obtain fused multimodal data;

[0109] Methods for obtaining fused multimodal data include:

[0110] Step 1. Obtain an image feature input sequence: divide the image data into T blocks of size P×P; input the T blocks into the convolutional neural network in sequence to obtain T corresponding image vectors, concatenate the T image vectors to obtain image features, and use the image features as the first input sequence of the transformer.

[0111] Obtain the input sequence of lidar point cloud features: divide the three-dimensional space corresponding to the lidar point cloud into spatial grids, calculate the average coordinates and point density of all lidar point clouds in each spatial grid, where the point density is the number of lidar point clouds in the spatial grid, concatenate the average coordinates and point density as spatial grid features, concatenate all spatial grid features to obtain lidar point cloud features, and use the lidar point cloud features as the second input sequence of the transformer.

[0112] Obtain an inertial measurement unit feature input sequence: concatenate the acceleration and angular velocity to obtain an inertial measurement unit feature, and use the inertial measurement unit feature as the third input sequence of the converter.

[0113] Step 2: Pass the first input sequence, the second input sequence, and the third input sequence through a linear embedding layer respectively to obtain a corresponding mapping sequence of a unified dimension;

[0114] Step 3: Add position codes to the vector features in each mapping sequence, add the vector features and position codes together to obtain the final embedding vector;

[0115] Methods for adding positional encoding include:

[0116] When The dimension of the vector feature at each position When it is an even number, the position encoding of the vector feature ; When the The dimension of the vector feature at each position When it is an odd number, the position encoding of the vector feature ;in, is a dimension parameter; for example, when the dimension of the vector feature is the first input sequence or the third input sequence, is an odd number; the dimension of the vector feature is the second input sequence, An even number.

[0117] Step 4: Perform three different linear transformations on the embedding vector to obtain the query vector, key vector, and value vector respectively;

[0118] Step 5: Obtain the attention score by calculating the dot product of the query vector and the key vector;

[0119] Step 6: Normalize the attention scores to obtain the attention weight of the vector feature at each position relative to the vector features at other positions;

[0120] Step 7: Calculate the weighted sum according to the attention weight and the value vector to obtain the output of the self-attention mechanism, and use the output of the self-attention mechanism as the fused multimodal data.

[0121] The fused multimodal data is used as the input of the deep neural network to obtain the obstacle position, obstacle type and confidence distribution value, and construct an obstacle confidence map;

[0122] Methods for training deep neural networks include:

[0123] Step 1: Divide the training set data containing the fused multimodal data according to a preset batch size, and extract a batch of fused multimodal data for training each time.

[0124] Step 2: Select a deconvolutional neural network as a generator, the input of the generator is a random noise vector, and the output is synthetic multimodal data with a similar distribution to the fused multimodal data; select a convolutional neural network as a discriminator, the input of the discriminator is the fused multimodal data or the synthetic multimodal data, and the output is the discrimination probability of discriminating that the input data is the fused multimodal data;

[0125] Step 3. Randomly introduce a batch of random noise vectors as the input of the generator, and the generator generates a batch of synthetic multimodal data; input the synthetic multimodal data into the discriminator, and the discriminator outputs the corresponding discrimination probability; according to the output of the discriminator, minimize the probability of synthetic multimodal data being judged as false as the loss function of the generator, and update the parameters of the generator through the back propagation algorithm.

[0126] Step 4: Randomly extract a batch of fused multimodal data from the training set, and use the extracted fused multimodal data and the synthetic multimodal data generated by the generator with updated parameters as the input of the discriminator, and the discriminator outputs the corresponding discrimination probability; the discriminator's loss function is to maximize the probability of correctly discriminating the fused multimodal data and the synthetic multimodal data, and the discriminator's parameters are updated through the back propagation algorithm.

[0127] Step 5: Randomly introduce pseudo-noise as the input of the updated generator and fuse it with the training set to obtain a new training set. Extract a batch of training data from the new training set and input it into the deep neural model. After calculations at each layer of the deep neural model, obtain the location of the obstacle, obstacle type and confidence distribution value.

[0128] Step 6. According to the total loss function, calculate the total loss function value corresponding to the prediction results of the current batch of training data; with the goal of minimizing the total loss function value, calculate the gradient of the total loss function value to all parameters in the deep neural model through the back propagation algorithm, and use the Adam optimizer to update the parameters according to the gradient.

[0129] Methods for obtaining the total loss function include:

[0130] Calculate the weighted cross entropy loss:

[0131] Calculate weighted cross entropy loss ;in, is the number of training data; is the number of obstacle categories; is the category weight, which can be obtained by optimizing the natural inspiration optimization algorithm; For the The real obstacle categories of the training data; The deep neural model predicts The training data belongs to Category obstacle probability;

[0132] Calculate the intersection-over-union loss ;in, To predict the obstacle position bounding box; is the bounding box of the actual obstacle position;

[0133] Calculate the confidence correction term ;in, is the penalty coefficient; is the confidence score;

[0134] Calculate the total loss function ;in, and is the balance coefficient, which can be obtained by optimizing the natural inspiration optimization algorithm.

[0135] Step 7: Repeat the above training cycle process for multiple rounds of training until the deep neural model reaches the preset optimal performance condition, and use the deep neural model corresponding to the preset optimal performance condition as the deep neural network.

[0136] Methods for obtaining obstacle confidence maps include:

[0137] Step 1: Discretize the spatial coordinates of the entire detection area according to the preset dimensions to form W spatial grids;

[0138] Step 2: Read the obtained obstacle positions, obstacle types and confidence distribution values ​​one by one, encode the obstacle types to obtain codes, and determine the spatial grid position corresponding to the obstacle according to each obstacle position.

[0139] Step 3: For the spatial grid position, if the same spatial grid position contains only one obstacle, the spatial grid position containing only one obstacle is assigned a value according to the obstacle type code and the corresponding confidence distribution value; if the same spatial grid position contains more than one obstacle, the maximum confidence distribution value of all the obstacles and the corresponding obstacle type code are taken to assign a value to the spatial grid position containing more than one obstacle.

[0140] Step 4: Obstacle confidence map is obtained after all spatial grids are assigned values. Figure 2 , Figure 2 A 2D obstacle confidence map of the top-down plane for planning the path to the roadblock.

[0141] Based on the obstacle confidence map obtained in real time, the obstacle avoidance path is dynamically planned in combination with the deep reinforcement learning algorithm, and the obstacle avoidance path is assisted in planning based on the MPC model;

[0142] Methods for dynamically planning obstacle avoidance paths using deep reinforcement learning algorithms include:

[0143] Step 1: Define the state space : State space Includes the robot's current position ,speed , obstacle confidence map And global path planning information ;in, The X-axis coordinate is , the Y-axis coordinate is , the Z-axis coordinate is The current position of the robot; The X-axis coordinate is , the Y-axis coordinate is , the Z-axis coordinate is The spatial grid; The X-axis coordinate is , the Y-axis coordinate is , the Z-axis coordinate is The waypoint coordinates of is the acceleration; is the angular velocity; get the state space ;

[0144] Step 2: Define the action space: Action space is a set of discrete actions, action space Includes the union of basic actions and obstacle avoidance action sequences, where the basic actions include forward actions , turning action and deceleration action etc. basic actions; the obstacle avoidance action sequence is a finite set of action subsequences ,in, and They are the first and second action subsequences respectively; each subsequence contains multiple consecutive actions, for example ;in, , and The first action subsequence contains three consecutive actions. For the first turning action, For the first forward movement, For the second turning action;

[0145] Step 3: Design a reward function: Methods for obtaining a reward function include:

[0146] Step 3.1: Design the distance reward for reaching the target point ;in, The distance weight is used to adjust the importance of the distance reward and can be obtained by searching the nature-inspired optimization algorithm; is the Euclidean distance between the robot's current position and the target point coordinates.

[0147] Step 3.2: Design obstacle avoidance safety reward: ;in, It is the obstacle avoidance safety weight coefficient, which is used to penalize the path selection in the high-risk area. It is obtained by comparing the confidence with the preset confidence threshold. If the confidence is higher than the preset confidence threshold, it means that it is in a high-risk area, and the obstacle avoidance safety weight coefficient is higher. To obtain the current position of the robot based on the preset robot safety obstacle avoidance distance The sum of the confidence scores of the surrounding areas.

[0148] Step 3.3: Design path smoothness reward ;in, is the path smoothness reward weight, which can be obtained by optimizing the natural inspiration optimization algorithm; is the action difference function; The action currently being taken; For the previous action.

[0149] Step 3.4: Design time-based rewards ;in, It is a timeliness weight, which is used to encourage quick decision-making and can be obtained by searching the optimization algorithm inspired by nature; Decision time for deep reinforcement learning algorithms.

[0150] Step 3.5: Calculate the reward function .

[0151] Step 4: Build a Q network based on DQN ,in, For status, For action, is the parameter of the network; it is used to learn the optimal Q function to maximize the value of the cumulative reward function.

[0152] Using the experience replay mechanism, the four-tuple in each decision-making process Stored in the preset experience playback buffer, quad Indicates the state, action, and reward corresponding to the current moment, as well as the state corresponding to the next moment.

[0153] Define the loss function of the Q network L ψ = E [( r + γ max a ' Q s ' , a ' ; ψ - - Q s , a ; ψ ) 2 ] ;in, is the mathematical expectation; For reward; are the parameters of the target network; is a discount factor used to balance current rewards and future rewards; For the next state Below, with parameters The Q network has the maximum Q value corresponding to all possible actions, that is, transferring from the current state to the next state After all the optional actions The maximum expected cumulative reward that can be obtained in . The Q network is optimized with the minimum loss function value as the optimization goal to obtain the corresponding optimal Q network. When it is necessary to plan an obstacle avoidance path, the optimal Q network is used to select and execute the optimal action according to the environmental state, and the optimal action is used as the obstacle avoidance path for dynamic planning.

[0154] The methods for assisting obstacle avoidance path planning based on model predictive control include:

[0155] Build a robot motion model ;in, , and The robot is at the time Euler angles around the X, Y and Z axes; , and The robot is at the time Angular velocity components about the X, Y, and Z axes; is the time interval; , and are the Euler angles of the robot around the X-axis, Y-axis, and Z-axis at the next moment.

[0156] Predicting the future The state of the time step is obtained by iterating the motion model based on the current state and action sequence. Assume that the prediction time domain is , the predicted state sequence is ;in, For the moment Predicting the future moment The state vector of .

[0157] The optimization objective function of the MPC model is to minimize the distance from the robot to the target point and the sum of the confidences. The optimization objective function is ;in, For the moment Predicting the future moment The distance to the target point; For the moment Predicting the future moment The sum of obstacle confidences; and They are and The corresponding weights can be obtained through the nature-inspired optimization algorithm.

[0158] The optimal action sequence is obtained by solving the optimization objective function through the nature-inspired optimization algorithm. In a low-confidence environment, if the deep reinforcement learning algorithm fails to output an obstacle avoidance path within the preset time, the optimal action sequence obtained by the MPC model is used as the obstacle avoidance path for auxiliary planning.

[0159] The obstacle confidence map is passed to the deep reinforcement learning algorithm in real time, and the priority of path planning is dynamically adjusted in combination with the deep reinforcement learning algorithm. The state information and action information generated when the obstacle avoidance path is dynamically planned in combination with the deep reinforcement learning algorithm are fed back to the deep neural network to update the obstacle position, obstacle type and confidence distribution value output by the deep neural network;

[0160] The output error of the deep neural network will directly affect the obstacle avoidance path planning results. It is necessary to feed back the state information and action information generated when dynamically planning the obstacle avoidance path to the deep neural network: when the path planning finds that the perception data does not match the actual environment, the deep neural network can be recalibrated and updated.

[0161] A joint training loss function is designed to jointly optimize the deep neural network and deep reinforcement learning algorithm with the goal of minimizing the joint training loss function.

[0162] Methods for obtaining the joint training loss function include:

[0163] Calculate obstacle recognition loss ;in, is the sample size; For the The confidence weight of each sample can be obtained by optimizing the natural inspiration optimization algorithm; For the The identification results of samples; The deep neural model predicts The identification results of samples;

[0164] Calculating distance loss ;in, is the number of positions in the robot position sequence; is the Euclidean distance calculation formula; is the first position in the sequence positions; is the target location;

[0165] Calculating obstacle avoidance loss ;in, For the The direction vector of the position; For the The direction vector of the position;

[0166] Calculate path planning loss ;in, is the number of obstacles; For the obstacle locations; Safety distance for obstacles;

[0167] Calculate the joint training loss function ;in, and is the weight balance coefficient, which can be obtained by optimizing the natural inspiration optimization algorithm.

[0168] Example 2

[0169] See also Figure 3 As shown, the navigation and obstacle avoidance system in the low confidence and feature-similar environment described in this embodiment includes:

[0170] Data acquisition module: uses sensors to collect multimodal sensor data of obstacles in the detection area under a preset low-confidence environment. The multimodal sensor data includes raw image data, raw lidar point cloud, and raw inertial measurement unit information.

[0171] Data processing module: pre-processes the multimodal sensor data to obtain denoised multimodal sensor data; the denoised multimodal sensor data includes denoised images, lidar point clouds and inertial measurement unit information;

[0172] The first processing module: introduce the generative adversarial network to perform image enhancement processing on the denoised image to obtain image data;

[0173] Data fusion module: uses the transformer architecture to perform cross-domain fusion of image data, lidar point cloud and inertial measurement unit information to obtain fused multi-modal data;

[0174] The second processing module: uses the fused multimodal data as the input of the deep neural network to obtain the obstacle position, obstacle type and confidence distribution value, and construct an obstacle confidence map;

[0175] Obstacle avoidance planning module: Based on the obstacle confidence map obtained in real time, the deep reinforcement learning algorithm is combined to dynamically plan the obstacle avoidance path, and the obstacle avoidance path is assisted in planning based on model predictive control;

[0176] Information feedback module: Feedback the state information and action information generated based on the dynamic planning of obstacle avoidance path to the deep neural network, so as to update the obstacle position, obstacle type and confidence distribution value output by the deep neural network;

[0177] Joint optimization module: Design a joint training loss function to jointly optimize the deep neural network and deep reinforcement learning algorithm with the goal of minimizing the joint training loss function.

[0178] According to the obstacle confidence map output by the second processing module, please refer to Figure 2, compare the confidence distribution value with the preset confidence threshold to obtain the corresponding obstacle position distribution, including position A (3.2, 3.4), position B (4.1, 3.9), position C (4.5, 5.2) and position D (6.4, 6.9), and the corresponding confidence distribution values ​​are 0.81, 0.83, 0.91 and 0.88. Among them, position A, position C and position D are all dynamic obstacles, and position B is a static obstacle; please refer to Figure 4 The system interface diagram can be obtained Figure 4 The path map of the obstacle detection and avoidance planning area on the upper right.

[0179] During the actual operation of the system in this embodiment, the output error of the second processing module will directly affect the obstacle avoidance path planning result. Therefore, a dynamic update mechanism based on feedback can be designed: when the path planning finds that the perception data does not match the actual environment, the recalibration and update of the second processing module is triggered.

[0180] In low confidence and feature similarity environments, the coordinated optimization of the second processing module and the obstacle avoidance planning module is particularly important. A feature similarity evaluation mechanism can be introduced, and the system can evaluate in real time which areas in the detection area have more ambiguous features, thereby dynamically adjusting the path planning strategy to avoid the algorithm from misleading the path due to insufficient environmental features.

[0181] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

[0182] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A navigation and obstacle avoidance method in a low confidence and feature-similar environment, characterized in that: The steps include: Using sensors to collect multimodal sensor data of obstacles in a detection area under a preset low confidence environment, the multimodal sensor data includes raw image data, raw lidar point cloud, and raw inertial measurement unit information; Preprocessing the multimodal sensor data to obtain denoised multimodal sensor data; Denoising multimodal sensor data including denoised images, LiDAR point clouds, and inertial measurement unit information; Generative adversarial networks are introduced to perform image enhancement processing on denoised images to obtain image data; Use the transformer architecture to perform cross-domain fusion of image data, lidar point cloud, and inertial measurement unit information to obtain fused multimodal data; The fused multimodal data is used as the input of the deep neural network to obtain the obstacle position, obstacle type and confidence distribution value, and construct an obstacle confidence map; Based on the obstacle confidence map obtained in real time, the obstacle avoidance path is dynamically planned in combination with the deep reinforcement learning algorithm, and the obstacle avoidance path is assisted in planning based on the MPC model.

2. The navigation and obstacle avoidance method in a low confidence and feature similar environment according to claim 1, characterized in that: Methods for dynamically planning obstacle avoidance paths using deep reinforcement learning algorithms include: Step 1: Define the state space , state space Including the robot's current position, speed, obstacle confidence map and global path planning information; Step 2: Define the action space , action space It includes the union of basic actions and obstacle avoidance action sequences; Step 3: Design reward function. Step 4: Build a Q network based on DQN ,in, For status, For action, is the parameter of the network; it is used to learn the optimal Q function to maximize the value of the cumulative reward function; using the experience replay mechanism, the four-tuple in each decision-making process is stored in the preset experience replay buffer, the four-tuple includes the state, action and reward corresponding to the current moment, and the state corresponding to the next moment; Define the loss function of the Q network; optimize the Q network with the minimum loss function value as the optimization goal to obtain the corresponding optimal Q network. When it is necessary to plan an obstacle avoidance path, use the optimal Q network to select and execute the optimal action according to the current state, and use the optimal action as the dynamically planned obstacle avoidance path.

3. The navigation and obstacle avoidance method in a low confidence and feature similar environment according to claim 2, characterized in that: Methods for obtaining reward functions include: Step 3.1, design the distance reward for reaching the target point based on the Euclidean distance between the robot's current position and the target point coordinates; Step 3.2, designing an obstacle avoidance safety reward based on the sum of confidences of the surrounding area of ​​the robot's current position obtained based on the preset robot safe obstacle avoidance distance; Step 3.3: Design a path smoothness reward based on the difference between the current action and the previous action. Step 3.4: Design time-effective rewards based on the decision time of the deep reinforcement learning algorithm; Step 3.5: Calculate the reward function based on the distance reward, obstacle avoidance safety reward, path smoothness reward, and timeliness reward.

4. The navigation and obstacle avoidance method in a low confidence and feature similar environment according to claim 3, characterized in that: The method for assisting obstacle avoidance path planning based on the MPC model includes: Based on the time Euler angles around the X-axis, Y-axis, and Z-axis, and the robot at time The robot's motion model is established by using the angular velocity components around the X-axis, Y-axis and Z-axis and the Euler angles of the robot around the X-axis, Y-axis and Z-axis at the next moment; Predicting the future The state of the time step is obtained by iterating the motion model based on the current state and action sequence. The prediction time domain is set to , the predicted state sequence is ;in, For the moment Predicting the future moment The state vector of Taking minimizing the distance from the robot to the target point and the sum of the confidence as the optimization objective function of the MPC model, the optimization objective function is obtained; The optimal action sequence is obtained by solving the optimization objective function through the nature-inspired optimization algorithm. In a low-confidence environment, if the deep reinforcement learning algorithm fails to output an obstacle avoidance path within the preset time, the optimal action sequence obtained by the MPC model is used as the obstacle avoidance path for auxiliary planning.

5. The navigation and obstacle avoidance method in a low confidence and feature similar environment according to claim 4, characterized in that: Feeding back the state information and action information generated when dynamically planning the obstacle avoidance path to the deep neural network, so as to update the obstacle position, obstacle type and confidence distribution value output by the deep neural network; Design a joint training loss function to jointly optimize the deep neural network and deep reinforcement learning algorithm with the goal of minimizing the joint training loss function; Methods for obtaining the joint training loss function include: Calculate obstacle recognition loss, distance loss, obstacle avoidance loss and path planning loss respectively; The obstacle recognition loss, distance loss, obstacle avoidance loss and path planning loss are comprehensively calculated to obtain the joint training loss function.

6. The navigation and obstacle avoidance method in a low confidence and feature similarity environment according to claim 1, characterized in that: Methods for obtaining obstacle confidence maps include: Step 1: Discretize the spatial coordinates of the entire detection area according to the preset dimensions to form W spatial grids; Step 2: Read the obtained obstacle positions, obstacle types and confidence distribution values ​​one by one, encode the obstacle types to obtain codes, and determine the spatial grid position corresponding to the obstacle according to each obstacle position; Step 3: For the spatial grid position, if the same spatial grid position contains only one obstacle, the spatial grid position containing only one obstacle is assigned a value according to the obstacle type code and the corresponding confidence distribution value; if the same spatial grid position contains more than one obstacle, the maximum confidence distribution value of all the obstacles and the corresponding obstacle type code are taken to assign a value to the spatial grid position containing more than one obstacle; Step 4: After all spatial grids are assigned values, the obstacle confidence map is obtained.

7. The navigation and obstacle avoidance method in a low confidence and feature similarity environment according to claim 1, characterized in that: Methods for training deep neural networks include: Step 1: Divide the training set data containing the fused multimodal data into a preset batch size, and extract a batch of fused multimodal data for training each time; Step 2: Select a deconvolutional neural network as a generator, the input of the generator is a random noise vector, and the output is synthetic multimodal data with a similar distribution to the fused multimodal data; select a convolutional neural network as a discriminator, the input of the discriminator is the fused multimodal data or the synthetic multimodal data, and the output is the discrimination probability of discriminating that the input data is the fused multimodal data; Step 3: A batch of random noise vectors are used as input of the generator, and the generator generates a batch of synthetic multimodal data; the synthetic multimodal data is input into the discriminator, and the discriminator outputs the corresponding discrimination probability; according to the output of the discriminator, the probability of minimizing the synthetic multimodal data being judged as false is used as the loss function of the generator, and the parameters of the generator are updated through the back propagation algorithm; Step 4: randomly extract a batch of fused multimodal data from the training set, and use the extracted fused multimodal data and the synthetic multimodal data generated by the generator after updating the parameters as the input of the discriminator, and the discriminator outputs the corresponding discrimination probability; the discriminator's loss function is to maximize the probability of correctly discriminating the fused multimodal data and the synthetic multimodal data, and the discriminator's parameters are updated through the back propagation algorithm; Step 5: Randomly introduce pseudo noise as the input of the updated generator, fuse it with the training set, obtain a new training set, extract a batch of training data from the new training set and input it into the deep neural model. After calculations at each layer of the deep neural model, obtain the location of the obstacle, obstacle type and confidence distribution value; Step 6: According to the total loss function, calculate the total loss function value corresponding to the prediction result of the current batch of training data; with the goal of minimizing the total loss function value, calculate the gradient of the total loss function value to all parameters in the deep neural model through the back propagation algorithm, and use the Adam optimizer to update the parameters according to the gradient; Step 7: Repeat the above training cycle process until the deep neural model reaches the preset optimal performance condition, and use the deep neural model corresponding to the preset optimal performance condition as the deep neural network.

8. The navigation and obstacle avoidance method in a low confidence and feature similar environment according to claim 7, characterized in that: Methods for obtaining the total loss function include: According to The training data belongs to The probability of class obstacles is calculated using weighted cross entropy loss; Calculate the intersection-over-union loss based on the obstacle position bounding box; Calculating a confidence correction term based on the confidence score; The total loss function is calculated based on weighted cross entropy loss, intersection-over-union loss, and confidence correction term.

9. The navigation and obstacle avoidance method in a low confidence and feature similarity environment according to claim 1, characterized in that: Methods for obtaining fused multimodal data include: Step 1: Obtain an image feature input sequence: divide the image data into T blocks of size P×P; input the T blocks into the convolutional neural network in sequence to obtain T corresponding image vectors, concatenate the T image vectors to obtain image features, and use the image features as the first input sequence of the transformer; Obtaining the input sequence of LiDAR point cloud features: Divide the three-dimensional space corresponding to the LiDAR point cloud into spatial grids, calculate the average coordinates of all LiDAR point clouds in each spatial grid, and the point density, where the point density is the number of LiDAR point clouds in the spatial grid, concatenate the average coordinates and the point density as spatial grid features, concatenate all spatial grid features to obtain LiDAR point cloud features, and use the LiDAR point cloud features as the second input sequence of the transformer; Obtaining an inertial measurement unit feature input sequence: concatenating the robot's acceleration and angular velocity to obtain an inertial measurement unit feature, and using the inertial measurement unit feature as the third input sequence of the converter; Step 2: Pass the first input sequence, the second input sequence, and the third input sequence through a linear embedding layer respectively to obtain a corresponding mapping sequence of a unified dimension; Step 3: Add position codes to the vector features in each mapping sequence, add the vector features and position codes together to obtain the final embedding vector; Step 4: Perform three different linear transformations on the embedding vector to obtain the query vector, key vector, and value vector respectively; Step 5: Obtain the attention score by calculating the dot product of the query vector and the key vector; Step 6: Normalize the attention scores to obtain the attention weight of the vector feature at each position relative to the vector features at other positions; Step 7: Calculate the weighted sum according to the attention weight and the value vector to obtain the output of the self-attention mechanism, and use the output of the self-attention mechanism as the fused multimodal data.

10. The navigation and obstacle avoidance method in a low confidence and feature similarity environment according to claim 9, characterized in that: Methods for adding positional encoding include: When The dimension of the vector feature at each position When is an even number, the position of the vector feature is encoded based on the sine function; The dimension of the vector feature at each position When is an odd number, the position of the vector feature is encoded based on the cosine function.

11. The navigation and obstacle avoidance method in a low confidence and feature similarity environment according to claim 1, characterized in that: Methods for obtaining image data include: Step 1: Data division: Divide the collected denoised images into training set, validation set and test set according to the preset ratio; Step 2: Model construction: Select the U-Net structure as the generator, use the random noise vector as the input of the generator, and obtain the generated image data with the same size and number of channels as the denoised image; select the convolutional neural network as the discriminator architecture, use the denoised image and the generated image data as the input of the discriminator, and obtain the probability that the input data is the denoised image; Step 3, network training: randomly extract M denoised images from the training set, marked as 1, indicating a real image; the generator generates corresponding generated image data according to the input random noise vector, marked as 0, indicating a generated image; the denoised image and the generated image data are respectively input into the discriminator, and the discriminator outputs the discrimination result; according to the output of the discriminator and the labels corresponding to the denoised image and the generated image data, the parameters of the generator and the discriminator are optimized by the back propagation algorithm with the goal of minimizing the value of the loss function; wherein the loss function is the sum of the generator loss and the discriminator loss, and the cross entropy loss function is used as the generator loss to minimize the probability that the generated image data is identified as false by the discriminator; the cross entropy loss function is used as the discriminator loss to maximize the probability of correctly classifying the denoised image data and the generated image data; repeat the above steps until the generative adversarial network reaches a convergence state; Step 4: Image enhancement: Input the denoised image into the generator of the generative adversarial network corresponding to the converged state to generate an enhanced image: use the enhanced image as image data.

12. A navigation and obstacle avoidance system in a low confidence and feature-similar environment, implementing the navigation and obstacle avoidance method in a low confidence and feature-similar environment as described in any one of claims 1 to 11, characterized in that: include: Data acquisition module: uses sensors to collect multimodal sensor data of obstacles in the detection area under a preset low-confidence environment. The multimodal sensor data includes raw image data, raw lidar point cloud, and raw inertial measurement unit information. Data processing module: pre-process the multimodal sensor data to obtain denoised multimodal sensor data; Denoising multimodal sensor data including denoised images, LiDAR point clouds, and inertial measurement unit information; The first processing module: introduce the generative adversarial network to perform image enhancement processing on the denoised image to obtain image data; Data fusion module: uses the transformer architecture to perform cross-domain fusion of image data, lidar point cloud and inertial measurement unit information to obtain fused multi-modal data; The second processing module: uses the fused multimodal data as the input of the deep neural network to obtain the obstacle position, obstacle type and confidence distribution value, and construct an obstacle confidence map; Obstacle avoidance planning module: Based on the obstacle confidence map obtained in real time, the deep reinforcement learning algorithm is combined to dynamically plan the obstacle avoidance path, and the obstacle avoidance path is assisted in planning based on model predictive control; Information feedback module: Feedback the state information and action information generated based on the dynamic planning of obstacle avoidance path to the deep neural network, so as to update the obstacle position, obstacle type and confidence distribution value output by the deep neural network; Joint optimization module: Design a joint training loss function to jointly optimize the deep neural network and deep reinforcement learning algorithm with the goal of minimizing the joint training loss function.

Images