Robot automatic navigation and homeward voyage method, device, equipment and storage medium

Abstract

The invention relates to the technical field of robot navigation, and discloses a robot automatic navigation and homeward voyage method, device and equipment and a storage medium, and the method comprises the steps: carrying out the multi-scale environment feature extraction of environment image data collected by a walking replacing robot, and obtaining environment feature data; constructing an initial navigation path; performing target detection and feature re-identification on the real-time environment image to obtain a real-time environment perception result; performing extended Kalman filtering fusion processing on the multi-sensor data to obtain robot-environment interaction state features; according to robot-environment interaction state characteristics, carrying out linear section and turning section trajectory analysis, and constructing a smooth navigation trajectory; continuously updating the environment characteristic data and the working state data of the walking-instead-of-walk robot in the navigation process, and constructing a dynamic environment map; and when a return flight signal is received, quickly planning an automatic return flight path based on the dynamic environment map. The autonomy and the working efficiency of the walking-replacing robot can be improved.

Description

Technical Field

[0001] The present invention relates to the field of robot navigation technology, and in particular to a robot automatic navigation and return method, device, equipment and storage medium. Background Art

[0002] Robots can not only provide services such as food delivery and guidance, but also greatly improve work efficiency and customer experience. However, existing mobility robots often have difficulty achieving stable and reliable autonomous navigation and return functions in complex and changing indoor environments, which seriously limits their practicality and work efficiency. Traditional navigation methods usually rely on pre-established static maps and cannot effectively deal with uncertain factors in dynamic environments, such as moving obstacles and crowded areas.

[0003] In addition, existing navigation algorithms often have low computational efficiency when processing large-scale environmental data, making it difficult to meet real-time requirements. At the same time, due to the lack of in-depth understanding of environmental characteristics and effective fusion of multi-sensor data, existing methods are prone to positioning deviations and unreasonable path planning in complex scenarios. These problems not only affect the accuracy and smoothness of navigation, but may also cause delays, collisions, and other situations that affect service quality when the robot is performing tasks or returning home. Summary of the invention

[0004] The present invention provides a method, device, equipment and storage medium for automatic navigation and return of a robot, and the present invention can improve the autonomy and work efficiency of the walking robot.

[0005] In a first aspect, the present invention provides a robot automatic navigation and return method, the robot automatic navigation and return method comprising: Perform multi-scale environmental feature extraction on the environmental image data collected by the walking robot to obtain environmental feature data; Constructing an initial navigation path according to the environmental feature data and the historical navigation data; Based on the initial navigation path, target detection and feature re-identification are performed on the real-time environment image to obtain a real-time environment perception result; According to the real-time environment perception results, the multi-sensor data is subjected to extended Kalman filter fusion processing to obtain the robot-environment interaction state characteristics; Performing straight line segment and turning segment trajectory analysis according to the robot-environment interaction state characteristics to construct a smooth navigation trajectory; During the navigation process, the environmental feature data and the working status data of the walking robot are continuously updated to construct a dynamic environmental map; when a return signal is received, an automatic return path is quickly planned based on the dynamic environmental map.

[0006] In a second aspect, the present invention provides a robot automatic navigation and return device, the robot automatic navigation and return device comprising: An extraction module is used to extract multi-scale environmental features from environmental image data collected by the walking robot to obtain environmental feature data; A construction module, used for constructing an initial navigation path according to the environmental feature data and the historical navigation data; A recognition module, used to perform target detection and feature re-recognition on the real-time environment image based on the initial navigation path to obtain a real-time environment perception result; A processing module, used to perform extended Kalman filter fusion processing on multi-sensor data according to the real-time environment perception result to obtain robot-environment interaction state characteristics; An analysis module, used for analyzing the trajectory of straight segments and turning segments according to the robot-environment interaction state characteristics, and constructing a smooth navigation trajectory; The planning module is used to continuously update the environmental feature data and the working status data of the walking robot during the navigation process to build a dynamic environmental map; when a return signal is received, the automatic return path is quickly planned based on the dynamic environmental map.

[0007] The third aspect of the present invention provides a computer device, comprising: a memory and at least one processor, wherein the memory stores instructions; the at least one processor calls the instructions in the memory so that the computer device executes the above-mentioned robot automatic navigation and return method.

[0008] A fourth aspect of the present invention provides a computer-readable storage medium, in which instructions are stored. When the computer-readable storage medium is run on a computer, the computer executes the above-mentioned robot automatic navigation and return method.

[0009] In the technical solution provided by the present invention, the multi-scale environmental feature extraction technology is combined with global-local contrast learning to capture environmental information more comprehensively and accurately, thereby improving the accuracy and robustness of environmental perception. The adaptive uncertainty path planning model effectively copes with various uncertain factors in complex dynamic environments by dynamically constructing an uncertain parameter set, thereby improving the adaptability and reliability of path planning. The TensorRT-accelerated YOLOv5 target detection and OSNet feature re-identification technology significantly improve the efficiency and accuracy of real-time environmental perception, and the extended Kalman filter fusion processes multi-sensor data, effectively improving the accuracy of robot-environment interaction state estimation. The differential flat trajectory generator combines polynomial interpolation and Bezier curves to generate smooth navigation trajectories that meet dynamic constraints, thereby improving the comfort and safety of navigation. The continuous update of the dynamic environment map and the fast return path planning technology enable the robot to respond to environmental changes in a timely manner and achieve safe and efficient automatic return. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0011] Figure 1 A schematic diagram of the steps of the robot automatic navigation and return method in an embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of the robot automatic navigation and return device in an embodiment of the present invention; Figure 3 It is a schematic block diagram of the structure of a computer device in an embodiment of the present invention. DETAILED DESCRIPTION

[0012] Embodiments of the present invention provide a method, device, equipment and storage medium for automatic navigation and return of a robot. The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0013] For ease of understanding, the specific process of the embodiment of the present invention is described below. Figure 1 , an embodiment of the robot automatic navigation and return method in the embodiment of the present invention includes: Step S1, extracting multi-scale environmental features from environmental image data collected by the walking robot to obtain environmental feature data; It is understandable that the execution subject of the present invention may be a robot automatic navigation and return device, or a terminal or a server, which is not limited here. The embodiment of the present invention is described by taking a server as the execution subject as an example.

[0014] Specifically, the environmental image data collected by the walking robot is processed, and the image is decomposed into multiple scales using the Gaussian pyramid decomposition method to obtain a multi-scale image set. Gaussian pyramid decomposition recursively downsamples the original image so that the system analyzes the image at different scales and captures the global and local information of the image. For the obtained multi-scale image set, a multi-layer convolutional neural network is used to extract convolutional features. The convolution operation includes shallow, medium and deep convolutions, thereby generating a multi-scale feature atlas. The features extracted by shallow convolution are usually low-level edge and texture information, while deep convolution extracts more abstract and high-level semantic information. The multi-scale feature atlas is input into the feature pyramid network, and the feature maps of different scales are upsampled and fused to generate a feature map with global semantic information, namely the global environmental feature map. The feature pyramid network adaptively fuses features of different scales so that the final feature map can retain both detail information and overall semantic information, including the overall layout of the scene and the position and shape of objects. The global environment feature map is input into the spatial attention module, which generates a key area weight map by calculating the spatial correlation of the feature map. The role of the spatial attention mechanism is to highlight particularly important areas in the environment, that is, areas that have a significant impact on the robot's navigation. By calculating the spatial correlation between different positions in the feature map, the most critical areas for understanding the environment are identified, and these areas are given higher weights to generate a key area weight map. Based on the key area weight map, the global environment feature map is weighted to obtain a local environment feature map. The global environment feature map is used as an anchor sample, the local environment feature map is used as a positive sample, and other feature maps in the batch are randomly selected as negative samples to construct a triple sample set. The InfoNCE loss function is calculated for the triple sample set. InfoNCE is a contrastive learning loss that learns a more discriminative feature representation by minimizing the distance between the anchor sample and the positive sample and maximizing the distance between the anchor sample and the negative sample. In this way, the ability to recognize similar environmental features is enhanced, and the discrimination between different environments is improved. Based on the feature similarity score, the weights of the global environment feature map and the key area weight map are adjusted to obtain more optimized environmental feature data. By providing feedback on feature similarity scores, the degree of attention paid to different features is continuously adjusted, so that the environmental feature data not only contains global information, but also can more effectively highlight the information of key areas.

[0015] Step S2: constructing an initial navigation path according to the environmental feature data and historical navigation data; Specifically, the current driving environment is spatially gridded based on the environmental feature data, and the continuous spatial environment is discretized into a series of fixed-size grid cells, so that the complex environment is converted into a discrete representation that is easy to analyze, forming a two-dimensional matrix containing the environmental state, that is, the two-dimensional environmental state matrix. Each grid cell in the two-dimensional environmental state matrix represents the state information of the corresponding position in the environment, such as the presence or absence of obstacles or the difficulty of passing. The two-dimensional environmental state matrix is ​​timestamped with the historical navigation data, and the current environment is combined with the previous navigation data to construct a spatiotemporal sequence data set containing the environmental state and historical trajectory. The spatiotemporal sequence data set is segmented by sliding windows, and multiple overlapping subsequence samples are extracted from the entire spatiotemporal sequence data set, so that the subsequent analysis can better capture the potential time correlation and change characteristics in the environment and navigation data. For each subsequence sample, a wavelet transform is performed to extract the characteristics of the subsequence in the time and frequency domain. Wavelet transform is a signal processing method that can effectively reveal the change pattern of data at different time scales. In order to reduce the computational complexity and remove noise, only the wavelet coefficients with significant significance are retained by threshold screening after the wavelet transform to obtain the feature vector after dimensionality reduction. According to the reduced dimensionality feature vector, the latent space representation and reconstruction error are generated. The latent space representation is a higher-level abstraction of the reduced dimensionality feature vector, while the reconstruction error is used to describe the reconstruction quality of these features. On this basis, the kernel density estimation algorithm is used to construct the uncertainty parameter distribution of the current driving environment. Kernel density estimation is a non-parametric statistical method that effectively estimates the probability density of sample data and obtains the probability distribution of uncertainty factors in the driving environment. The uncertainty parameter distribution describes the various uncertain factors and their possibilities in the current environment, such as the possible location of obstacles and the speed of dynamic changes. Based on the uncertainty parameter distribution, the Latin hypercube sampling method is used to generate multiple groups of representative uncertainty parameter samples. Latin hypercube sampling is an effective sampling technique that can generate uniformly distributed samples in multidimensional parameter space, thereby ensuring that different uncertainty factors can be fully explored. For the generated multiple groups of uncertainty parameter samples, they are respectively input into the path planner based on the rapidly expanding random tree. The rapidly expanding random tree is a path planning algorithm that gradually expands the path by random sampling, and can quickly find a pass path in a complex environment. For each set of uncertainty parameter samples, the rapid expansion random tree planner generates a candidate path, thereby obtaining multiple candidate paths, each of which corresponds to a possible environmental state. In order to select the best initial navigation path from the candidate paths, all candidate paths are evaluated for multiple objectives. The evaluation objectives include path length, travel time, energy consumption, safety and other factors. In order to comprehensively consider the advantages and disadvantages of different objectives, the comprehensive score of each candidate path is calculated by weighted summation.According to the comprehensive score, all candidate paths are sorted in descending order, and the candidate path with the highest score is selected as the initial navigation path.

[0016] Step S3: Based on the initial navigation path, target detection and feature re-identification are performed on the real-time environment image to obtain a real-time environment perception result; Specifically, based on the initial navigation path, the real-time environment image is collected and input into the TensorRT-accelerated YOLOv5 network for processing. The YOLOv5 network is an efficient target detection model, which includes three main parts: the backbone network, the neck network, and the detection head network. The backbone network is used to extract the basic features of the environment image. It adopts the CSPDarknet53 structure. CSPDarknet53 contains 5 CSP modules. Each module consists of multiple residual units and 1×1 convolutional layers. It can efficiently extract low-level and high-level features of the image and generate the first multi-scale feature map. Through these feature maps, different objects and feature information in the environment are extracted. The first multi-scale feature map is input into the neck network of YOLOv5. The neck network adopts a feature pyramid structure, including 3 bottom-up feature fusion layers and 3 top-down feature fusion layers. The function of these fusion layers is to effectively fuse features of different scales to generate the second multi-scale feature map. By fusing multi-scale features, we can capture target features of different sizes and scales in the environment, so that target detection can adapt to complex environmental conditions and ensure that the robot can detect various important objects and obstacles during navigation. The second multi-scale feature map is processed using the detection head network of YOLOv5. The detection head network contains three parallel detection branches, each of which includes two convolutional layers and one output layer to output the target detection results. The detection results include the bounding box coordinates, confidence, and category probability of the target object. The detection results are processed by non-maximum suppression to remove redundant detection frames and retain only the most confident target detection frames to obtain the filtered target detection results. The image area corresponding to the filtered target detection frame is cropped and resized, and the cropped image area contains the detected target object. In order to identify and distinguish the target object, the cropped image area is input into the OSNet network accelerated by TensorRT for feature re-identification. The OSNet network consists of a lightweight backbone network and a feature aggregation module. The lightweight backbone network adopts a deep separable convolution structure and contains 4 stages, each of which consists of multiple lightweight residual units. The third multi-scale feature map is obtained by processing the lightweight backbone network. After the feature extraction of the lightweight backbone network is completed, these features are processed by the feature aggregation module. The feature aggregation module adopts an adaptive feature fusion strategy, including a channel attention mechanism and a spatial attention mechanism. The channel attention mechanism is used to highlight the feature channels that are important for classification and re-identification, while the spatial attention mechanism is used to enhance the significant features at a specific spatial position. Through the adaptive feature fusion strategy, a fused global feature vector is generated. The fused global feature vector is L2 normalized to obtain the target re-identification feature. The target detection result after non-maximum suppression is associated and matched with the normalized target re-identification feature to obtain the real-time environment perception result.

[0017] Step S4: Perform extended Kalman filter fusion processing on multi-sensor data according to the real-time environment perception results to obtain the robot-environment interaction state characteristics; Specifically, the object detection boxes and re-identification features in the real-time environmental perception results are aligned in time and space to obtain feature-enhanced environmental perception data. The feature-enhanced environmental perception data is synchronized with the original sensor data from the inertial measurement unit, encoder, lidar, and depth camera to obtain a multi-source heterogeneous sensor data set, which contains visual information of the robot's interaction with the environment, as well as other important sensor measurement data such as posture and distance. The noise characteristics of various sensor data in the multi-source heterogeneous sensor data set are analyzed. For example, the angular velocity data measured by the IMU is usually affected by drift noise, while the accuracy of the lidar distance measurement is affected by the ambient light and surface reflection characteristics. The noise of each sensor data is modeled, and the sensor measurement noise covariance matrix and process noise covariance matrix are established to describe the uncertainty of the sensor measurement value and to weigh the trust level of different sensors in the Kalman filtering process. The nonlinear state transfer equation and observation equation are constructed. The state transfer equation is used to describe the dynamic change of the robot state, while the observation equation is used to describe the relationship between the measurement value and state of each sensor. In this method, the state vector contains information such as the robot's position, posture, velocity, and angular velocity, and the observation vector includes the actual measurement value of each sensor. In the framework of extended Kalman filtering, since the state transfer equation and the observation equation are nonlinear, these equations are linearized. The nonlinear state transfer equation is expanded by a first-order Taylor to obtain the Jacobian matrix of the state prediction equation; similarly, the nonlinear observation equation is expanded by a first-order Taylor to obtain the Jacobian matrix of the observation prediction equation. The Jacobian matrix is ​​used to describe the linear characteristics of the system near the current state, so as to apply the theoretical framework of Kalman filtering in a nonlinear environment. Based on the state estimate at the previous moment and the current control input, the Jacobian matrix of the state prediction equation is used to predict the state of the robot, and the prior state estimate and the prior error covariance matrix are obtained. The prior state estimate is a prediction of the current state of the robot, and the prior error covariance matrix describes the credibility of this prediction. The prior state estimate is substituted into the observation prediction equation to calculate the observation prediction value. The observation prediction value is subtracted from the actual observation value to obtain the observation residual. The observation residual reflects the difference between the actual observation value and the prediction value. This difference is usually caused by the incomplete accuracy of the system model and sensor noise, so it is used to further correct the state estimate. In order to improve the accuracy of state estimation, the Kalman gain is calculated based on the prior error covariance matrix, the Jacobian matrix of the observation prediction equation, and the observation noise covariance matrix. The calculation of the Kalman gain is one of the core steps of the extended Kalman filter, which is used to weigh the trust between the prior prediction and the actual observation. If the observation noise is small, the Kalman gain will rely more on the observation value; if the predicted prior error is small, the Kalman gain will trust the prior prediction more.By using the Kalman gain, the prior state estimate is corrected to obtain the robot-environment interaction state characteristics.

[0018] Step S5, analyzing the straight segment and turning segment trajectories according to the robot-environment interaction state characteristics, and constructing a smooth navigation trajectory; Specifically, the robot-environment interaction state characteristics are analyzed, and the robot's current position, posture, velocity and acceleration are extracted. This information is used as the constraint conditions for constructing the starting point of the trajectory. At the same time, the constraint conditions of the end point of the trajectory are determined based on the initial navigation path, and a complete set of trajectory boundary conditions is obtained. The trajectory boundary condition set contains the state information of the starting point and end point of the trajectory, including position, velocity and acceleration. These constraints are used to ensure that the trajectory can be smoothly connected and meet the requirements of navigation. Based on the trajectory boundary condition set, the initial navigation path is segmented and divided into straight segments and turning segments to obtain a path segment set. The path segment set is used to describe the types of each section that the robot needs to pass through during its movement. Different types of path segments correspond to different trajectory construction methods. For the straight segments in the path segment set, the fifth-order polynomial interpolation method is used to construct the trajectory equation. The fifth-order polynomial interpolation method can meet the constraints of position, velocity and acceleration while ensuring the smoothness of the trajectory. In order to simplify the calculation, the time is normalized to the [0, 1] interval, and the position, velocity and acceleration information of the starting point and end point of the trajectory are used to solve the coefficients of the polynomial to obtain the trajectory equation of the straight segment. For the turning segments in the path segment set, the third-order Bezier curve is used to construct the trajectory equation. The characteristic of the third-order Bezier curve is that the shape of the curve can be flexibly adjusted by setting control points, which is suitable for describing the turning trajectory. When constructing the turning segment trajectory, the four control points are set as the starting point of the turning segment, the point in the tangent direction of the starting point, the point in the tangent direction of the end point, and the end point. By adjusting the positions of the two middle control points, the curvature of the turning segment is flexibly controlled to ensure that the robot can pass smoothly without a sharp change of direction during the turning process. The obtained turning segment trajectory equation can better adapt to the dynamic characteristics of the robot and meet the requirements for smoothness. The trajectory equations of the straight segment and the turning segment are combined to construct a complete trajectory equation. In order to make the trajectory adapt to the motion state of the robot at different time points, the complete trajectory equation is time-parameterized to obtain a parameterized trajectory equation. Time parameterization enables the trajectory to be adjusted according to the actual speed of the robot, ensuring that the trajectory planning is not only reasonable in space, but also meets the time constraints of the robot. Differentiation operations are performed on the parameterized trajectory equation to obtain the velocity, acceleration, and acceleration expression of the trajectory. Through these expressions, the dynamic characteristics of the trajectory are analyzed to ensure that the changes in velocity and acceleration of the trajectory conform to the physical characteristics of the robot. Combined with the dynamic constraints of the robot, the feasibility of the trajectory is verified to check whether the trajectory conforms to the dynamic performance of the robot. If the trajectory does not meet the constraints in some aspects, such as too high velocity or too large acceleration, the trajectory parameters are adjusted and the trajectory is regenerated to ensure that the final generated trajectory is feasible and safe. After generating a parameterized trajectory equation that meets the constraints, a discrete trajectory point sequence is generated from the parameterized trajectory equation using equal time interval sampling.Each trajectory point contains the robot's position, posture, velocity, and acceleration information, which is used to control the robot's state at each point in time. After generating the trajectory point sequence, the trajectory points are checked for smoothness and continuity to ensure that the position, velocity, and acceleration changes between adjacent trajectory points meet the smoothness requirements. If the changes between adjacent trajectory points are too drastic, it will cause unstable movement during the robot's driving process. These points are adjusted to ensure the smoothness and continuity of the trajectory and obtain a smooth navigation trajectory.

[0019] Step S6: During the navigation process, the environmental feature data and the working status data of the walking robot are continuously updated to construct a dynamic environmental map; when a return signal is received, an automatic return path is quickly planned based on the dynamic environmental map.

[0020] Specifically, during the robot navigation process, the image data of the motion environment is collected in real time, and the image data is subjected to multi-scale convolution processing and feature fusion to obtain a motion multi-scale feature map. The motion multi-scale feature map is input into the global-local contrast learning module for processing, and more refined and differentiated features are extracted through global and local contrast learning to obtain the latest environmental feature data. The environmental feature data is continuously updated and can dynamically reflect the changes in the robot's environment. The latest environmental feature data is temporally and spatially aligned to construct a short-term environmental feature sequence matrix. The temporal and spatial alignment process arranges the environmental feature data in chronological order to form a short-term feature sequence to capture the temporal characteristics of environmental changes. By performing temporal and spatial correlation feature analysis on the short-term environmental feature sequence matrix, a dynamic environmental feature representation vector is extracted to reflect the dynamic changes and various important spatiotemporal information in the current environment. At the same time, the robot is estimated in real time based on the odometer data and IMU data of the walking robot to obtain the robot's position information. The pose estimation combines the data of the odometer and inertial measurement unit to provide accurate position information for the robot. The dynamic environmental feature representation vector is associated with the robot's position information to obtain a local dynamic environment map. The local dynamic environment map is a detailed description of the robot's current environment, including the dynamic characteristics of the robot's current position and its surrounding environment. The local dynamic environment map is spatially registered and merged with the global environment map to update the global dynamic environment map. The global dynamic environment map is a dynamic description of the entire navigation area. As the local dynamic environment map is continuously updated, the global map can dynamically reflect the changes in the environment. Data association and tracking are performed on dynamic obstacles in the real-time motion environment to obtain the movement trajectory set of the obstacles. When the system receives the return signal, the current position coordinates of the replacement robot are extracted as the starting point of the return path, and the coordinates of the navigation start position are taken as the end point of the return path to construct the return mission target point pair. The return mission target point pair and the global dynamic environment map are input into the A algorithm for path search to obtain the initial return path composed of grid coordinates. The A algorithm is a heuristic path search algorithm that can find a feasible path from the starting point to the end point on a known map to ensure the optimality and feasibility of the path. In order to make the initial return path smoother, a cubic B-spline curve is fitted, and the control point interval is set to 0.5 meters to generate a smooth path curve. The application of cubic B-spline curves enables the path to maintain smooth curvature changes between various control points, avoiding sharp turns or sudden changes in direction in the path. The system uniformly samples every 0.1 meters along the fitted path to obtain a path point sequence containing multiple path points, each of which contains information such as position, posture, speed, and acceleration. At the same time, the path point sequence is combined with the dynamic environment feature representation vector and the obstacle motion trajectory set to construct a return scene description tensor that fully describes the return scene.The return scene description tensor contains all the environment and path information required for the robot's current return mission. The return scene description tensor is input into the adaptive uncertainty path planning model, which adopts a variational inference network structure and consists of an encoder and a decoder. The encoder is used to probabilistically encode the return scene, generate the latent variable distribution of the scene, and sample multiple sets of scene samples from it. In this way, different environmental uncertainties are modeled to generate a variety of possible scenarios. Multiple sets of scene samples are input into the decoder, and the decoder generates multiple candidate return paths. For the generated candidate return paths, an optimization solution is performed to evaluate the feasibility and advantages and disadvantages of each path in different environments, and finally an optimal automatic return path is obtained.

[0021] In the embodiment of the present invention, the multi-scale environmental feature extraction technology is combined with global-local contrast learning to capture environmental information more comprehensively and accurately, thereby improving the accuracy and robustness of environmental perception. The adaptive uncertainty path planning model effectively copes with various uncertain factors in complex dynamic environments by dynamically constructing an uncertain parameter set, thereby improving the adaptability and reliability of path planning. The TensorRT-accelerated YOLOv5 target detection and OSNet feature re-identification technology significantly improve the efficiency and accuracy of real-time environmental perception, and the extended Kalman filter fusion processes multi-sensor data, effectively improving the accuracy of robot-environment interaction state estimation. The differential flat trajectory generator combines polynomial interpolation and Bezier curves to generate smooth navigation trajectories that meet dynamic constraints, thereby improving the comfort and safety of navigation. The continuous update of the dynamic environment map and the fast return path planning technology enable the robot to respond to environmental changes in a timely manner and achieve safe and efficient automatic return.

[0022] In a specific embodiment, the process of executing step S1 may specifically include the following steps: Perform Gaussian pyramid decomposition on the environmental image data to obtain a multi-scale image set, and perform multi-layer convolution feature extraction on the multi-scale image set to obtain a multi-scale feature atlas containing shallow, middle and deep features; The multi-scale feature atlas is input into the feature pyramid network for upsampling and fusion of features of different scales to obtain a global environment feature map. The global environment feature map is then input into the spatial attention module to obtain a key area weight map by calculating the spatial correlation of the feature map. The global environment feature map is weighted based on the key area weight map to obtain the local environment feature map, and the global environment feature map is used as the anchor sample, the local environment feature map is used as the positive sample, and other samples in the batch are randomly selected as negative samples to construct a triplet sample set; Calculate the InfoNCE loss for the triple sample set, and obtain the feature similarity score by minimizing the distance between the anchor sample and the positive sample and maximizing the distance between the anchor sample and the negative sample; Based on the feature similarity score, the global environment feature map and the key area weight map are weighted to obtain the environment feature data.

[0023] Specifically, the environmental image data is processed at multiple scales to capture environmental features at different scales. Gaussian pyramid decomposition is a multi-scale image processing technique that recursively smooths and downsamples the original image to obtain a series of images with gradually decreasing resolutions. Each layer of images represents a different scale, and these images form a multi-scale image set. The process of Gaussian pyramid is expressed as: ; in, The pyramid layer image, represents Gaussian smoothing and downsampling operations, and Represents the original image. In this way, a multi-scale environment representation is obtained. Multi-layer convolution feature extraction is performed on the multi-scale image set, and the convolution layer in the convolutional neural network is used to extract features from each layer of the image. Through convolution layers of different depths, shallow features, middle features, and deep features are extracted respectively to form a multi-scale feature atlas. Shallow features correspond to edge and texture information in the image, while deep features can capture more abstract semantic information. Assume Indicates The layer image is in The output features of the convolutional layers are then used to obtain the feature representation of each layer: ; in, represents the activation function (such as the ReLU function), Indicates convolution kernels, * indicates convolution operation, Represents the bias term. Through this step, feature maps of different levels are generated and integrated into a multi-scale feature atlas, which reflects the local details and global structure of the image at the same time. The multi-scale feature atlas is input into the feature pyramid network for upsampling and fusion of features of different scales. The feature pyramid network is a structure used for target detection tasks. Through upsampling operations, low-resolution, deep feature maps are fused with high-resolution, shallow feature maps to generate a global environment feature map containing rich contextual information. The upsampling process increases the low-resolution feature map to a higher resolution through interpolation, and at the same time, features of different scales are added and fused through jump connections to obtain a global environment feature map. The global environment feature map is input into the spatial attention module to identify the most critical areas in the environment by calculating the spatial correlation of the feature map. The spatial attention mechanism strengthens the salient areas in the feature map by learning weights, highlighting areas that are important for navigation tasks. The calculation of the spatial attention module is expressed as: ; in, and They represent the query matrix and key matrix obtained by convolution transformation respectively. Softmax is a standardized operation used to map the relevance of features to the range of 0 to 1. Represents the obtained key area weight map. Through the weight map, the global environment feature map is weighted to obtain the local environment feature map. The global environment feature map is used as the anchor sample, the local environment feature map is used as the positive sample, and other samples are randomly selected from the current batch as negative samples to construct a triple sample set. This sample construction method can effectively enhance the discriminative ability of the model and ensure that the model can correctly distinguish between key areas and non-key areas in the environment. Calculate the InfoNCE loss for the triple sample set. InfoNCE loss is a contrastive learning loss function, whose goal is to minimize the distance between the anchor sample and the positive sample, and maximize the distance between the anchor sample and the negative sample, thereby improving the distinguishability of the feature. The loss function is expressed as: ; in, Represents the similarity calculation between features (such as cosine similarity), , , Represent the anchor sample, positive sample and The feature representation of negative samples is is a temperature coefficient used to control the difficulty of contrastive learning. By minimizing the loss function, the features of positive samples and anchor samples are closer, while the features of negative samples and anchor samples are farther away, thereby improving the reliability of feature representation. Based on the feature similarity score, the weights of the global environment feature map and the key area weight map are adjusted to optimize the environment feature data and obtain the environment feature data.

[0024] In a specific embodiment, the process of executing step S2 may specifically include the following steps: Based on the environmental feature data, the current driving environment is spatially gridded, the continuous space is discretized into fixed-size grid cells, and a two-dimensional environmental state matrix is ​​obtained. The two-dimensional environmental state matrix is ​​time-stamp aligned with the historical navigation data to construct a spatiotemporal series data set containing environmental states and historical trajectories. Sliding window segmentation is performed on the spatiotemporal sequence data set to extract multiple overlapping subsequence samples. Wavelet transform is performed on each subsequence sample to extract time-frequency domain features, and significant coefficients are retained through threshold screening to obtain a reduced-dimensional feature vector. Generate a latent space representation and a reconstruction error according to the reduced dimension feature vector, and construct the uncertainty parameter distribution of the current driving environment using a kernel density estimation algorithm based on the latent space representation and the reconstruction error; Latin hypercube sampling is performed based on the uncertainty parameter distribution to generate multiple groups of representative uncertainty parameter samples, and the multiple groups of uncertainty parameter samples are respectively input into the path planner based on the rapidly expanding random tree to generate multiple candidate paths; A multi-objective evaluation and weighted summation are performed on multiple candidate paths to calculate the comprehensive score of each candidate path. Based on the comprehensive score, the multiple candidate paths are sorted in descending order, and the candidate path with the highest comprehensive score is selected as the initial navigation path.

[0025] Specifically, the current driving environment is spatially gridded based on the environmental feature data. By dividing the continuous space into grid cells of fixed size, the entire environment is discretized into a two-dimensional grid matrix. Assume that the size of the grid cell is , by spatially discretizing the environmental features, we obtain a two-dimensional environmental state matrix , where each element Represented in the grid The state information on the robot, such as whether there are obstacles and the difficulty of passing, is obtained. The two-dimensional environment state matrix is ​​timestamped with the historical navigation data, the current environment state is associated with the past navigation trajectory, and a spatiotemporal sequence data set containing the environment state and historical trajectory is constructed. Through the alignment operation, the dynamic changes of the environment and the position information of the robot at different time points are captured, forming a more comprehensive description of the environment. represents a time series, where each time point There is a corresponding environment state matrix The spatial-temporal sequence data set is composed of the position trajectory information of the robot and the position trajectory information of the robot. The spatial-temporal sequence data set is segmented by sliding windows to extract multiple overlapping subsequence samples. Sliding window segmentation can capture the short-term change characteristics of the environment. Each window contains the state information within a continuous period of time to form a subsequence sample. Assume that the sliding window size is , the sliding step length is , multiple overlapping subsequence samples are obtained by segmentation , reflecting the characteristics of the environment and trajectory in different time periods. For each subsequence sample, wavelet transform is performed to extract its characteristics in the time-frequency domain. Wavelet transform is a time-frequency analysis tool that decomposes the signal into different frequency components and reveals the characteristics of the signal at different time scales. Let the subsequence sample be , the wavelet coefficient matrix is ​​obtained through wavelet transform ,in represents the scale parameter, represents the time shift parameter. The formula of wavelet transform is expressed as: ; in, is the mother wavelet function, and * represents complex conjugate. The wavelet coefficient matrix obtained by wavelet transform contains the characteristic information of the subsequence at different frequencies and time points. In order to simplify the calculation and remove noise, the significant wavelet coefficients are retained by threshold screening to obtain the reduced dimension feature vector , the feature vector can effectively describe the changing characteristics of the environment. According to the reduced dimension feature vector, the latent space representation and reconstruction error are generated. The latent space representation is a further compression and abstraction of the original features, which can reveal the potential structure of the feature data. At the same time, the reconstruction error is used to measure the difference between the original features and the reconstructed features. Based on the latent space representation and reconstruction error, the kernel density estimation algorithm is used to construct the uncertainty parameter distribution of the current driving environment. Kernel density estimation is a non-parametric statistical method used to estimate the probability distribution of data. Let the set of reduced dimension feature vectors be , the kernel density estimate is expressed as: ; in, is a kernel function (e.g. a Gaussian kernel), is the bandwidth parameter. Through kernel density estimation, the probability distribution of the uncertainty parameters in the environment is obtained. Latin hypercube sampling is performed based on the uncertainty parameter distribution to generate multiple groups of representative uncertainty parameter samples. Latin hypercube sampling is an effective sampling method that generates samples uniformly in a high-dimensional parameter space to ensure full exploration of environmental uncertainty. For the generated uncertainty parameter samples, each group of samples is input into a path planner based on a rapidly expanding random tree (RRT) to generate multiple candidate paths. RRT is a random sampling path planning algorithm that can quickly find a feasible path from the starting point to the target point in a complex environment. For the multiple candidate paths generated, a multi-objective evaluation is performed, and the comprehensive score of each candidate path is calculated by a weighted summation method. The objectives of the multi-objective evaluation include path length, driving time, energy consumption, safety, etc., and each objective has a different weight. Assume that the candidate path is , its comprehensive score It is expressed as: ; in, Indicates The weight of the target, Represents candidate paths In the Through the weighted summation method, a comprehensive evaluation is performed on different candidate paths, and multiple candidate paths are sorted in descending order according to the comprehensive scores, and the candidate path with the highest comprehensive score is selected as the initial navigation path.

[0026] In a specific embodiment, the process of executing step S3 may specifically include the following steps: Based on the initial navigation path, real-time environment images are collected and input into the YOLOv5 network accelerated by TensorRT, where the YOLOv5 network includes a backbone network, a neck network, and a detection head network; The backbone network adopts the CSPDarknet53 structure, which contains 5 CSP modules. Each CSP module contains multiple residual units and 1×1 convolutional layers. The first multi-scale feature map is obtained through the backbone network. The neck network adopts a feature pyramid structure, which includes 3 bottom-up feature fusion layers and 3 top-down feature fusion layers. The second multi-scale feature map is obtained through the neck network. The detection head network contains three parallel detection branches, each of which contains two convolutional layers and one output layer. The target detection results, including bounding box coordinates, confidence and category probability, are obtained through the detection head network. Perform non-maximum suppression on the target detection results to obtain the filtered target detection frame, crop and resize the image area corresponding to the filtered target detection frame, and input it into the OSNet network accelerated by TensorRT, where the OSNet network includes a lightweight backbone network and a feature aggregation module; The lightweight backbone network adopts a deep separable convolutional structure, which consists of 4 stages. Each stage contains multiple lightweight residual units. The third multi-scale feature map is obtained through the lightweight backbone network. The feature aggregation module adopts an adaptive feature fusion strategy, including channel attention mechanism and spatial attention mechanism, and obtains the fused global feature vector through the feature aggregation module; The fused global feature vector is L2 normalized to obtain the target re-identification feature, and the target detection result and the target re-identification feature are associated and matched to obtain the real-time environment perception result.

[0027] Specifically, ensure that the robot or autonomous vehicle can continuously collect real-time image data of the surrounding environment during driving. These image data are captured by the camera and used as input for subsequent target detection and recognition. The YOLOv5 network accelerated by TensorRT improves the speed and efficiency of reasoning. The YOLOv5 network has a three-part structure of the backbone network, the neck network, and the detection head network. These modules work together to efficiently complete the task of detecting targets. The backbone network is the basic part of feature extraction of YOLOv5 and adopts the CSPDarknet53 structure. CSPDarknet53 contains 5 CSP modules, each of which consists of multiple residual units and 1 The residual unit effectively solves the gradient vanishing problem in deep neural networks, making the network stable for deeper training. Through the residual unit and convolution operation, the backbone network extracts feature information at different levels and generates the first multi-scale feature map. Assume that the input real-time image is , then it is expressed through the backbone network as: ; in, Represents the convolution operation and residual structure in the backbone network, It is a multi-scale feature map extracted from the input image, including basic edge, texture and other features in the image. The first multi-scale feature map is input into the neck network of YOLOv5. The neck network adopts a feature pyramid structure, including 3 bottom-up feature fusion layers and 3 top-down feature fusion layers. These feature fusion layers combine feature maps of different scales through up and down sampling operations to achieve cross-level information integration and obtain the second multi-scale feature map. . The second multi-scale feature map The input is sent to the detection head network of YOLOv5. The detection head network contains 3 parallel detection branches, each of which contains 2 convolutional layers and 1 output layer. Each branch is responsible for detecting objects at different scales, thereby detecting objects of different sizes at the same time. The detection head network finally outputs the target detection result, including the coordinates of the bounding box, confidence, and category probability. Suppose the target detection result is ,in Indicates The bounding box coordinates of the objects, Indicates category, Indicates the confidence of the target. In order to remove redundant and inaccurate detection results, non-maximum suppression is performed on the target detection results, the frame with the highest confidence is selected from the overlapping detection frames, and other redundant frames are removed to obtain the filtered target detection frame. . The image area corresponding to the target detection box is cropped and resized for feature re-identification. In order to achieve feature re-identification of the target, the cropped image area is input into the OSNet network accelerated by TensorRT. The OSNet network consists of two parts: a lightweight backbone network and a feature aggregation module. The lightweight backbone network adopts a deep separable convolution structure, which includes 4 stages, and each stage consists of multiple lightweight residual units. Deep separable convolution can effectively reduce the computational complexity of the model while maintaining good feature extraction capabilities. Through the processing of the lightweight backbone network, the third multi-scale feature map is obtained , these feature maps contain the basic appearance features and structural information of the target. The feature aggregation module is used to aggregate the third multi-scale feature map The feature aggregation module adopts an adaptive feature fusion strategy, including channel attention mechanism and spatial attention mechanism. The channel attention mechanism is used to adaptively adjust the importance of each feature channel and highlight the channels that have made significant contributions to target recognition; the spatial attention mechanism is used to emphasize the significant features in the spatial position of the feature map. In this way, the fused global feature vector is generated. , expressed as: ; in, Represents the combination of channel attention and spatial attention operations. The fused global feature vector contains the salient feature information of the target and is used for the target re-identification task. In order to ensure the consistency of feature comparison between different targets, the fused global feature vector is L2 normalized. The normalized feature vector satisfy: ; in, Represents the L2 norm of the feature vector. Through normalization, the length of the feature vector is standardized to ensure that the features extracted under different scales and lighting conditions have the same metric, making the similarity comparison between different targets more fair and effective. The normalized target re-identification features are associated and matched with the target detection results to obtain the perception results of the real-time environment. The perception results contain basic information such as the location and category of the target, as well as the feature description of the target, which is used to identify and distinguish different targets.

[0028] In a specific embodiment, the process of executing step S4 may specifically include the following steps: The object detection boxes and re-identification features in the real-time environment perception results are spatiotemporally aligned to obtain feature-enhanced environment perception data, and the feature-enhanced environment perception data is time-synchronized with the original sensor data from the inertial measurement unit, encoder, lidar, and depth camera to obtain a multi-source heterogeneous sensor dataset. Analyze the noise characteristics of various sensor data in multi-source heterogeneous sensor data sets, and establish the sensor measurement noise covariance matrix and process noise covariance matrix; Construct nonlinear state transfer equations and observation equations, where the state vector includes the robot's position, posture, velocity and angular velocity, and the observation vector includes the measurement values ​​of each sensor; Perform a first-order Taylor expansion on the nonlinear state transfer equation to obtain the Jacobian matrix of the state prediction equation, and perform a first-order Taylor expansion on the nonlinear observation equation to obtain the Jacobian matrix of the observation prediction equation; Based on the state estimation value and control input at the previous moment, the Jacobian matrix of the state prediction equation is used to perform state prediction to obtain the prior state estimation value and the prior error covariance matrix; Substitute the prior state estimate into the observation prediction equation, calculate the observation prediction value, and subtract it from the actual observation value to obtain the observation residual; The Kalman gain is calculated according to the prior error covariance matrix, the Jacobian matrix of the observation prediction equation and the observation noise covariance matrix, and the Kalman gain is used to correct the prior state estimate to obtain the robot-environment interaction state characteristics.

[0029] Specifically, the visual information of environmental perception is sorted and aligned. The real-time environmental perception results include the position of the target detection box and the feature information obtained by re-identification, which are from different time points. These data are aligned in time and space to ensure that all information can jointly describe the environmental state at the same time point. The time and space alignment unifies the data collected at different times to the same moment through interpolation and matching to obtain feature-enhanced environmental perception data. The feature-enhanced environmental perception data is synchronized with the original sensor data from other sensors (such as inertial measurement unit (IMU), encoder, lidar and depth camera). The time synchronization process is achieved through timestamp matching, which unifies all data on the same time axis to obtain a multi-source heterogeneous sensor data set. The noise characteristics of various sensor data in the multi-source heterogeneous sensor data set are analyzed to establish the sensor measurement noise covariance matrix and process noise covariance matrix. Different types of sensors are affected by noise from different sources during the measurement process. For example, the acceleration data of the IMU is affected by drift noise, while the lidar is affected by ambient light and the reflection characteristics of the object surface when measuring distance. In order to accurately describe the statistical characteristics of these noises, the noise of each sensor is modeled and the measurement noise covariance matrix is ​​obtained. and the process noise covariance matrix Each element of the covariance matrix describes the degree of mutual dependence between the measurements, which can be used to weigh the trust of different sensors in the filtering process. Construct nonlinear state transfer equations and observation equations to describe the change of the robot state over time and the measurement process of each sensor. State vector Contains information such as the robot's position, posture, speed, and angular velocity, and the observation vector Including the actual measurement values ​​of each sensor. The state transfer equation is expressed as: ; in, Indicates at time status, represents the state transfer function, which describes the control input Under the action, time Status How to transfer to Moment . represents process noise, describing the random disturbance in state transition. The observation equation is expressed as: ; in, Indicates at time The observed value of Represents the observation function, describing the state How to measure with various sensors, Represents the measurement noise. Since both the state transfer equation and the observation equation are nonlinear, these equations are linearized so that the extended Kalman filter can be used for state estimation. The nonlinear state transfer equation is expanded by first order Taylor to obtain the Jacobian matrix of the state prediction equation , and perform a first-order Taylor expansion on the nonlinear observation equation to obtain the Jacobian matrix of the observation prediction equation The Jacobian matrix is ​​used to describe the linear characteristics of the system near the current state, thus applying the theoretical framework of Kalman filtering in a nonlinear environment. Based on the state estimate at the previous moment and control input , using the Jacobian matrix of the state prediction equation Predict the robot's state and obtain a priori state estimate and the prior error covariance matrix ; ; in, represents the error covariance matrix of the previous moment, Represents the process noise covariance matrix. Substitute the prior state estimate into the observation prediction equation and calculate the observation prediction value , and the actual observed value Subtract and get the observed residual ; The observation residual reflects the difference between the actual observation value and the predicted value. This difference is caused by the incomplete accuracy of the system model and sensor noise, and is used to further correct the state estimation. In order to improve the accuracy of state estimation, based on the prior error covariance matrix , the Jacobian matrix of the observation prediction equation and the observation noise covariance matrix , calculate the Kalman gain ; The role of Kalman gain is to weigh the trust between the prior prediction and the actual observation. If the observation noise is small, the Kalman gain will rely more on the observation value; if the predicted prior error is small, the Kalman gain will trust the prior prediction more. The Kalman gain is used to correct the prior state estimate to obtain the final robot-environment interaction state characteristics: ; In a specific embodiment, the process of executing step S5 may specifically include the following steps: The robot-environment interaction state characteristics are analyzed, and the robot's current position, posture, speed and acceleration are extracted as the trajectory starting point constraint conditions. The trajectory end point constraint conditions are determined according to the initial navigation path to obtain the trajectory boundary condition set. Based on the trajectory boundary condition set, the initial navigation path is segmented into straight segments and turning segments to obtain a path segment set; For the straight line segments in the path segment set, the trajectory equation is constructed using the quintic polynomial interpolation method, the time is normalized to the interval [0, 1], and the position, velocity, and acceleration constraints of the trajectory start and end points are used to solve the polynomial coefficients to obtain the straight line segment trajectory equation; For the turning segments in the path segment set, the third-order Bezier curve is used to construct the trajectory equation, in which the four control points are set as the starting point of the turning segment, the point in the tangent direction of the starting point, the point in the tangent direction of the end point, and the end point, respectively. The curvature is controlled by adjusting the positions of the two middle control points to obtain the trajectory equation of the turning segment; The straight segment trajectory equation and the turning segment trajectory equation are combined to construct a complete trajectory equation, and the complete trajectory equation is time-parameterized to obtain a parameterized trajectory equation; Differentiate the parameterized trajectory equation to obtain the expressions of velocity, acceleration and acceleration, and verify the feasibility of the trajectory in combination with the robot dynamics constraints. If the constraints are not met, adjust the trajectory parameters and regenerate it; Based on the parameterized trajectory equation, an equal time interval sampling method is adopted to generate a discrete trajectory point sequence. Each trajectory point contains position, attitude, velocity and acceleration information. The smoothness and continuity of the trajectory point sequence are checked to ensure that the position, velocity and acceleration changes between adjacent trajectory points meet the smoothness requirements and obtain a smooth navigation trajectory.

[0030] Specifically, the robot's current environment interaction state characteristics are analyzed to extract the robot's current position ,attitude ,speed and acceleration As the constraint condition of the starting point of the trajectory. At the same time, according to the terminal position, posture, speed and other information of the initial navigation path, the terminal constraint conditions of the trajectory are determined, including the terminal position , Ending posture Terminal velocity and acceleration . Through these constraints, a trajectory boundary condition set is defined for subsequent trajectory planning. The initial navigation path is segmented based on the trajectory boundary condition set. The initial navigation path consists of multiple continuous points, and the path is divided into straight segments and turning segments to obtain a path segment set. Different trajectory construction methods are used for different types of road segments to improve the accuracy and adaptability of trajectory planning. For the straight segments in the path segment set, the trajectory equation is constructed using the quintic polynomial interpolation method. The quintic polynomial interpolation has sufficient degrees of freedom to ensure the smoothness of the trajectory while satisfying the boundary constraints of position, velocity and acceleration. Assuming the time Normalized to the interval [0, 1], the quintic polynomial trajectory equation is expressed as: ; ; in, and are the polynomial coefficients to be determined. By using the position, velocity, acceleration and other constraints of the starting and ending points of the trajectory, these coefficients are solved to obtain a fifth-order polynomial equation for describing the trajectory of the straight line segment. For the turning segment in the path segment set, the third-order Bezier curve is used to construct the trajectory equation. The characteristic of the Bezier curve is that the shape of the curve is adjusted by control points, which is suitable for describing the trajectory of the robot during the turning process. The third-order Bezier curve trajectory of the turning segment is expressed as: ; in, There are four control points, is the time parameter. Control point and They are the starting point and end point of the turning segment, and the control point and Located at the tangent direction of the starting point and the end point respectively, by adjusting and The position of the robot is controlled to control the curvature of the curve, so that the trajectory of the turning section is smoother and meets the dynamic requirements. The trajectory equations of the straight segment and the turning segment are combined to construct a complete trajectory equation. In order to make the trajectory adapt to the motion state of the robot at different time points, the complete trajectory equation is time-parameterized to obtain a parameterized trajectory equation. The purpose of time parameterization is to associate the trajectory with time so that the system can execute the trajectory according to the actual motion time, ensuring the smoothness and dynamic consistency of the trajectory. Differentiate the parameterized trajectory equation to obtain expressions for velocity and acceleration. For the trajectory equation and , velocity and acceleration are expressed as: ; ; Through these expressions, the velocity and acceleration of the trajectory at each time point are analyzed. Combined with the dynamic constraints of the robot, such as maximum velocity, maximum acceleration, and maximum steering angle, the feasibility of the trajectory is verified. If the trajectory does not meet these dynamic constraints, for example, the acceleration of a certain section exceeds the maximum acceleration that the robot can withstand, the parameters of the trajectory are adjusted and the trajectory is regenerated until the trajectory meets all constraints. After generating a parameterized trajectory equation that meets the constraints, a discrete sequence of trajectory points is generated from the parameterized trajectory equation using equal time interval sampling. Each trajectory point contains information such as the robot's position, posture, velocity, and acceleration. For the time interval , the trajectory point sequence is expressed as The generated trajectory point sequence is checked for smoothness and continuity to ensure that the changes in position, velocity, and acceleration between adjacent trajectory points are continuous and smooth. The smoothness check avoids unstable situations when the robot executes the trajectory, such as sudden changes in velocity or direction, and obtains a smooth navigation trajectory.

[0031] In a specific embodiment, the process of executing step S6 may specifically include the following steps: The motion environment images collected in real time during the navigation process are subjected to multi-scale convolution processing and feature fusion to obtain a motion multi-scale feature map, which is then input into the global-local contrast learning module for processing to obtain the latest environmental feature data; Perform spatiotemporal registration on the latest environmental feature data, construct a short-term environmental feature sequence matrix, and perform spatiotemporal correlation feature analysis on the short-term environmental feature sequence matrix to obtain a dynamic environmental feature representation vector; Based on the odometer data and IMU data of the walking robot, the robot's real-time posture is estimated to obtain the robot's position information, and the dynamic environment feature representation vector is associated with the robot's position information to obtain a local dynamic environment map; Perform spatial registration on the local dynamic environment map, update the global dynamic environment map, and perform data association and tracking on the dynamic obstacles in the real-time motion environment to obtain the obstacle motion trajectory set; When the return signal is received, the current position coordinates of the replacement robot are taken as the starting point, the navigation start position coordinates are taken as the end point, the return mission target point pair is constructed, and the return mission target point pair and the global dynamic environment map are input into the A* algorithm for path search to obtain the initial return path composed of grid coordinates; The initial return path is fitted with a cubic B-spline curve, with the control point interval set to 0.5 meters to generate a smooth path curve. The path point sequence is uniformly sampled every 0.1 meters along the path. The path point sequence, dynamic environment feature representation vector and obstacle motion trajectory set are combined to construct the return scene description tensor. The return scene description tensor is input into the adaptive uncertainty path planning model. The adaptive uncertainty path planning model adopts a variational inference network structure, which includes an encoder and a decoder. The encoder is used to probabilistically encode the return scene to generate the latent variable distribution of the scene, and multiple groups of scene samples are sampled therefrom. The multiple groups of scene samples are input into the decoder to generate multiple candidate return paths, and the multiple candidate return paths are optimized to obtain an automatic return path.

[0032] Specifically, a vehicle camera or a camera on a walking robot is used to collect real-time image data of the surrounding environment. The real-time image is input into a multi-scale convolutional network for processing. The purpose of multi-scale convolutional processing is to extract features from different scales so as to capture both detailed information and global semantic information. Assume that the input environment image is , through multi-scale convolutional network processing, we get the motion multi-scale feature map . The motion multi-scale feature map is input into the global-local contrastive learning module for processing, and the discrimination ability of environmental feature data is enhanced through contrastive learning. In the contrastive learning module, the global features and local features are compared, and the features that can effectively distinguish different environments are learned through contrastive loss to obtain the latest environmental feature data. The contrastive learning process is described by the InfoNCE loss function, which aims to minimize the distance between positive samples and maximize the distance between negative samples. Through contrastive learning, more robust environmental features are extracted. The latest environmental feature data are temporally and spatially aligned to construct a short-time environmental feature sequence matrix. The environmental feature data collected at different time points are aligned and sorted to form an environmental feature sequence matrix containing multiple time segments. Assume that the environmental feature data is ,in Represents the number of time steps, and a feature matrix is ​​obtained through spatiotemporal registration ,in The dimension of the feature is represented. The feature matrix is ​​analyzed for time and space correlation features to extract the dynamic change pattern in the environment and obtain the dynamic environment feature representation vector , effectively describing the dynamic changing characteristics of the current environment. At the same time, based on the odometer data and inertial measurement unit (IMU) data of the walking robot, the robot's real-time posture estimation is performed to obtain the robot's current position and posture. The odometer data provides information about the robot's travel distance and direction, while the IMU data provides measurements of the robot's acceleration and angular velocity. By fusing these data, the robot's posture at the current moment is estimated to obtain the robot's position information ,in represents the two-dimensional plane position of the robot, Represents the robot's attitude angle. Associate the dynamic environment feature representation vector with the robot's position information to obtain a local dynamic environment map. Perform spatial registration on the local dynamic environment map and merge it with the global environment map to update the global dynamic environment map. The global dynamic environment map is used to represent the environmental state of the entire navigation area and reflects environmental changes by continuously updating the global map. At the same time, data association and tracking of dynamic obstacles in the real-time motion environment are performed to obtain a set of obstacle motion trajectories. ,in Indicates The trajectory of obstacles is calculated, and these trajectory data are used for obstacle avoidance in subsequent path planning. When the system receives the return signal, the current position coordinates of the replacement robot are extracted as the starting point of the return path, and the coordinates of the navigation start position are used as the end point to construct the return mission target point pair. The return mission target point pair and the global dynamic environment map are input into the A* algorithm for path search. The algorithm is a classic heuristic path search algorithm that can find the optimal path from the starting point to the end point on a known map. Through the A* algorithm, an initial return path consisting of grid coordinates is generated. In order to make the initial return path smoother, a cubic B-spline curve is fitted. The cubic B-spline curve constructs a smooth curve by setting control points, and the control point interval is set to 0.5 meters, so as to ensure that the generated path curve is smooth enough in space and can meet the driving requirements of the robot. The representation of the spline curve is: ; in, represents a spline curve, represents the control point, is the cubic B-spline basis function, are the parameters of the curve. By controlling the positions of these control points, the shape of the spline curve is adjusted so that the path has smooth speed and acceleration characteristics at the start and end points. Along the fitted path, the system uniformly samples every 0.1 meters to obtain a path point sequence containing multiple path points. , where each path point contains position, attitude, velocity and acceleration information. At the same time, the path point sequence and dynamic environment feature representation vector and obstacle trajectory set Combined together, construct the return scene description tensor , which contains all the environmental and dynamic information that needs to be considered during the return process. The return scene description tensor is input into the adaptive uncertainty path planning model. The model adopts a variational inference network structure, which includes an encoder and a decoder. The encoder is used to probabilistically encode the return scene and generate the hidden variable distribution of the scene. Suppose the return scene is described as , through the encoder Get the latent variable distribution: ; in, represents a hidden variable, and Represent the mean and covariance of the latent variables respectively, and the encoder parameters Decision. By sampling from this latent variable distribution, multiple sets of scene samples are obtained to model the uncertainty in the environment. Multiple sets of scene samples are input into the decoder, and multiple candidate return paths are generated through the decoder. The role of the decoder is to map the latent variables back to specific path representations, and the generated candidate return paths can cover different possible environmental changes and obstacle movements. For the multiple candidate return paths generated, an optimization solution is performed to find an optimal return path. The goal of the optimization solution is to achieve the best balance between the return path in terms of safety, driving time, and energy consumption.

[0033] The above describes the robot automatic navigation and return method in the embodiment of the present invention. The following describes the robot automatic navigation and return device in the embodiment of the present invention. Figure 2 In one embodiment of the present invention, the robot automatic navigation and return device comprises: An extraction module is used to extract multi-scale environmental features from environmental image data collected by the walking robot to obtain environmental feature data; A construction module, used for constructing an initial navigation path according to environmental characteristic data and historical navigation data; The recognition module is used to perform target detection and feature re-identification on the real-time environment image based on the initial navigation path to obtain the real-time environment perception result; A processing module is used to perform extended Kalman filter fusion processing on multi-sensor data according to real-time environmental perception results to obtain robot-environment interaction state characteristics; An analysis module is used to analyze the trajectory of straight segments and turning segments according to the characteristics of the robot-environment interaction state and construct a smooth navigation trajectory; The planning module is used to continuously update the environmental feature data and the working status data of the walking robot during the navigation process to build a dynamic environmental map; when a return signal is received, the automatic return path is quickly planned based on the dynamic environmental map.

[0034] Through the collaboration of the above components, the use of multi-scale environmental feature extraction technology combined with global-local contrast learning can capture environmental information more comprehensively and accurately, improving the accuracy and robustness of environmental perception. The adaptive uncertainty path planning model effectively copes with various uncertain factors in complex dynamic environments by dynamically constructing an uncertain parameter set, improving the adaptability and reliability of path planning. TensorRT-accelerated YOLOv5 target detection and OSNet feature re-identification technology significantly improve the efficiency and accuracy of real-time environmental perception. The extended Kalman filter fusion processes multi-sensor data and effectively improves the accuracy of robot-environment interaction state estimation. The differential flat trajectory generator combines polynomial interpolation and Bezier curves to generate smooth navigation trajectories that meet dynamic constraints, improving the comfort and safety of navigation. The continuous update of the dynamic environment map and the fast return path planning technology enable the robot to respond to environmental changes in a timely manner and achieve safe and efficient automatic return.

[0035] Reference Figure 3 In an embodiment of the present invention, a computer device is also provided. The computer device may be a server, and its internal structure may be as follows: Figure 3 As shown. The computer device includes a processor, a memory, a display screen, an input device, a network interface and a database connected through a system bus. Among them, the processor designed by the computer is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store the corresponding data in this embodiment. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the above method is implemented.

[0036] Those skilled in the art will understand that Figure 3 The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present invention, and does not constitute a limitation on the computer device to which the solution of the present invention is applied.

[0037] An embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the above method is implemented. It can be understood that the computer-readable storage medium in this embodiment can be a volatile readable storage medium or a non-volatile readable storage medium.

[0038] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media provided by the present invention and used in the embodiments may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double-speed data rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM.

[0039] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described systems, systems and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0040] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the whole or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0041] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A robot automatic navigation and return method, characterized in that: The method comprises: Perform multi-scale environmental feature extraction on the environmental image data collected by the walking robot to obtain environmental feature data; Constructing an initial navigation path according to the environmental feature data and the historical navigation data; Based on the initial navigation path, target detection and feature re-identification are performed on the real-time environment image to obtain a real-time environment perception result; According to the real-time environment perception results, the multi-sensor data is subjected to extended Kalman filter fusion processing to obtain the robot-environment interaction state characteristics; Performing straight line segment and turning segment trajectory analysis according to the robot-environment interaction state characteristics to construct a smooth navigation trajectory; During the navigation process, the environmental feature data and the working status data of the walking robot are continuously updated to construct a dynamic environmental map; when a return signal is received, an automatic return path is quickly planned based on the dynamic environmental map.

2. The robot automatic navigation and return method according to claim 1, characterized in that: The multi-scale environmental feature extraction is performed on the environmental image data collected by the walking robot to obtain environmental feature data, including: Performing Gaussian pyramid decomposition on the environmental image data to obtain a multi-scale image set, and performing multi-layer convolution feature extraction on the multi-scale image set to obtain a multi-scale feature atlas containing shallow, middle and deep features; Input the multi-scale feature atlas set into the feature pyramid network to perform upsampling and fusion of features of different scales to obtain a global environment feature map, and input the global environment feature map into the spatial attention module to obtain a key area weight map by calculating the spatial correlation of the feature map; The global environment feature map is weighted based on the key area weight map to obtain a local environment feature map, and the global environment feature map is used as an anchor sample, the local environment feature map is used as a positive sample, and other samples in the batch are randomly selected as negative samples to construct a triplet sample set; Calculate the InfoNCE loss for the triple sample set, and obtain the feature similarity score by minimizing the distance between the anchor sample and the positive sample and maximizing the distance between the anchor sample and the negative sample; The global environment feature map and the key area weight map are weighted based on the feature similarity score to obtain environment feature data.

3. The robot automatic navigation and return method according to claim 2, characterized in that: The constructing the initial navigation path according to the environmental feature data and the historical navigation data comprises: Based on the environmental feature data, the current driving environment is spatially gridded, the continuous space is discretized into grid units of fixed size, a two-dimensional environmental state matrix is ​​obtained, and the two-dimensional environmental state matrix is ​​time-stamp aligned with the historical navigation data to construct a spatiotemporal sequence data set including environmental states and historical trajectories; Sliding window segmentation is performed on the spatiotemporal sequence data set to extract multiple overlapping subsequence samples, wavelet transform is performed on each subsequence sample to extract time-frequency domain features, and significant coefficients are retained through threshold screening to obtain a reduced-dimensional feature vector; Generating a latent space representation and a reconstruction error according to the dimension reduction feature vector, and constructing an uncertainty parameter distribution of the current driving environment using a kernel density estimation algorithm based on the latent space representation and the reconstruction error; Performing Latin hypercube sampling based on the uncertainty parameter distribution to generate multiple groups of representative uncertainty parameter samples, and inputting the multiple groups of uncertainty parameter samples into a path planner based on a rapidly expanding random tree to generate multiple candidate paths; The multiple candidate paths are evaluated by multiple objectives and weighted summation is performed to calculate a comprehensive score for each candidate path, and the multiple candidate paths are sorted in descending order based on the comprehensive score, and the candidate path with the highest comprehensive score is selected as the initial navigation path.

4. The robot automatic navigation and return method according to claim 3, characterized in that: The performing target detection and feature re-identification on the real-time environment image based on the initial navigation path to obtain a real-time environment perception result includes: Based on the initial navigation path, a real-time environment image is collected, and the real-time environment image is input into a YOLOv5 network accelerated by TensorRT, wherein the YOLOv5 network includes a backbone network, a neck network, and a detection head network; The backbone network adopts the CSPDarknet53 structure, including 5 CSP modules, each CSP module includes multiple residual units and 1×1 convolutional layers, and the first multi-scale feature map is obtained through the backbone network; The neck network adopts a feature pyramid structure, including 3 bottom-up feature fusion layers and 3 top-down feature fusion layers, and a second multi-scale feature map is obtained through the neck network; The detection head network includes three parallel detection branches, each branch includes two convolutional layers and one output layer, and the target detection results, including bounding box coordinates, confidence and category probability, are obtained through the detection head network; Performing non-maximum suppression processing on the target detection result to obtain a filtered target detection frame, cropping and resizing the image area corresponding to the filtered target detection frame, and inputting it into the OSNet network accelerated by TensorRT, wherein the OSNet network includes a lightweight backbone network and a feature aggregation module; The lightweight backbone network adopts a deep separable convolution structure, including 4 stages, each stage including multiple lightweight residual units, and a third multi-scale feature map is obtained through the lightweight backbone network; The feature aggregation module adopts an adaptive feature fusion strategy, including a channel attention mechanism and a spatial attention mechanism, and obtains a fused global feature vector through the feature aggregation module; The fused global feature vector is subjected to L2 normalization processing to obtain a target re-identification feature, and the target detection result and the target re-identification feature are associated and matched to obtain a real-time environment perception result.

5. The robot automatic navigation and return method according to claim 4, characterized in that: According to the real-time environment perception result, the multi-sensor data is subjected to extended Kalman filter fusion processing to obtain the robot-environment interaction state characteristics, including: Performing spatiotemporal alignment on the target detection boxes and re-identification features in the real-time environment perception results to obtain feature-enhanced environment perception data, and performing time synchronization on the feature-enhanced environment perception data with the original sensor data from the inertial measurement unit, encoder, lidar, and depth camera to obtain a multi-source heterogeneous sensor data set; Performing noise characteristic analysis on various sensor data in the multi-source heterogeneous sensor data set, and establishing a sensor measurement noise covariance matrix and a process noise covariance matrix; Construct nonlinear state transfer equations and observation equations, where the state vector includes the robot's position, posture, velocity and angular velocity, and the observation vector includes the measurement values ​​of each sensor; Performing a first-order Taylor expansion on the nonlinear state transfer equation to obtain a Jacobian matrix of the state prediction equation, and performing a first-order Taylor expansion on the nonlinear observation equation to obtain a Jacobian matrix of the observation prediction equation; Based on the state estimation value and control input at the previous moment, the Jacobian matrix of the state prediction equation is used to perform state prediction to obtain a priori state estimation value and a priori error covariance matrix; Substituting the prior state estimate into the observation prediction equation, calculating the observation prediction value, and subtracting it from the actual observation value to obtain the observation residual; The Kalman gain is calculated according to the prior error covariance matrix, the Jacobian matrix of the observation prediction equation and the observation noise covariance matrix, and the prior state estimation value is corrected by using the Kalman gain to obtain the robot-environment interaction state characteristics.

6. The robot automatic navigation and return method according to claim 5, characterized in that: The step of analyzing the trajectory of straight segments and turning segments according to the robot-environment interaction state characteristics to construct a smooth navigation trajectory includes: Analyze the robot-environment interaction state characteristics, extract the robot's current position, posture, speed and acceleration as the trajectory starting point constraint conditions, and determine the trajectory end point constraint conditions according to the initial navigation path to obtain a trajectory boundary condition set; Based on the trajectory boundary condition set, segmenting the initial navigation path, dividing the initial navigation path into straight segments and turning segments, and obtaining a path segment set; For the straight line segments in the path segment set, a trajectory equation is constructed using a quintic polynomial interpolation method, the time is normalized to the interval [0, 1], and the position, velocity, and acceleration constraints of the trajectory start and end points are used to solve the polynomial coefficients to obtain the straight line segment trajectory equation; For the turning segment in the path segment set, a third-order Bezier curve is used to construct a trajectory equation, wherein four control points are respectively set as the starting point of the turning segment, a point in the tangent direction of the starting point, a point in the tangent direction of the end point, and the end point, and the curvature is controlled by adjusting the positions of the two middle control points to obtain the turning segment trajectory equation; Combining the straight segment trajectory equation and the turning segment trajectory equation to construct a complete trajectory equation, and performing time parameterization on the complete trajectory equation to obtain a parameterized trajectory equation; Perform differential operation on the parameterized trajectory equation to obtain expressions of velocity, acceleration and acceleration, and verify the feasibility of the trajectory in combination with the robot dynamics constraints. If the constraints are not met, adjust the trajectory parameters and regenerate them; Based on the parameterized trajectory equation, an equal time interval sampling method is adopted to generate a discrete trajectory point sequence, each trajectory point contains position, attitude, velocity and acceleration information, and the trajectory point sequence is checked for smoothness and continuity to ensure that the position, velocity and acceleration changes between adjacent trajectory points meet the smoothness requirements, so as to obtain a smooth navigation trajectory.

7. The robot automatic navigation and return method according to claim 6, characterized in that: The environmental feature data and the working status data of the walking robot are continuously updated during the navigation process to construct a dynamic environmental map; When a return signal is received, an automatic return path is quickly planned based on the dynamic environment map, including: The motion environment image collected in real time during the navigation process is subjected to multi-scale convolution processing and feature fusion to obtain a motion multi-scale feature map, and the motion multi-scale feature map is input into a global-local contrast learning module for processing to obtain the latest environment feature data; Performing spatiotemporal registration on the latest environmental feature data, constructing a short-term environmental feature sequence matrix, and performing spatiotemporal correlation feature analysis on the short-term environmental feature sequence matrix to obtain a dynamic environmental feature representation vector; Based on the odometer data and IMU data of the walking robot, the real-time posture estimation of the robot is performed to obtain the position information of the robot, and the dynamic environment feature representation vector is associated with the robot position information to obtain a local dynamic environment map; Performing spatial registration on the local dynamic environment map, updating the global dynamic environment map, and performing data association and tracking on the dynamic obstacles in the real-time motion environment to obtain an obstacle motion trajectory set; When a return signal is received, the current position coordinates of the walking robot are extracted as the starting point, the navigation start position coordinates are taken as the end point, a return mission target point pair is constructed, and the return mission target point pair and the global dynamic environment map are input into the A* algorithm for path search to obtain an initial return path composed of grid coordinates; The initial return path is fitted with a cubic B-spline curve, with the control point interval set to 0.5 meters, to generate a smooth path curve, and a path point sequence is obtained by uniformly sampling every 0.1 meters along the path, and the path point sequence, the dynamic environment feature representation vector and the obstacle motion trajectory set are combined to construct a return scene description tensor; The return scene description tensor is input into an adaptive uncertainty path planning model, which adopts a variational inference network structure and includes an encoder and a decoder. The encoder is used to probabilistically encode the return scene to generate a latent variable distribution of the scene, and multiple groups of scene samples are sampled therefrom. The multiple groups of scene samples are input into a decoder to generate multiple candidate return paths, and the multiple candidate return paths are optimized to obtain an automatic return path.

8. A robot automatic navigation and return device, characterized in that: Used to execute the robot automatic navigation and return method according to any one of claims 1 to 7, the device comprises: An extraction module is used to extract multi-scale environmental features from environmental image data collected by the walking robot to obtain environmental feature data; A construction module, used for constructing an initial navigation path according to the environmental feature data and the historical navigation data; A recognition module, used to perform target detection and feature re-recognition on the real-time environment image based on the initial navigation path to obtain a real-time environment perception result; A processing module, used to perform extended Kalman filter fusion processing on multi-sensor data according to the real-time environment perception result to obtain robot-environment interaction state characteristics; An analysis module, used for analyzing the trajectory of straight segments and turning segments according to the robot-environment interaction state characteristics, and constructing a smooth navigation trajectory; The planning module is used to continuously update the environmental feature data and the working status data of the walking robot during the navigation process to build a dynamic environmental map; when a return signal is received, the automatic return path is quickly planned based on the dynamic environmental map.

9. A computer device, characterized in that: It comprises a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and is characterized in that when the processor executes the computer program, the robot automatic navigation and return method described in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the processor is enabled to execute the robot automatic navigation and return method according to any one of claims 1 to 7.

Images