Robot positioning state analysis method, device, equipment and medium

Abstract

The application relates to the technical field of robots and discloses a robot positioning state analysis method, a robot positioning state analysis device, equipment and a medium, the method comprising the following steps: acquiring estimated poses and laser scanning data of a mobile robot; performing multidimensional disturbance sampling with the estimated poses as the original points to construct a disturbance sampling grid; for each sampling pose node, projecting the laser scanning data to a pre-built map to calculate a matching degree measurement value and generate a three-dimensional matching degree tensor; compressing and projecting the tensor along multiple orthogonal directions to obtain multiple two-dimensional projection matrices and fuse the two-dimensional projection matrices into a multi-channel pseudo-color image; and inputting the image into a two-dimensional convolutional neural network classification model for analysis to obtain a positioning state classification result. The application solves the problem of high false alarm rate of the existing positioning failure detection method in a dynamic scene.

Description

Technical Field

[0001] This invention relates to the field of robotics, specifically to methods, devices, equipment, and media for analyzing robot positioning status. Background Technology

[0002] In warehousing and logistics, autonomous handling, and industrial automation, autonomous mobile robots (AMRs / AGVs) typically rely on 2D LiDAR combined with pre-built maps (grid maps or point cloud maps) for real-time localization and motion control. The accuracy of their localization estimation is a core prerequisite for ensuring operational safety and efficiency. Currently, mainstream localization solutions output the robot's real-time pose by fusing laser scanning data with odometry information. However, when robots encounter cumulative errors caused by wheel slippage, highly dynamic pedestrian or cargo obstructions, or degenerate scenarios with limited geometric features such as long corridors and symmetrical tunnels, localization failures are highly likely to occur, leading to collisions, equipment damage, or production downtime.

[0003] In existing technologies, the determination of positioning failure is usually based on the detection mechanism of instantaneous absolute matching degree to realize the anomaly. However, this method cannot effectively distinguish whether the decrease in matching degree is due to deviation from the actual pose or only due to temporary dynamic occlusion. In industrial logistics scenarios with frequent dynamic changes, the absolute matching degree of a single frame is prone to sudden drop due to temporary occlusion, which frequently triggers false alarms, causing the robot to stop or slow down for no reason, seriously affecting the smoothness of operation and system availability. Summary of the Invention

[0004] In view of this, embodiments of the present invention provide a robot positioning status analysis method, apparatus, device and medium to solve the problem of high false alarm rate in existing positioning failure detection methods in dynamic scenarios.

[0005] In a first aspect, embodiments of the present invention provide a robot positioning state analysis method, the method comprising:

[0006] Obtain the estimated pose of the mobile robot at the current moment and the laser scan data at the current moment;

[0007] Using the estimated pose as the origin, multi-dimensional perturbation sampling is performed in the pose state space of the mobile robot to construct a perturbation sampling grid containing multiple sampled pose nodes.

[0008] For each sampling pose node in the perturbation sampling grid, the laser scanning data is projected onto a pre-built map, the matching degree metric value under the sampling pose node is calculated, and a three-dimensional matching degree tensor is generated based on the matching degree metric values ​​of all sampling pose nodes.

[0009] The three-dimensional matching degree tensor is compressed and projected along multiple orthogonal projection directions to obtain multiple two-dimensional matching degree projection matrices, and the multiple two-dimensional matching degree projection matrices are fused into a multi-channel pseudo-color image;

[0010] The multi-channel pseudo-color image is input into a pre-trained two-dimensional convolutional neural network classification model for analysis to obtain the localization state classification result of the estimated pose.

[0011] Furthermore, the step of performing multi-dimensional perturbation sampling in the pose state space of the mobile robot, using the estimated pose as the origin, to construct a perturbation sampling mesh containing multiple sampled pose nodes, includes:

[0012] Using the estimated pose as the origin, translational perturbation deviations are applied in the translational dimension of the mobile robot and rotational perturbation deviations are applied in the rotational dimension of the mobile robot to generate multiple perturbation poses.

[0013] Based on the deviation relationship between the estimated pose and each of the perturbation poses, a three-dimensional perturbation sampling grid is constructed;

[0014] The translation dimension includes the lateral translation dimension and the longitudinal translation dimension of the mobile robot.

[0015] Furthermore, for each sampling pose node in the perturbation sampling grid, projecting the laser scanning data onto a pre-built map and calculating the matching degree metric value under the sampling pose node includes:

[0016] Obtain the laser coordinates of multiple laser points in the laser scanning data in the laser coordinate system;

[0017] For each sampling pose node in the perturbation sampling grid, a rotation transformation matrix and a translation vector are constructed based on the pose information of the sampling pose node;

[0018] Based on the rotation transformation matrix and the translation vector, the laser coordinates of each laser point are converted into global projected coordinates in the global coordinate system of the pre-built map;

[0019] Determine the map occupancy information corresponding to each of the global projection coordinates in the pre-built map, and calculate the proportion of effective laser points that meet the preset matching conditions among each of the laser points based on the map occupancy information. Use the proportion of effective laser points as the matching degree metric value under the sampling pose node.

[0020] Furthermore, the three-dimensional matching degree tensor is compressed and projected along multiple orthogonal projection directions to obtain multiple two-dimensional matching degree projection matrices, including:

[0021] Obtain the multiple dimensional axes corresponding to the three-dimensional matching degree tensor;

[0022] Max pooling projection is performed on the three-dimensional matching degree tensor along the first dimension axis of multiple dimensions to obtain the first two-dimensional matching degree projection matrix;

[0023] Max pooling projection is performed on the three-dimensional matching tensor along the direction of the second dimension axis among multiple dimension axes to obtain the second two-dimensional matching degree projection matrix;

[0024] Max pooling projection is performed on the three-dimensional matching degree tensor along the direction of the third dimension axis among multiple dimensions to obtain the third two-dimensional matching degree projection matrix.

[0025] Furthermore, fusing the multiple two-dimensional matching degree projection matrices into a multi-channel pseudo-color image includes:

[0026] The first two-dimensional matching degree projection matrix, the second two-dimensional matching degree projection matrix, and the third two-dimensional matching degree projection matrix are normalized respectively to obtain a first normalized matrix, a second normalized matrix, and a third normalized matrix.

[0027] The first normalized matrix, the second normalized matrix, and the third normalized matrix are respectively mapped to the first color channel component, the second color channel component, and the third color channel component of the color image;

[0028] The first color channel component, the second color channel component, and the third color channel component are fused along the channel dimension to generate a multi-channel pseudo-color image of the original size.

[0029] Furthermore, after generating the original-size multi-channel pseudo-color image, the method further includes:

[0030] Obtain the pre-defined target size of the input layer of the two-dimensional convolutional neural network classification model;

[0031] The original multi-channel pseudo-color image is upsampled to generate a multi-channel pseudo-color image that conforms to the target size.

[0032] Furthermore, the step of inputting the multi-channel pseudo-color image into a pre-trained two-dimensional convolutional neural network classification model for analysis to obtain the localization state classification result of the estimated pose includes:

[0033] The multi-channel pseudo-color image is input into the two-dimensional convolutional neural network classification model. The two-dimensional convolutional neural network classification model extracts spatial features from the multi-channel pseudo-color image through multiple cascaded feature extraction modules to obtain a multi-scale topological feature map.

[0034] The multi-scale topological feature map is input into the fully connected classification module of the two-dimensional convolutional neural network classification model, and the fully connected classification module outputs the first confidence level of the estimated pose belonging to the normal positioning state and the second confidence level of the pose belonging to the positioning failure state.

[0035] The localization state classification result of the estimated pose is determined based on the relative magnitude relationship between the first confidence level and the second confidence level.

[0036] Furthermore, the training method for the two-dimensional convolutional neural network classification model includes:

[0037] Obtain a simulation dataset, wherein the simulation dataset includes simulated laser scanning data, simulated odometry data, and simulated absolute true pose of the virtual robot at multiple moments in the simulation environment;

[0038] The simulated odometer data and the simulated laser scanning data are fused together for localization to obtain the simulated estimated pose at each time point;

[0039] Based on the deviation between the simulated estimated pose and the corresponding simulated absolute true pose at each time step, the localization state label corresponding to each time step is determined.

[0040] Based on the simulated estimated pose and the simulated laser scanning data at each time point, sample multi-channel pseudo-color images corresponding to each time point are generated.

[0041] By utilizing the mapping relationship between the positioning status labels corresponding to each time point and the sample multi-channel pseudo-color image, a training sample set is constructed, and the initial two-dimensional convolutional neural network classification model is trained under supervision using the training sample set to obtain the pre-trained two-dimensional convolutional neural network classification model.

[0042] Furthermore, after obtaining the localization state classification result of the estimated pose, the method further includes:

[0043] When the positioning status classification result indicates that the estimated pose is in a positioning failure state, the first level of safety control operation is executed. The first level of safety control operation includes: discarding the laser scanning data at the current moment, switching the positioning solution method to dead reckoning based on odometry data, and reducing the movement speed of the mobile robot.

[0044] If the cumulative duration of the dead reckoning reaches a preset time threshold, a second-level safety control operation is performed. The second-level safety control operation includes: acquiring visual image frames collected by the visual sensors on the mobile robot, matching and locating the visual image frames with the visual information in the pre-built map to obtain the visual positioning pose, and using the visual positioning pose to perform a correction operation on the cumulative deviation of the dead reckoning.

[0045] If the cumulative drift calculated by dead reckoning is greater than or equal to a preset distance threshold, then a third-level safety control operation is executed. The third-level safety control operation includes: controlling the mobile robot to decelerate and stop, and initiating a global relocation operation based on the laser scanning data and the pre-built map.

[0046] Secondly, embodiments of the present invention provide a robot positioning state analysis device, the device comprising:

[0047] The acquisition module is used to acquire the estimated pose of the mobile robot at the current moment and the laser scan data at the current moment;

[0048] The construction module is used to perform multi-dimensional perturbation sampling in the pose state space of the mobile robot with the estimated pose as the origin, and construct a perturbation sampling grid containing multiple sampled pose nodes.

[0049] The generation module is used to project the laser scanning data onto a pre-built map for each sampling pose node in the perturbation sampling grid, calculate the matching degree metric value under the sampling pose node, and generate a three-dimensional matching degree tensor based on the matching degree metric values ​​of all sampling pose nodes.

[0050] The fusion module is used to compress and project the three-dimensional matching degree tensor along multiple orthogonal projection directions to obtain multiple two-dimensional matching degree projection matrices, and fuse the multiple two-dimensional matching degree projection matrices into a multi-channel pseudo-color image.

[0051] The analysis module is used to input the multi-channel pseudo-color image into a pre-trained two-dimensional convolutional neural network classification model for analysis, and obtain the localization state classification result of the estimated pose.

[0052] Thirdly, embodiments of the present invention provide a computer device, including: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to perform the method described in the first aspect or any corresponding embodiment thereof.

[0053] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing computer instructions that cause a computer to perform the method described in the first aspect or any of its corresponding embodiments.

[0054] The method provided in this application has the following beneficial effects:

[0055] The method provided in this application acquires the estimated pose and laser scanning data of a mobile robot, providing a real-time data foundation for subsequent localization state analysis. It constructs a sampling grid by performing multi-dimensional perturbation sampling in the pose state space with the estimated pose as the origin, introducing spatial context information within the local pose neighborhood to provide a perturbation comparison benchmark for distinguishing the causes of decreased matching degree. It calculates the matching degree between the laser scanning data and the pre-built map for each sampled pose node and generates a three-dimensional matching degree tensor, quantizing the spatial distribution of matching degree within the robot's pose neighborhood into computable three-dimensional cost volume data. It compresses and projects the three-dimensional matching degree tensor along multiple orthogonal directions to obtain multiple two-dimensional projection matrices and fuses them into a multi-channel pseudo-color image, reducing the high-dimensional cost distribution to an image representation that retains spatial topological features, thus visualizing the spatial structure of localization uncertainty. Finally, it inputs the multi-channel pseudo-color image into a two-dimensional convolutional neural network classification model for analysis and outputs the localization state classification result, utilizing the convolutional neural network's ability to extract image topological features to achieve intelligent discrimination of the localization state. Attached Figure Description

[0056] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0057] Figure 1 This is a flowchart illustrating the robot positioning state analysis method according to an embodiment of the present invention;

[0058] Figure 2 This is a flowchart illustrating another robot positioning state analysis method according to an embodiment of the present invention;

[0059] Figure 3 This is a flowchart illustrating another robot positioning state analysis method according to an embodiment of the present invention;

[0060] Figure 4 This is a schematic diagram of the algorithm flow and data conversion of the robot positioning state analysis method according to an embodiment of the present invention;

[0061] Figure 5 This is a schematic diagram of the structure of a robot positioning state analysis system according to an embodiment of the present invention;

[0062] Figure 6 This is a structural block diagram of a robot positioning state analysis device according to an embodiment of the present invention;

[0063] Figure 7 This is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. Detailed Implementation

[0064] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0065] According to embodiments of the present invention, a robot positioning state analysis method, apparatus, device, and medium are provided. It should be noted that the steps shown in the flowcharts in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowcharts, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0066] This embodiment provides a robot positioning state analysis method. Figure 1 This is a flowchart of a robot localization state analysis method according to an embodiment of the present invention, such as... Figure 1 As shown, the process includes the following steps:

[0067] Step S101: Obtain the estimated pose of the mobile robot at the current moment and the laser scan data at the current moment.

[0068] In this embodiment, the estimated pose refers to the pose estimate of the robot in the map coordinate system output by the localization solution module of the mobile robot (such as Adaptive Monte Carlo Localization AMCL, graph optimization method, or Extended Kalman Filter EKF) at the current moment. It is usually represented as a three-dimensional state vector containing lateral translation x, longitudinal translation y, and heading angle θ. Laser scanning data refers to the real-time laser scanning frames output by the two-dimensional lidar installed on the mobile robot. These frames consist of H polar coordinate beam ranging values ​​in the lidar's own coordinate system. After polar-to-Cartesian coordinate transformation, they can be represented as a set of two-dimensional Cartesian coordinates for H laser points, denoted as: The pre-built map refers to a raster map that is stored in a non-volatile medium in advance. Its resolution is 0.02 m. Each raster cell stores an occ(·) occ(·) occ(·) between 0 (free area) and 1 (obstacle area), which is used for matching degree calculation during subsequent laser point projection.

[0069] During normal operation of the mobile robot, a real-time pose estimation module (such as an onboard industrial computer or microcontroller) periodically receives current frame laser scan data from a two-dimensional LiDAR and odometer data from a combination of wheel photoelectric encoders and an inertial measurement unit. The module then fuses these multi-source sensor data for positioning, thereby outputting the estimated pose at the current moment. Simultaneously, the original laser scan frame at the current moment is directly read from the driver of the 2D lidar, H polar coordinate ranging values ​​are obtained and converted into a set of 2D Cartesian coordinate points in the laser coordinate system { } (i = 1,2,…,H).

[0070] Step S102: Using the estimated pose as the origin, perform multi-dimensional perturbation sampling in the pose state space of the mobile robot to construct a perturbation sampling grid containing multiple sampled pose nodes.

[0071] In this embodiment of the application, multi-dimensional perturbation sampling is performed in the pose state space of the mobile robot, with the estimated pose as the origin, to construct a perturbation sampling mesh containing multiple sampled pose nodes, including:

[0072] Step A1: Using the estimated pose as the origin, apply translational perturbation deviations to the translational dimension of the mobile robot and rotational perturbation deviations to the rotational dimension of the mobile robot to generate multiple perturbation poses; wherein, the translational dimension includes the lateral translational dimension and the longitudinal translational dimension of the mobile robot.

[0073] Specifically, the translational perturbation deviation refers to the small positional offset artificially applied in both the lateral translation dimension (x-direction) and the longitudinal translation dimension (y-direction) based on the estimated pose, denoted as and Rotational disturbance deviation refers to the small angular offset artificially applied in the heading angle dimension (θ direction) of the estimated pose, denoted as The preset positive odd number n is a parameter used to control the number of perturbation samples in each dimension. In this embodiment, n is 11, that is, 11 sampling points are collected symmetrically with the estimated pose as the center in each dimension. The three-dimensional perturbation sampling grid is a three-dimensional sampling space spanned by the horizontal translation perturbation dimension, the vertical translation perturbation dimension, and the heading angle rotation perturbation dimension. Each node in the grid corresponds to a sampling pose node that has been perturbed in the neighborhood of the estimated pose.

[0074] To estimate pose As the origin of the perturbation sampling space, equally spaced perturbation biases are applied in the lateral translation dimension x, the longitudinal translation dimension y, and the heading angle dimension θ. The lateral translation perturbation bias δx_u is uniformly sampled according to the following formula:

[0075]

[0076] The longitudinal translational disturbance deviation δy_v is uniformly sampled according to the following formula:

[0077]

[0078] The heading angle rotation disturbance deviation δθ_w is uniformly sampled according to the following formula:

[0079]

[0080] In this embodiment, the translation step size Δx and Δy are both 0.02 m, and the rotation step size Δθ is 1.5°. Therefore, the perturbation range in both the lateral and longitudinal directions is ±0.1 m (i.e., ±5 steps), and the perturbation range in the heading angle is ±7.5° (i.e., ±5 steps). By combining the Cartesian products in the above three dimensions, n×n×n = 1331 perturbation poses can be generated.

[0081] It should be noted that when a mobile robot is equipped with a 3D LiDAR for localization, the robot is in a six-degree-of-freedom pose state in three-dimensional space (including lateral translation x, longitudinal translation y, vertical translation z, and six dimensions: roll, pitch, and yaw). In this application scenario, three key dimensions most prone to drift degradation (e.g., lateral translation x, longitudinal translation y, and yaw) can be selected to construct a 3D perturbation sampling grid. The perturbation deviations of the other three dimensions can be ignored or set as constant compensation values. The three-channel projection method of this embodiment can still be used to fuse the 3D matching degree tensor into an RGB pseudo-color image along the three selected dimensional axes using max-pooling projection matrices, which can then be used to regress and evaluate the pose anomalies or slippage collapse of the 3D localization system. This alternative extends the core idea of ​​this invention from 2D laser localization to 3D laser localization scenarios and is an adaptive variation within the scope of protection of this invention.

[0082] In this embodiment, the perturbation step size does not need to be a fixed parameter. It can be adaptively and linearly scaled with the current linear velocity of the mobile robot. For example, when the robot is running at high speed, the accumulated positional drift per unit time is greater. The translation step size Δx and Δy are adaptively increased proportionally (e.g., the step size is increased by 50% when the speed doubles). This allows the perturbation sampling grid to cover a wider range during high-speed motion, capturing a larger-scale cost surface landscape and improving the ability to predict positioning drift risk under high-speed handling conditions. Conversely, when the robot is running at low speed or stationary, the perturbation step size can be adaptively reduced to improve the resolution of the local cost landscape. This alternative allows the sampling range of the perturbation grid to dynamically adapt to the robot's motion state, optimizing the positioning failure detection sensitivity under different speed conditions while keeping computational overhead controllable. It is an equivalent variation of the step size setting method of this invention.

[0083] Step A2: Construct a three-dimensional perturbation sampling grid based on the deviation relationship between the estimated pose and each perturbation pose.

[0084] Specifically, based on the perturbation deviations in each dimension, each perturbation pose node is constructed according to the following formula: the perturbation deviations corresponding to each lateral perturbation index u, longitudinal perturbation index v, and rotational perturbation index w are respectively superimposed on the estimated pose's lateral translation x, longitudinal translation y, and heading angle θ.

[0085]

[0086] Let each u, v, w index combination traverse the integer set, thus obtaining all pose nodes. This forms a three-dimensional perturbation sampling grid centered on the estimated pose and containing n×n×n sampling points. This grid spatially covers all sampling nodes within the estimated pose neighborhood, including a lateral translation range of ±0.1 m, a longitudinal translation range of ±0.1 m, and a heading angle range of ±7.5°. This provides input data for calculating the matching degree metric under each perturbation pose in subsequent steps.

[0087] Step S103: For each sampling pose node in the perturbation sampling grid, project the laser scanning data onto the pre-built map, calculate the matching degree metric value under the sampling pose node, and generate a three-dimensional matching degree tensor based on the matching degree metric values ​​of all sampling pose nodes.

[0088] In this embodiment of the application, for each sampling pose node in the perturbation sampling grid, the laser scanning data is projected onto a pre-built map, and the matching degree metric value under the sampling pose node is calculated, including:

[0089] Step B1: Obtain the laser coordinates of multiple laser points in the laser scanning data in the laser coordinate system.

[0090] Specifically, the laser coordinate system is a local two-dimensional Cartesian coordinate system established with the geometric center of the two-dimensional lidar as the origin and the direction of the lidar itself as the reference direction. Each ranging value output by the lidar is polar coordinate data described in this coordinate system. Laser coordinates refer to the position coordinates of a single laser point in the laser coordinate system after polar coordinate-Cartesian coordinate transformation, and are represented by a two-dimensional vector.

[0091] The system reads the laser scan data at the current moment, which consists of H polar coordinate beam ranging values. Each beam contains the ranging distance and corresponding angle information. The system converts these polar coordinate data one by one into two-dimensional Cartesian coordinates in the laser coordinate system, following the standard polar coordinate transformation formula, i.e., the horizontal coordinate... y-axis After this transformation, the set of laser coordinates for all H laser points is obtained, denoted as: This provides the basic data for subsequent coordinate projection transformations.

[0092] Step B2: For each sampling pose node in the perturbation sampling grid, construct a rotation transformation matrix and a translation vector based on the pose information of the sampling pose node.

[0093] Specifically, a sampling pose node refers to any node in the 3D perturbation sampling mesh, corresponding to a perturbation pose x_{u,v,w} determined by a specific set of lateral perturbation indices u, longitudinal perturbation indices v, and rotational perturbation indices w. It contains the lateral translation, longitudinal translation, and heading angle information of that node. The rotation transformation matrix is ​​a 2×2 orthogonal matrix used to describe the rotation relationship between the laser coordinate system and the global coordinate system, and is composed of the sine and cosine values ​​of the heading angle. The translation vector is a 2×1 column vector used to describe the translation offset of the origin of the laser coordinate system relative to the origin of the global coordinate system.

[0094] For the current sampling pose node in the perturbed sampling grid Extract the heading angle component (θ + δθ_w) of the node, and construct the rotation transformation matrix accordingly:

[0095]

[0096] This matrix rotates the coordinate vectors in the laser coordinate system to align with the directions of the global coordinate system;

[0097] Simultaneously, extract the lateral translation component (x + ) of this node. ) and longitudinal translation component (y + Construct the translation vector:

[0098]

[0099] The rotation transformation matrix and translation vector together constitute the complete rigid body transformation parameters from the laser coordinate system to the global coordinate system, which are used for the coordinate projection of all H laser points in subsequent steps.

[0100] Step B3: Based on the rotation transformation matrix and translation vector, convert the laser coordinates of each laser point into global projected coordinates in the global coordinate system of the pre-built map.

[0101] Specifically, the global coordinate system is the fixed world coordinate system in which the pre-built map is located. The positioning pose of the mobile robot and the position of the laser point cloud in the map are described in this coordinate system. The global projection coordinates refer to the position coordinates of a single laser point in the global coordinate system after rotation and translation transformation. It represents the spatial landing point of the laser point projected onto the map with the current sampling pose.

[0102] For each laser coordinate si (i=1,2,…,H), use the rotation transformation matrix Translation vector The coordinate transformation calculations are performed one by one according to the rigid body transformation:

[0103]

[0104] By sequentially performing the above transformation on all H laser points, the set of global projected coordinates of each laser point under the current sampling pose node is obtained. These coordinates will be used to query the occupation information of each landing point on the map later.

[0105] Step B4: Determine the map occupancy information corresponding to each global projection coordinate in the pre-built map, and calculate the proportion of effective laser points that meet the preset matching conditions among each laser point based on the map occupancy information. Use the proportion of effective laser points as the matching degree metric value under the sampling pose node.

[0106] Specifically, the pre-built map is a raster map pre-stored in a non-volatile medium with a resolution of 0.02m. Each raster cell stores an occupancy probability value between 0 (completely free) and 1 (completely obstructed), denoted as occ(·). Map occupancy information refers to the occupancy probability value stored in the raster cell corresponding to a certain global projection coordinate in the pre-built map, used to quantify the confidence level that the location is occupied by an obstacle. The preset matching condition refers to the criterion for determining whether a single projection point constitutes a valid match, specifically whether the map occupancy probability value is greater than or equal to the preset map occupancy threshold γ. The effective laser point ratio refers to the ratio of the number of laser points that meet the preset matching condition to the total number of laser points among all H laser points for a certain sampling pose node. This ratio is the matching degree metric for that node.

[0107] For each global projection coordinate Map it to the corresponding raster cell in the pre-built map and query the occupancy probability value stored in that raster cell. Subsequently, the system performs cumulative calculations according to the matching degree metric calculation formula:

[0108]

[0109] in, γ is the indicative index function (outputs 1 when the expression in parentheses is true, otherwise outputs 0), and γ is the preset map occupancy threshold (0.65 in this embodiment).

[0110] For each laser point, if the grid occupancy probability at its global projected coordinates is greater than or equal to 0.65, the point is considered a valid match point, and the indicator function outputs 1; otherwise, the indicator function outputs 0. The summation of the indicator function outputs for all H laser points, divided by H, yields the proportion of valid laser points at that sampling pose node. This proportion is the matching degree metric M(u,v,w) for that node. After processing this sampling pose node, the process returns to process the next sampling pose node in the perturbed sampling grid, until all 1331 nodes have obtained their corresponding matching degree metrics, ultimately generating a three-dimensional real tensor of size n×n×n. Each element It fully describes the spatial distribution of matching degree in each perturbation direction within the estimated pose neighborhood.

[0111] It should be noted that this embodiment uses the proportion of effective laser points falling on the map grid (i.e., the proportion of effective laser beams to all laser beams) as the matching degree metric for the sampling pose node. The matching similarity metric function can be replaced by other forms of registration similarity calculation models. For example, the mean Euclidean distance from the global projected coordinates of all laser points to the nearest map wall point cloud can be calculated, and the average distance between all laser points and map obstacles can be used as a measure of matching degree—the smaller the mean distance, the higher the registration similarity. Alternatively, the matching degree can be calculated using the Gaussian Likelihood Field function, which involves pre-converting the map grid into a Gaussian likelihood field, storing a Gaussian probability value based on its distance to the nearest obstacle for each grid position, then mapping the global projected coordinates of the laser points to this likelihood field, and averaging or multiplying the Gaussian probability values ​​corresponding to all laser points as the matching degree metric. When using the above-mentioned alternative metric function, if the smaller the function output value, the higher the matching degree, then it is only necessary to perform an inverse proportional mapping transformation on each of the finally generated two-dimensional projection matrices to restore the mapping relationship where the matching degree is proportional to the pixel value. The essence of CNN in judging the spatial symmetry and extreme value topological trend of the relative distribution of matching degree in each channel image has not changed, so it still falls within the scope of the equivalent substitution of this invention.

[0112] Step S104: Compress and project the three-dimensional matching degree tensor along multiple orthogonal projection directions to obtain multiple two-dimensional matching degree projection matrices, and fuse the multiple two-dimensional matching degree projection matrices into a multi-channel pseudo-color image.

[0113] In this embodiment of the application, the three-dimensional matching degree tensor is compressed and projected along multiple orthogonal projection directions to obtain multiple two-dimensional matching degree projection matrices, including:

[0114] Step C1: Obtain the multiple dimension axes corresponding to the three-dimensional matching degree tensor.

[0115] Specifically, the dimension axis refers to the coordinate axis direction used to index different dimensions in a high-dimensional tensor. In this embodiment, the three-dimensional matching degree tensor contains three dimension axes, namely the u-axis (lateral translation perturbation dimension), the v-axis (vertical translation perturbation dimension), and the w-axis (heading angle rotation perturbation dimension).

[0116] Obtain the 3D matching degree tensor V, identify its dimensional structure as n×n×n (n=11 in this embodiment), and extract its corresponding three dimensional axes: the first dimensional axis is the u-axis (horizontal translation perturbation index dimension), the second dimensional axis is the v-axis (vertical translation perturbation index dimension), and the third dimensional axis is the w-axis (heading angle rotation perturbation index dimension). The purpose of identifying these three dimensional axes is to provide an index basis for subsequently determining the compression direction of max pooling projection, enabling dimensionality reduction and compression of the 3D tensor along the specified dimensional axis direction.

[0117] Step C2: Perform max pooling projection on the three-dimensional matching degree tensor along the first dimension axis of the multiple dimensions to obtain the first two-dimensional matching degree projection matrix.

[0118] Specifically, max pooling projection is a dimensionality reduction operation. In a 3D tensor, along a certain dimensional axis, while keeping the indices of the other two dimensions unchanged, the maximum value among all elements along that dimensional axis is taken as the projection output value, thereby compressing the 3D tensor into a 2D matrix. The 2D matching degree projection matrix is ​​the 2D matrix obtained after max pooling projection of the 3D matching degree tensor along a certain dimensional axis. This matrix retains the matching degree spatial distribution information corresponding to the remaining two dimensions other than the compressed dimension. The first dimensional axis (u-axis) corresponds to the horizontal translation perturbation index, the second dimensional axis (v-axis) corresponds to the vertical translation perturbation index, and the third dimensional axis (w-axis) corresponds to the heading angle rotation perturbation index.

[0119] Compression is performed along the third-dimensional axis (w-axis, i.e., the dimension of the heading angle rotation perturbation). For each fixed pair of lateral perturbation indices u and longitudinal perturbation indices v, the maximum value of the matching degree metric at all w-index positions is taken, and the first two-dimensional matching degree projection matrix of size n×n is obtained by calculating according to the following formula. :

[0120]

[0121] This matrix represents the projection of the xy translational uncertainty surface, retaining the maximum matching degree information under lateral and longitudinal translational perturbations, and will be mapped to the R channel (red channel) of the pseudo-color image in subsequent steps.

[0122] Step C3: Perform max pooling projection on the three-dimensional matching degree tensor along the second dimension axis of the multiple dimensions to obtain the second two-dimensional matching degree projection matrix.

[0123] Specifically, compression is performed along the first dimension axis (u-axis, i.e., the horizontal translational perturbation dimension). For each fixed pair of longitudinal perturbation index v and yaw angle rotation perturbation index w, the maximum value of the matching degree metric at all u-index positions is taken, and the second two-dimensional matching degree projection matrix of size n×n is obtained by calculating according to the following formula. :

[0124]

[0125] This matrix represents the coupled projection of longitudinal displacement and heading angle, reflecting the misalignment cost landscape of forward slip and angular rotation, and will be mapped to the G channel (green channel) of a pseudo-color image in subsequent steps.

[0126] Step C4: Perform max pooling projection on the three-dimensional matching degree tensor along the direction of the third dimension axis among multiple dimensions to obtain the third two-dimensional matching degree projection matrix.

[0127] Specifically, compression is performed along the second dimension axis (v-axis, i.e., the longitudinal translational perturbation dimension). For each fixed pair of lateral perturbation index u and yaw angle rotation perturbation index w, the maximum value of the matching degree metric at all v-index positions is taken, and the result is calculated using the following formula to obtain a third two-dimensional matching degree projection matrix of size n×n. :

[0128]

[0129] This matrix represents the coupled projection of lateral displacement and heading angle, reflecting the cost distribution of lateral drift and angular rotation, and will be mapped to the B channel (blue channel) of the pseudo-color image in subsequent steps.

[0130] Through the max pooling projection operations in the three directions mentioned above, the three-dimensional matching degree tensor V is compressed into three two-dimensional matching degree projection matrices. Each matrix has a size of 11×11, and these three matrices together preserve the key spatial topological information of the original three-dimensional tensor in the three orthogonal projection directions.

[0131] In this embodiment of the application, multiple two-dimensional matching degree projection matrices are fused into a multi-channel pseudo-color image, including:

[0132] Step D1: Normalize the first two-dimensional matching degree projection matrix, the second two-dimensional matching degree projection matrix, and the third two-dimensional matching degree projection matrix respectively to obtain the first normalized matrix, the second normalized matrix, and the third normalized matrix.

[0133] Specifically, the first two-dimensional matching degree projection matrix This refers to a two-dimensional matrix of size n×n obtained by max-pooling projection along the perturbation dimension (w-axis) rotated along the heading angle, where its elements... This represents the value obtained by taking the maximum matching degree along the heading angle perturbation direction with the lateral perturbation index u and the longitudinal perturbation index v fixed. This matrix is ​​subsequently mapped to the R channel (red channel) of a pseudo-color image, representing the projection of the xy translational uncertainty surface; the second two-dimensional matching degree projection matrix. This refers to the two-dimensional matrix obtained by max-pooling projection along the horizontal translation perturbation dimension (u-axis), whose elements... This represents the value obtained by taking the maximum matching degree along the lateral disturbance direction with the longitudinal disturbance index v and the heading angle disturbance index w fixed. This matrix is ​​subsequently mapped to the G channel (green channel) of a pseudo-color image, representing the coupled projection of longitudinal displacement and heading angle; the third two-dimensional matching degree projection matrix This refers to the two-dimensional matrix obtained by max-pooling projection along the longitudinal translation perturbation dimension (v-axis), whose elements... This represents the value after taking the maximum matching degree along the longitudinal perturbation direction with the lateral perturbation index u and the heading angle perturbation index w fixed. This matrix is ​​subsequently mapped to the B channel (blue channel) of the pseudo-color image, representing the coupled projection of lateral displacement and heading angle. The normalization process linearly maps the element values ​​of each projection matrix from the original matching degree metric space (whose value range is constrained by the effective proportion of the laser point and is between 0 and 1) to the standard 8-bit image pixel value space (integers from 0 to 255), so that the pixel values ​​of each channel can be visualized and used as RGB components of the standard color image in subsequent steps for network input.

[0134] Obtain the first two-dimensional matching degree projection matrix Second two-dimensional matching degree projection matrix and the third two-dimensional matching degree projection matrix Perform normalization operations independently on each of these three matrices. (Using any channel matrix...) Taking (C∈{R,G,B}) as an example, first iterate through all n×n elements of the matrix and determine the maximum value among them. and minimum value Then, a linear mapping calculation is performed on each element in the matrix according to the following formula:

[0135]

[0136] Where (i,j) represent the row and column indices in the matrix. As a preset constant to prevent division by zero, `round(·)` represents the rounding operation. After this normalization process, the element values ​​of each of the three channel matrices are independently mapped to the integer range, resulting in the first normalized matrix. Second normalized matrix and the third normalized matrix Each normalization matrix has a size of 11×11, which prepares the numerical range for subsequent color channel mapping.

[0137] Step D2: Map the first normalized matrix, the second normalized matrix, and the third normalized matrix to the first color channel component, the second color channel component, and the third color channel component of the color image, respectively.

[0138] Specifically, a color channel component refers to a two-dimensional pixel value matrix of a specific color channel in a color digital image. Each channel component corresponds to the brightness value of each pixel in the image under that channel. The three channel components together constitute a complete color image. The first color channel component corresponds to the red channel (R channel), which is directly mapped from the first normalization matrix. The second color channel component corresponds to the green channel (G channel), which is directly mapped from the second normalization matrix. The third color channel component corresponds to the blue channel (B channel), which is directly mapped from the third normalization matrix.

[0139] The first normalized matrix is ​​used as the first color channel component (i.e., the red channel) of the color image. Each element of the matrix represents the brightness value of the pixel located in the i-th row and j-th column of the pseudo-color image in the red channel (0 represents no red component, and 255 represents the maximum red component). The second normalized matrix is ​​used as the second color channel component (i.e., the green channel) of the color image. Each element of the matrix represents the brightness value of the corresponding pixel in the green channel. The third normalized matrix is ​​used as the third color channel component (i.e., the blue channel) of the color image. Each element of the matrix represents the brightness value of the corresponding pixel in the blue channel. All three normalized matrices are 11×11 two-dimensional matrices with the same number of rows and columns. Therefore, they have a one-to-one correspondence in spatial position. That is, the three normalized values ​​located at the same index (i,j) jointly determine the final color of the pixel in the i-th row and j-th column of the pseudo-color image.

[0140] Step D3: The first color channel component, the second color channel component, and the third color channel component are fused along the channel dimension to generate a multi-channel pseudo-color image of the original size.

[0141] Specifically, channel dimension refers to the dimension used to distinguish different color components or feature channels in a digital image. In a multi-channel color image, the channel dimension organizes the three single-channel components of red, green, and blue into a unified data structure, so that the brightness values ​​of the three channels are stored at each spatial location simultaneously. A multi-channel pseudo-color image is a color image synthesized by assigning projection information with three different physical meanings to the three physical color channels (red, green, and blue). The color of each pixel does not reflect the true color of the scene, but rather encodes the spatial topological information of the matching degree tensor in the three orthogonal projection directions. The original size refers to the spatial resolution of the pseudo-color image before channel fusion, i.e., 11×11 pixels, corresponding to the number of sampling points n of the three-dimensional perturbation sampling grid in each dimension.

[0142] The first, second, and third color channel components are concatenated and fused along the channel dimension. Specifically, for a pixel at spatial location (i,j) (where i∈[1,11], j∈[1,11]), the luminance value for the red channel is extracted from the first color channel component, the luminance value for the green channel is extracted from the second color channel component, and the luminance value for the blue channel is extracted from the third color channel component. These three luminance values ​​are combined to form the final RGB color value of the pixel. After performing this operation on all 121 pixel locations, an original multi-channel pseudo-color image of size 11×11×3 is obtained (i.e., 11 rows, 11 columns, 3 channels). The red channel of this image encodes the maximum matching degree distribution on the lateral-longitudinal translational uncertainty surface, the green channel encodes the maximum matching degree distribution on the longitudinal-heading angle coupling surface, and the blue channel encodes the maximum matching degree distribution on the lateral-heading angle coupling surface.

[0143] In this embodiment of the application, after generating the original-size multi-channel pseudo-color image, the method further includes: obtaining the target size of the input layer of the two-dimensional convolutional neural network classification model; upsampling the original-size multi-channel pseudo-color image to generate a multi-channel pseudo-color image that conforms to the target size.

[0144] Specifically, after generating the original size pseudo-color image, the target size (128×128 pixels in this embodiment) of the input layer of the two-dimensional convolutional neural network classification model is obtained. The original 11×11×3 image is losslessly upsampled using a bilinear interpolation operator to generate a 128×128×3 multi-channel pseudo-color image that meets the target size, so as to satisfy the requirements of the convolutional neural network for the receptive field size of the input layer. Then, the normalized image is sent into the two-dimensional convolutional neural network classification model for online inference and judgment.

[0145] It should be noted that after generating the original-sized (11×11×3) multi-channel pseudo-color image, a bilinear interpolation operator is used to upsample the original image, enlarging its size from 11×11 to 128×128 to meet the input layer size requirements of the two-dimensional convolutional neural network classification model. Alternatively, other interpolation algorithms can be used to achieve the same upsampling function. For example, Nearest Neighbor Interpolation can be used to directly copy each original pixel to the neighboring region of the target image; this method is the fastest but produces jagged edges. Alternatively, Bicubic Interpolation can be used, which uses the 16 nearest neighbor pixels around the target pixel to perform cubic polynomial weighted interpolation, generating a smoother, higher-resolution image. Furthermore, as an alternative, the image upsampling step can be completely skipped. Three n×n (i.e., 11×11) channel matrices can be directly concatenated along the channel dimension and fed into an ultra-lightweight network (such as a multilayer perceptron (MLP) or a miniature fully connected network) specifically designed to receive extremely small input images for classification, without needing to enlarge the image size to 128×128 through interpolation. These alternatives are equivalent in achieving upsampling normalization or direct classification and all fall within the scope of this invention.

[0146] Step S105: Input the multi-channel pseudo-color image into the pre-trained two-dimensional convolutional neural network classification model for analysis to obtain the localization state classification result of the estimated pose.

[0147] In this embodiment, a multi-channel pseudo-color image is input into a pre-trained two-dimensional convolutional neural network classification model for analysis to obtain a localization state classification result with estimated pose, including:

[0148] Step E1: Input the multi-channel pseudo-color image into the two-dimensional convolutional neural network classification model. The two-dimensional convolutional neural network classification model extracts spatial features from the multi-channel pseudo-color image through multiple cascaded feature extraction modules to obtain a multi-scale topological feature map.

[0149] Specifically, a multi-channel pseudo-color image refers to a normalized pseudo-color image with a size of 128×128×3, whose three color channels encode the maximum matching degree projection distribution of the three-dimensional matching degree tensor in three orthogonal directions (xy translational uncertainty surface, yθ longitudinal-heading angle coupling surface, and xθ lateral-heading angle coupling surface); a two-dimensional convolutional neural network classification model refers to a lightweight two-dimensional convolutional neural network pre-deployed and trained at the edge, which extracts spatial features from the input image through multiple convolutional operations and finally outputs the probability distribution of the current pose belonging to different localization states; the feature extraction module refers to a module consisting of two-dimensional convolutional layers and batch normalization layers (Batch The basic network unit, consisting of a normalization layer, a ReLU activation function layer, and a max pooling layer cascaded in sequence, is used to extract spatial features at different levels of abstraction from the input image layer by layer. The multi-scale topological feature map refers to the multi-dimensional feature map tensor output by the last feature extraction module after being processed by multiple cascaded feature extraction modules. This feature map has been gradually compressed in the spatial dimension and gradually expanded in the channel dimension, condensing the spatial topological structure information (such as bright spot shape, ridge direction, peak shift, etc.) in the input image that is related to the localization state discrimination.

[0150] A 128×128×3 multi-channel pseudo-color image is fed into the input layer of a two-dimensional convolutional neural network (CNN) classification model. The image data then flows sequentially through multiple cascaded feature extraction modules for spatial feature extraction. In this embodiment, the specific architecture of the two-dimensional CNN classification model is configured as follows: The first feature extraction module uses 32 convolutional kernels of size 3×3 to perform two-dimensional spatial convolution, followed by a batch normalization layer, a ReLU activation function layer, and a 2×2 max pooling layer, at which point the feature map size is reduced to 64×64×32. The second feature extraction module uses 64 convolutional kernels of size 3×3 for convolution, followed by batch normalization, ReLU activation, and 2×2 max pooling, while adding 25% Dropout regularization to suppress overfitting, resulting in an output feature map size of 32×32×64. The third feature extraction module uses 128 convolutional kernels of size 3×3 for convolution, with the same structure as above, and after 2×2 max pooling, the output feature map size is 16×16×128. After layer-by-layer processing by the three cascaded feature extraction modules, a 16×16×128 multi-scale topological feature map is finally obtained. This feature map retains the spatial topological features in the input image that are highly related to the localization status (such as the symmetry of bright spots, the stretching direction of ridges, and the distance from the origin), providing rich feature basis for the subsequent discrimination decision of the fully connected classification module.

[0151] Step E2: Input the multi-scale topological feature map into the fully connected classification module of the two-dimensional convolutional neural network classification model. The fully connected classification module outputs the first confidence score of the estimated pose belonging to the normal positioning state and the second confidence score of the pose belonging to the positioning failure state.

[0152] Specifically, the fully connected classification module refers to the classifier part in a two-dimensional convolutional neural network classification model, located after the feature extraction module. It consists of a feature map flattening operation, a fully connected hidden layer, and an output layer, and is responsible for mapping the extracted multi-scale topological feature maps to the final classification probability output. The feature map flattening operation is the process of unfolding all elements of the multi-dimensional feature map tensor (16×16×128) into a one-dimensional vector (containing 32768 neurons), so that the feature data can be input into the fully connected layer for processing. The confidence score is the probability value between 0 and 1 output by the classification model, representing the current estimated pose. The confidence level of a specific positioning state is defined as follows: a higher value indicates a stronger certainty of the model regarding the classification result. A normal positioning state refers to a state where the deviation between the current estimated pose and the actual pose of the mobile robot is within an acceptable range, and the positioning system is functioning normally. A positioning failure state refers to a state where the deviation between the current estimated pose and the actual pose of the mobile robot exceeds a safety threshold, and the positioning system experiences positioning loss or drift failure. The first confidence level P0 represents the probability that the model determines the current estimated pose belongs to the normal positioning state; the second confidence level P1 represents the probability that the model determines the current estimated pose belongs to the positioning failure state, and satisfies P0 + P1 = 1. The preset detection threshold α is the alarm trigger threshold used to determine positioning failure, and in this embodiment, it is set to a conservative alarm threshold of 0.70.

[0153] A multi-scale topological feature map (16×16×128 pixels) is input into the fully connected classification module of a two-dimensional convolutional neural network classification model. First, the fully connected classification module flattens the feature map, transforming the 16×16×128 feature map into a one-dimensional tensor containing 16×16×128 = 32768 neurons. This one-dimensional tensor is then connected to a fully connected hidden layer with 128 neurons and Dropout regularization. After activation function processing, higher-order semantic features are further extracted. Finally, this 128-dimensional feature vector is input to a two-node output layer using the Softmax activation function. The Softmax function maps the output values ​​of the two nodes to a probability distribution, outputting a first confidence level P0 indicating the current estimated pose belongs to the normal localization state and a second confidence level P1 indicating the localization failure state, with the sum of the two always equal to 1. ).

[0154] Step E3: Based on the relative magnitudes of the first confidence level and the second confidence level, determine the localization state classification result of the estimated pose.

[0155] Specifically, the positioning status classification result is determined based on the relative magnitude of the first confidence level P0 and the second confidence level P1 output by the output layer: when P1≥α (i.e., the second confidence level is greater than or equal to the preset detection threshold α=0.70), the current estimated pose is determined to be in a positioning failure state and an "positioning failure" alarm signal is immediately triggered; otherwise, when P1<α, the current estimated pose is determined to be in a normal positioning state, and the positioning failure detection device continues to operate normally and monitor.

[0156] In this embodiment of the application, after obtaining the localization state classification result of the estimated pose, as follows: Figure 2 As shown, the method also includes:

[0157] Step S201: When the positioning state classification result indicates that the estimated pose is in a positioning failure state, the first level of safety control operation is executed. The first level of safety control operation includes: discarding the laser scanning data at the current moment, switching the positioning solution method to dead reckoning based on odometry data, and reducing the movement speed of the mobile robot.

[0158] In this embodiment, the positioning failure state refers to the state where the deviation between the current estimated pose and the true pose, as determined by the two-dimensional convolutional neural network classification model, exceeds a safety threshold, and the positioning system experiences positioning loss or drift failure. Laser scanning data refers to the laser scanning frame output in real time by the two-dimensional lidar mounted on the mobile robot body. This frame data consists of the polar coordinate ranging values ​​of H laser points. Odometry data refers to the relative motion estimation information jointly provided by the photoelectric encoder and inertial measurement unit (IMU) mounted on the mobile robot wheels, including the displacement increment and heading angle change of the mobile robot in a short period of time. Dead reckoning is a relative positioning method based on odometry data. It recursively estimates the current pose by accumulating the displacement increment and angle increment based on the known pose at the previous moment. Its cumulative error increases with time, but it can maintain trajectory continuity in a short period. Motion speed refers to the linear velocity control command value of the mobile robot, which determines the speed of the robot's movement at the current moment.

[0159] When the localization state classification result output by the 2D convolutional neural network classification model indicates that the current estimated pose is in a localization failure state (i.e., the second confidence level P1 ≥ the preset detection threshold α = 0.70), a "localization failure" alarm signal is triggered, and the safety degradation and relocation control module immediately initiates the first-level safety degradation protection operation. First, the laser scan data frame at the current moment is rejected and discarded to prevent abnormal matching points caused by wheel slippage or large-scale dynamic occlusion from polluting the global sub-map and to prevent erroneous data from being written to the map update module of the localization solver. Second, the localization solution method is temporarily downgraded from the current laser localization mode to high-frequency inertial dead reckoning based on wheel photoelectric encoders and IMU to maintain trajectory continuity for a short period of time and avoid loss of positioning leading to loss of motion control. At the same time, the motion controller forcibly reduces the maximum linear velocity of the mobile robot to 30% of the original limit speed and narrows the deceleration monitoring threshold of the safety obstacle avoidance sensor. By reducing the robot's movement speed, the emergency braking distance is shortened to prevent the risk of collision caused by localization anomalies. Once the above three protective measures are implemented simultaneously, the positioning status will continue to be monitored. If the positioning status classification result recovers to the normal positioning state in subsequent moments, the first level of security protection will be exited and the normal operation mode will be restored. If the positioning failure state continues, the second level of security control operation will be entered.

[0160] Step S202: If the cumulative duration of dead reckoning reaches a preset time threshold, then the second-level safety control operation is executed. The second-level safety control operation includes: acquiring visual image frames collected by the visual sensor on the mobile robot, matching and locating the visual image frames with the visual information in the pre-built map to obtain the visual positioning pose, and using the visual positioning pose to perform a correction operation on the cumulative deviation of dead reckoning.

[0161] In this embodiment, the cumulative duration of dead reckoning refers to the time elapsed from the start of switching from the positioning solution method to the dead reckoning mode based on odometry data to the current moment; the preset time threshold refers to a pre-set time threshold. When the time for maintaining positioning solely through dead reckoning exceeds this threshold, the pose reliability of dead reckoning will decrease to an unacceptable level due to the gradual increase in odometry cumulative error. At this point, it is necessary to introduce other sensor information for intervention and correction; the second-level safety control operation is an intermediate level of protection in the graded safety mechanism. It is activated when the first-level safety protection (discarding laser frames and switching dead reckoning) is insufficient to restore positioning reliability. It corrects the cumulative deviation by introducing visual sensor information; the visual sensor refers to an onboard camera (such as a monocular camera, binocular camera, or RGB-D camera) installed on the mobile robot body, used to collect visual image information of the robot's surrounding environment; the visual image frame refers to the two-dimensional image data output by the visual sensor at the current moment; Visual information in the pre-built map refers to standard visual feature information (such as landmark identifiers, natural road sign feature points, or QR code identifiers) corresponding to the global coordinate system, which is stored in the map database in advance and used for matching and positioning with real-time acquired visual image frames. Matching and positioning refers to the process of matching the visual features extracted from the visual image frames with the visual information in the pre-built map, and determining the robot's current absolute pose by solving the pose of the visual features in the global coordinate system. Visual positioning pose refers to the absolute pose estimate of the robot in the map coordinate system obtained by visual matching and positioning, represented by a homogeneous transformation matrix. Cumulative deviation refers to the deviation between the pose estimate and the true pose caused by factors such as wheel slippage, encoder quantization error, and IMU zero bias during the continuous operation of dead reckoning mode. This deviation accumulates and increases over time. Correction operation refers to the process of correcting the cumulative deviation of dead reckoning by using the absolute position constraints of the visual positioning pose, so that the positioning system reconverges to the correct pose.

[0162] First, continuously monitor the cumulative duration of the dead reckoning mode. When this cumulative duration reaches the system's preset time threshold (i.e., operation relying solely on dead reckoning has exceeded the safe duration), and the mobile robot is equipped with an onboard camera, initiate a second-level multi-sensor redundancy switching operation to implement fault-tolerant recovery. First, acquire the visual image frames collected by the mobile robot's visual sensors at the current moment, and obtain the corresponding pre-stored visual information (such as landmark identifiers, artificial road signs, or global coordinates of natural visual feature points) from the map database. Second, extract features from the visual image frames, match the extracted visual features with the visual information in the pre-built map, calculate the relative pose Tra of the visual identifier relative to the robot using a visual localization algorithm, and combine it with the global pose Twt of the visual identifier in the pre-built map, using the global pose transformation formula to perform coordinate transformation:

[0163]

[0164] Where Twt represents the robot's global pose in the map coordinate system obtained by visual localization, Twr represents the fixed pose of the visual identifier in the global map, Tra represents the relative pose of the visual identifier with respect to the robot, and Tat is the auxiliary transformation matrix. The final visual localization pose Twt serves as an absolute pose constraint.

[0165] Finally, the visual positioning pose is used to correct the accumulated deviation of the current dead reckoning: the visual positioning pose is input as an observation to the positioning solver, and the deviation between the visual positioning pose and the current dead reckoning pose is calculated and used as a correction to compensate for the accumulated error in the dead reckoning, so that the robot's pose estimation converges back to near the true pose, thereby achieving autonomous recovery of positioning under multi-sensor redundancy switching. If the correction is successful and the positioning status returns to normal, the safety protection mode is exited and laser positioning is restored; if the positioning failure status persists after correction or visual positioning fails, the third-level safety control operation is entered.

[0166] Step S203: If the cumulative drift calculated by dead reckoning is greater than or equal to a preset distance threshold, then execute the third-level safety control operation. The third-level safety control operation includes: controlling the mobile robot to decelerate and stop, and initiating a global relocation operation based on laser scanning data and a pre-built map.

[0167] In this embodiment, the cumulative drift of dead reckoning refers to the positional deviation of the dead reckoning pose relative to the last valid absolute positioning pose (i.e., the pose after laser positioning or visual correction before the switch) from the positioning solution method to the dead reckoning mode based on odometry data up to the current moment. This deviation reflects the cumulative effect of factors such as wheel slippage, encoder quantization error, and IMU zero bias over time. The preset distance threshold refers to the safety threshold of the cumulative drift amount preset by the system. In this embodiment, the threshold value is set to 1.5 meters. When the cumulative drift of dead reckoning reaches or exceeds this threshold, it indicates that the pose error of dead reckoning has far exceeded the acceptable range, and safe operation can no longer be maintained by dead reckoning alone. The third-level safety control operation is the lowest level in the graded safety mechanism. Other ultimate protection measures are activated when both the first level (discarding laser frames and switching dead reckoning) and the second level (visual correction) fail to restore positioning reliability. These measures achieve a final safety fallback through deceleration, shutdown, and global repositioning. Global repositioning refers to the process where, after the mobile robot has come to a stop, it directly registers and aligns the real-time scanning data collected by the LiDAR with a pre-built map in the global coordinate system. The robot's absolute pose in the global map is then redefined through search or optimization methods. Bitmap-based global repositioning refers to a repositioning method that directly aligns the laser point cloud with the grid map. By using strategies such as multi-resolution search or particle filtering, the robot finds the pose with the highest laser scan matching degree in the global map space as the repositioning result, allowing the robot to autonomously recover its positioning without human intervention.

[0168] First, continuously monitor the cumulative drift in the dead reckoning mode. If step S202 cannot be executed due to the lack of an onboard camera, or if the positioning failure persists after the visual correction operation in step S202 fails, then calculate the cumulative positional deviation between the dead reckoning pose and the last valid absolute positioning pose. When the cumulative drift Δd ≥ 1.5 m (preset distance threshold), it is determined that dead reckoning can no longer maintain a reliable positioning state, and the third-level ultimate protection operation is immediately executed: First, the motion controller controls the mobile robot to safely decelerate in place until it comes to a smooth stop, avoiding the risk of collision caused by continued movement in a severely positioning failure state, and at the same time reports the shutdown anomaly to the central dispatch system and records the fault context information; then, the system triggers the global repositioning mechanism in place at high frequency when the robot is stationary, and performs a bitmap-based global repositioning operation using the real-time laser scanning data collected by the lidar at the current moment and the pre-built map. Meanwhile, after successful relocation, the observation covariance matrix parameter of the Kalman filter in the positioning solver can be adaptively adjusted, or a regularization factor scale_factor can be used to limit the pose update amplitude along the degradation axis to prevent relocation from converging to a local optimum in a geometrically degraded environment. The global relocation mode is exited and normal laser positioning and motion control are restored until the positioning state classification result is restored to the normal positioning state and the relocation pose confidence meets the safety requirements. This allows for autonomous recovery of positioning without manual intervention to push the device back to the power-on origin.

[0169] In the embodiments of this application, such as Figure 3 As shown, the training method for a two-dimensional convolutional neural network classification model includes:

[0170] Step S301: Obtain the simulation dataset, which includes the simulation laser scan data, simulation odometry data, and simulation absolute true pose of the virtual robot at multiple moments in the simulation environment.

[0171] In this embodiment, the simulation dataset refers to the training data set collected in the virtual robot operating environment constructed using a physical simulation engine (such as NVIDIA Isaac Sim). This dataset includes simulated laser scanning data, simulated odometry data, and simulated absolute ground truth poses of the virtual robot at multiple time points. The simulated absolute ground truth poses are noise-free, true poses directly output by the physical simulator. It can be used as a reference benchmark for training label generation; simulated laser scanning data refers to the laser scanning frames output by the virtual two-dimensional lidar on the virtual robot in the simulation environment at a specific moment, and its data format is consistent with the lidar output on the actual robot; simulated odometer data refers to the relative motion estimation information output by the wheel encoder and IMU of the virtual robot in the simulation environment.

[0172] In physics simulation engines such as NVIDIA Isaac Sim, diverse industrial scenes, including degraded scenarios such as symmetrical corridors and empty warehouses, as well as highly dynamic environments, were reconstructed. Twenty-five spherical obstacles with random trajectory movements were configured to simulate dynamic pedestrian flow in a real environment. Subsequently, a virtual robot was controlled to perform multiple long-path navigation tasks. During the operation, simulated laser scanning data output by a virtual 2D LiDAR, simulated odometer data from the wheel encoder and IMU, and noiseless absolute true pose directly output by the physics simulator were recorded at a fixed frequency. , and construct a complete simulation dataset.

[0173] Step S302: The simulated odometer data and simulated laser scanning data are fused for positioning to obtain the simulated estimated pose at each time point.

[0174] In this embodiment, the simulated pose estimation refers to the estimated pose value of the virtual robot at each moment obtained after fusing simulated odometry data and simulated laser scanning data through a positioning and calculation module. The simulated odometry data and simulated laser scanning data recorded at each moment are input into the positioning and calculation module for fusion positioning to obtain the simulated pose estimation at each moment.

[0175] Step S303: Determine the localization status label corresponding to each time step based on the deviation between the simulated estimated pose and the corresponding simulated absolute true pose at each time step.

[0176] In this embodiment of the application, the positioning status label refers to the truth label automatically generated according to the preset judgment rules based on the deviation between the simulation estimated pose and the corresponding simulation absolute true pose, which indicates that the positioning status at that moment is either normal positioning (label value 0) or positioning failure (label value 1).

[0177] Based on the deviation between the simulated estimated pose and the corresponding simulated absolute true pose at each time step, the localization status label is determined according to the following automatic labeling logic:

[0178] Calculate the position drift:

[0179]

[0180] Calculate the direction deviation angle:

[0181]

[0182] When Δd≥0.1m or Δθ≥3.0°, the sample label corresponding to that moment is automatically defined as positioning failure (label value 1); other cases are defined as positioning normal (label value 0). At the same time, the proportion of failure samples is controlled at around 23.1% to ensure sample balance.

[0183] Step S304: Based on the simulated pose estimation and simulated laser scanning data at each time point, generate sample multi-channel pseudo-color images corresponding to each time point.

[0184] In this embodiment, the sample multi-channel pseudo-color image refers to a multi-channel pseudo-color image generated according to the simulated estimated pose and simulated laser scanning data at each time step, following a complete process, and used as the training input sample for the classification model. Based on the simulated estimated pose and corresponding simulated laser scanning data at each time step, following the complete process of steps S101 to S104 (including acquiring pose and laser data, constructing a perturbation sampling grid, calculating the matching degree tensor, three-dimensional tensor projection compression, and pseudo-color image generation and upsampling normalization), sample multi-channel pseudo-color images of 128×128×3 corresponding to each time step are generated.

[0185] Step S305: Utilize the mapping relationship between the positioning status labels at each time point and the sample multi-channel pseudo-color images to construct a training sample set, and use the training sample set to perform supervised training on the initial two-dimensional convolutional neural network classification model to obtain a pre-trained two-dimensional convolutional neural network classification model.

[0186] In this embodiment, the training sample set refers to a dataset used for supervised training, which consists of the mapping relationship between the positioning status labels corresponding to multiple time points and the sample multi-channel pseudo-color images; the initial two-dimensional convolutional neural network classification model refers to an untrained two-dimensional convolutional neural network structure, whose network architecture is the same as the inference model structure, and the network parameters are continuously updated through backpropagation during the training process.

[0187] A training sample set is constructed by utilizing the mapping relationship between the localization status labels at each time step and the multi-channel pseudo-color images of the samples. The cross-entropy loss function is used as the optimization objective, and the initial two-dimensional convolutional neural network classification model is trained end-to-end using the backpropagation algorithm. At the same time, real robot operation log data (including maps, laser scanning data, SLAM localization poses, and manually annotated localization status labels) is used as a supplementary dataset for model fine-tuning and generalization verification. Finally, a fully trained two-dimensional convolutional neural network classification model with extremely high robustness is obtained. This model can run continuously in real time in an edge embedded controller for online analysis and judgment.

[0188] Figure 4 This is a schematic diagram illustrating the algorithm flow and data conversion of the robot localization state analysis method provided in the embodiments of this application, as shown below. Figure 4As shown, the process includes: First, using the estimated pose (x, y, θ) at the current moment as the origin, active perturbation sampling is performed in the pose state space to construct a three-dimensional perturbation sampling mesh containing n×n×n sampling pose nodes; then, by performing multi-point reprojection similarity calculation between the laser point cloud and the pre-built map on each sampling pose node, a three-dimensional similarity volume (i.e., a three-dimensional matching tensor) is generated; this three-dimensional tensor is subjected to max pooling projection along three orthogonal axes: the R channel (xy translational uncertainty surface) is obtained by projection along the θ axis, the G channel (yθ longitudinal-heading angle coupling surface) is obtained by projection along the x axis, and the B channel (xθ transverse-heading angle coupling surface) is obtained by projection along the y axis, realizing the dimensionality reduction mapping of the three-dimensional cost volume to the two-dimensional multi-channel spatial bright spot and flat ridge topological features; then, the above three two-dimensional projection matrices are assembled and stitched into an RGB three-channel pseudo-color image, and the image size is upsampled and normalized to 128×128 pixels by bilinear interpolation; finally, the pseudo-color image is sent to a 2D... The CNN network extracts the topological features of uncertainty caused by positioning deviation in the image, and outputs the first confidence P (normal) of the current pose belonging to the normal positioning state and the second confidence P (abnormal) of the positioning failure state through fully connected layers and softmax layers. According to the decision judgment rules, it outputs positioning failure alarm to trigger hierarchical safety control operations such as shutdown degradation or abnormal protection.

[0189] Figure 5 This is a schematic diagram of the structure of the robot positioning state analysis system provided in the embodiments of this application, as shown below. Figure 5 As shown, the system includes: a 2D lidar (11), an odometer and IMU (12), a pre-built map database (13), a real-time pose estimation module (20), a positioning failure detection device (30), a 2D CNN positioning failure classifier (40), and a security degradation and relocation control module (50).

[0190] A 2D lidar (11) is installed on the mobile robot body to collect laser scanning data at the current moment and output it to the real-time pose estimation module and the positioning failure detection device.

[0191] The odometer and IMU (12) are installed on the mobile robot body and consist of a wheel photoelectric encoder and an inertial measurement unit. They are used to collect the current odometer status data and output it to the real-time pose estimation module.

[0192] A pre-built map database (13) is stored in a non-volatile medium to store pre-built grid maps, providing map data for the positioning calculation of the real-time pose estimation module and the matching degree calculation of the positioning failure detection device;

[0193] The real-time pose estimation module (20) is connected to the 2D LiDAR, odometer and IMU and the pre-built map database respectively. It is used to receive the laser scanning data and odometer status data at the current time, and solve them by fusion positioning algorithm (including adaptive Monte Carlo positioning AMCL, graph optimization method or extended Kalman filter EKF) to output the estimated pose of the mobile robot at the current time in the map coordinate system.

[0194] A positioning failure detection device (30) is connected to a real-time pose estimation module, a 2D lidar, and a pre-built map database. It receives the estimated pose, laser scanning data, and pre-built map data at the current moment to assess whether a positioning failure has occurred. The positioning failure detection device (30) specifically includes:

[0195] The pose 3D perturbation mesh construction module (31) is used to apply equally spaced perturbation deviations in the lateral translation dimension, longitudinal translation dimension and heading angle dimension with the estimated pose as the origin, and construct a 3D perturbation sampling mesh containing multiple sampling pose nodes.

[0196] The multidimensional matching degree reprojection calculation module (32) is used to project the laser scanning data onto the pre-built map for each sampling pose node in the perturbation sampling grid, calculate the matching degree metric value under each sampling pose node, and generate a three-dimensional matching degree tensor based on the matching degree metric value of all sampling pose nodes.

[0197] The RGB multi-channel projection mapping module (33) is used to perform maximum pooling compression projection on the three-dimensional matching degree tensor along three orthogonal directions to obtain three two-dimensional matching degree projection matrices.

[0198] The RGB pseudo-color image assembly module (34) is used to normalize the three two-dimensional matching degree projection matrices and map them into three color channel components. It then fuses the components in the channel dimension to generate a multi-channel pseudo-color image of the original size and generates a normalized multi-channel pseudo-color image that conforms to the input size of the classification model by upsampling.

[0199] A 2D CNN positioning failure classifier (40) is connected to a positioning failure detection device. It is used to receive a normalized multi-channel pseudo-color image, perform online reasoning through a pre-trained two-dimensional convolutional neural network classification model, output the first confidence level of the current estimated pose belonging to the normal positioning state and the second confidence level belonging to the positioning failure state, and determine the positioning state classification result based on the relative size relationship between the first confidence level and the second confidence level.

[0200] The safety degradation and relocation control module (50) is connected to the 2D CNN positioning failure classifier. When the positioning status classification result indicates positioning failure, it receives an abnormal alarm and performs hierarchical safety control operations, including: discarding the laser scanning data at the current moment and switching to dead reckoning based on odometry data, reducing the speed of movement, performing cumulative deviation correction based on visual sensor information, and controlling deceleration and stopping and initiating a global relocation operation based on laser scanning data and a pre-built map.

[0201] This embodiment also provides a robot positioning state analysis device, which is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0202] This embodiment provides a robot positioning status analysis device, such as... Figure 6 As shown, it includes:

[0203] The acquisition module 61 is used to acquire the estimated pose of the mobile robot at the current moment and the laser scanning data at the current moment;

[0204] Module 62 is used to perform multi-dimensional perturbation sampling in the pose state space of the mobile robot with the estimated pose as the origin, and to construct a perturbation sampling mesh containing multiple sampled pose nodes.

[0205] The generation module 63 is used to project laser scanning data onto a pre-built map for each sampling pose node in the perturbation sampling grid, calculate the matching degree metric value under the sampling pose node, and generate a three-dimensional matching degree tensor based on the matching degree metric values ​​of all sampling pose nodes.

[0206] The fusion module 64 is used to compress and project the three-dimensional matching degree tensor along multiple orthogonal projection directions to obtain multiple two-dimensional matching degree projection matrices, and then fuse the multiple two-dimensional matching degree projection matrices into a multi-channel pseudo-color image.

[0207] Analysis module 65 is used to input multi-channel pseudo-color images into a pre-trained two-dimensional convolutional neural network classification model for analysis, and obtain the localization state classification result of the estimated pose.

[0208] In this embodiment of the application, the construction module 62 is specifically used to apply translational perturbation deviations and rotational perturbation deviations to the translational dimension of the mobile robot, respectively, with the estimated pose as the origin, to generate multiple perturbation poses; and to construct a three-dimensional perturbation sampling mesh based on the deviation relationship between the estimated pose and each perturbation pose; wherein, the translational dimension includes the lateral translational dimension and the longitudinal translational dimension of the mobile robot.

[0209] In this embodiment, the generation module 63 is specifically used to obtain the laser coordinates of multiple laser points in the laser scanning data in the laser coordinate system; for each sampling pose node in the perturbation sampling grid, a rotation transformation matrix and a translation vector are constructed based on the pose information of the sampling pose node; based on the rotation transformation matrix and the translation vector, the laser coordinates of each laser point are converted into global projection coordinates in the global coordinate system of the pre-built map; the map occupancy information corresponding to each global projection coordinate in the pre-built map is determined, and the proportion of effective laser points that meet the preset matching conditions in each laser point is calculated based on the map occupancy information, and the proportion of effective laser points is used as the matching degree metric value under the sampling pose node.

[0210] In this embodiment of the application, the fusion module 64 is specifically used to obtain multiple dimensional axes corresponding to the three-dimensional matching degree tensor; perform max pooling projection on the three-dimensional matching degree tensor along the direction of the first dimensional axis among the multiple dimensional axes to obtain a first two-dimensional matching degree projection matrix; perform max pooling projection on the three-dimensional matching degree tensor along the direction of the second dimensional axis among the multiple dimensional axes to obtain a second two-dimensional matching degree projection matrix; and perform max pooling projection on the three-dimensional matching degree tensor along the direction of the third dimensional axis among the multiple dimensional axes to obtain a third two-dimensional matching degree projection matrix.

[0211] In this embodiment, the fusion module 64 is specifically used to normalize the first two-dimensional matching degree projection matrix, the second two-dimensional matching degree projection matrix, and the third two-dimensional matching degree projection matrix respectively to obtain a first normalized matrix, a second normalized matrix, and a third normalized matrix; map the first normalized matrix, the second normalized matrix, and the third normalized matrix to the first color channel component, the second color channel component, and the third color channel component of the color image respectively; and fuse the first color channel component, the second color channel component, and the third color channel component in the channel dimension to generate a multi-channel pseudo-color image of the original size.

[0212] In this embodiment of the application, the device further includes: an upsampling module, used to obtain a pre-set target size of the input layer of the two-dimensional convolutional neural network classification model; and to upsample the original size of the multi-channel pseudo-color image to generate a multi-channel pseudo-color image that conforms to the target size.

[0213] In this embodiment, the analysis module 65 is used to input a multi-channel pseudo-color image into a two-dimensional convolutional neural network classification model. The two-dimensional convolutional neural network classification model extracts spatial features from the multi-channel pseudo-color image through multiple cascaded feature extraction modules to obtain a multi-scale topological feature map. The multi-scale topological feature map is then input into the fully connected classification module of the two-dimensional convolutional neural network classification model. The fully connected classification module outputs a first confidence level for the estimated pose belonging to the normal positioning state and a second confidence level for the estimated pose belonging to the positioning failure state. Based on the relative magnitude relationship between the first confidence level and the second confidence level, the positioning state classification result of the estimated pose is determined.

[0214] In this embodiment, the apparatus further includes: a training module for acquiring a simulation dataset, wherein the simulation dataset includes simulated laser scanning data, simulated odometry data, and simulated absolute true pose of the virtual robot at multiple moments in the simulation environment; fusing the simulated odometry data and simulated laser scanning data to obtain the simulated estimated pose at each moment; determining the localization state label corresponding to each moment based on the deviation between the simulated estimated pose at each moment and the corresponding simulated absolute true pose; generating sample multi-channel pseudo-color images corresponding to each moment based on the simulated estimated pose and simulated laser scanning data at each moment; constructing a training sample set using the mapping relationship between the localization state label at each moment and the sample multi-channel pseudo-color images, and using the training sample set to perform supervised training on the initial two-dimensional convolutional neural network classification model to obtain a pre-trained two-dimensional convolutional neural network classification model.

[0215] In this embodiment, the device further includes: an execution module, configured to execute a first-level safety control operation when the positioning state classification result indicates that the estimated pose is in a positioning failure state. The first-level safety control operation includes: discarding the laser scanning data at the current moment, switching the positioning solution method to dead reckoning based on odometry data, and reducing the movement speed of the mobile robot; if the cumulative duration of dead reckoning reaches a preset time threshold, then executing a second-level safety control operation. The second-level safety control operation includes: acquiring visual image frames collected by the visual sensor on the mobile robot, matching and locating the visual image frames with visual information in a pre-built map to obtain a visual positioning pose, and using the visual positioning pose to perform a correction operation on the cumulative deviation of the dead reckoning; if the cumulative drift of the dead reckoning is greater than or equal to a preset distance threshold, then executing a third-level safety control operation. The third-level safety control operation includes: controlling the mobile robot to decelerate and stop, and initiating a global repositioning operation based on laser scanning data and a pre-built map.

[0216] Please see Figure 7 , Figure 7This is a schematic diagram of the structure of a computer device provided in an optional embodiment of the present invention, such as... Figure 7 As shown, the computer device includes one or more processors 10, memory 20, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components communicate with each other via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces). In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system).

[0217] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.

[0218] The memory 20 stores instructions executable by at least one processor 10 to cause at least one processor 10 to perform the method shown in the above embodiments.

[0219] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the computer device as shown by a landing page for an app. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, which can be connected to the computer device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0220] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0221] The computer device also includes a communication interface 30 for communicating with other devices or communication networks.

[0222] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the above embodiments.

[0223] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A method for analyzing the localization state of a robot, characterized in that, The method includes: Obtain the estimated pose of the mobile robot at the current moment and the laser scan data at the current moment; Using the estimated pose as the origin, multi-dimensional perturbation sampling is performed in the pose state space of the mobile robot to construct a perturbation sampling grid containing multiple sampled pose nodes. For each sampling pose node in the perturbation sampling grid, the laser scanning data is projected onto a pre-built map, the matching degree metric value under the sampling pose node is calculated, and a three-dimensional matching degree tensor is generated based on the matching degree metric values ​​of all sampling pose nodes. The three-dimensional matching degree tensor is compressed and projected along multiple orthogonal projection directions to obtain multiple two-dimensional matching degree projection matrices, and the multiple two-dimensional matching degree projection matrices are fused into a multi-channel pseudo-color image; The multi-channel pseudo-color image is input into a pre-trained two-dimensional convolutional neural network classification model for analysis to obtain the localization state classification result of the estimated pose.

2. The method according to claim 1, characterized in that, The step of performing multi-dimensional perturbation sampling in the pose state space of the mobile robot, using the estimated pose as the origin, and constructing a perturbation sampling mesh containing multiple sampled pose nodes, includes: Using the estimated pose as the origin, translational perturbation deviations are applied in the translational dimension of the mobile robot and rotational perturbation deviations are applied in the rotational dimension of the mobile robot to generate multiple perturbation poses. Based on the deviation relationship between the estimated pose and each of the perturbation poses, a three-dimensional perturbation sampling grid is constructed; The translation dimension includes the lateral translation dimension and the longitudinal translation dimension of the mobile robot.

3. The method according to claim 1, characterized in that, For each sampling pose node in the perturbation sampling grid, the laser scanning data is projected onto a pre-built map, and the matching degree metric value under the sampling pose node is calculated, including: Obtain the laser coordinates of multiple laser points in the laser scanning data in the laser coordinate system; For each sampling pose node in the perturbation sampling grid, a rotation transformation matrix and a translation vector are constructed based on the pose information of the sampling pose node; Based on the rotation transformation matrix and the translation vector, the laser coordinates of each laser point are converted into global projected coordinates in the global coordinate system of the pre-built map; Determine the map occupancy information corresponding to each of the global projection coordinates in the pre-built map, and calculate the proportion of effective laser points that meet the preset matching conditions among each of the laser points based on the map occupancy information. Use the proportion of effective laser points as the matching degree metric value under the sampling pose node.

4. The method according to claim 1, characterized in that, The three-dimensional matching degree tensor is compressed and projected along multiple orthogonal projection directions to obtain multiple two-dimensional matching degree projection matrices, including: Obtain the multiple dimensional axes corresponding to the three-dimensional matching degree tensor; Max pooling projection is performed on the three-dimensional matching degree tensor along the first dimension axis of multiple dimensions to obtain the first two-dimensional matching degree projection matrix; Max pooling projection is performed on the three-dimensional matching tensor along the direction of the second dimension axis among multiple dimension axes to obtain the second two-dimensional matching degree projection matrix; Max pooling projection is performed on the three-dimensional matching degree tensor along the direction of the third dimension axis among multiple dimensions to obtain the third two-dimensional matching degree projection matrix.

5. The method according to claim 4, characterized in that, The step of fusing the multiple two-dimensional matching degree projection matrices into a multi-channel pseudo-color image includes: The first two-dimensional matching degree projection matrix, the second two-dimensional matching degree projection matrix, and the third two-dimensional matching degree projection matrix are normalized respectively to obtain a first normalized matrix, a second normalized matrix, and a third normalized matrix. The first normalized matrix, the second normalized matrix, and the third normalized matrix are respectively mapped to the first color channel component, the second color channel component, and the third color channel component of the color image; The first color channel component, the second color channel component, and the third color channel component are fused along the channel dimension to generate a multi-channel pseudo-color image of the original size.

6. The method according to claim 5, characterized in that, After generating the original-size multi-channel pseudo-color image, the method further includes: Obtain the pre-defined target size of the input layer of the two-dimensional convolutional neural network classification model; The original multi-channel pseudo-color image is upsampled to generate a multi-channel pseudo-color image that conforms to the target size.

7. The method according to claim 1, characterized in that, The step of inputting the multi-channel pseudo-color image into a pre-trained two-dimensional convolutional neural network classification model for analysis to obtain the localization state classification result of the estimated pose includes: The multi-channel pseudo-color image is input into the two-dimensional convolutional neural network classification model. The two-dimensional convolutional neural network classification model extracts spatial features from the multi-channel pseudo-color image through multiple cascaded feature extraction modules to obtain a multi-scale topological feature map. The multi-scale topological feature map is input into the fully connected classification module of the two-dimensional convolutional neural network classification model, and the fully connected classification module outputs the first confidence level of the estimated pose belonging to the normal positioning state and the second confidence level of the pose belonging to the positioning failure state. The localization state classification result of the estimated pose is determined based on the relative magnitude relationship between the first confidence level and the second confidence level.

8. The method according to claim 1, characterized in that, The training method for the two-dimensional convolutional neural network classification model includes: Obtain a simulation dataset, wherein the simulation dataset includes simulated laser scanning data, simulated odometry data, and simulated absolute true pose of the virtual robot at multiple moments in the simulation environment; The simulated odometer data and the simulated laser scanning data are fused together for localization to obtain the simulated estimated pose at each time point; Based on the deviation between the simulated estimated pose and the corresponding simulated absolute true pose at each time step, the localization state label corresponding to each time step is determined. Based on the simulated estimated pose and the simulated laser scanning data at each time point, sample multi-channel pseudo-color images corresponding to each time point are generated. By utilizing the mapping relationship between the positioning status labels corresponding to each time point and the sample multi-channel pseudo-color image, a training sample set is constructed, and the initial two-dimensional convolutional neural network classification model is trained under supervision using the training sample set to obtain the pre-trained two-dimensional convolutional neural network classification model.

9. The method according to claim 1, characterized in that, After obtaining the localization state classification result of the estimated pose, the method further includes: When the positioning status classification result indicates that the estimated pose is in a positioning failure state, the first level of safety control operation is executed. The first level of safety control operation includes: discarding the laser scanning data at the current moment, switching the positioning solution method to dead reckoning based on odometry data, and reducing the movement speed of the mobile robot. If the cumulative duration of the dead reckoning reaches a preset time threshold, a second-level safety control operation is performed. The second-level safety control operation includes: acquiring visual image frames collected by the visual sensors on the mobile robot, matching and locating the visual image frames with the visual information in the pre-built map to obtain the visual positioning pose, and using the visual positioning pose to perform a correction operation on the cumulative deviation of the dead reckoning. If the cumulative drift calculated by dead reckoning is greater than or equal to a preset distance threshold, then a third-level safety control operation is executed. The third-level safety control operation includes: controlling the mobile robot to decelerate and stop, and initiating a global relocation operation based on the laser scanning data and the pre-built map.

10. A robot positioning state analysis device, characterized in that, The device includes: The acquisition module is used to acquire the estimated pose of the mobile robot at the current moment and the laser scan data at the current moment; The construction module is used to perform multi-dimensional perturbation sampling in the pose state space of the mobile robot with the estimated pose as the origin, and construct a perturbation sampling grid containing multiple sampled pose nodes. The generation module is used to project the laser scanning data onto a pre-built map for each sampling pose node in the perturbation sampling grid, calculate the matching degree metric value under the sampling pose node, and generate a three-dimensional matching degree tensor based on the matching degree metric values ​​of all sampling pose nodes. The fusion module is used to compress and project the three-dimensional matching degree tensor along multiple orthogonal projection directions to obtain multiple two-dimensional matching degree projection matrices, and fuse the multiple two-dimensional matching degree projection matrices into a multi-channel pseudo-color image. The analysis module is used to input the multi-channel pseudo-color image into a pre-trained two-dimensional convolutional neural network classification model for analysis, and obtain the localization state classification result of the estimated pose.

11. A computer device, characterized in that, include: A memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, the processor executing the computer instructions to perform the method of any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a computer to perform the method of any one of claims 1 to 9.

Images