Overlay detection method and map optimization method of mobile robot

Abstract

The invention discloses an overlay detection method and a map optimization method of a mobile robot. The overlay detection method comprises the following steps: acquiring an environment map constructed by the robot; detecting the area occupied by the neural network model and / or the track intersection point set of the robot on the environment map according to a pre-constructed overlay map, and judging whether the environment map generates the overlay map or not; when it is determined that the environment map constructed by the robot generates a stacked map, the environment map is disassembled into a plurality of sub-maps in an inverted sequence according to a robot pose sequence stored during construction; sequentially detecting whether each sub-graph in the plurality of sub-graphs generates a stacked graph or not; and removing the sub-graphs which generate the overlapped graph in the plurality of sub-graphs, and constructing a new map according to the remaining sub-graphs. By means of the method, the wrong map generating the overlapped map can be detected and optimized, the new map is constructed, the mobile robot can normally execute the cleaning task, and therefore the product experience of a user is improved.

Description

Technical Field

[0001] The present application belongs to the field of mobile robot mapping technology, and specifically relates to a mobile robot overlay detection method and a map optimization method. Background Art

[0002] Building a map of the current working environment of a mobile robot (such as a cleaning robot) is the basis for mobile robot navigation. Simultaneous Localization and Mapping (SLAM) is the most commonly used mapping technology. Robot wheel slippage, tilt, mirror refraction, etc. can cause mismatching in SLAM mapping.

[0003] In complex scenes with multiple obstacles, loop detection technology can be used to optimize the posture of the mobile robot, thereby reducing the mismatch rate of SLAM mapping in scenes with multiple obstacles. However, in scenes with a single obstacle, loop detection technology is ineffective. Due to the similarity of obstacle features, the mismatch rate of SLAM mapping is still very high.

[0004] At the same time, the existing SLAM mapping technology cannot detect the mismatch errors that occur during mapping, which will cause the mobile robot to continue mapping on the wrong map, eventually resulting in map overlap. When the map used by the mobile robot is an overlapped map, it will lose its own posture data and thus be unable to complete the cleaning task along the original route, causing problems in the use of mobile robot products and reducing the user's product experience. Summary of the invention

[0005] The present application provides a map overlay detection method and a map optimization method for a mobile robot to solve at least one of the above-mentioned technical problems.

[0006] The technical solution adopted in this application is:

[0007] A method for detecting overlapping images of a mobile robot, comprising:

[0008] Get the environment map constructed by the robot;

[0009] Whether the environmental map generates an overlay is determined based on a pre-constructed overlay detection neural network model and / or the area occupied by the robot's trajectory intersection set on the environmental map; wherein the overlay detection neural network model is obtained by training abnormal maps that generate overlays and normal maps that do not generate overlays, and the trajectory intersection set is a set of trajectory intersections generated by trajectory intersections in the robot's driving trajectory due to sudden changes in the robot's position.

[0010] In a possible implementation of the present application, judging whether the environment map generates an overlay according to a pre-built overlay detection neural network model and / or an area occupied by a set of trajectory intersections of the robot on the environment map includes:

[0011] Inputting the environment map into the overlay detection neural network model to determine whether a recognition result of the overlay detection neural network model is an overlay;

[0012] If yes, determining whether the environment map generates an overlay according to the area occupied by the set of trajectory intersections of the robot on the environment map;

[0013] or,

[0014] Determining whether there is an overlapping area in the environment map according to the area occupied by the set of trajectory intersections of the robot on the environment map;

[0015] If so, the overlay area is input into the overlay detection neural network model for identification, and it is determined whether the environment map generates an overlay based on the identification result.

[0016] In a possible implementation of the present application, before obtaining the environment map constructed by the robot, the method further includes:

[0017] In the pixel coordinate system, obtain the coordinates of each laser point in the current point cloud data that falls into the grid map;

[0018] Calculating the matching score between the coordinates corresponding to each laser point and the coordinates of the obstacle in the grid map;

[0019] When the average matching score of each laser point is higher than a preset score threshold, the current point cloud data is inserted into the grid map to complete the construction of the environment map.

[0020] In a possible implementation of the present application, the overlay detection neural network model includes 8 convolutional residual block structures and 1 fully connected layer, and the fully connected layer is used to output a binary classification result of the environment map, and the binary classification result includes an overlay image and a non-overlay image;

[0021] The activation function of the overlay image detection neural network model is implemented by using the relu function, and the loss function is implemented by using the binary cross entropy function.

[0022] In a possible implementation of the present application, judging whether the environment map generates an overlay according to the area occupied by the set of trajectory intersections of the robot on the environment map includes:

[0023] Get the robot's driving trajectory;

[0024] Determine a jump point on the robot's driving trajectory according to a pixel distance between adjacent trajectory points on the robot's driving trajectory;

[0025] Splitting the robot's driving trajectory into a first trajectory sequence and a second trajectory sequence through the jumping point;

[0026] Determining a set of trajectory intersection points between the first trajectory sequence and the second trajectory sequence;

[0027] Performing contour fitting on the trajectory intersection set along the outermost intersection points, and calculating the contour area obtained by fitting;

[0028] When the contour area is greater than a preset area threshold, it is determined that the environment map generates an overlay.

[0029] A map optimization method for a mobile robot, comprising:

[0030] Determine whether the environment map constructed by the robot produces overlapping images;

[0031] When it is determined that the environment map generates an overlay, the environment map is decomposed into a plurality of sub-maps in reverse order according to the robot posture sequence saved during construction;

[0032] sequentially detecting whether each of the plurality of sub-images generates overlapping images;

[0033] Sub-maps that produce overlapping images among the plurality of sub-maps are removed, and a new map is constructed according to the remaining sub-maps.

[0034] In a possible implementation of the present application, sequentially detecting whether each of the plurality of sub-images generates an overlapping image includes:

[0035] Traversing the plurality of subgraphs, extracting the longest straight line segment and the second longest straight line segment in the subgraphs;

[0036] Extending the longest straight line segment and the second longest straight line segment, and determining whether there is an angle between the extended lines of the two;

[0037] If so, when the angle of the included angle is greater than a first preset angle threshold, determining that the sub-image generates an overlapping image;

[0038] If not, the pixel distance between the longest straight line segment and the second longest straight line segment is calculated in the sub-image, and when the pixel distance is greater than a preset distance threshold, it is determined that the sub-image generates an overlapping image.

[0039] In a possible implementation of the present application, extracting the longest straight line segment and the second longest straight line segment in the subgraph includes:

[0040] Extracting a straight line segment in a sub-image whose length is greater than a preset length threshold and whose inclination angle with the horizontal direction or the vertical direction of the sub-image is greater than a second preset angle threshold;

[0041] The longest straight line segment and the second longest straight line segment among the straight line segments are selected.

[0042] In a possible implementation of the present application, removing the sub-images that generate overlapping images from the plurality of sub-images, and constructing a new map based on the remaining sub-images includes:

[0043] When it is detected that any sub-image generates overlapping images, the sub-image is removed from the plurality of sub-images;

[0044] When a preset number of consecutive sub-images in the remaining sub-images do not generate overlapping images, a new map is constructed using the remaining sub-images.

[0045] A map optimization method for a mobile robot, comprising:

[0046] During the movement of the robot, detecting whether an environment map constructed by the robot generates an overlapping image;

[0047] When the generation of the overlay image is detected, the environment map generating the overlay image is displayed to the user through the client of the mobile robot, and the mobile robot is controlled to return to the charging station;

[0048] After the mobile robot returns to the charging station, the map that generates the overlay is optimized to construct a new map;

[0049] The new map is presented to the user via the client of the mobile robot.

[0050] Due to the adoption of the above technical solution, the beneficial effects achieved by this application are as follows:

[0051] 1. After obtaining the original map constructed by the robot, the pre-built overlay detection neural network model and / or the area occupied by the robot trajectory intersection set on the original map will be used to determine whether the original map is an overlay. This allows the robot to detect the wrong map when an overlay occurs due to mismatching during map construction, thereby realizing overlay detection and avoiding the robot's failure to perform tasks on the overlay and reducing the user experience.

[0052] 2. When detecting robot overlays, you can first use the overlay detection neural network model for identification, and then use the area occupied by the robot track intersection set on the environment map to make a judgment, so as to improve the accuracy of overlay detection. You can also first use the area occupied by the robot track intersection set on the environment map to judge the overlay area on the environment map, and then use the overlay detection neural network model to identify the overlay area, which can reduce the amount of calculation of the overlay detection neural network model and improve the calculation speed, so as to obtain the overlay detection result faster. These two overlay detection schemes can complement each other, thereby improving the accuracy of overlay detection and reducing the probability of misidentification.

[0053] 3. When it is recognized that the robot map generates an overlay, the overlay can be reversely decomposed into several sub-images according to the posture data saved when the robot builds the map. This is because when an overlay is generated, it must be the subsequent sub-image that generates an error. After that, the overlay detection is performed on several sub-images, the sub-images that generate the overlay are removed, and the remaining sub-images are used to build a new map to achieve optimized processing of the overlay so that the robot can perform the cleaning task normally on the new map.

[0054] 4. After optimizing the original map that produces the overlay to obtain a new map, the new map will continue to be detected for overlays. If overlays are detected, the new map will continue to be optimized. The detection-optimization process will be repeated until the new map is obtained without any overlays, thereby ensuring that the robot can perform cleaning work on a normal map and improving the user experience.

[0055] 5. When constructing the environment map in the robot, the robot decides whether to insert the current point cloud data into the grid map based on the matching score between the coordinates of each laser point in the current point cloud data falling into the grid map and the coordinates of the existing obstacles in the grid map. When the matching score is low, the current point cloud data will not be inserted into the grid map. Only when the matching score between the coordinates is high, the current point cloud data will be inserted into the grid map to complete the construction of the environment map. In this construction scheme, noise points will get low scores or no scores, thereby avoiding the introduction of noise points into the grid map, which can improve the accuracy of the constructed environment map, so that the environment map can accurately reflect the current working environment of the robot, and the calculation process of calculating the matching score is short, thereby improving the efficiency of map construction. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0057] Figure 1 A flow chart of a method for detecting overlapping images of a mobile robot provided in an embodiment of the present application;

[0058] Figure 2 A schematic diagram of a method for calibrating a pitch angle of a single-line laser radar in an application scenario provided in an embodiment of the present application;

[0059] Figure 3 A schematic diagram of a method for calibrating the roll angle of a single-line laser radar in an application scenario provided in an embodiment of the present application;

[0060] Figure 4 A schematic diagram of a method for calibrating the yaw angle of a single-line laser radar in an application scenario provided in an embodiment of the present application;

[0061] Figure 5 A schematic diagram of three coordinate systems included in the mobile robot provided in the embodiment of the present application;

[0062] Figure 6 A posture graph of a mobile robot provided in an embodiment of the present application;

[0063] Figure 7 A posture diagram of another mobile robot provided in an embodiment of the present application;

[0064] Figure 8 A schematic diagram of a contour fitting process of a set of robot trajectory intersection points provided in an embodiment of the present application;

[0065] Fig. 9 A comparison diagram of a normal image and an overlay image in a robot SLAM mapping process provided in an embodiment of the present application;

[0066] Fig.10 A schematic diagram of a convolution residual block structure provided in an embodiment of the present application;

[0067] Fig.11 A schematic diagram of a neural network structure provided in an embodiment of the present application;

[0068] Fig.12 A flow chart of a map optimization method for a mobile robot provided in an embodiment of the present application;

[0069] Fig.13 A schematic diagram of an overlay optimization process provided in an embodiment of the present application;

[0070] Fig.14 A flowchart of another mobile robot map optimization method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0071] In order to more clearly illustrate the overall concept of the present application, a detailed description is given below in an illustrative manner in conjunction with the accompanying drawings.

[0072] In the following description, many specific details are set forth to facilitate a full understanding of the present application. However, the present application may also be implemented in other ways different from those described herein. Therefore, the scope of protection of the present application is not limited to the specific embodiments disclosed below.

[0073] In addition, in the description of the present application, it should be understood that the terms "top", "bottom", "inside", "outside", "axial", "radial", "circumferential", etc., indicating orientations or positional relationships are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0074] In this application, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection, an electrical connection, or a communication; it can be a direct connection, or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0075] In the present application, unless otherwise clearly specified and limited, the first feature "above" or "below" the second feature may be that the first and second features are in direct contact, or the first and second features are in indirect contact through an intermediate medium. In the description of this specification, the description with reference to the terms "implementation method", "example", "one embodiment", "example" or "specific example" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in an appropriate manner in any one or more embodiments or examples.

[0076] The image overlap detection method of a mobile robot proposed in the embodiments of the present application is explained below with reference to the accompanying drawings.

[0077] Figure 1 A flow chart of a method for detecting overlapping images of a mobile robot provided in an embodiment of the present application is as follows: Figure 1 As shown, the overlay detection method in the embodiment of the present application at least includes the following execution steps:

[0078] Step 101: Obtain an environment map constructed by the robot.

[0079] When a robot in an embodiment of the present application, such as a cleaning robot, performs a cleaning task, it will first construct a map of the robot's current working environment. For the convenience of distinction, the first map constructed by the robot in the embodiment of the present application is recorded as the original map. In an example of the present application, the process of the robot constructing the original map mainly includes the following contents:

[0080] First, calibrate the robot sensor and convert various coordinate systems:

[0081] The inertial measurement unit (IMU), lidar and other sensors on the robot are calibrated, the robot's prior posture is obtained based on the robot's sensors, and the conversion equations between the robot's plane coordinate system, the radar point cloud polar coordinate system, and the pixel plane coordinate system are established.

[0082] 1) IMU sensor parameter calibration

[0083] The purpose of IMU calibration is to eliminate the parameter errors of IMU, including zero bias α, scale factor β and other parameters. The IMU calibration process can be divided into two methods: static calibration and dynamic calibration. Static calibration is suitable for machines in a stationary state, and dynamic calibration is suitable for machines in a moving state. Because the parameter calibration of IMU involves the derivation of complex mathematical formulas, only a brief introduction is given in the embodiments of this application. Based on the dynamic calibration method, the IMU is fixed on a horizontally rotatable platform and rotated at different speeds v. i and direction g i Rotate n times and read the speed of the IMU measurement each time direction Readings, respectively calculate IMU measurement values With the platform setting value v i , g i The values ​​of parameters α and β when the difference is minimum are transformed into a nonlinear optimization problem, as shown in the following formula:

[0084]

[0085] The above equation can be solved by a nonlinear optimization solver to obtain α and β, thus completing the IMU calibration.

[0086] 2) LiDAR parameter calibration

[0087] LiDAR parameter calibration is divided into internal parameter calibration and external parameter calibration. The internal parameters of LiDAR have been calibrated before leaving the factory and can be used directly. The external parameter calibration of LiDAR is to calculate the difference between the polar coordinate system of the radar point cloud corresponding to the LiDAR and the robot's plane coordinate system, so as to obtain the LiDAR's position and posture data in the robot's plane coordinate system.

[0088] Coordinate system conversion can be viewed as a rotation and translation problem. The difference can be specifically divided into rotation angle error and translation error. The translation error can be measured by measuring tools such as rangefinders. For example, the distance from the installation center of the laser radar to the center of the robot structure can be measured. However, the rotation angle errors in the three directions of roll, pitch, and yaw cannot be directly measured by tools and need to be calibrated and calculated. The accuracy of roll, pitch, and yaw measurements determines the accuracy of the parameter θ for converting the polar coordinate system of the radar point cloud to the plane coordinate system of the robot.

[0089] ① Calibration of the pitch angle of a single-line laser radar:

[0090] Figure 2 A schematic diagram of a method for calibrating the pitch angle of a single-line laser radar in an application scenario provided in an embodiment of the present application. Figure 2 In the figure, 1 is the laser radar, and 2 is a calibration plate made of two different materials ( Figure 2 The right part shows the front view of the calibration plate 2, and the middle part shows the side view of the calibration plate 2). During calibration, the calibration plate 2 is placed in front of the laser radar 1. The straight line 4 represents the distance from the center of the laser radar 1 to the calibration plate 2, which is set to L. 1 , straight line 3 represents the distance obtained by the laser radar 1 test, set to L 2 . Then the absolute value of the pitch angle is Then, because the materials of the calibration plate 2 are different, the reflection value intensity of the laser radar can be used to determine which half of the calibration plate 2 the laser hits, thereby determining the positive or negative pitch angle. Finally, L 1 ,L 2 The deviation between them is the error to be eliminated.

[0091] ②Calibration of the roll angle of a single-line laser radar:

[0092] Figure 3 A schematic diagram of a method for calibrating the roll angle of a single-line laser radar in an application scenario provided in an embodiment of the present application. Figure 3 In the figure, 1 is a robot including a single-line laser radar, 3 is the x-axis of the robot's plane coordinate system, and during calibration, the robot 1 is placed facing a wall, and then the point cloud data collected by the robot 1 is fitted into a straight line 2. The angle between the x-axis 3 of the robot's plane coordinate system and the straight line 2 is the actual roll angle, and the deviation between the actual roll angle and the wall straight line is the error to be eliminated.

[0093] ③Calibration of the yaw angle of the single-line laser radar:

[0094] Figure 4 A schematic diagram of a method for calibrating the yaw angle of a single-line laser radar in an application scenario provided in an embodiment of the present application. Figure 4 In the figure, 1 is a robot with a single-line laser radar, and 2 is a column. During calibration, the robot 1 is facing a column 2. The laser radar on the robot 1 scans the column 2 to obtain a plane point cloud arc. The arc takes the straight line facing the column 2 as the origin 0, and the angles of the two end points of the arc are θ 1 ,θ 2 , the measured origin angle is but This is the error in the yaw angle that needs to be eliminated.

[0095] 3) Multi-coordinate system conversion

[0096] The robot's prior posture is obtained according to the robot sensor, and the conversion equations among the robot's plane coordinate system, the radar point cloud polar coordinate system, and the pixel plane coordinate system are established.

[0097] The main components of IMU sensors include gyroscopes and accelerometers. The gyroscope can obtain the angular velocity of each axis, and can calculate the rotation angle of the object in combination with time, while the accelerometer can obtain the acceleration of each axis; the wheel odometer has a built-in speed measurement unit, which can be used to measure the number of circles the wheel has turned in a certain period of time. When the wheel radius is known, the distance traveled can be calculated.

[0098] The robot's rotation angle δ is measured by the IMU sensor, and the wheel odometer is used to measure the robot's travel distance D. According to the conversion relationship between polar coordinates and rectangular coordinates, the robot's prior position in the robot's plane coordinate system can be calculated as (x, y, θ), where (x, y) is the robot's horizontal and vertical coordinates in the robot's plane coordinate system, and x = D*cosθ, y = D*sinθ, θ is the robot's orientation in the robot's plane coordinate system, which can be calculated based on the robot's rotation angle δ.

[0099] The mobile robot contains three coordinate systems, such as Figure 5 As shown in the figure, the three coordinate systems include: the robot plane coordinate system, which takes the working base station as the coordinate origin, including the X-axis, Y-axis and Z-axis; the radar point cloud polar coordinate system, which takes the position of the laser radar as the coordinate origin, and uses ρ and θ to represent the polar coordinates; the pixel plane coordinate system, which takes the upper left corner of the image as the origin (0, 0), the upper boundary of the image is the u-axis, and the left boundary is the v-axis.

[0100] The polar coordinates of the point cloud are point(ρ, θ), which are converted to the rectangular coordinate system as point(x, y, θ), where x = ρ*cosθ, y = ρ*sinθ, the robot plane coordinates are word(x, y), and the pixel plane coordinate system is (u, v); then, the conversion formula from the polar coordinate system of the radar point cloud to the robot plane coordinate system is:

[0101] world.x i=world.x i-1 +c i *point.x i -s i *point.y i

[0102] world.y i =world.x i-1 +s i *point.x i +c i *point.y i

[0103] c i = sin(θ i ), s i =cos(θ i )

[0104] The conversion formula from the robot plane coordinate system to the pixel plane coordinate system is:

[0105] u=(world.x-world.x c ) / σ+ε

[0106]

[0107] Among them, world.x c is the horizontal coordinate of the center point of the robot's plane coordinate system; σ is the pixel unit, and its value is 0.05 meters, that is, 1 pixel unit is 0.05 meters, but it can be adjusted according to actual needs.

[0108] Secondly, front-end matching and mapping are performed based on SLAM mapping technology:

[0109] Front-end matching is the main process of establishing a SLAM grid map. The matching algorithm used in the embodiment of the present application is mainly the grid 8-neighborhood point cloud matching method: the average matching score between the known obstacle and the radar point cloud in the pixel coordinate system is calculated, and the one with a high matching average score is inserted into the map. The matching here can be understood as matching the 8-neighborhood of the laser point and the obstacle point, that is, calculating whether the laser point exists in the 8-neighborhood of the obstacle point, and calculating different matching scores according to the distance between the laser point and the obstacle point when the laser point is in the 8-neighborhood of the obstacle point.

[0110] In a possible implementation of the embodiment of the present application, the main implementation process of the grid 8 neighborhood point cloud matching method is as follows: first, a pixel coordinate system is constructed in the grid map of the robot. After the construction is completed, each laser point in the current laser point cloud collected by the robot is placed in the aforementioned pixel coordinate system to obtain the coordinates of each laser point in the pixel coordinate system; secondly, in the pixel coordinate system corresponding to the grid map, the coordinates corresponding to the known obstacles are extracted; thereafter, the matching score between the coordinates of the known obstacles and the coordinates of each laser point is calculated. When the average matching score between the laser points is higher than the preset score threshold, the current point cloud data is inserted into the grid map. At the same time, when the average matching score is lower than the preset score threshold, the current point cloud data is discarded. It should be noted that the average matching score between the laser points can be obtained by first calculating the matching score of each laser point and then calculating the average score between all matching scores. The aforementioned process is performed for the point cloud data collected by the robot. After the robot has collected all the laser point clouds in the working environment, the construction of the environment map can be completed.

[0111] Furthermore, the above calculation of the matching score between the coordinates of the known obstacle and the coordinates of each laser point can be achieved by the following formula:

[0112]

[0113] Among them, (u z , v z ) is the coordinate of the laser point, (u p , v p ) is the coordinate of the obstacle point, and S is the matching score between the laser point and the obstacle point.

[0114] Furthermore, according to the above formula, the calculated matching scores between each laser point and the obstacle point can include the following three cases:

[0115] 1) At the laser point coordinates (u z , v z ) and the obstacle point coordinates (u p , v p ) completely overlap, the matching score between the laser point and the obstacle point is the first score, and the first score is 1.

[0116] 2) At the laser point coordinates (u z , v z ) is located at the obstacle point coordinate (u p , v p ), the matching score between the laser point and the obstacle point is the second score, and the first score is 0.5. It should be noted that the four neighborhoods here are the upper, lower, left, and right neighborhoods, which refer to the four adjacent grids corresponding to the upper, lower, left, and right of the obstacle point on the grid map.

[0117] 3) At the laser point coordinates (u z , v z ) is located at the obstacle point coordinate (u p , v p ), the matching score between the laser point and the obstacle point is the third score, and the third score is 0.293. It should be noted that the 8 neighborhoods here are the upper, lower, left, and right neighborhoods, as well as the upper left, lower left, upper right, and lower right neighborhoods, which refer to the 8 adjacent grids corresponding to the upper, lower, left, and right positions of the obstacle point on the grid map.

[0118] Therefore, in one example of the present application, the preset score threshold may be 0.4. That is, when the average matching score of each laser point in the current laser point cloud is greater than 0.4, the current point cloud data is inserted into the grid map, and when the average matching score is less than 0.4, the current grid map is discarded.

[0119] The 8-neighborhood point cloud matching method used in the embodiment of the present application has good robustness to the noise point cloud formed by mirror refraction. The noise will get a low score or no score during matching, which can avoid introducing noise point clouds into the raster map, thereby improving the mapping accuracy.

[0120] Finally, the back-end closed-loop detection is optimized based on SLAM mapping technology, which mainly includes the following execution processes:

[0121] 1)Store key laser frames

[0122] In the process of matching and building maps in the front end of the mobile robot, the image entropy can be used to describe the change in map size. If the image entropy changes greatly, the current laser frame data is stored, otherwise it is not stored. The image entropy is defined as:

[0123]

[0124] Among them, H is the image entropy, P i is the pixel ratio of gray value i in the image. It should be noted that image entropy is a statistical form of image features that can reflect the average amount of information in the image.

[0125] 2) Create a pose graph

[0126] Let the pose of the mobile robot when detecting the key frame be P(x, y, θ). Then, a pose graph can be established based on the poses P1 and P2 of the previous and next key frames. The pose P(x, y, θ) is the graph node, and the difference between adjacent poses is the graph edge. Figure 6 A posture graph of a mobile robot provided in an embodiment of the present application, Figure 6In , P1, P2, P3 and P4 are graph nodes, and P1, P2, P3 and P4 are the poses corresponding to the adjacent key frames in the process of mobile robot mapping. Moreover, C1, C2 and C3 can represent the difference between the adjacent poses of the mobile robot. Figure 6 In the figure, it is represented as a graph edge.

[0127] 3) Posture closed-loop detection

[0128] Figure 7 Another posture graph of a mobile robot provided in an embodiment of the present application, Figure 7 In the figure, the initial keyframe pose of the robot is P1, and the final keyframe pose is P7. If it is detected that the distance between P1 and P7 is close, and the laser points at P1 and P7 match well with the obstacle points, and the matching score is high, it is considered that the pose points P1, P2, P3, P4, P5, P6 and P7 and their connecting edges can form a closed loop.

[0129] 4) Optimize robot posture

[0130] From the above content, the cumulative error of the robot's posture can be calculated as:

[0131] error=C1+C2+C3+C4+C5+C6>0

[0132] When a closed loop is detected, the robot's starting position P1 is equal to the final position P7, and the following equation holds:

[0133] error=P7-P1>0

[0134] Loop=P7-P1=0

[0135] At this time, the tool library g2o can be used to iteratively solve the problem. There exist P1′, P2′, P3′, P4′, P5′, P6′, P7′ such that error=P7-P1=0. At this time, the graph nodes in the aforementioned pose graph are updated to P1′, P2′, P3′, P4′, P5′, P6′, P7′, and the optimization process of the robot pose is completed.

[0136] At this point, the process of building the robot's environment map is completed and the environment map is obtained.

[0137] Step 102: Determine whether the environment map generates an overlay based on a pre-built overlay detection neural network model and / or the area occupied by the robot's trajectory intersection set on the environment map.

[0138] In one embodiment, it is possible to determine whether the environmental map generates overlays only based on a pre-constructed overlay detection neural network model. After acquiring the environmental map constructed by the robot, the acquired environmental map can be input into a pre-trained overlay detection neural network model. The training process of the overlay detection neural network model can be found in the relevant description below, and the embodiments of the present application will not be repeated here. Obtain the classification result corresponding to the environmental map output by the overlay detection neural network model, where the classification result includes overlays and non-overlays. When the classification result output by the overlay detection neural network model is an overlay, it is determined that the environmental map generates overlays; when the classification result output by the overlay detection neural network model is non-overlays, it is determined that the environmental map does not generate overlays.

[0139] In one embodiment, it is possible to determine whether an overlay is generated in the environment map only based on the area occupied by the robot's trajectory intersection set on the environment map. The specific execution process of determining an overlay using the area occupied by the robot's trajectory intersection set on the environment map can refer to the relevant description in the following embodiment, which will not be repeated here.

[0140] In one embodiment, it is possible to comprehensively determine whether an overlay is generated in the environment map based on a pre-built overlay detection neural network model and the area occupied by the set of trajectory intersections of the robot on the environment map.

[0141] Method 1: First use the overlay detection neural network model to perform overlay recognition, and then use the area occupied by the robot's trajectory intersection set on the environment map to make overlay judgment.

[0142] Specifically, the acquired environment map is first input into a pre-trained overlay detection neural network model. The training process of the overlay detection neural network model can be found in the relevant description below, and the embodiments of the present application will not be described in detail here. The classification result corresponding to the environment map output by the overlay detection neural network model is obtained. The classification result here includes overlays and non-overlaps. If the output result of the overlay detection neural network model is non-overlaps, it can be considered that there is no overlay in the environment map, and there is no need to make further judgment through the trajectory intersection set.

[0143] If the output result of the overlay detection neural network model is an overlay, it is necessary to perform trajectory analysis on the robot again, and use the area occupied by the robot's trajectory intersection set on the environmental map to further determine whether there is an overlay. If it is determined to be an overlay based on the trajectory intersection set, it can be determined that the environmental map is indeed an overlay; if it is determined not to be an overlay based on the trajectory intersection set, it can be considered that the overlay detection neural network model has made a misjudgment, and it is determined that the environmental map is not an overlay, or the environmental map can be regarded as a suspected overlay map, and the environmental map is re-detected under preset conditions. The preset conditions may include the robot walking a preset distance, or after a preset time, or the map change exceeds a preset area, etc. This application does not limit this.

[0144] In a possible implementation of the embodiment of the present application, the robot is subjected to trajectory analysis, and the overlay is determined by using the area occupied by the set of trajectory intersections of the robot on the environment map, which specifically includes the following execution process:

[0145] 1) The trajectory sequence formed by each trajectory point of the robot on the environment map is {t 1 , t 2 ,…,t n}, find the jump point t in the trajectory sequence i , then, based on the jump point t i The aforementioned trajectory sequence {t 1 , t 2 ,…,t n} is split into {t 1 , t 2 ,…,t i} and {t i+1 , t i+2 ,…,t n}.

[0146] In an example of the present application, when the robot is cleaning normally, the track points on the environment map are continuous and uninterrupted, that is, the spacing between each track point is 0 pixel, and when the robot has a positioning error, the coordinates of the next track point will deviate from the previous track sequence. Therefore, in an embodiment of the present application, the pixel distance between adjacent track points in the track sequence can be traversed and calculated, and when the pixel distance is greater than a preset distance threshold, the track point is determined as a jump point. The aforementioned preset distance threshold can be 5 pixels.

[0147] 2) Calculate the two trajectory sequences {t 1 , t 2 ,…,t i} and {t i+1 , t i+2 ,…,t n} is the set of trajectory intersection points between them.

[0148] In an example of the present application, when the robot jumps its trajectory, the robot will suddenly change its position. In this way, there will be at least one intersection point on the map between the newly generated erroneous trajectory sequence and the original normal trajectory sequence. Therefore, the number of intersection points between the two trajectory sequences can be determined by calculating the pixel distance between the corresponding trajectory points on the two segmented trajectory sequences, thereby establishing a trajectory intersection point set.

[0149] It should be noted that the pixel distance between the corresponding trajectory points on the two trajectory sequences can be expressed as follows:

[0150] {d(t 1 , t i+1 ), d(t 1 , t i+2 ),…,d(t 2 , t i+1 ), d(t 2 , t i+2 ),…,d(t i , t i+1 ), d(t i , t i+2 )}

[0151] For example, the trajectory point t 1 With t i+1 is a pair of corresponding trajectory points, the pixel distance d(t 1 , t i+1 )=0, the intersection count is increased by one, and the trajectory point t 1 Add to the trajectory intersection collection.

[0152] 3) Performing iconographic polygonal contour fitting on the aforementioned trajectory intersection set along the outermost trajectory points, that is, marking each trajectory point in the aforementioned trajectory intersection set on the environment map, and then fitting the contour along the outermost trajectory points, with an error of 1 pixel during fitting.

[0153] 4) Calculate the ratio of the contour area obtained by fitting to the area of ​​the environment map. If the contour area obtained by fitting is larger than a preset area threshold, where the preset area threshold can be 1 / 4 of the environment map area, then it is determined that the environment map produces an overlay. Otherwise, it is determined that the environment map does not produce an overlay.

[0154] Figure 8 A schematic diagram of a contour fitting process of a robot trajectory intersection set provided in an embodiment of the present application. Figure 8 It can be seen that the robot has two trajectory sequences {t 1 , t 2 ,…,t i} and {t i+1 , ti+2 ,…,t n}, shown in two rectangular boxes respectively, these two trajectory sequences start from t i Start segmentation, and there are multiple trajectory intersections between these two trajectory sequences, that is, Figure 8 The black dots in the rectangular box are shown in the figure. By fitting the contours of these trajectory intersections, we can get Figure 8 The fitted contour is shown on the right.

[0155] Method 2: First, use the area occupied by the robot's trajectory intersection set on the environment map to identify the overlapping area in the environment map, and then use the overlapping detection neural network model to identify the overlapping area.

[0156] Specifically, the robot is first analyzed for its trajectory on the environment map, and the area occupied by the robot's trajectory intersection set on the environment map is used to detect whether there is an overlay area on the environment map. It should be noted that the robot trajectory analysis process here is the same or similar to the robot trajectory analysis process described in the above embodiment, and the embodiment of the present application is not repeated here. If it is detected that there is no overlay area on the environment map based on the trajectory intersection set, it can be considered that there is no overlay in the environment map, and there is no need to input the environment map into the overlay detection neural network model for detection.

[0157] If it is detected that there is an overlapping area on the environmental map based on the trajectory intersection set, then after removing the non-overlapping area on the environmental map, the remaining part of the environmental map (at least including the aforementioned overlapping area) is input into the overlay detection neural network model for identification, and further determines whether the initial overlay generates an overlay. If the output classification result of the overlay detection neural network model is an overlay, it is determined that the environmental map generates an overlay. If the output classification result of the overlay detection neural network model is not an overlap, it can be considered that the previous detection based on the trajectory intersection set produced a misjudgment, and it is determined that the environmental map does not generate an overlay; alternatively, the complete environmental map without removing the non-overlapping area can be input into the overlay detection neural network model for secondary detection; alternatively, the environmental map can be re-detected for overlays under preset conditions, and the preset conditions can include the robot walking a preset distance, or after a preset time, or the map change exceeds a preset area, etc. This application does not impose any restrictions on this.

[0158] Method 3: Detect the environment map based on the pre-built overlay detection neural network model and the area occupied by the robot's trajectory intersection set on the environment map at the same time. Determine the first overlay detection result based on the area occupied by the robot's trajectory intersection set on the environment map; input the overlay area into the overlay detection neural network model for identification, and determine the second overlay detection result based on the identification result; determine whether the environment map generates an overlay based on the first detection result and the second detection result.

[0159] In this embodiment, the confidence level can be set in advance for the first detection result and the second detection result. If both the first detection result and the second detection result show that the environment map generates an overlay, it can be determined that the environment map does generate an overlay. If both the first detection result and the second detection result show that the environment map does not generate an overlay, it can be determined that the environment map does not generate an overlay. If the first detection result and the second detection result are inconsistent, the detection result with a higher confidence level can be used as the final detection result.

[0160] In a possible implementation method of an embodiment of the present application, whether it is the above-mentioned method 1, method 2 or method 3, if it is detected that the map is not overlapped (the non-overlapping result here can be the final detection result of method 1, method 2, and method 3, or it can be the intermediate detection result of a certain detection step executed in method 1 and / or method 2, for example, the detection result of the overlay detection neural network model in method 1 is not overlapped), then it means that the map at this time is normal and no subsequent optimization is required. At this time, the robot will continue to execute the normal SLAM mapping process.

[0161] The above two algorithms for detecting overlays (overlay detection neural network model and robot trajectory analysis) in the embodiment of the present application can compensate for each other and eliminate errors, thereby improving the accuracy of identifying overlays and reducing the probability of misidentification. By arranging the execution order of the above two algorithms, detection schemes with different effects can be generated, namely the above-mentioned method 1 and method 2: the recognition accuracy of the overlay detection neural network model is high, but there are many parameters and the resource consumption is large. Therefore, the overlay detection neural network model is first executed to preliminarily screen and identify overlays, and then the robot trajectory analysis is executed to further judge the overlays. This will obtain higher accuracy, but will cause greater computing consumption, making the operation slower; if the robot trajectory analysis is first executed to preliminarily screen and identify overlays, and a part of the map data without overlays is excluded, the computing consumption will be reduced due to the reduction of data when the overlay detection neural network model is executed later, and the operation is faster, but the recognition accuracy is greatly affected by the initial screening of the robot trajectory analysis, which will lose part of the recognition accuracy.

[0162] In a possible implementation of the embodiment of the present application, whether it is method 1 or method 2, before using the overlay detection neural network model, it is necessary to first construct and train the overlay detection neural network model, which mainly includes the following execution process:

[0163] 1) Constructing an overlay dataset

[0164] The robot was controlled to run 305 times in the experimental field, and 4030 images were collected. The data set was constructed according to the ratio of training set to validation set of 5:1, the ratio of overlay image to normal image in the training set of 1:2, and the ratio of overlay image to normal image in the validation set of 1:2: the training set has 3300 images, including 1100 overlay images and 2200 normal images; the validation set has a total of 730 images, including 500 normal images and 230 overlay images.

[0165] Fig. 9 A comparison diagram of a normal image and an overlay image in a robot SLAM mapping process provided in an embodiment of the present application, Fig. 9 In the figure, the left side is the normal image in the SLAM mapping process, and the right side is the overlay image generated in the SLAM mapping process. That is, the normal image in the aforementioned data set is as follows: Fig. 9 As shown on the left, the overlay image is Fig. 9 Shown on the right.

[0166] 2) Constructing a neural network model

[0167] In the embodiment of the present application, a neural network model is built based on a convolutional residual block structure, and the convolutional residual block structure used is as follows: Fig.10 shown. Fig.10 A schematic diagram of a convolution residual block structure provided in an embodiment of the present application, in Fig.10 In the figure, X represents the input image, which can be converted into 64-dimensional data after the convolution operation Conv(X); Conv(64-d,3,3,64-d) represents the input data dimension 64-d, the convolution kernel size (3,3), and the output data dimension 64-d; Represents the second convolution value Conv(64-d,3,3,64-d) added to the input value Conv(x); the expression of the activation function relu(x) is as follows:

[0168]

[0169] Considering the real-time performance and computing consumption issues, the lightweight network design principle is adopted, that is, the number of convolution operation layers is small and the data dimension is small. The final neural network model structure is as follows: Fig.11 shown.

[0170] Fig.11 A schematic diagram of a neural network structure provided in an embodiment of the present application, in Fig.11 In the example, the input X is a 3D SLAM image, and F(·) is a residual block structure operation. Fig.11 It can be seen that the neural network model in the embodiment of the present application adopts 8 convolution residual block structures; FC(·) is a fully connected layer, and its expression is as follows:

[0171]

[0172] Among them, n takes the value of 2, corresponding to the two categories: overlay image and non-overlay image; w i With b i Represents the weight matrix and bias term of the i-th neural node.

[0173] 3) Define the loss function

[0174] Define the category corresponding to the overlay image as 1, and the category corresponding to the non-overlay image, that is, the normal image, as 0. The loss function loss is defined based on the binary cross entropy function as follows:

[0175]

[0176] Among them, M is 2, y i,c represents the category of the i-th sample image (value is 0 or 1), p(x) i,c Represents the category probability value predicted by the neural network for the i sample images.

[0177] 4) Training the Neural Network

[0178] Based on the least squares method, the loss function loss in step ③ is iteratively minimized, so that the distribution of the predicted value p(x) is infinitely close to the distribution of the true value y(x), which can be expressed as solving the parameter set w, b that minimizes the loss:

[0179]

[0180] After the training is completed, the aforementioned overlay image detection neural network model can be obtained.

[0181] Fig.12 A flow chart of a map optimization method for a mobile robot provided in an embodiment of the present application, such as Fig.12 As shown, the map optimization method in the embodiment of the present application at least includes the following execution steps:

[0182] Step 1201, determining whether the environment map constructed by the robot generates an overlay;

[0183] Among them, judging whether the environment map constructed by the robot generates an overlay can be performed as described above. Figure 1 The implementation of the related embodiments of the method for detecting overlapping images of the mobile robot shown is not described in detail here.

[0184] Step 1202: When it is determined that the environment map constructed by the robot generates an overlay, the environment map is decomposed into a plurality of sub-maps in reverse order according to the robot posture sequence saved during construction.

[0185] After determining that the robot's environment map generates an overlay image through the method 1 and / or the method 2 described in the above embodiment, the overlay image is optimized. Specifically, when generating the overlay image, the environment map is first decomposed into several sub-images in reverse order according to the posture sequence saved by the robot when building the map. The reverse order decomposition is selected here because when the overlay image is generated, the subsequent sub-images Figure 1 It must be wrong, you can test it first.

[0186] In one example of the present application, when the environment map is disassembled in reverse order, it can be disassembled according to the posture sequence of the robot cleaning: Assuming that the posture sequence of the robot is {p 1 ,p 2 ,…,p n-1 ,p n}, when disassembling, 5 consecutive positions can be taken {p n ,p n-1 ,p n-2 ,p n-3 ,p n-4}Build the first sub-graph, and then build a sub-graph for every 5 poses in reverse order to obtain several sub-graphs; it can also be disassembled according to the number of point cloud frames: according to the point cloud frames retained during map construction, build a sub-graph for every 20 frames of point cloud in reverse order to obtain several sub-graphs.

[0187] Step 1203: sequentially detect whether each of the plurality of sub-images has overlapping images.

[0188] After obtaining a plurality of sub-images, an overlap detection is performed on each of the plurality of sub-images to determine whether the disassembled sub-images generate overlaps.

[0189] Specifically, for each sub-image, firstly, an image algorithm is used to extract the straight line segments contained in the sub-image. The image algorithm used here can be implemented by existing technologies or algorithms, and the embodiments of the present application will not be described in detail. The straight line segments whose length is greater than a preset length threshold (which can be set to 1m) and whose inclination angle with the horizontal direction or vertical direction of the sub-image is greater than a first preset angle threshold (which can be set to 10°) are screened out; then, the longest straight line segment and the second longest straight line segment among the screened straight line segments are extended, and it is determined whether there is an angle between the extended lines of the two. If there is, the angle between the two is calculated, and when the angle is greater than a second preset angle threshold (which can be set to 15°), the sub-image is determined to be an overlapping image; if there is no angle between the extended lines of the longest straight line segment and the second longest straight line segment, or if there is an angle but the angle is less than the second preset angle threshold, the overlapping image can be determined by calculating the pixel distance between the longest straight line segment and the second longest straight line segment. If the actual distance corresponding to the pixel distance between the two (the actual distance can be calculated by the resolution of the sub-image and the pixel distance) is greater than the preset distance threshold, it is considered that the sub-image generates overlapping images. In an example of the present application, the aforementioned preset distance threshold may be 15 m. This is because the robot's working environment is generally a room environment, but the north-south or east-west distance of a single room generally does not exceed 15 m.

[0190] Of course, whether each sub-image generates an overlapping image can also be determined by the above Figure 1 The overlay detection method shown is implemented, and this application does not impose any limitation on this.

[0191] Step 1204: remove the sub-images that produce overlapping images from the plurality of sub-images, and construct a new map based on the remaining sub-images.

[0192] After determining the sub-images that generate overlapping images among the several sub-images, the sub-images that generate overlapping images will be eliminated from the several sub-images, and then the remaining sub-images will be used to construct a new map to complete the overlapping image optimization process of the environment map. In an example of the present application, after obtaining the new map, the overlapping image detection process described in the aforementioned step 102 will continue to be performed on the new map. If it is detected that the new map has overlapping images, the new map will continue to be optimized. That is, in the embodiment of the present application, the overlapping image detection and optimization process of the map will be repeatedly executed until the new map obtained does not generate overlapping images, thereby ensuring that the robot can perform the cleaning task normally and improving the user's product experience.

[0193] In a possible implementation of an embodiment of the present application, when a new map is constructed using the remaining sub-graphs, there is a requirement for the remaining sub-graphs, requiring that a consecutive number of sub-graphs do not produce overlapping images. Only when this requirement is met, the remaining sub-graphs are used to construct a new map, so as to avoid constructing a new map using sub-graphs with overlapping images, thereby reducing the probability of overlapping images appearing again in the constructed new map.

[0194] Fig.13 A schematic diagram of an overlay optimization process provided in an embodiment of the present application, in Fig.13 In the process, the erroneous map that produces overlays will be decomposed into N consecutive sub-maps in reverse order. After that, the overlay detection will be performed on these N consecutive sub-maps respectively to determine whether the sub-maps produce overlays. Finally, the sub-maps that do not produce overlays are used to construct a new map to complete the overlay optimization process.

[0195] The map optimization method provided in the embodiment of the present application can make up for the defects of the current SLAM system (including two modules: a laser front-end matching module and a back-end robot posture optimization module. The laser front-end matching module is mainly responsible for establishing the surrounding environment map perceived by the robot, while the back-end robot posture optimization module optimizes the robot posture through loop detection technology to further eliminate the drift and error of the map) when the environment is single. It is unable to detect the correctness of map construction and eliminate the defect of erroneous sub-maps. It can realize self-detection and optimization processing of erroneous maps that have already overlapped, so that the robot can continue to perform cleaning tasks, thereby improving user experience.

[0196] Fig.14 A flowchart of another mobile robot map optimization method provided in an embodiment of the present application, such as Fig.14 As shown, the map optimization method in the embodiment of the present application at least includes the following execution steps:

[0197] Step 1401: During the movement of the robot, detect whether the map constructed by the robot generates overlapping images.

[0198] The robot can detect the current environment and build or update the map in real time through the lidar during movement. The detection of whether the map built by the robot has overlapping images can be done as mentioned above. Figure 1 The implementation of the embodiment related to the dripping detection method of the mobile robot shown is not repeated here.

[0199] Step 1402: When it is detected that an overlay map is generated, the map generating the overlay map is displayed to the user through the client of the mobile robot, and the mobile robot is controlled to return to the charging station.

[0200] Since the robot will lose its position data when the map is overlaid and will not be able to complete the cleaning task according to the original route, the robot can be controlled to return to the charging station based on the map that generates the overlay. During this process, since the client of the mobile robot can display the robot's map to the user in real time, the user can find through the client that the map has an overlay and that some areas in the map displayed by the client do not match the actual environment.

[0201] In one example of the present application, if the robot cannot accurately navigate back to the charging pile due to map overlay, the robot will first navigate to the vicinity of the charging pile, and then enter the wall driving model, controlling the robot to drive along the wall and search for the charging pile until it returns to the charging pile.

[0202] Step 1403: After the mobile robot returns to the charging station, the map that generates the overlay is optimized to construct a new map.

[0203] After the mobile robot returns to the charging station, the robot can optimize the map that produces the overlay into a new map without the overlay by using the map optimization method of the mobile robot. Fig.12 The corresponding embodiments will not be described in detail here.

[0204] Step 1404: Display the new map to the user through the client of the mobile robot.

[0205] After the mobile robot optimizes the map that produces the overlay to a new map without the overlay, the new map can be displayed to the user through the client of the mobile robot, and the user can see that the map has changed through the client.

[0206] That is, those skilled in the art can understand that if during the movement of the robot, the user discovers through the mobile robot client that the robot's map is superimposed, and after the robot returns to the charging station, the user discovers through the mobile robot client that the superimposed map displayed has changed into an optimized correct map, then it can be considered that the mobile robot has applied this solution and is considered to have infringed.

[0207] Anything not described in this application can be achieved by adopting or drawing on existing technologies.

[0208] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

[0209] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A method for detecting overlapping images of a mobile robot, comprising: Get the environment map constructed by the robot; It is characterized in that the method also includes: judging whether the environmental map generates overlays based on a pre-constructed overlay detection neural network model and / or the area occupied by the robot's trajectory intersection set on the environmental map; wherein the overlay detection neural network model is obtained by training abnormal maps that generate overlays and normal maps that do not generate overlays, and the trajectory intersection set is a set of trajectory intersections generated by trajectory intersections in the robot's driving trajectory due to sudden changes in the robot's position.

2. The method for detecting overlapping images of a mobile robot according to claim 1, characterized in that: Judging whether the environment map generates an overlay according to a pre-built overlay detection neural network model and an area occupied by a set of trajectory intersections of the robot on the environment map includes: Inputting the environment map into the overlay detection neural network model to determine whether a recognition result of the overlay detection neural network model is an overlay; If yes, determining whether the environment map generates an overlay according to the area occupied by the set of trajectory intersections of the robot on the environment map; or, Determining whether there is an overlapping area in the environment map according to the area occupied by the set of trajectory intersections of the robot on the environment map; If yes, the overlay area is input into the overlay detection neural network model for identification, and whether the environment map generates an overlay is determined according to the identification result; or, Determining a first stack detection result according to an area occupied by a set of trajectory intersections of the robot on the environment map; Inputting the overlay image area into the overlay image detection neural network model for identification, and determining a second overlay image detection result according to the identification result; It is determined whether an overlapping image is generated in the environment map according to the first detection result and the second detection result.

3. The method for detecting overlapping images of a mobile robot according to claim 1, characterized in that: Before obtaining the environment map constructed by the robot, the method further includes: In the pixel coordinate system, obtain the coordinates of each laser point in the current point cloud data that falls into the grid map; Calculating the matching score between the coordinates corresponding to each laser point and the coordinates of the obstacle in the grid map; When the average matching score of each laser point is higher than a preset score threshold, the current point cloud data is inserted into the grid map to complete the construction of the environment map.

4. The method for detecting overlapping images of a mobile robot according to claim 1, characterized in that: The overlay detection neural network model includes 8 convolutional residual block structures and 1 fully connected layer, and the fully connected layer is used to output a binary classification result of the environment map, and the binary classification result includes an overlay image and an unoverlay image; The activation function of the overlay image detection neural network model is implemented by using the relu function, and the loss function is implemented by using the binary cross entropy function.

5. The method for detecting overlapping images of a mobile robot according to claim 1, characterized in that: Judging whether the environment map generates an overlay according to the area occupied by the set of trajectory intersections of the robot on the environment map comprises: Get the robot's driving trajectory; Determine a jump point on the robot's driving trajectory according to a pixel distance between adjacent trajectory points on the robot's driving trajectory; Splitting the robot's driving trajectory into a first trajectory sequence and a second trajectory sequence through the jumping point; Determining a set of trajectory intersection points between the first trajectory sequence and the second trajectory sequence; Performing contour fitting on the trajectory intersection set along the outermost intersection points, and calculating the contour area obtained by fitting; When the contour area is greater than a preset area threshold, it is determined that the environment map generates an overlay.

6. A map optimization method for a mobile robot, characterized in that: The method comprises: Determine whether the environment map constructed by the robot produces overlapping images; When it is determined that the environment map generates an overlay, the environment map is decomposed into a plurality of sub-maps in reverse order according to the robot posture sequence saved during construction; sequentially detecting whether each of the plurality of sub-images generates overlapping images; Sub-maps that produce overlapping images among the plurality of sub-maps are removed, and a new map is constructed according to the remaining sub-maps.

7. A map optimization method for a mobile robot according to claim 6, characterized in that: The step of sequentially detecting whether overlapping images are generated in each of the plurality of sub-images comprises: Traversing the plurality of subgraphs, extracting the longest straight line segment and the second longest straight line segment in the subgraphs; Extending the longest straight line segment and the second longest straight line segment, and determining whether there is an angle between the extended lines of the two; If so, when the angle of the included angle is greater than a first preset angle threshold, determining that the sub-image generates an overlapping image; If not, the pixel distance between the longest straight line segment and the second longest straight line segment is calculated in the sub-image, and when the pixel distance is greater than a preset distance threshold, it is determined that the sub-image generates an overlapping image.

8. A map optimization method for a mobile robot according to claim 7, characterized in that: Extracting the longest straight line segment and the second longest straight line segment in a subgraph includes: Extracting a straight line segment in a sub-image whose length is greater than a preset length threshold and whose inclination angle with the horizontal direction or the vertical direction of the sub-image is greater than a second preset angle threshold; The longest straight line segment and the second longest straight line segment among the straight line segments are selected.

9. The map optimization method for a mobile robot according to claim 6, characterized in that: Eliminating the sub-images that produce overlapping images among the plurality of sub-images, and constructing a new map based on the remaining sub-images includes: When it is detected that any sub-image generates overlapping images, the sub-image is removed from the plurality of sub-images; When a preset number of consecutive sub-images in the remaining sub-images do not generate overlapping images, a new map is constructed using the remaining sub-images.

10. A map optimization method for a mobile robot, characterized in that: The method comprises: During the movement of the robot, detecting whether an environment map constructed by the robot generates an overlapping image; When the generation of the overlay image is detected, the environment map generating the overlay image is displayed to the user through the client of the mobile robot, and the mobile robot is controlled to return to the charging station; After the mobile robot returns to the charging station, the map that generates the overlay is optimized to construct a new map; The new map is presented to the user via the client of the mobile robot.

Images