Detection method and device for mobile robot assembly

Abstract

The invention discloses a detection method for a mobile robot component, and the method comprises the steps: collecting the detection data of a mobile robot body in a detection region in a detection motion execution process, and enabling the detection region to comprise a positioning region used for detecting a navigation positioning component of the mobile robot, the service area is used for detecting a service component for executing services of the mobile robot, and the obstacle avoidance area is used for detecting an obstacle avoidance component of the mobile robot, and the detection actions comprise a first detection action for acquiring detection data of a navigation positioning component, a second detection action for acquiring detection data of the service component and a third detection action for acquiring detection data of the obstacle avoidance component. And the third detection module is used for acquiring at least one of third detection actions of detection data of the obstacle avoidance module, the detection data comprises report data of sensors in modules installed on the mobile robot body, and at least one of a navigation positioning module, a service module and the obstacle avoidance module of the mobile robot is detected by using the acquired detection data. According to the invention, comprehensive detection of the component is realized.

Description

Technical Field

[0001] The present invention relates to the field of robots, and in particular to a method for detecting components of a mobile robot. Background Art

[0002] With the development of technology, more and more robots are being used in automation fields such as logistics and warehousing. Different business applications require different robot functions. Taking mobile robots as an example, a mobile robot system includes at least positioning, control, business functions, and obstacle avoidance. The implementation of these functions depends on components installed on the mobile robot body, such as the navigation and positioning component for navigation and positioning, the obstacle avoidance component for obstacle avoidance, and the business components for business execution.

[0003] As mobile robots operate for a long time, the maintenance and upkeep of their components becomes a necessity. Existing maintenance methods for mobile robots usually involve testing the sensors installed on the robot body one by one. This method is not only inefficient but also lacks comprehensiveness in the detection of mobile robots. Summary of the Invention

[0004] The present invention provides a method for detecting components of a mobile robot, so as to improve the comprehensiveness of the detection of the mobile robot.

[0005] A first aspect of the present invention provides a method for detecting a mobile robot component, the method comprising:

[0006] Collect detection data of the mobile robot body during the detection action in the detection area,

[0007] The detection area includes at least one of: a positioning area for detecting a navigation and positioning component of a mobile robot, a service area for detecting a service component of a mobile robot that performs services, and an obstacle avoidance area for detecting an obstacle avoidance component of a mobile robot.

[0008] The detection action includes at least one of: a first detection action for obtaining detection data of a navigation and positioning component, a second detection action for obtaining detection data of a business component, and a third detection action for obtaining detection data of an obstacle avoidance component.

[0009] The detection data includes: the reported data of each sensor installed in each component of the mobile robot body,

[0010] The collected detection data is used to detect at least one of the navigation and positioning component, the business component, and the obstacle avoidance component of the mobile robot.

[0011] Preferably, the positioning area includes: a positioning mark for identifying the actual position information of each position on the set detection path,

[0012] The first detection action at least includes: a movement detection action of moving a set first distance along a set detection path, and a rotation detection action of rotating at a set first rotation angle.

[0013] The detection data at least includes: synchronous reporting data of each sensor in the navigation and positioning component installed on the mobile robot body,

[0014] The detection of the navigation and positioning component includes:

[0015] Using the identified location information to obtain verification data, and based on the detection data collected in the positioning area, detecting the navigation and positioning component of the mobile robot;

[0016] in,

[0017] The detection data collected during the execution of the mobile detection action is used to detect the position accuracy of the navigation and positioning component and / or the external reference positioning posture of the navigation and positioning component.

[0018] The detection data collected during the execution of the rotation detection action is used to detect the angular accuracy of the navigation and positioning component.

[0019] Preferably, the business area includes a shelf, and the bottom surface of the shelf has a cargo code mark for business component detection, and the cargo code mark includes actual global position information of the shelf;

[0020] The second detection action includes: rotating at a set second rotation angle under the shelf and performing a business operation action to observe the cargo code identification;

[0021] The detection data includes: synchronous reporting data of each sensor in the business component installed in the mobile robot body;

[0022] The detection of the business component includes:

[0023] Use the global position information in the cargo code identification to obtain verification data, and detect the business components of the mobile robot based on the detection data collected in the business area;

[0024] in,

[0025] The detection data collected during the execution of the second detection action is used to detect at least one of the external parameter positioning accuracy and the business operation accuracy of the business component.

[0026] Preferably, the obstacle avoidance area includes: an obstacle facade;

[0027] The third detection action includes: a detection action of collecting image data of obstacle facade information at a set obstacle detection position;

[0028] The detection data includes: synchronous reporting data of each sensor in the obstacle avoidance component installed in the mobile robot body;

[0029] The detection of the obstacle avoidance component includes:

[0030] Obstacle detection location information is used to obtain verification data, and the obstacle avoidance component of the mobile robot is tested based on the detection data collected in the obstacle avoidance area;

[0031] in,

[0032] The detection data collected during the execution of the third detection action is used to detect at least one of the installation posture accuracy and the ranging accuracy of the obstacle avoidance component;

[0033] The service area, positioning area, and obstacle avoidance area are adjacent in sequence.

[0034] Preferably, the detection path is a straight path; the positioning marks are distributed on the straight path at equal intervals; the navigation and positioning component includes: at least one of a visual module for visual navigation and positioning, an odometer, and an inertial measurement unit;

[0035] The detection data includes: at least one of image data, odometer data, and inertial measurement unit data for visual navigation positioning;

[0036] The detecting of the navigation and positioning component of the mobile robot includes at least one of the following:

[0037] Based on the image data in the detection data, the first verification data is obtained using the position information or movement trajectory of the positioning mark to detect the positioning posture of the external reference mark of the visual module.

[0038] Based on the odometer data and image data in the detection data, the second verification data is obtained using the position information of the marker, and the odometer is tested, the test including: at least one of the odometer rotation angle accuracy test and the odometer position accuracy test,

[0039] Based on the inertial measurement unit data and the image data in the detection data, the third verification data is obtained using the position information of the marker to detect the heading angle accuracy of the inertial measurement unit.

[0040] Preferably, the visual module includes a first laser radar for laser navigation and positioning;

[0041] The image data is the first laser point cloud image data reported by the first laser radar,

[0042] The detection of the external reference positioning posture of the visual module includes: at least one of the position calibration detection of the first laser radar and the posture calibration detection,

[0043] in,

[0044] Position calibration detection includes:

[0045] Based on the point cloud image data of the positioning marker included in the first laser point cloud image data, the observed global position information of the positioning marker is determined,

[0046] Calculating the position standard deviation between the observed global positions of all positioning markers and the actual position information of all positioning markers, wherein the actual position information of all positioning markers is used as the first verification data,

[0047] Determine whether the position standard deviation is less than a set position standard deviation threshold; if so, determine that the position calibration of the first laser radar is accurate; otherwise, determine that the position calibration of the first laser radar is inaccurate;

[0048] Attitude calibration detection includes:

[0049] Determine the moving trajectory of the mobile robot based on the navigation positioning information determined by the laser point cloud image data reported by the first laser radar,

[0050] Intercept the approximate straight line trajectory data of the mobile robot's moving trajectory,

[0051] Based on the intercepted trajectory data, a straight line is fitted to obtain a fitting straight line.

[0052] Calculate the linear angle of the fitted line in the world coordinate system to obtain the trajectory angle, which is used as the first calibration data.

[0053] Select any trajectory point in the intercepted trajectory data, and obtain the observed global posture information of the trajectory point based on the navigation positioning information of the trajectory point.

[0054] Determine whether the angle error between the observed global posture information of the trajectory point and the trajectory angle is less than the set angle error threshold,

[0055] If so, it is determined that the attitude calibration of the first laser radar is accurate; otherwise, it is determined that the attitude calibration of the first laser radar is inaccurate.

[0056] Preferably, the positioning mark is a downward-looking code located in a bearing surface for supporting the mobile robot, and the visual module includes a downward-looking camera for reading the downward-looking code, and the downward-looking camera is installed at the bottom of the mobile robot body;

[0057] The image data is downward-viewing coded image data reported by the downward-viewing camera;

[0058] The detection of the external reference positioning posture of the visual module includes: position calibration detection of the downward-looking camera,

[0059] in,

[0060] Position calibration detection includes:

[0061] For each frame of the lower view code image data collected during the movement along the set detection path, it is identified whether the frame contains the lower view code image.

[0062] If yes, then

[0063] Determine the observed global position information of the lower view code corresponding to the lower view code image in the frame, and obtain the scanning lateral deviation distance and forward deviation distance of the lower view code, wherein the scanning lateral deviation distance is used to characterize the distance between the observed global position of the lower view code and the actual position of the lower view code, and the actual position of the lower view code is used as the first verification data. The forward deviation distance is used to characterize the distance error between the distance between the observed global positions of the two lower view codes and the distance between the actual positions of the two lower view codes, and the distance between the actual positions of the two lower view codes is used as the first verification data.

[0064] Calculate the average lateral deviation distance and the average forward deviation distance of all scanned codes.

[0065] Otherwise, the navigation positioning information of the frame is determined based on the frame, and the distance between the position of the navigation positioning information and the detection path at the position is calculated to obtain the code-free deviation distance.

[0066] Determine whether the average value of the lateral deviation distance of the code scanning is less than the set lateral deviation distance threshold of the code scanning, whether the average value of the forward deviation distance is less than the set forward deviation distance threshold, and whether the no-code deviation distance is less than the set no-code deviation distance threshold.

[0067] If so, it is determined that the downward-looking camera extrinsic calibration is accurate; otherwise, it is determined that the downward-looking camera extrinsic calibration is inaccurate.

[0068] Preferably, the positioning mark is a downward-looking code located on the bearing surface for supporting the mobile robot, and the visual module includes a downward-looking camera for reading the downward-looking code and a first laser radar for laser navigation and positioning.

[0069] The image data is downward-looking code image data reported by the downward-looking camera and first laser point cloud image data reported by the first laser radar;

[0070] The detecting of the external reference positioning posture of the visual module includes: at least one of position calibration detection of the first laser radar, attitude calibration detection, and position calibration detection of the downward-looking camera;

[0071] in,

[0072] The first laser radar position calibration detection includes:

[0073] For each frame of the lower view code image data, the global position information of the lower view code is determined according to the local position of the lower view code corresponding to the lower view code image in the frame in the mobile robot coordinate system and the navigation positioning information determined based on the first laser point cloud image data.

[0074] Calculate the position standard deviation between the observed global position of all the lower view codes and the actual position information of all the lower view codes, wherein the actual position information of all the lower view codes is used as the first verification data,

[0075] Determine whether the position standard deviation is less than a set position standard deviation threshold; if so, determine that the position calibration of the first laser radar is accurate; otherwise, determine that the position calibration of the first laser radar is inaccurate;

[0076] The attitude calibration detection of the first laser radar includes:

[0077] Determine the moving trajectory of the mobile robot based on the navigation positioning information determined by the laser point cloud image data reported by the first laser radar,

[0078] Intercept the approximate straight line trajectory data of the mobile robot's moving trajectory,

[0079] Based on the intercepted trajectory data, a straight line is fitted to obtain a fitting straight line.

[0080] Calculate the linear angle of the fitted line in the world coordinate system to obtain the trajectory angle, which is used as the first calibration data.

[0081] Select any trajectory point in the intercepted trajectory data, and obtain the observed global posture information of the trajectory point based on the navigation positioning information of the trajectory point.

[0082] Determine whether the angle error between the observed global posture information of the trajectory point and the trajectory angle is less than the set angle error threshold,

[0083] If yes, it is determined that the attitude calibration of the first laser radar is accurate; otherwise, it is determined that the attitude calibration of the first laser radar is inaccurate;

[0084] The downward-looking camera position calibration test includes:

[0085] For each frame of the lower view code image data collected during the movement along the set detection path, it is identified whether the frame contains the lower view code image.

[0086] If yes, then

[0087] Determine the observed global position information of the lower view code corresponding to the lower view code image in the frame, and obtain the scanning lateral deviation distance and forward deviation distance of the lower view code, wherein the scanning lateral deviation distance is used to characterize the distance between the observed global position of the lower view code and the actual position of the lower view code, and the actual position of the lower view code is used as the first verification data. The forward deviation distance is used to characterize the distance error between the distance between the observed global positions of the two lower view codes and the distance between the actual positions of the two lower view codes, and the distance between the actual positions of the two lower view codes is used as the first verification data.

[0088] Calculate the average lateral deviation distance and the average forward deviation distance of all scanned codes.

[0089] Otherwise, the navigation positioning information of the frame is determined based on the frame, and the distance between the position of the navigation positioning information and the detection path at the position is calculated to obtain the code-free deviation distance.

[0090] Determine whether the average value of the lateral deviation distance of the code scanning is less than the set lateral deviation distance threshold of the code scanning, whether the average value of the forward deviation distance is less than the set forward deviation distance threshold, and whether the no-code deviation distance is less than the set no-code deviation distance threshold.

[0091] If so, it is determined that the downward-looking camera extrinsic calibration is accurate; otherwise, it is determined that the downward-looking camera extrinsic calibration is inaccurate.

[0092] Preferably, the detecting of the navigation and positioning component of the mobile robot further comprises: detecting the recognition rate of the downward-looking camera based on the downward-looking code image frame data,

[0093] The downward-looking camera recognition rate test includes:

[0094] Based on the lower view code image frame data, identify the lower view code image in each frame, and count the number of different lower view codes.

[0095] Check whether the number of down-view codes counted is the same as the actual number of down-view codes,

[0096] If yes, then the recognition rate of the camera is determined to be accurate.

[0097] Otherwise, it is determined that the recognition rate of the lower view code camera is inaccurate.

[0098] Preferably, the odometer rotation angle accuracy detection includes:

[0099] Based on any two odometer data collected during the same direction rotation process, the odometer cumulative angle change of the two odometer data is determined.

[0100] Based on the navigation positioning information determined during the rotation process, two navigation positioning information having the same time information as the two odometer data are obtained, and an angle change is determined based on the two navigation positioning information, and the angle change is used as the second verification data.

[0101] Calculating a first deviation between the odometer cumulative angle change and the second verification data,

[0102] determining whether the first deviation value is less than a set first deviation threshold; if so, determining that the odometer rotation angle is accurate; otherwise, determining that the odometer rotation angle is inaccurate;

[0103] The odometer position accuracy detection includes:

[0104] Based on any two odometer data collected during the straight path movement of the mobile robot, the odometer cumulative distance of the two odometer data is determined.

[0105] Based on the navigation positioning information determined during the straight path movement, two navigation positioning information having the same time information as the two odometer data are obtained, and the distance between the two navigation positioning positions is determined based on the two navigation positioning information, and the distance is used as the second verification data,

[0106] Calculate the distance difference between the odometer accumulated distance and the second verification data,

[0107] Determine whether the distance difference is less than the set distance threshold. If so, the odometer position is determined to be accurate; otherwise, the odometer position is determined to be inaccurate.

[0108] The heading angle accuracy detection includes:

[0109] Based on any two gyroscope data collected from the inertial measurement unit data during the same direction of rotation, the gyroscope cumulative angle change of the two data is determined.

[0110] Based on the navigation positioning information determined during the rotation process, two navigation positioning information having the same time information as the two gyroscope data are obtained, and an angle change is determined based on the two navigation positioning information, and the angle change is used as the third verification data.

[0111] Calculating a second deviation between the odometer cumulative angle change and the second verification data,

[0112] Determine whether the second deviation value is less than a set second deviation threshold value, if so, determine that the gyroscope heading angle is accurate, otherwise, determine that the gyroscope heading angle is inaccurate;

[0113] in,

[0114] Navigation positioning information is determined as follows:

[0115] When the down-view code image data is collected, the change in the scanning angle determined based on the down-view code image data is used as navigation positioning information.

[0116] In a case where the first laser point cloud image data is collected but the down-view code image data is not collected, the navigation positioning information is determined based on the first laser point cloud image data.

[0117] Preferably, the mobile robot is a lifting type mobile robot, and the mobile robot body is equipped with an upward-looking camera for reading cargo code identification;

[0118] The bearing surface below the shelf in the business area includes a positioning mark;

[0119] The second rotation angle is achieved in the following manner: the mobile robot moves at the same position at at least three different orientation angles, wherein the sum of the circular angles formed by the orientation angles is equal to the second rotation angle;

[0120] The business operation actions include: lifting and lowering operations;

[0121] The detection data includes: cargo code image data reported by the upward-looking camera;

[0122] The business component detection includes: at least one of: upward-looking camera extrinsic parameter position calibration detection, upward-looking camera extrinsic parameter attitude calibration detection, and upward-looking camera recognition rate detection;

[0123] The business operation accuracy detection includes: posture consistency detection before and after lifting;

[0124] in,

[0125] Position calibration detection includes:

[0126] For each frame of the cargo code image data, the observed global position of the cargo code is obtained based on the frame.

[0127] Determine the circular radius of the trajectory formed by each observed global position according to each observed global position, wherein the number of observed global positions is at least three,

[0128] Determine whether the circle radius is greater than the set radius threshold. If so, determine that the position calibration is accurate; otherwise, determine that the position calibration is inaccurate.

[0129] Attitude calibration detection includes:

[0130] Based on the preceding and following frames including the positioning marker information in the image data, two local postures of the positioning marker observed in the preceding and following frames in the mobile robot coordinate system are obtained, and based on the two local postures, a posture change amount between the first frames is determined,

[0131] Based on the preceding and following frames in the cargo code image data, two local postures of the cargo code mark observed in the preceding and following frames in the coordinate system of the mobile robot are obtained, and based on the two local postures, a posture change amount between the second frames is determined, wherein the preceding and following frames including the positioning mark information have the same time information as the preceding and following frames in the cargo code image data.

[0132] Calculate the posture change error between the first frame posture change and the second frame posture change,

[0133] Determine whether the attitude change error is less than the set attitude change threshold. If so, the attitude calibration is determined to be accurate; otherwise, the attitude calibration is determined to be inaccurate.

[0134] Posture consistency testing before and after lifting includes:

[0135] For each frame of the cargo pallet image data collected before lifting, obtain the first position of the cargo pallet in the frame, and calculate the average value of all the first position values.

[0136] For each frame of cargo pallet image data collected after lifting, obtain the second posture of the cargo pallet in the frame, and calculate the average value of all the second postures.

[0137] Determine whether the pose error between the average value of the first pose and the average value of the second pose is lower than the set pose error threshold. If so, it is determined that there is consistency; otherwise, it is determined that there is no consistency;

[0138] Upward-looking camera recognition rate detection includes:

[0139] For each frame of cargo code image data collected before lifting, identify the cargo code in the frame.

[0140] For each frame of cargo code image data collected after lifting, identify the cargo code in the frame.

[0141] Mark each image frame where recognition fails,

[0142] It is counted whether the number of consecutive frames in the marked image frames is greater than a set frame number threshold. If so, it is determined that the recognition rate is inaccurate; otherwise, it is determined that the recognition rate is accurate.

[0143] Preferably, the obstacle detection position is located on the perpendicular midline of the projection line segment of the obstacle facade on the bearing surface, and the distance between the obstacle detection position and the obstacle facade meets the set obstacle avoidance detection distance;

[0144] The obstacle avoidance component includes: a second laser radar for obstacle avoidance;

[0145] The detection data is the second laser point cloud image data reported by the second laser radar;

[0146] The third detection action includes: rotating clockwise and counterclockwise at least once at the detection position;

[0147] The installation posture accuracy detection includes:

[0148] For each frame in the second laser point cloud image data,

[0149] Convert the coordinates of the point cloud in the frame in the second laser radar coordinate system to the coordinates in the obstacle avoidance coordinate system.

[0150] Based on the distribution of obstacles in the obstacle avoidance area, the point cloud of the obstacle and the point cloud of the obstacle avoidance area in the non-obstacle area are screened out.

[0151] Count the number of point clouds in the filtered area of ​​the frame to get the number of point clouds in the obstacle avoidance area of ​​the frame.

[0152] Perform point cloud feature extraction on the point cloud of the screened obstacles.

[0153] Based on the extracted point cloud features, the point cloud straight line of the obstacle is fitted to obtain the fitted straight line.

[0154] Determine the angle between the fitted straight line and the horizontal direction of the obstacle avoidance coordinate system to obtain the fitted straight line angle;

[0155] Summarize the number of obstacle avoidance area point clouds and the angle of the fitted line for each frame.

[0156] Determine whether the difference between the average of all fitted line angles and the theoretical line angle exceeds the set angle threshold. If so, determine that the second lidar calibration is abnormal. The theoretical line angle is the angle between the projected line segment in the obstacle avoidance coordinate system and the horizontal direction of the obstacle avoidance coordinate system. This angle is used as verification data.

[0157] Determine whether the number of point clouds in the obstacle avoidance area of ​​all frames is greater than a set first number threshold. If so, if the number of point clouds within the boundary of the mobile robot body is greater than a set second number threshold, determine that the second laser radar is interfered with by the mobile robot body; if the number of point clouds within the boundary of the mobile robot body is not greater than the set second number threshold, determine that the second laser radar is interfered with by the bearing surface of the mobile robot body;

[0158] The ranging accuracy detection includes:

[0159] For each frame of the second laser point cloud image data, determine the measured distance of each detection angle to the obstacle within the theoretical scanning angle range, and obtain the measured distance of each detection angle in the frame.

[0160] Summarize the measured distance of obstacles at each detection angle in each frame,

[0161] For each detection angle,

[0162] The average value of each measured distance of the detection angle in each frame is calculated to obtain the measured distance of the detection angle.

[0163] Determine whether the error between the measured distance of the detection angle and the theoretical measured distance of the detection angle exceeds the set distance error threshold. If so, determine that the second laser radar is abnormal in the distance measurement of the detection angle, and use the theoretical measured distance of the detection angle as verification data.

[0164] in,

[0165] The theoretical scanning angle range is calculated based on the coordinates of the two endpoints of the projection line segment in the second lidar coordinate system.

[0166] Preferably, the first detection action, the second detection action, and the third detection action are performed in the following consecutive actions:

[0167] In the business area, at the starting position mark, take the forward direction detection path of the mobile robot body as the starting direction, rotate 90 degrees, move the set second distance in the current direction, rotate 180 degrees and return to the starting position mark, then move the set second distance in the current direction and rotate 180 degrees and return to the starting position mark, then rotate 90 degrees to the starting direction, after completing the first detection action and returning to the starting position mark, perform the lifting and lowering operation to complete the second detection action;

[0168] In the positioning area, at the starting point positioning mark, move along the detection path to the end point positioning mark in the starting direction, and rotate clockwise and counterclockwise at the end point positioning mark to complete the third detection action, then rotate 180 degrees and return to the starting point positioning mark along the detection path to complete the first detection action.

[0169] in,

[0170] The starting point positioning mark is located under the shelf, and the end point positioning mark is located at the obstacle detection position.

[0171] The accumulation of each forward direction in the service area forms a second rotation angle.

[0172] A second aspect of the present application provides a detection device for a mobile robot assembly, the device comprising:

[0173] The detection data acquisition module is used to collect detection data of the mobile robot body during the detection action in the detection area.

[0174] The detection area includes at least one of: a positioning area for detecting a navigation and positioning component of a mobile robot, a service area for detecting a service component of a mobile robot that performs services, and an obstacle avoidance area for detecting an obstacle avoidance component of a mobile robot.

[0175] The detection action includes at least one of: a first detection action for obtaining detection data of a navigation and positioning component, a second detection action for obtaining detection data of a business component, and a third detection action for obtaining detection data of an obstacle avoidance component.

[0176] The detection data includes: the reported data of each sensor installed in each component of the mobile robot body,

[0177] The component detection module uses the collected detection data to detect at least one of the mobile robot's navigation and positioning component, business component, and obstacle avoidance component.

[0178] The present application provides a method for detecting mobile robot components. By collecting and analyzing detection data during the detection action, at least one of the mobile robot's navigation and positioning components, business components, and obstacle avoidance components is detected, thereby avoiding the inefficiency and incompleteness caused by detecting sensors one by one, and improving the efficiency and reliability of mobile robot operation and maintenance. BRIEF DESCRIPTION OF THE DRAWINGS

[0179] Figure 1 A flow chart of a method for detecting a mobile robot component according to an embodiment of the present application.

[0180] Figures 2a to 2c A schematic diagram of a process for detecting components of a mobile robot according to this embodiment.

[0181] Figure 3 A three-dimensional schematic diagram of the detection area environment.

[0182] Figure 4 A schematic diagram of a top view of the detection area and the detection action.

[0183] Figure 5 A schematic diagram of the first laser radar attitude calibration detection.

[0184] Figure 6 A schematic diagram of lateral deviation caused by scanning deviation.

[0185] Figure 7 A schematic diagram of upward camera posture error detection.

[0186] Figure 8 A schematic diagram of the obstacle avoidance coordinate system.

[0187] Figure 9A schematic diagram of a detection device for a mobile robot component according to an embodiment of the present application.

[0188] Figure 10 Another schematic diagram of the detection device for the mobile robot component according to an embodiment of the present application. DETAILED DESCRIPTION

[0189] In order to make the purpose, technical means and advantages of this application more clear, this application is further described in detail below with reference to the accompanying drawings.

[0190] An embodiment of the present application provides a method for detecting components of a mobile robot, which can at least detect the navigation and positioning components by collecting and analyzing detection data during the detection action performed by the mobile robot body in the detection area.

[0191] See also Figure 1 As shown, Figure 1 The following is a flow chart of a method for detecting a mobile robot component according to an embodiment of the present application. The method includes:

[0192] Step 11: Collect detection data of the mobile robot body during the detection action in the detection area.

[0193] The detection area includes at least one of: a positioning area for detecting a navigation and positioning component of a mobile robot, a service area for detecting a service component of a mobile robot that performs services, and an obstacle avoidance area for detecting an obstacle avoidance component of a mobile robot.

[0194] The detection action includes at least one of: a first detection action for obtaining detection data of a navigation and positioning component, a second detection action for obtaining detection data of a business component, and a third detection action for obtaining detection data of an obstacle avoidance component.

[0195] As an example, the positioning area includes: a positioning identifier for identifying actual position information of each position on the set detection path;

[0196] The first detection action includes at least: a movement detection action of moving a set first distance along a set detection path, and a rotation detection action of rotating at a set first rotation angle.

[0197] As an example, the business area includes a shelf, and the bottom surface of the shelf has a cargo code mark for business component detection, and the cargo code mark includes actual global pose information of the shelf;

[0198] As an example, the obstacle avoidance area includes: an obstacle facade.

[0199] The business area, positioning area, and obstacle avoidance area are adjacent in sequence.

[0200] As an example, the second detection action includes: rotating at a set second rotation angle below the shelf and performing a business operation action to observe the cargo code identification. To obtain multi-angle detection data and improve detection accuracy, the second rotation angle is achieved as follows: the mobile robot moves at at least three different orientation angles at the same position, wherein the sum of the circular angles formed by the orientation angles is greater than or equal to the second rotation angle;

[0201] The third detection action includes: a detection action of collecting image data of obstacle facade information at a set obstacle detection position;

[0202] To improve detection efficiency, the first detection action, the second detection action, and the third detection action form a continuous action. As an example, in the business area, at the starting position mark, the forward direction detection path of the mobile robot body is used as the starting direction, and the robot rotates 90 degrees. After moving the set second distance in the current direction, the robot rotates 180 degrees and returns to the starting position mark. After moving the set second distance in the current direction, the robot rotates 180 degrees and returns to the starting position mark. Then, the robot rotates 90 degrees to the starting direction. After the first detection action is completed and the robot returns to the starting position mark, the robot performs the lifting and lowering operations to complete the second detection action.

[0203] In the positioning area, move from the starting position mark to the end position mark along the detection path in the starting direction, and rotate clockwise and counterclockwise at the end position mark to complete the third detection action, then rotate 180 degrees and return to the starting position mark along the detection path to complete the first detection action.

[0204] As an example, the detection data includes: the detection data includes: the reported data of each sensor in each component installed in the mobile robot body, the synchronously reported data of each sensor in the business component installed in the mobile robot body, and at least one of the synchronously reported data of each sensor in the obstacle avoidance component installed in the mobile robot body.

[0205] Step 12: Use the collected detection data to detect at least one of the navigation and positioning component, the business component, and the obstacle avoidance component of the mobile robot.

[0206] As an example, verification data is obtained using the identified position information, and a navigation and positioning component of the mobile robot is tested based on the detection data collected in the positioning area;

[0207] Use the global position information in the cargo code identification to obtain verification data, and detect the business components of the mobile robot based on the detection data collected in the business area;

[0208] Obstacle detection location information is used to obtain verification data, and the obstacle avoidance component of the mobile robot is tested based on the detection data collected in the obstacle avoidance area;

[0209] in,

[0210] The detection data collected during the execution of the mobile detection action is used to detect the position accuracy of the navigation and positioning component and / or the external reference positioning posture of the navigation and positioning component.

[0211] The detection data collected during the execution of the rotation detection action is used to detect the angular accuracy of the navigation and positioning component.

[0212] The detection data collected during the execution of the second detection action is used to detect at least one of the external reference positioning accuracy and the business operation accuracy of the business component.

[0213] The detection data collected during the execution of the third detection action is used to detect at least one of the installation posture accuracy and the ranging accuracy of the obstacle avoidance component.

[0214] As an example, the navigation and positioning component includes: at least one of a visual module, an odometer, and an inertial measurement unit for visual navigation and positioning; the detection data includes: at least one of image data, odometer data, and inertial measurement unit data for visual navigation and positioning;

[0215] The navigation and positioning component performs at least one of the following detections:

[0216] Based on the image data in the detection data, the first verification data is obtained using the position information or movement trajectory of the positioning mark to detect the positioning posture of the external reference mark of the visual module.

[0217] Based on the odometer data and image data in the detection data, the second verification data is obtained using the position information of the marker, and the odometer is tested, the test including: at least one of the odometer rotation angle accuracy test and the odometer position accuracy test,

[0218] Based on the inertial measurement unit data and the image data in the detection data, the third verification data is obtained using the position information of the marker to detect the heading angle accuracy of the inertial measurement unit.

[0219] As an example, the visual module includes a first laser radar for laser navigation positioning, the positioning marker can be located in the environment within the detection area, the image data used for visual navigation positioning is the first laser point cloud image data reported by the first laser radar, and the detection of the external reference positioning posture of the visual module includes: at least one of the position calibration detection and the posture calibration detection of the first laser radar,

[0220] in,

[0221] Position calibration detection includes:

[0222] Based on the point cloud image data of the positioning marker included in the first laser point cloud image data, the observed global position information of the positioning marker is determined,

[0223] Calculating the position standard deviation between the observed global positions of all positioning markers and the actual position information of all positioning markers, wherein the actual position information of all positioning markers is used as the first verification data,

[0224] Determine whether the position standard deviation is less than a set position standard deviation threshold; if so, determine that the position calibration of the first laser radar is accurate; otherwise, determine that the position calibration of the first laser radar is inaccurate;

[0225] Attitude calibration detection includes:

[0226] Determine the moving trajectory of the mobile robot based on the navigation positioning information determined by the laser point cloud image data reported by the first laser radar,

[0227] Intercept the approximate straight line trajectory data of the mobile robot's moving trajectory,

[0228] Based on the intercepted trajectory data, a straight line is fitted to obtain a fitting straight line.

[0229] Calculate the linear angle of the fitted line in the world coordinate system to obtain the trajectory angle, which is used as the first calibration data.

[0230] Select any trajectory point in the intercepted trajectory data, and obtain the observed global posture information of the trajectory point based on the navigation positioning information of the trajectory point.

[0231] Determine whether the angle error between the observed global posture information of the trajectory point and the trajectory angle is less than the set angle error threshold,

[0232] If so, it is determined that the attitude calibration of the first laser radar is accurate; otherwise, it is determined that the attitude calibration of the first laser radar is inaccurate.

[0233] As another example, the visual module includes a downward-looking camera for reading downward-looking codes, and the downward-looking camera is installed at the bottom of the mobile robot body. The positioning identifier is the downward-looking code located in the bearing surface for supporting the mobile robot, and the image data used for visual navigation positioning is the downward-looking code image data reported by the downward-looking camera. The detection of the external parameter positioning posture of the visual module includes: at least one of: position calibration detection of the downward-looking camera and detection of the recognition rate of the downward-looking camera.

[0234] in,

[0235] Position calibration detection includes:

[0236] For each frame of the lower view code image data collected during the movement along the set detection path, it is identified whether the frame contains the lower view code image.

[0237] If yes, then

[0238] Determine the observed global position information of the lower view code corresponding to the lower view code image in the frame, and obtain the scanning lateral deviation distance and forward deviation distance of the lower view code, wherein the scanning lateral deviation distance is used to characterize the distance between the observed global position of the lower view code and the actual position of the lower view code, and the actual position of the lower view code is used as the first verification data. The forward deviation distance is used to characterize the distance error between the distance between the observed global positions of the two lower view codes and the distance between the actual positions of the two lower view codes, and the distance between the actual positions of the two lower view codes is used as the first verification data.

[0239] Calculate the average lateral deviation distance and the average forward deviation distance of all scanned codes.

[0240] Otherwise, the navigation positioning information of the frame is determined based on the frame, and the distance between the position of the navigation positioning information and the detection path at the position is calculated to obtain the code-free deviation distance.

[0241] Determine whether the average value of the lateral deviation distance of the code scanning is less than the set lateral deviation distance threshold of the code scanning, whether the average value of the forward deviation distance is less than the set forward deviation distance threshold, and whether the no-code deviation distance is less than the set no-code deviation distance threshold.

[0242] If so, it is determined that the downward-looking camera extrinsic calibration is accurate; otherwise, it is determined that the downward-looking camera extrinsic calibration is inaccurate.

[0243] Downward-looking camera recognition rate detection includes:

[0244] Based on the lower view code image frame data, identify the lower view code image in each frame, and count the number of different lower view codes.

[0245] Check whether the number of down-view codes counted is the same as the actual number of down-view codes,

[0246] If yes, then the recognition rate of the camera is determined to be accurate.

[0247] Otherwise, it is determined that the recognition rate of the lower view code camera is inaccurate.

[0248] As another example, the vision module includes a first laser radar and a downward-looking camera for reading downward-looking codes.

[0249] The odometer rotation angle accuracy detection includes:

[0250] Based on any two odometer data collected during the same direction rotation process, the odometer cumulative angle change of the two odometer data is determined.

[0251] Based on the navigation positioning information determined during the rotation process, two navigation positioning information having the same time information as the two odometer data are obtained, and an angle change is determined based on the two navigation positioning information, and the angle change is used as the second verification data.

[0252] Calculating a first deviation between the odometer cumulative angle change and the second verification data,

[0253] It is determined whether the first deviation value is less than a set first deviation threshold value. If so, it is determined that the odometer rotation angle is accurate; otherwise, it is determined that the odometer rotation angle is inaccurate.

[0254] The odometer position accuracy detection includes:

[0255] Based on any two odometer data collected during the straight path movement of the mobile robot, the odometer cumulative distance of the two odometer data is determined.

[0256] Based on the navigation positioning information determined during the straight path movement, two navigation positioning information having the same time information as the two odometer data are obtained, and a distance change is determined based on the two navigation positioning information, and the distance change is used as the second verification data.

[0257] Calculate the distance difference between the odometer accumulated distance and the second verification data,

[0258] Determine whether the distance difference is less than the set distance threshold. If so, the odometer position is determined to be accurate; otherwise, the odometer position is determined to be inaccurate.

[0259] The heading angle accuracy detection includes:

[0260] Based on any two gyroscope data collected from the inertial measurement unit data during the same direction of rotation, the gyroscope cumulative angle change of the two data is determined.

[0261] Based on the navigation positioning information determined during the rotation process, two navigation positioning information having the same time information as the two gyroscope data are obtained, and an angle change is determined based on the two navigation positioning information, and the angle change is used as the third verification data.

[0262] Calculating a second deviation between the odometer cumulative angle change and the second verification data,

[0263] Determine whether the second deviation value is less than a set second deviation threshold value, if so, determine that the gyroscope heading angle is accurate, otherwise, determine that the gyroscope heading angle is inaccurate;

[0264] in,

[0265] Navigation positioning information is determined as follows:

[0266] When the down-view code image data is collected, the change in the scanning angle determined based on the down-view code image data is used as navigation positioning information.

[0267] In a case where the first laser point cloud image data is collected but the down-view code image data is not collected, the navigation positioning information is determined based on the first laser point cloud image data.

[0268] The embodiment of the present application can realize the detection of the navigation component of the mobile robot through the detection data collected in the positioning area in the detection area, can realize the detection of the business component through the detection data collected in the business area, and can realize the detection of the obstacle avoidance component through the detection data collected in the obstacle avoidance area. The embodiment of the present application can realize comprehensive detection of the mobile robot, which is beneficial to improving the operation and maintenance efficiency of the mobile robot and reducing duplication of investment.

[0269] To facilitate understanding of the present application, the following description will be given using a lifting mobile robot as an example. It should be understood that the present application is not limited thereto, and the detection of any other type of mobile robot may also be applicable.

[0270] As an example, the main body of a lifting mobile robot is equipped with a navigation and positioning component for navigation and positioning, as well as a business component for executing business operations. The navigation and positioning component includes: an inertial measurement unit (IMU) sensor, an odometer, a first laser radar for navigation and positioning, a second laser radar for obstacle avoidance, and a downward-looking barcode reading camera for downward-looking barcode reading. Depending on the navigation method, either the first laser radar or the downward-looking barcode reading camera is selected as one of the navigation and positioning components. Preferably, both the first laser radar and the downward-looking barcode reading camera are installed. As an example, the first and second laser radars can be combined to form a laser radar for both navigation and positioning and obstacle avoidance. The business component includes an upward-looking barcode reading camera for cargo code reading.

[0271] The triggering conditions for detecting the mobile robot include one of the following conditions:

[0272] 1) The vehicle reaches a set maintenance time. As an example, the vehicle's cumulative operation time and / or travel distance are obtained based on the vehicle's operation record. When the cumulative time and / or travel distance reaches a set threshold, the mobile robot is triggered to detect the vehicle.

[0273] 2) Abnormal deviation of the vehicle body. As an example, if the frequency of deviation alarms reported by the mobile robot reaches a set frequency threshold, or if the mobile robot is observed to be deviating on site, the mobile robot detection is triggered.

[0274] 3) The vehicle's operating accuracy does not meet usage requirements. For example, if the vehicle is observed to be not properly placed or not running to the designated location, the mobile robot will trigger a detection.

[0275] In this embodiment, the detection of mobile robot components includes: navigation component detection and business component detection, wherein:

[0276] Navigation component detection includes: navigation and positioning component detection, and obstacle avoidance component detection.

[0277] The navigation and positioning component detection includes at least one of: the first laser radar external parameter positioning posture detection, the downward camera position calibration detection, the odometer detection, the IMU detection, and the downward code reading camera recognition rate detection.

[0278] The obstacle avoidance component detection includes: at least one of the second laser radar working condition detection and frame data detection;

[0279] The business component detection includes: at least one of: upward-looking code reading camera recognition rate detection and upward-looking camera external reference mark positioning posture detection.

[0280] See also Figures 2a to 2c As shown, Figures 2a to 2c This is a schematic diagram of a process for detecting components of a mobile robot according to this embodiment. The detection method includes:

[0281] Step 21: Acquire detection data of the mobile robot during the detection action within the detection area.

[0282] See also Figure 3 As shown, Figure 3 This is a 3D diagram of the inspection area. As an example, the inspection area is an independent area embedded in the on-site environment. To prevent interference during the inspection process, it is generally located in an area distinct from the on-site business path, such as a corner or the end of a corridor. To highlight the independence of this area, fixed fences can be placed around it. These fences are optional and only serve to prevent interference.

[0283] The detection area includes: a business area for obtaining business component detection data, a positioning area for obtaining navigation positioning component detection data, and an obstacle avoidance area for obtaining obstacle avoidance component detection data. The business area, positioning area, and obstacle avoidance area are adjacent in sequence.

[0284] The bearing surface for supporting the mobile robot in the positioning area has a linear code array composed of several equally spaced positioning markers. As an example, the positioning marker is a down-view code, also known as a ground code. Each positioning marker includes the actual position information of the positioning marker. The down-view code close to the obstacle avoidance area in the linear code array is the end down-view code, and the down-view code close to the business area is the starting down-view code. The number of down-view codes can be set as needed. The linear code array can be used as a detection path.

[0285] The business area includes a shelf, which is located above a lower visual code. Preferably, the shelf is located directly above the lower visual code of the starting point. The bottom surface of the shelf has a cargo code identification for business component detection, and the cargo code identification includes the actual location information of the cargo code. Preferably, the cargo code identification is located at the center of the bottom surface of the shelf, and the projection of the cargo code identification on the bearing surface coincides with the lower visual code of the starting point.

[0286] The obstacle avoidance area includes an obstacle facade, the linear code array is located on the perpendicular bisector of the projection line segment of the facade on the bearing surface, the distance between the end point lower view code and the obstacle facade meets the set obstacle detection distance, and there are no other obstacles in the obstacle avoidance area.

[0287] As an example, the code information of the cargo code and the cargo code is a QR code.

[0288] See also Figure 4 As shown, Figure 4 This is a schematic diagram of a top view of the detection area and the detection actions. As an example, the detection actions include:

[0289] A movement detection action of moving a set first distance along a set detection path, and a rotation detection action of rotating at a set first rotation angle, wherein the detection data collected during the execution of the movement detection action is used to detect the position accuracy of the navigation and positioning component and / or the external parameter positioning posture of the navigation and positioning component, and the detection data collected during the execution of the rotation detection action is used to detect the angular accuracy of the navigation and positioning component, for example, a movement detection action of moving in a straight line from the starting lower view code position to the end lower view code position and back along the detection path composed of a straight line code array, and a rotation detection action of rotating 180 degrees when returning to the starting lower view code position at the end lower view code position.

[0290] A second detection action is performed in which the robot rotates at a set second rotation angle under the shelf and performs a business operation action to observe the cargo code identification. The detection data collected during the execution of the detection action is used to detect at least one of the external parameter positioning posture accuracy and the business operation accuracy of the business component. For example, the mobile robot moves left and right one time according to the set moving distance at the starting point lower visual code position under the shelf, and performs one lifting and lowering, so that the image data of the obstacle facade information is collected at the set obstacle detection position. A third detection action is performed in which the detection data collected during the execution of the action is used to detect at least one of the installation posture accuracy and ranging accuracy of the obstacle avoidance component, for example, the action of collecting obstacle information at the end point lower visual code position.

[0291] As an example, in order to improve the efficiency of acquiring detection data, the above detection action can form a continuous detection action. Specifically,

[0292] Set the state of the mobile robot's head facing the end point's lower view code as the initial state, that is, the forward direction of the mobile robot body facing the end point's lower view code is the starting direction,

[0293] The mobile robot rotates 90 degrees in any clockwise direction, moves the set second distance in the direction of the current vehicle head, rotates 180 degrees, moves to the lower visual code position of the starting point in the direction of the current vehicle head, then moves to the direction of the current vehicle head according to the set second distance, rotates 180 degrees again, and moves to the lower visual code position of the starting point in the direction of the current vehicle head; Figure 4 In the process, the mobile robot first rotates 90 degrees counterclockwise, moves to the left by the second distance, then rotates 180 degrees, moves to the right to the lower view code position of the starting point, then moves to the right by the second distance, and finally rotates 180 degrees and moves to the left to the lower view code position of the starting point to perform the second detection action;

[0294] The vehicle rotates 90 degrees at the starting point's lower visual code position so that the vehicle's head faces the end point's lower visual code position, and moves from the starting point's lower visual code position to the end point's lower visual code position along the detection path composed of the linear code array to perform the mobile detection action.

[0295] At the end point, the vehicle rotates clockwise and counterclockwise for one circle respectively, thereby completing the fourth detection task to collect obstacle information.

[0296] The vehicle rotates 180 degrees at the end point's lower visual code position so that the vehicle's head faces the starting point's lower visual code position, thereby completing the second detection task. The vehicle then returns from the end point's lower visual code position to the starting point's lower visual code position along the detection path formed by the linear code array to complete the mobile detection action.

[0297] At the starting point, the lowering action is performed once to complete the second detection action.

[0298] in,

[0299] Rotation refers to the rotation in place around the central axis of the mobile robot body;

[0300] Under the shelf, by moving left and right and returning to the starting point, the lifting and lowering actions enable the mobile robot to obtain at least three different forward facing angles at the same position. These angles form a second rotation angle in the form of accumulation of intermittent rotation angles, for example, Figure 4 The second rotation angle can reach one full rotation. During the detection process, laser point cloud image data, down-view code image data, cargo code image data, odometer data, and IMU data are synchronously collected so that the synchronously collected detection data have the same time information. The laser point cloud image data includes first laser point cloud image data from the first laser radar and second laser point cloud image data from the second laser radar.

[0301] Step 22: Detect the navigation and positioning component using the detection data obtained by the mobile robot in the positioning area.

[0302] Specifically,

[0303] Step 221 , using the lower view code image data and the first laser point cloud image data collected during the rotation and the straight line movement along the lower view code in the detection action, perform external reference mark positioning pose detection on the first laser radar.

[0304] The first laser radar's external parameter calibration posture detection for navigation and positioning includes position calibration and posture calibration. Specifically, the first laser radar's local posture calibration in the mobile robot's coordinate system affects positioning feedback accuracy and multi-vehicle consistency. Errors in the external parameter calibration position can cause the positioning information fed back during the mobile robot's rotation to differ from the original position. Errors in the external parameter calibration posture can also cause the feedback positioning angle to differ from the actual angle, leading to skewed positioning after positioning, resulting in the vehicle body deviating to one side during movement and an S-shaped trajectory.

[0305] As an example, the position calibration detection method includes:

[0306] Step 2211: For each frame of the n frames of the lower view code image data, the observed global position information of the lower view code corresponding to the lower view code image is calculated based on the local position of the lower view code in the mobile robot coordinate system and the SLAM navigation positioning information of the mobile robot obtained by the first laser radar. The mathematical formula is:

[0307] wm = wb × b1 Formula 1

[0308] Among them, b1 is the local position of the lower view code in the mobile robot coordinate system, wb is the navigation positioning information, and wm is the observed global position of the lower view code in this frame.

[0309] Thus, the n observed global positions of the lower viewing code can be obtained.

[0310] In step 2212, since the lower view code is attached to the supporting surface, its actual global position will not change. The position standard deviation between the lower view code observed global position and the lower view code actual global position obtained by formula 1 is expressed as:

[0311]

[0312] Among them, x and y are the actual global position coordinates of the lower view code, wm n .x, wm n .y is the observed global position wm n The actual global position of the lower view code is used as the first verification data.

[0313] Step 2213: determine whether the position standard deviation is less than the set position standard deviation threshold.

[0314] If yes, it is determined that the position calibration of the first laser radar is accurate.

[0315] Otherwise, it is determined that the position calibration of the first laser radar is inaccurate.

[0316] See also Figure 5 As shown, Figure 5 The figure below is a schematic diagram of the first LiDAR attitude calibration test. Since the detection action in the positioning area includes linear movement at the lower view code, the mobile robot uses the lower view code for global positioning and navigation during this movement, and the movement trajectory along the center of the lower view code approximates a straight line. At the same time, the SLAM navigation positioning information obtained from the laser point cloud image of the first LiDAR can be collected during the movement. This can be used to verify whether the position change direction and heading angle match. Therefore, the attitude calibration test includes:

[0317] Step 2214, based on the navigation positioning information determined by the laser point cloud image data reported by the first laser radar, determine the moving trajectory of the mobile robot, intercept the trajectory data of a section of approximate straight line in the actual trajectory of the mobile robot, perform straight line fitting on the intercepted trajectory data, and obtain a fitted straight line.

[0318] Step 2215 , calculating the linear angle of the fitted line, which may be the angle between the fitted line and the coordinate axis in the global coordinate system, to obtain a trajectory angle, which serves as the first verification data.

[0319] Step 2216: Select a trajectory point on the intercepted trajectory, obtain the observed global posture information of the trajectory point based on the navigation positioning information of the trajectory point, and calculate the angle error between the trajectory angle and the observed global posture information at the trajectory point.

[0320] Step 2217: determine whether the angle error is less than the set angle error threshold.

[0321] If yes, it is determined that the attitude calibration of the first laser radar is accurate.

[0322] Otherwise, it is determined that the attitude calibration of the first laser radar is inaccurate.

[0323] It should be understood that there is no order relationship between the above steps 2211-2213 and steps 2214-2217, and they can be two processes executed in parallel.

[0324] Step 222 , performing downward-looking camera extrinsic calibration detection using the downward-looking code image data and the first laser point cloud image data collected during the straight-line movement along the downward-looking code in the detection action.

[0325] The downward-looking camera extrinsic reference refers to the local pose of the center of the downward-looking camera lens in the mobile robot coordinate system, which can be recorded as (x, y, theta).

[0326] The mobile robot moves in a straight line in the positioning area. If the calibration accuracy of the lower reading terminal is poor, the scanning angle accuracy will be reduced, the code output angle will be deviated, and the 0-degree direction considered by the mobile robot will be angularly different from the actual 0-degree direction. The running direction of the mobile robot will deviate from the center line of the lower viewing code. When scanning the next code to update the position information, there will be a large lateral jump, making the positioning accuracy not meet the usage requirements. Figure 6 As shown, Figure 6 A schematic diagram of lateral deviation caused by code scanning deviation. Because the mobile robot's perceived angle deviates from its positioning angle, this deviation results in a significant lateral deviation when updating the next code scanned.

[0327] Therefore, the position calibration detection method of the downward-looking camera includes:

[0328] Step 2221: for each frame of the lower view code image data collected during the movement of the set detection path, identify whether the frame contains the lower view code image.

[0329] If yes, then execute step 2222,

[0330] Otherwise, the navigation positioning information is determined based on the first laser point cloud image frame having the same time information as the lower view code image frame, and the distance between the position of the navigation positioning information and the detection path is calculated to obtain the code-free deviation distance.

[0331] Step 2222: Obtain the lateral deviation distance and forward deviation distance of the code scanned in the frame.

[0332] The lateral deviation distance of the code scanning is used to represent the distance between the observed global position of the lower view code and the actual position of the lower view code. The actual position of the lower view code is used as the first verification data.

[0333] The forward deviation distance is used to characterize the distance error between the distance between the observed global positions of the two visual codes and the distance between the actual positions of the two visual codes, and the distance between the actual positions of the two visual codes is used as the first verification data.

[0334] Step 2223: Count the average of the lateral deviation distances, the average of the forward deviation distances, and the average of the no-code deviation distances of all scanned codes.

[0335] Step 2224: Determine whether the average value of the lateral deviation distance of the code scanning is less than the set lateral deviation distance threshold of the code scanning, and whether the average value of the forward deviation distance is less than the set forward deviation distance threshold, and determine whether the average value of the non-code deviation distance is less than the set non-code deviation distance threshold.

[0336] If yes, the downward camera position calibration is determined to be accurate.

[0337] Otherwise, it is determined that the position calibration of the downward-looking camera is inaccurate.

[0338] Step 223 : Detect the odometer using the lower view code image data and odometer data in the detection data.

[0339] The purpose of odometer testing is to detect abnormal integral angle accuracy caused by severe rotation slippage and abnormal calibration parameters. Such problems can easily lead to abnormal positioning angles when rotating outside the lower view code, causing deviation.

[0340] To better understand the accuracy of the odometer itself, the odometer rotation accuracy and distance accumulation accuracy are tested based on the accuracy of the downward camera scanning observation. Given that the mobile robot will perform a reciprocating linear movement in the downward view code area, at least at the end point of the downward view code, it will rotate 180 degrees on the downward view code and turn around to return. The odometer rotation accuracy can be tested by collecting the downward view code image data during the rotation process and comparing and analyzing the odometer posture accumulation data. Since the downward camera scan angle increment accuracy is high, by comparing the scan angle change with the odometer angle accumulation change, the odometer angle accumulation deviation value can be obtained. The smaller the deviation value, the higher the odometer rotation accumulation accuracy. See Figure 5 As shown, Figure 5 A schematic diagram showing the difference between the odometer's cumulative rotation angle and the scanned code angle.

[0341] Therefore, the odometer rotation angle accuracy test includes:

[0342] Step 2231: Determine the cumulative odometer angle change based on the odometer data collected during the same direction rotation at the end point lower view code, and determine the scanning angle change based on the lower view code image data collected during the same rotation at the end point lower view code.

[0343] Among them, the odometer data and the lower view code image data have the same time information. As an example, the rotation process in the same direction includes: one of: rotating 90 degrees or 180 degrees in place, rotating 360 degrees clockwise, and rotating 360 degrees counterclockwise.

[0344] For example, based on any two odometer data, the odometer cumulative angle change of the two odometer data is determined.

[0345] Based on the navigation positioning information determined during the rotation process, two navigation positioning information having the same time information as the two odometer data are obtained, and an angle change is determined based on the two navigation positioning information, and the angle change is used as the second verification data.

[0346] Step 2232: Calculate a first deviation between the odometer's cumulative angle change and the barcode scanning angle change.

[0347] Step 2233: Determine whether the first deviation value is less than a set first deviation threshold. The first deviation threshold is determined based on the proportion of the total rotation angle when rotating in the same direction. For example, 1% of the total rotation angle is the first deviation threshold.

[0348] If yes, it is determined that the odometer rotation angle accuracy is accurate.

[0349] Otherwise, the odometer rotation angle accuracy is judged to be inaccurate.

[0350] When a mobile robot is traveling straight ahead at the lower view code, it can obtain a relatively accurate position by observing the lower view code. As an example, align the odometer's starting point with the lower view code at the starting point. At the end point, compare the odometer's position with the actual position scanned by the lower view code to obtain the odometer position error. A smaller error indicates a higher odometer position accuracy.

[0351] As an example, the odometer distance accumulation accuracy test includes:

[0352] Step 2234, based on the lower view code image frame data, determine the positions of the first lower view code and the second lower view code, wherein the first lower view code and the second lower view code are any two lower view codes on the moving path, and the moving path points from the first lower view code to the second lower view code, according to the position of the first lower view code, determine the starting point of the odometer based on the odometer data so that the starting point of the odometer is aligned with the lower view code, and according to the position of the second lower view code, determine the end point of the odometer based on the odometer data.

[0353] Step 2235, calculate the distance between the first lower view code and the second lower view code based on the positions of the first lower view code and the second lower view code, use this distance as the second verification data, and calculate the accumulated distance of the odometer based on the starting point and end point of the odometer.

[0354] Step 2236: determine whether the distance difference between the two is less than the set distance threshold.

[0355] If yes, it is determined that the odometer distance accumulation accuracy is accurate.

[0356] Otherwise, it is determined that the odometer distance accumulation accuracy is inaccurate.

[0357] Step 224: Use the lower view code image data and IMU data in the detection data to detect the IMU.

[0358] The IMU consists of a gyroscope and an accelerometer. The gyroscope measures angular velocity around its central axis, while the accelerometer measures acceleration along that axis. Given that current mobile robot positioning focuses on steering accuracy, the gyroscope's zero drift value can change over time or with temperature fluctuations, affecting the accuracy of the positioning angle integration. Therefore, the heading angle accuracy must be tested.

[0359] Considering that the mobile robot collects gyroscope data and downward-view code image data during its rotation in place in the downward-view code area, and since the incremental accuracy of the downward-view camera's code scanning angle is high, the change in the code scanning angle is compared with the cumulative change in the gyroscope angle to obtain the cumulative deviation value of the gyroscope heading angle. The smaller the deviation value, the higher the accuracy of the gyroscope's heading angle.

[0360] Similar to the odometer rotation angle detection, the IMU heading angle detection method includes:

[0361] Step 2241, based on the IMU data collected during the 180-degree rotation at the end point lower view code, determine the gyroscope cumulative angle change, and based on the lower view code image data collected during the 180-degree rotation at the end point lower view code, determine the code scanning angle change.

[0362] Among them, the IMU data and the down-view code image data have the same time information,

[0363] For example, based on any two gyroscope data collected from the inertial measurement unit data during the same direction of rotation, the gyroscope cumulative angle change of the two data is determined.

[0364] Based on the navigation positioning information determined during the rotation process, two navigation positioning information having the same time information as the two gyroscope data are obtained, and an angle change is determined based on the two navigation positioning information, and the angle change is used as the third verification data.

[0365] Step 2242, calculate the second deviation value between the cumulative angle change of the gyroscope and the angle change of the code scanning.

[0366] Step 2243: determine whether the second deviation value is less than the set second deviation threshold.

[0367] If yes, it is determined that the gyroscope heading angle accuracy is accurate.

[0368] Otherwise, the gyroscope heading angle accuracy is judged to be inaccurate.

[0369] Step 225 : detecting the recognition rate of the down-view code camera based on the down-view code image frame data.

[0370] Considering that the camera may have problems such as unreasonable exposure gain parameter configuration, untorn camera film, unreasonable size of the lower view code and cargo code, resulting in poor lower view code recognition effect, and lower view code recognition is the prerequisite for autonomous movement tasks and parameter verification, the recognition rate of the lower view code is tested.

[0371] The recognition rate detection method of the lower view code includes:

[0372] Step 2251 : Based on the lower view code image frame data, identify the lower view code in each frame and count the number of different lower view codes.

[0373] Step 2252: Check whether the number of the counted lower view codes is the same as the number of lower view codes arranged in the environment.

[0374] If yes, then the recognition rate of the camera is determined to be accurate.

[0375] Otherwise, it is determined that the recognition rate of the lower view code camera is inaccurate.

[0376] Step 23: Detect the service components using the detection data acquired by the mobile robot in the service area.

[0377] Calibration errors in the upward-looking camera can reduce shelf positioning accuracy, leading to problems such as tilted or misplaced shelves. Furthermore, shelf tasks typically verify the shelf's local pose relative to the mobile robot before lifting to determine whether to lift the shelf. This pose is also verified after lifting to determine whether to continue the task. If the difference before and after lifting is significant, exceeding the accuracy threshold, repeated lifting and lowering actions may occur. Therefore, service component testing includes: upward-looking camera extrinsic parameter positioning pose detection, shelf pose consistency detection before and after lifting, and upward-looking camera recognition rate detection. Upward-looking camera extrinsic parameter calibration testing includes position calibration and attitude calibration.

[0378] Given that the cargo code data is collected at the starting point by intermittently rotating 360 degrees at the downward-looking code, if the extrinsic position of the upward-looking camera is not calibrated accurately, it means that the position deviation of the camera relative to the center of the mobile robot is inconsistent with the actual situation. The global position of the shelf obtained by calculating the mobile robot body at different angles will be different, and the shelf positioning will basically appear as a circle. The radius of the circle is the calibration error of the extrinsic position.

[0379] Therefore, as an example, the position calibration detection method includes:

[0380] Step 231: for each frame of the cargo code image data, determine the observed global position of the cargo code corresponding to the frame based on the frame. Thus, the global positions of m cargo codes can be obtained from m frames of cargo code images, where m is a natural number greater than or equal to 3.

[0381] Step 232: Determine the radius of the circle where the observed global position trajectory lies based on the observed global positions of all cargo pallets.

[0382] Step 233: determine whether the radius of the circle is greater than a set radius threshold.

[0383] If yes, the position calibration is accurate.

[0384] Otherwise, the position calibration is determined to be inaccurate.

[0385] As another example, the position calibration detection method includes:

[0386] Step 231′: For each frame of the m frames of cargo code image data, the observed global position of the cargo code corresponding to the frame is calculated based on the local position of the cargo code in the mobile robot coordinate system and the SLAM navigation positioning information of the mobile robot obtained by the first laser radar. The mathematical formula is:

[0387] wn=wb×b2 Formula 3

[0388] Among them, b2 is the local position of the cargo code in the mobile robot coordinate system, wb is the navigation positioning information, and wn is the observed global position of the cargo code in this frame.

[0389] Thus, m observed global positions of the cargo pallet can be obtained.

[0390] In step 232′, since the cargo tags are attached to the bottom of the shelf, their actual global position will not change. Therefore, the cargo tag position standard deviation between the observed global position of each cargo tag obtained by formula 3 and its actual global position is calculated, which can be expressed as follows:

[0391]

[0392] Among them, x and y are the actual global position coordinates of the down-view code, wn m .x、wn m .y is the observed global position wn m The location coordinates of the cargo code and the actual global location of the cargo code are used as verification data.

[0393] Step 233', determine whether the standard deviation of the cargo code position is less than the set cargo code position standard deviation threshold,

[0394] If yes, the position calibration is accurate.

[0395] Otherwise, the position calibration is determined to be inaccurate.

[0396] See also Figure 7 As shown, Figure 7 This is a schematic diagram of upward camera posture error detection. Given that the downward camera has been calibrated, the mobile robot moves and rotates beneath the shelf. The local postures of the starting downward view code in the mobile robot coordinate system, se2_db0 and se2_db1, are obtained from the preceding and following frames of the downward view code image captured at the starting downward view code position. From this, the first relative posture change of the mobile robot body, se2_b0b1, can be obtained. Similarly, the local postures of the cargo code in the mobile robot coordinate system, se2_hb0 and se2_hb1, can be obtained from the preceding and following frames of the cargo code image. From this, the second relative posture change of the mobile robot body, se2_b0b1′, can be obtained. If the upward camera posture calibration is inaccurate, there will be a deviation in the posture change between the two data sources. This can be used to test the accuracy of the extrinsic reference posture calibration.

[0397] Therefore, the attitude calibration detection method includes:

[0398] Step 234 , based on the previous and next frames of the lower view code image data collected at the starting lower view code position, determine the two local postures se2_db0 and se2_db1 of the starting lower view code in the previous and next frames in the mobile robot coordinate system, and calculate the posture change between the first frames.

[0399] Step 235: Based on the previous and next frames in the cargo pallet image, determine the two local postures se2_hb0 and se2_hb1 of the cargo pallet in the previous and next frames in the mobile robot coordinate system, and calculate the posture change between the second frames.

[0400] The preceding and following frames in the lower view code image data and the preceding and following frames in the cargo code image have the same time information.

[0401] Step 236, calculate the posture change error between the first inter-frame posture change and the second inter-frame posture change, which can be expressed as:

[0402] err=se2_b0b1-se2_b0b1′

[0403] Step 237: determine whether the attitude change error is less than the set attitude change threshold.

[0404] If yes, the upward camera attitude calibration is determined to be accurate.

[0405] Otherwise, it is determined that the upward camera posture calibration is inaccurate.

[0406] Since the test data includes the lifting and lowering of the shelf, the consistency of the shelf posture before and after lifting is detected by comparing the shelf posture during lifting and lowering. Therefore, the consistency test of the shelf posture before and after lifting includes:

[0407] Step 238, for each frame of cargo code image data collected by the mobile robot in the lifted state (after lifting), determine the first position pose of the cargo code in the frame. Thus, the first position poses of multiple cargo codes can be obtained from multiple frames of cargo code images, and the average value of all the first position poses is calculated.

[0408] Step 239: For each frame of cargo code image data collected by the mobile robot in the lowered state (before lifting), determine the second pose of the cargo code in the frame. Thus, the second poses of multiple cargo code images can be obtained from multiple frames, and the average value of all the second poses is calculated.

[0409] Step 240: Determine whether the posture error between the average value of the first posture and the average value of the second posture is less than a set posture error threshold.

[0410] Specifically, if the position error in the error is smaller than a set position error threshold, and if the posture error in the error is smaller than a set posture error threshold, it is determined that there is consistency; otherwise, it is determined that there is no consistency.

[0411] Given that in the business area, the mobile robot will perform detection actions such as left and right movement, rotation, lifting, and lowering under the shelf, the upward camera recognition rate detection can be performed by counting the number of consecutive frames of unrecognized cargo before and after lifting. Therefore, the upward camera recognition rate detection method includes:

[0412] Step 241 : for each frame of cargo code image data collected by the mobile robot in the lifting state, identify the cargo code in the frame. If the recognition fails, mark the frame as a first recognition failure.

[0413] Step 242 : for each frame of cargo code image data collected by the mobile robot in the lowered state, identify the second cargo code in the frame. If the identification fails, mark the frame as a second identification failure.

[0414] Step 243: Count the number of consecutive frames in the marked first recognition failure frames and the number of consecutive frames in the marked second recognition failure frames to see whether they exceed a set frame number threshold.

[0415] If yes, it is determined that the upward camera recognition rate is inaccurate.

[0416] Otherwise, it is determined that the upward camera recognition rate is accurate.

[0417] The navigation positioning information used in the above-mentioned odometer, gyroscope, and upward-looking camera detection process is determined by the downward-looking image data. It should be understood that the navigation positioning information determined by the first lidar image data can also be used. Step 24 is to perform a second lidar detection using the second laser point cloud image data in the detection data.

[0418] The second laser radar detection includes: detection of at least one of frame data and working status. The frame data detection is used to detect the reported data caused by the failure of the laser radar device itself. The working status detection is used to extract features based on the obstacle point cloud information and / or non-obstacle point cloud information in the frame data, and use the extracted features to detect the working status of the laser radar.

[0419] The types of frame data anomaly detection include: at least one of: time information anomaly between frame data, point cloud anomaly between frame data, point cloud anomaly in frame data, obstacle distance anomaly represented by frame data, and obstacle distance anomaly represented between frame data. Working condition anomaly detection includes: at least one of: lidar configuration parameter detection, dirtiness detection, ranging detection, and installation posture detection.

[0420] in,

[0421] The time information between frame data is abnormal, which is characterized by the time information of the frame data not being updated. For example, the current frame timestamp is equal to the previous frame timestamp.

[0422] Point cloud anomalies between frame data are characterized by the point cloud in the frame data not being updated, for example, the point cloud in the current frame is the same as the point cloud in the previous frame.

[0423] The point cloud in the frame data is abnormal, which is characterized by all 0s in the frame data. For example, the frame data reported by the lidar at each detection angle is always 0, that is, there is no point cloud in the frame data reported at each detection angle, which indicates that there is an abnormality in the detection angle.

[0424] The obstacle distance represented by the frame data is abnormal, which is characterized by missing frame data. For example, within the set detection angle threshold range, the obstacle detection distance is always 0 or not updated;

[0425] The obstacle distances represented by the frame data are abnormal, which is characterized by frame data disorder. For example, the difference between the obstacle distances of two adjacent frames exceeds the set distance threshold.

[0426] Configuration parameter detection is used to check whether the installation parameters and attribute parameters of the LiDAR are reasonable. The installation parameters are the 6D position (x, y, z, yaw, pitch, roll) of the LiDAR relative to the robot. The detection methods include:

[0427] Based on any single frame, determine whether the field of view of the laser radar in the frame is outside the robot body, for example, outside the robot body. If so, the installation parameters are determined to be reasonable, otherwise, they are determined to be unreasonable. This test is used to perform an initial inspection of the installation posture of the second laser radar to roughly detect the rationality of the installation posture.

[0428] Attribute parameters include the resolution and scanning range of the lidar, and detection methods include:

[0429] Based on any single frame, it is determined whether the attribute parameters of the frame meet the set threshold range. If yes, the attribute parameters are determined to be reasonable; otherwise, they are determined to be unreasonable.

[0430] Dirt detection is used to detect the dirtiness of the second LiDAR. Detection methods include:

[0431] For each point in each frame of data,

[0432] Extract the continuous angle feature, intensity feature, and geometric distribution feature of the point. The continuous angle feature is used to characterize: the angle between the obstacle surface formed by the previous point adjacent to the point in the scanning sequence and the detection ray of the point, and the angle between the obstacle surface formed by the next point adjacent to the point in the scanning sequence and the detection ray of the point. The intensity feature is used to characterize the signal strength of the point.

[0433] If the point's continuous angle feature is less than the set angle feature threshold, the intensity feature is less than the set intensity feature threshold, or the geometric distribution feature is sparsely distributed, then the point is determined to be a suspected dust point.

[0434] Count the distribution of suspected dust points in each frame data,

[0435] If the proportion of the counted suspected dust points in all point clouds of each frame data exceeds a set proportion threshold, the second laser radar is determined to be in a dirty state.

[0436] Among them, the angle feature threshold and the intensity feature threshold can be set according to the obstacle avoidance detection distance.

[0437] Installation posture detection methods include:

[0438] For each frame of data,

[0439] Convert the coordinates of the point cloud in the frame data in the second laser radar coordinate system to the coordinates of the obstacle avoidance coordinate system. Figure 8 As shown, Figure 8 This is a schematic diagram of the obstacle avoidance coordinate system. The obstacle avoidance coordinate system is defined as a right-handed Cartesian coordinate system with the endpoint's lower viewpoint as the origin and the perpendicular bisector of the projected line segment as the x-axis.

[0440] Based on the distribution of obstacle facades in the obstacle avoidance area, the point cloud of the obstacle and the point cloud of the obstacle avoidance area in the non-obstacle area are screened out.

[0441] Count the number of point clouds in the obstacle avoidance area selected in the frame data to obtain the number of point clouds in the obstacle avoidance area of ​​the frame data.

[0442] Perform point cloud feature extraction on the point cloud of the screened obstacles.

[0443] Based on the extracted point cloud features, the point cloud straight line of the obstacle is fitted to obtain the fitted straight line.

[0444] Determine the angle between the fitted straight line and the horizontal direction of the obstacle avoidance coordinate system to obtain the fitted straight line angle;

[0445] Summarize the number of obstacle avoidance area point clouds and the angle of the fitted line for each frame of data.

[0446] Determine whether the difference between the average of all fitted straight line angles and the theoretical straight line angle exceeds a set angle threshold. If so, determine that the second lidar calibration is abnormal. The theoretical straight line angle is the angle between the projected line segment in the obstacle avoidance coordinate system and the horizontal direction of the obstacle avoidance coordinate system.

[0447] Determine whether the number of point clouds in the obstacle avoidance area of ​​all frame data is greater than the set first number threshold. If so, when the number of point clouds within the boundary of the mobile robot body is greater than the set second number threshold, determine that the second laser radar is interfered with by the mobile robot body. When the number of point clouds within the boundary of the mobile robot body is not greater than the set second number threshold, determine that the second laser radar is interfered with by the bearing surface of the mobile robot body.

[0448] The ranging detection includes:

[0449] For each frame of data, determine the measured distance of each detection angle to the obstacle within the theoretical scanning angle range, and obtain the measured distance of each detection angle in the frame of data.

[0450] Summarize the measured distance of obstacles at each detection angle in each frame data,

[0451] For each detection angle,

[0452] The average value of each measured distance of the detection angle in each frame data is calculated to obtain the measured distance of the detection angle.

[0453] Determine whether the error between the measured distance at the detection angle and the theoretical measured distance at the detection angle exceeds a set error threshold. If so, determine that the distance measurement of the second laser radar at the detection angle is abnormal.

[0454] in,

[0455] The theoretical scanning angle range is calculated based on the coordinates of the two endpoints of the projection line segment in the second lidar coordinate system.

[0456] The theoretical scanning angle range is from the starting angle to the ending angle.

[0457] in,

[0458] The starting and ending angles are determined as follows:

[0459] Convert the coordinates of the two endpoints of the projection line segment in the obstacle avoidance coordinate system to the coordinates of the second lidar coordinate system.

[0460] In the second laser radar coordinate system, the starting angle is determined according to the coordinates of one end point, and the ending angle is determined according to the coordinates of the other end point.

[0461] The embodiment of the present application realizes comprehensive detection of the entire mobile robot by analyzing the detection data collected during the detection action, thereby improving the maintenance efficiency of the mobile robot.

[0462] See also Figure 9 As shown, Figure 9 This is a schematic diagram of a detection device for a mobile robot assembly according to an embodiment of the present application. The device includes:

[0463] The detection action execution module is used to execute the detection action in response to the detection task instruction, wherein the detection task instruction is triggered when the detection trigger condition of the mobile robot is met.

[0464] The detection data acquisition module is used to collect detection data of the mobile robot body during the detection action in the detection area.

[0465] The component detection module uses the collected detection data to detect at least one of the mobile robot's navigation and positioning component, business component, and obstacle avoidance component.

[0466] As an example, the detection task instruction includes: a first detection instruction for detecting a navigation and positioning component of a mobile robot, a second detection instruction for detecting a business component of a mobile robot that performs a business, and a third detection instruction for detecting an obstacle avoidance component of a mobile robot.

[0467] The detection action execution module includes:

[0468] The first detection action execution submodule is used to execute the first detection action.

[0469] The second detection action execution submodule is used to execute the second detection action,

[0470] The third detection action execution submodule is used to execute the third detection action.

[0471] The component detection module includes:

[0472] The navigation and positioning component detection submodule is used to obtain verification data using the identified location information and detect the navigation and positioning component of the mobile robot based on the detection data collected in the positioning area.

[0473] The business component detection submodule is used to obtain verification data using the global posture information in the cargo code identification, and detect the business components of the mobile robot based on the detection data collected in the business area.

[0474] The obstacle avoidance component detection submodule uses the obstacle detection position information to obtain verification data and detects the obstacle avoidance component of the mobile robot based on the detection data collected in the obstacle avoidance area.

[0475] See also Figure 10 As shown, Figure 10 Another schematic diagram of a detection device for a mobile robot assembly according to an embodiment of the present application. The device includes a memory and a processor, wherein the memory stores a computer program and the processor is configured to execute the computer program to implement the steps of the detection method for a mobile robot assembly according to an embodiment of the present application.

[0476] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage. Alternatively, the memory may be at least one storage device located away from the processor.

[0477] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components.

[0478] An embodiment of the present invention further provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of the detection method of the mobile robot component described in the embodiment of the present application are implemented.

[0479] As for the apparatus / network-side device / storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0480] In this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus comprising the element.

[0481] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for detecting a mobile robot component, characterized in that: The method includes: Collect detection data of the mobile robot body during the detection action in the detection area, The detection area includes at least one of: a positioning area for detecting a navigation and positioning component of a mobile robot, a service area for detecting a service component of a mobile robot that performs services, and an obstacle avoidance area for detecting an obstacle avoidance component of a mobile robot. The detection action includes at least one of: a first detection action for obtaining detection data of a navigation and positioning component, a second detection action for obtaining detection data of a business component, and a third detection action for obtaining detection data of an obstacle avoidance component. The detection data includes: the reported data of each sensor installed in each component of the mobile robot body, The collected detection data is used to detect at least one of the navigation and positioning component, the business component, and the obstacle avoidance component of the mobile robot.

2. The detection method according to claim 1, wherein The positioning area includes: a positioning identifier for identifying the actual position information of each position on the set detection path, The first detection action at least includes: a movement detection action of moving a set first distance along a set detection path, and a rotation detection action of rotating at a set first rotation angle. The detection data at least includes: synchronous reporting data of each sensor in the navigation and positioning component installed on the mobile robot body, The detection of the navigation and positioning component includes: Using the identified location information to obtain verification data, and based on the detection data collected in the positioning area, detecting the navigation and positioning component of the mobile robot; in, The detection data collected during the execution of the mobile detection action is used to detect the position accuracy of the navigation and positioning component and / or the external reference positioning posture of the navigation and positioning component. The detection data collected during the execution of the rotation detection action is used to detect the angular accuracy of the navigation and positioning component.

3. The detection method according to claim 1, wherein The business area includes a shelf, and the bottom surface of the shelf has a cargo code mark for business component detection, and the cargo code mark includes actual global position information of the shelf; The second detection action includes: rotating at a set second rotation angle under the shelf and performing a business operation action to observe the cargo code identification; The detection data includes: synchronous reporting data of each sensor in the business component installed in the mobile robot body; The detection of the business component includes: Use the global position information in the cargo code identification to obtain verification data, and detect the business components of the mobile robot based on the detection data collected in the business area; in, The detection data collected during the execution of the second detection action is used to detect at least one of the external parameter positioning accuracy and the business operation accuracy of the business component.

4. The detection method according to claim 1, wherein The obstacle avoidance area includes: the obstacle facade; The third detection action includes: a detection action of collecting image data of obstacle facade information at a set obstacle detection position; The detection data includes: synchronous reporting data of each sensor in the obstacle avoidance component installed in the mobile robot body; The detection of the obstacle avoidance component includes: Obstacle detection location information is used to obtain verification data, and the obstacle avoidance component of the mobile robot is tested based on the detection data collected in the obstacle avoidance area; in, The detection data collected during the execution of the third detection action is used to detect at least one of the installation posture accuracy and the ranging accuracy of the obstacle avoidance component; The service area, positioning area, and obstacle avoidance area are adjacent in sequence.

5. The detection method according to claim 2, wherein The detection path is a straight path; the positioning marks are distributed on the straight path at equal intervals; The navigation and positioning component includes: at least one of a vision module, an odometer, and an inertial measurement unit for visual navigation and positioning; The detection data includes: at least one of image data, odometer data, and inertial measurement unit data for visual navigation positioning; The detecting of the navigation and positioning component of the mobile robot includes at least one of the following: Based on the image data in the detection data, the first verification data is obtained using the position information or movement trajectory of the positioning mark to detect the positioning posture of the external reference mark of the visual module. Based on the odometer data and image data in the detection data, the second verification data is obtained using the position information of the marker, and the odometer is tested, the test including: at least one of the odometer rotation angle accuracy test and the odometer position accuracy test, Based on the inertial measurement unit data and the image data in the detection data, the third verification data is obtained using the position information of the marker to detect the heading angle accuracy of the inertial measurement unit.

6. The detection method according to claim 5, wherein The visual module includes a first laser radar for laser navigation and positioning; The image data is the first laser point cloud image data reported by the first laser radar, The detection of the external reference positioning posture of the visual module includes: at least one of the position calibration detection of the first laser radar and the posture calibration detection, in, Position calibration detection includes: Determine the observed global position information of the positioning marker based on the point cloud image data of the positioning marker included in the first laser point cloud image data, Calculating the position standard deviation between the observed global positions of all positioning markers and the actual position information of all positioning markers, wherein the actual position information of all positioning markers is used as the first verification data, Determine whether the position standard deviation is less than a set position standard deviation threshold; if so, determine that the position calibration of the first laser radar is accurate; otherwise, determine that the position calibration of the first laser radar is inaccurate; Attitude calibration detection includes: Determine the moving trajectory of the mobile robot based on the navigation positioning information determined by the laser point cloud image data reported by the first laser radar, Intercept the approximate straight line trajectory data of the mobile robot's moving trajectory, Based on the intercepted trajectory data, a straight line is fitted to obtain a fitting straight line. Calculate the linear angle of the fitted line in the world coordinate system to obtain the trajectory angle, which is used as the first calibration data. Select any trajectory point in the intercepted trajectory data, and obtain the observed global posture information of the trajectory point based on the navigation positioning information of the trajectory point. Determine whether the angle error between the observed global posture information of the trajectory point and the trajectory angle is less than the set angle error threshold, If so, it is determined that the attitude calibration of the first laser radar is accurate; otherwise, it is determined that the attitude calibration of the first laser radar is inaccurate.

7. The detection method according to claim 5, wherein The positioning mark is a downward-looking code located in a bearing surface for supporting the mobile robot, and the visual module includes a downward-looking camera for reading the downward-looking code, and the downward-looking camera is installed at the bottom of the mobile robot body; The image data is downward-viewing coded image data reported by the downward-viewing camera; The detection of the external reference positioning posture of the visual module includes: position calibration detection of the downward-looking camera, in, Position calibration detection includes: For each frame of the lower view code image data collected during the movement along the set detection path, it is identified whether the frame contains the lower view code image. If yes, then Determine the observed global position information of the lower view code corresponding to the lower view code image in the frame, and obtain the scanning lateral deviation distance and forward deviation distance of the lower view code, wherein the scanning lateral deviation distance is used to characterize the distance between the observed global position of the lower view code and the actual position of the lower view code, and the actual position of the lower view code is used as the first verification data. The forward deviation distance is used to characterize the distance error between the distance between the observed global positions of the two lower view codes and the distance between the actual positions of the two lower view codes, and the distance between the actual positions of the two lower view codes is used as the first verification data. Calculate the average lateral deviation distance and the average forward deviation distance of all scanned codes. Otherwise, the navigation positioning information of the frame is determined based on the frame, and the distance between the position of the navigation positioning information and the detection path at the position is calculated to obtain the code-free deviation distance. Determine whether the average value of the lateral deviation distance of the code scanning is less than the set lateral deviation distance threshold of the code scanning, whether the average value of the forward deviation distance is less than the set forward deviation distance threshold, and whether the no-code deviation distance is less than the set no-code deviation distance threshold. If so, it is determined that the downward-looking camera extrinsic calibration is accurate; otherwise, it is determined that the downward-looking camera extrinsic calibration is inaccurate.

8. The detection method according to claim 5, wherein The positioning mark is a downward-looking code located in the bearing surface for supporting the mobile robot. The visual module includes a downward-looking camera for reading the downward-looking code and a first laser radar for laser navigation and positioning. The image data is downward-looking code image data reported by the downward-looking camera and first laser point cloud image data reported by the first laser radar; The detecting of the external reference positioning posture of the visual module includes: at least one of position calibration detection of the first laser radar, attitude calibration detection, and position calibration detection of the downward-looking camera; in, The first laser radar position calibration detection includes: For each frame of the lower view code image data, the global position information of the lower view code is determined according to the local position of the lower view code corresponding to the lower view code image in the frame in the mobile robot coordinate system and the navigation positioning information determined based on the first laser point cloud image data. Calculate the position standard deviation between the observed global position of all the lower view codes and the actual position information of all the lower view codes, wherein the actual position information of all the lower view codes is used as the first verification data, Determine whether the position standard deviation is less than a set position standard deviation threshold; if so, determine that the position calibration of the first laser radar is accurate; otherwise, determine that the position calibration of the first laser radar is inaccurate; The attitude calibration detection of the first laser radar includes: Determine the moving trajectory of the mobile robot based on the navigation positioning information determined by the laser point cloud image data reported by the first laser radar, Intercept the approximate straight line trajectory data of the mobile robot's moving trajectory, Based on the intercepted trajectory data, a straight line is fitted to obtain a fitting straight line. Calculate the linear angle of the fitted line in the world coordinate system to obtain the trajectory angle, which is used as the first calibration data. Select any trajectory point in the intercepted trajectory data, and obtain the observed global posture information of the trajectory point based on the navigation positioning information of the trajectory point. Determine whether the angle error between the observed global posture information of the trajectory point and the trajectory angle is less than the set angle error threshold, If yes, it is determined that the attitude calibration of the first laser radar is accurate; otherwise, it is determined that the attitude calibration of the first laser radar is inaccurate; The downward-looking camera position calibration test includes: For each frame of the lower view code image data collected during the movement along the set detection path, it is identified whether the frame contains the lower view code image. If yes, then Determine the observed global position information of the lower view code corresponding to the lower view code image in the frame, and obtain the scanning lateral deviation distance and forward deviation distance of the lower view code, wherein the scanning lateral deviation distance is used to characterize the distance between the observed global position of the lower view code and the actual position of the lower view code, and the actual position of the lower view code is used as the first verification data. The forward deviation distance is used to characterize the distance error between the distance between the observed global positions of the two lower view codes and the distance between the actual positions of the two lower view codes, and the distance between the actual positions of the two lower view codes is used as the first verification data. Calculate the average lateral deviation distance and the average forward deviation distance of all scanned codes. Otherwise, the navigation positioning information of the frame is determined based on the frame, and the distance between the position of the navigation positioning information and the detection path at the position is calculated to obtain the code-free deviation distance. Determine whether the average value of the lateral deviation distance of the code scanning is less than the set lateral deviation distance threshold of the code scanning, whether the average value of the forward deviation distance is less than the set forward deviation distance threshold, and whether the no-code deviation distance is less than the set no-code deviation distance threshold. If so, it is determined that the downward-looking camera extrinsic calibration is accurate; otherwise, it is determined that the downward-looking camera extrinsic calibration is inaccurate.

9. The detection method according to claim 7 or 8, wherein The detection of the navigation and positioning component of the mobile robot further includes: detecting the recognition rate of the downward-looking camera based on the downward-looking code image frame data, The downward-looking camera recognition rate test includes: Based on the lower view code image frame data, identify the lower view code image in each frame, and count the number of different lower view codes. Check whether the number of down-view codes counted is the same as the actual number of down-view codes, If yes, then the recognition rate of the camera is determined to be accurate. Otherwise, it is determined that the recognition rate of the lower view code camera is inaccurate.

10. The detection method according to any one of claims 5 to 8, characterized in that: The odometer rotation angle accuracy detection includes: Based on any two odometer data collected during the same direction rotation process, the odometer cumulative angle change of the two odometer data is determined. Based on the navigation positioning information determined during the rotation process, two navigation positioning information having the same time information as the two odometer data are obtained, and an angle change is determined based on the two navigation positioning information, and the angle change is used as the second verification data. Calculating a first deviation between the odometer cumulative angle change and the second verification data, determining whether the first deviation value is less than a set first deviation threshold; if so, determining that the odometer rotation angle is accurate; otherwise, determining that the odometer rotation angle is inaccurate; The odometer position accuracy detection includes: Based on any two odometer data collected during the straight path movement of the mobile robot, the odometer cumulative distance of the two odometer data is determined. Based on the navigation positioning information determined during the straight path movement, two navigation positioning information having the same time information as the two odometer data are obtained, and the distance between the two navigation positioning positions is determined based on the two navigation positioning information, and the distance is used as the second verification data, Calculate the distance difference between the odometer accumulated distance and the second verification data, Determine whether the distance difference is less than the set distance threshold. If so, the odometer position is determined to be accurate; otherwise, the odometer position is determined to be inaccurate. The heading angle accuracy detection includes: Based on any two gyroscope data collected from the inertial measurement unit data during the same direction of rotation, the gyroscope cumulative angle change of the two data is determined. Based on the navigation positioning information determined during the rotation process, two navigation positioning information having the same time information as the two gyroscope data are obtained, and an angle change is determined based on the two navigation positioning information, and the angle change is used as the third verification data. Calculating a second deviation between the odometer cumulative angle change and the second verification data, Determine whether the second deviation value is less than a set second deviation threshold value, if so, determine that the gyroscope heading angle is accurate, otherwise, determine that the gyroscope heading angle is inaccurate; in, Navigation positioning information is determined as follows: When the down-view code image data is collected, the change in the scanning angle determined based on the down-view code image data is used as navigation positioning information. In a case where the first laser point cloud image data is collected but the down-view code image data is not collected, the navigation positioning information is determined based on the first laser point cloud image data.

11. The detection method according to claim 3, wherein The mobile robot is a lifting type mobile robot, and the mobile robot body is equipped with an upward-looking camera for reading cargo code identification; The bearing surface below the shelf in the business area includes a positioning mark; The second rotation angle is achieved in the following manner: the mobile robot moves at the same position at at least three different orientation angles, wherein the sum of the circular angles formed by the orientation angles is equal to the second rotation angle; The business operation actions include: lifting and lowering operations; The detection data includes: cargo code image data reported by the upward-looking camera; The business component detection includes: at least one of: upward-looking camera extrinsic parameter position calibration detection, upward-looking camera extrinsic parameter attitude calibration detection, and upward-looking camera recognition rate detection; The business operation accuracy detection includes: posture consistency detection before and after lifting; in, Position calibration detection includes: For each frame of the cargo code image data, the observed global position of the cargo code is obtained based on the frame. Determine the circular radius of the trajectory formed by each observed global position according to each observed global position, wherein the number of observed global positions is at least three, Determine whether the circle radius is greater than the set radius threshold. If so, determine that the position calibration is accurate; otherwise, determine that the position calibration is inaccurate. Attitude calibration detection includes: Based on the preceding and following frames including the positioning marker information in the image data, two local postures of the positioning marker observed in the preceding and following frames in the mobile robot coordinate system are obtained, and based on the two local postures, a posture change amount between the first frames is determined, Based on the preceding and following frames in the cargo code image data, two local postures of the cargo code mark observed in the preceding and following frames in the mobile robot coordinate system are obtained, and based on the two local postures, a posture change amount between the second frames is determined, wherein the preceding and following frames including the positioning mark information have the same time information as the preceding and following frames in the cargo code image data. Calculate the posture change error between the first frame posture change and the second frame posture change, Determine whether the attitude change error is less than the set attitude change threshold. If so, the attitude calibration is determined to be accurate; otherwise, the attitude calibration is determined to be inaccurate. Posture consistency testing before and after lifting includes: For each frame of the cargo pallet image data collected before lifting, obtain the first position of the cargo pallet in the frame, and calculate the average value of all the first position values. For each frame of cargo pallet image data collected after lifting, obtain the second posture of the cargo pallet in the frame, and calculate the average value of all the second postures. Determine whether the pose error between the average value of the first pose and the average value of the second pose is lower than the set pose error threshold. If so, it is determined that there is consistency; otherwise, it is determined that there is no consistency; Upward-looking camera recognition rate detection includes: For each frame of cargo code image data collected before lifting, identify the cargo code in the frame. For each frame of cargo code image data collected after lifting, identify the cargo code in the frame. Mark each image frame where recognition fails, It is counted whether the number of consecutive frames in the marked image frames is greater than a set frame number threshold. If so, it is determined that the recognition rate is inaccurate; otherwise, it is determined that the recognition rate is accurate.

12. The detection method according to claim 4, wherein The obstacle detection position is located on the perpendicular midline of the projection line segment of the obstacle facade on the bearing surface, and the distance between the obstacle facade and the obstacle detection position meets the set obstacle avoidance detection distance; The obstacle avoidance component includes: a second laser radar for obstacle avoidance; The detection data is the second laser point cloud image data reported by the second laser radar; The third detection action includes: rotating clockwise and counterclockwise at least once at the detection position; The installation posture accuracy detection includes: For each frame in the second laser point cloud image data, Convert the coordinates of the point cloud in the frame in the second laser radar coordinate system to the coordinates in the obstacle avoidance coordinate system. Based on the distribution of obstacles in the obstacle avoidance area, the point cloud of the obstacle and the point cloud of the obstacle avoidance area in the non-obstacle area are screened out. Count the number of point clouds in the filtered area of ​​the frame to get the number of point clouds in the obstacle avoidance area of ​​the frame. Perform point cloud feature extraction on the point cloud of the screened obstacles. Based on the extracted point cloud features, the point cloud straight line of the obstacle is fitted to obtain the fitted straight line. Determine the angle between the fitted straight line and the horizontal direction of the obstacle avoidance coordinate system to obtain the fitted straight line angle; Summarize the number of obstacle avoidance area point clouds and the angle of the fitted line for each frame. Determine whether the difference between the average of all fitted line angles and the theoretical line angle exceeds the set angle threshold. If so, determine that the second lidar calibration is abnormal. The theoretical line angle is the angle between the projected line segment in the obstacle avoidance coordinate system and the horizontal direction of the obstacle avoidance coordinate system. This angle is used as verification data. Determine whether the number of point clouds in the obstacle avoidance area of ​​all frames is greater than a set first number threshold. If so, if the number of point clouds within the boundary of the mobile robot body is greater than a set second number threshold, determine that the second laser radar is interfered with by the mobile robot body; if the number of point clouds within the boundary of the mobile robot body is not greater than the set second number threshold, determine that the second laser radar is interfered with by the bearing surface of the mobile robot body; The ranging accuracy detection includes: For each frame of the second laser point cloud image data, determine the measured distance of each detection angle to the obstacle within the theoretical scanning angle range, and obtain the measured distance of each detection angle in the frame. Summarize the measured distance of obstacles at each detection angle in each frame, For each detection angle, The average value of each measured distance of the detection angle in each frame is calculated to obtain the measured distance of the detection angle. Determine whether the error between the measured distance of the detection angle and the theoretical measured distance of the detection angle exceeds the set distance error threshold. If so, determine that the second laser radar is abnormal in the distance measurement of the detection angle, and use the theoretical measured distance of the detection angle as verification data. in, The theoretical scanning angle range is calculated based on the coordinates of the two endpoints of the projection line segment in the second lidar coordinate system.

13. The detection method according to claim 4, wherein The first detection action, the second detection action, and the third detection action are performed in the following consecutive actions: In the business area, at the starting position mark, take the forward direction detection path of the mobile robot body as the starting direction, rotate 90 degrees, move the set second distance in the current direction, rotate 180 degrees and return to the starting position mark, then move the set second distance in the current direction and rotate 180 degrees and return to the starting position mark, then rotate 90 degrees to the starting direction, after completing the first detection action and returning to the starting position mark, perform the lifting and lowering operation to complete the second detection action; In the positioning area, at the starting point positioning mark, move along the detection path to the end point positioning mark in the starting direction, and rotate clockwise and counterclockwise at the end point positioning mark to complete the third detection action, then rotate 180 degrees and return to the starting point positioning mark along the detection path to complete the first detection action. in, The starting point positioning mark is located under the shelf, and the end point positioning mark is located at the obstacle detection position. The accumulation of each forward direction in the service area forms a second rotation angle.

14. A detection device for a mobile robot component, characterized in that: The device includes: The detection data acquisition module is used to collect detection data of the mobile robot body during the detection action in the detection area. The detection area includes at least one of: a positioning area for detecting a navigation and positioning component of a mobile robot, a service area for detecting a service component of a mobile robot that performs services, and an obstacle avoidance area for detecting an obstacle avoidance component of a mobile robot. The detection action includes at least one of: a first detection action for obtaining detection data of a navigation and positioning component, a second detection action for obtaining detection data of a business component, and a third detection action for obtaining detection data of an obstacle avoidance component. The detection data includes: the reported data of each sensor installed in each component of the mobile robot body, The component detection module uses the collected detection data to detect at least one of the mobile robot's navigation and positioning component, business component, and obstacle avoidance component.

Images