Robotic asset inventory method based on multi-sensor fusion

Abstract

The application relates to a robot asset inventory method based on multi-sensor fusion, and relates to the technical field of artificial intelligence.The robot asset inventory method based on multi-sensor fusion and a system thereof adopt an RFID, multi-line laser radar, IMU multi-sensor fusion mode to acquire asset information of an inventory area, input the asset information as a point cloud strong connection point into a mapping algorithm to complete the construction of a real-time high-precision asset point cloud map, and directly visualize the asset position information in the asset point cloud map, thereby improving the traditional inventory mode of manually memorizing asset positions, solving the problem of unclear fixed asset positions, quickly responding to asset dynamic adjustment and the like, comparing the asset information of corresponding assets in a robot recognition area with a reference sequence, quickly giving a missed detection, false detection and abnormal report, realizing asset inventory, saving the manual inventory step in the whole inventory process, improving the intelligent degree, and effectively shortening the fixed asset inventory period.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and specifically to a method for inventorying robot assets based on multi-sensor fusion. Background Technology

[0002] With the continuous advancement of new and old growth drivers, intelligentization, and automation in various sectors, and the ongoing expansion of enterprise scale, the types and quantities of fixed assets such as daily office equipment, high-precision instruments, and production line equipment are rapidly increasing. Due to their wide distribution, high mobility, and diverse attributes, asset management is quite challenging. The quantity, quality, and technical structure of an enterprise's fixed assets are indispensable to its production capacity and management level. If fixed asset management is chaotic, and the physical assets do not match the accounting records, it will not only lead to asset loss but also affect the enterprise's daily production and operation activities. Therefore, strengthening the rational and efficient classification and organization of fixed assets can optimize fixed asset utilization efficiency, clarify the details of enterprise asset attributes, and further save on enterprise capital losses.

[0003] In existing technologies, asset inventory often relies on manual scanning with handheld RFID receivers, followed by manual registration of asset information into a computer for comparison. This process is time-consuming, labor-intensive, and prone to missed or incorrect detections. Furthermore, traditional asset inventory methods lack dynamic management. When asset transfers are frequent, inventory personnel become unfamiliar with the new asset status, leading to industry pain points such as unclear fixed asset locations and inaccurate asset tag data. This compromises inventory efficiency and reliability, creating management challenges. Traditional asset inventory methods often rely on manual memorization of asset locations. In production line environments, with a wide variety of equipment and the need to record equipment usage time, type, and other attributes, and given the broad and dense distribution of assets, inventory personnel cannot quickly handle asset changes, severely impacting work efficiency. Summary of the Invention

[0004] In order to overcome the shortcomings of the above technologies, this invention provides a robot asset inventory method that shortens the fixed asset inventory cycle and solves problems such as unclear fixed asset locations.

[0005] The technical solution adopted by this invention to overcome its technical problems is:

[0006] A method for inventorying robot assets based on multi-sensor fusion includes:

[0007] An RFID tag is set on each asset device, the UID of the RFID tag is bound to the corresponding asset device, and the attribute information is registered in the JSON file of the asset entry processing module to obtain the asset information reference sequence; the asset entry processing module sends the JSON file to the computing unit of the mobile robot.

[0008] As the mobile robot passes through the work area, it perceives the environment and outputs raw laser point cloud data to its computing unit.

[0009] The IMU sensor installed on the mobile robot sends the robot's posture information to the robot's computing unit;

[0010] After the mobile robot triggers the asset patrol and inventory task instruction, the mobile robot inspects the asset equipment and obtains the RFID signal by reading the RFID patch of the asset equipment through the set RFID receiver. It then obtains the attribute information of the asset equipment corresponding to the RFID signal through the asset information reference sequence.

[0011] The patrol and inventory module consists of an asset point cloud generation unit and a 3D point cloud mapping unit. The mobile robot's computing unit sends the original laser point cloud data, RFID signals, and the mobile robot's posture information to the patrol and inventory module to generate an asset point cloud map.

[0012] The asset self-inspection module provides intelligent reminders after generating an asset point cloud map.

[0013] Furthermore, the assets and equipment are production line equipment or daily office equipment.

[0014] Furthermore, the attribute information includes: UID information, asset type, name, and usage time of the asset device. Additionally, the mobile robot moves through the work area via remote control or manual pushing.

[0015] Furthermore, the mobile robot is equipped with multi-line lidar, which allows it to perceive the environment and output raw lidar point cloud data. RFID receivers are installed on the left and right sides of the mobile robot, respectively.

[0016] Furthermore, the processing steps of the asset point cloud generation unit are as follows:

[0017] After the mobile robot triggers the asset patrol and inventory task instruction, the mobile robot checks whether the asset information reference sequence JSON file is valid. If it is invalid, the asset self-check module outputs an asset information reference sequence error. If it is valid, the robot drives the multi-line LiDAR, IMU sensor, and RFID receiver. If the mobile robot fails to obtain the original LiDAR point cloud data and / or the mobile robot's attitude information, the asset self-check module outputs a sensor error. If the mobile robot obtains the original LiDAR point cloud data and the mobile robot's attitude information, the corresponding data queue is set to the original LiDAR point cloud data and the mobile robot's attitude information thin value under the current timestamp.

[0018] During the mobile robot's inspection process, the RFID receiver scans the RFID patches in the work area. If the RFID receiver does not identify relevant information within the tolerance time for the current frame of the laser point cloud, the raw laser point cloud data is directly output. If the RFID receiver identifies relevant information within the tolerance time for the current frame of the laser point cloud, the identified scan information is filtered and added to the asset information reference queue. RFID signals that do not belong to the asset information reference queue but belong to the asset information reference sequence are filtered out. If there are no valid RFID signals in the filtered scan information, the raw laser point cloud data is directly output. If there are unregistered valid RFID signals in the filtered scan information, the UID of the RFID signal is stored in the asset information reference sequence, and the RFID signal is converted to the lidar coordinates and then processed using the formula... World coordinate system calculated In the formula The coordinate system from the RFID receiver to the robot body is as follows. This represents the coordinate system from the lidar coordinate system to the robot's body coordinate system, where T is the transpose. This is the transformation matrix from the lidar coordinate system to the world coordinate system;

[0019] world coordinate system The RFID signal is simulated as a scanning point in a frame of LiDAR, and this scanning point is taken as a strong connection point. The LiDAR point cloud with strong connection points is taken as the asset point cloud.

[0020] Preferably, the tolerance time is 0.1s.

[0021] Furthermore, it also includes saving the strong connection points and laser point cloud indexes in the lidar coordinate system to the asset information reference sequence with the corresponding UID.

[0022] Furthermore, the processing steps for the 3D point cloud mapping unit are as follows:

[0023] S01. Continuously acquire raw data from the IMU sensor and save the raw data to the IMU receiving queue;

[0024] S02. Through the formula The original angular velocity observation value of the robot was calculated. In the formula ω t The value is the raw angular velocity measurement of the IMU sensor at time t in the robot coordinate system. Let be the random walk value of the angular velocity at time t. The white noise represents the angular velocity at time t, expressed by the formula... The original acceleration observations of the robot were calculated. R t BWLet a be the rotation matrix from the world coordinate system to the body coordinate system at time t. t Let be the initial acceleration value at time t, and g be the invariant gravitational acceleration in the world coordinate system. Let be the random walk value of acceleration at time t. The white noise represents the angular velocity at time t, expressed by the formula... The robot's velocity v at time t is calculated based on the measurements taken by the IMU sensor. t In the formula, v t-1 R is the robot's predicted velocity at time t-1 based on IMU sensor measurements, where Δt is the time difference between time t and t-1. t-1 The robot's pose information at time t-1 is obtained through the formula... The position information p predicted by the robot at time t using the IMU sensor measurements is calculated. t In the formula p t-1 The position information predicted by the robot at time t-1 using IMU sensor measurements is expressed by the formula. The robot's attitude information R, predicted by the IMU sensor measurements at time t, is calculated. t Output the robot's raw angular velocity observation value at the current time t to the 3D point cloud mapping unit. raw acceleration observations speed v t Location information p t Attitude information R t Complete the robot pose calculation;

[0025] S03. The 3D point cloud mapping unit uses motion filtering to remove part of the point cloud from the asset point cloud to obtain the optimized point cloud, and calculates the robot's position information p at time t. t Attitude information R t The position information p at time t-1 in the relative point cloud optimization part t-1 Attitude information R t-1 If the asset point cloud at time t does not contain strong connections or the position and attitude increments are both less than the set position increment threshold of 0.1m and the attitude increment threshold of 5°, then step S02 is used to complete the robot's predicted pose calculation, and it does not participate in the mapping operation. If the asset point cloud at time t contains strong connections or the calculated position increment is greater than or equal to the set position increment threshold of 0.1m or the attitude increment is greater than or equal to the attitude increment threshold of 5°, then a body velocity filtering operation with a body velocity mesh size of 4cm is performed on the current asset point cloud. Strong connections do not participate in the filtering process. The robot's position information p at time t output in step S02 is then processed. tThe filtered current frame asset point cloud is used as the input of the NDT matching algorithm. After scanning and matching calculation by the NDT algorithm, the relative incremental information of the current frame asset point cloud relative to the key asset point cloud frame recorded at time t-1 is obtained, and the key asset point cloud frame queue is obtained.

[0026] S04. The position increment and attitude increment values ​​of the key asset point cloud frames recorded from time t to time t-1 are added to the GTSAM factor graph as laser odometry factors. The SCAN CONTEXT loop closure algorithm is used to search for loops in the key asset point cloud frame queue. If the loop closure algorithm detects that the robot is in a loop environment, the loop constraint is added to the GTSAM factor graph to form a loop closure factor. The factor graph optimization operation is performed to save the asset point cloud information, coordinate transformation from the laser radar coordinate system to the world coordinate system, pose information of strong connection points of the laser point cloud, and other effective information of the key asset point cloud frame at time t to the key asset point cloud frame queue.

[0027] S05. After the robot inspection completes the scanning of the area to be inspected, the 3D point cloud mapping unit stitches together the asset point cloud information recorded in the key asset point cloud frame and the pose information of the strong connection points of the laser point cloud based on the coordinate transformation from the laser radar coordinate system to the world coordinate system, thus completing the construction of the 3D point cloud map.

[0028] Furthermore, the asset self-inspection module provides intelligent reminders after generating the asset point cloud map as follows:

[0029] The robot reads the pose information and UID information of all strong connection points, transforms the pose of the previous connection points to the world coordinate system, uses the UID information of all strong connection points transformed to the world coordinate system as the key index, and adds the pose information of the strong connection points as the value to the asset information comparison sequence.

[0030] Determine if the asset information comparison sequence matches the asset information reference sequence. If they match, update the corresponding asset location information to the asset information reference sequence to complete the inventory task. If they do not match and there is no asset location information in the asset information reference sequence, find the points with missed detection anomalies and perform asset rescanning. If they do not match but there is asset pose information in the asset information reference sequence, find the global pose of the undetected asset based on the asset information reference sequence and provide the host computer with the robot's route feedback value to the undetected asset based on the A* algorithm, until the asset information comparison sequence matches the asset information reference sequence.

[0031] The beneficial effects of this invention are as follows: The robot asset inventory method and system based on multi-sensor fusion uses RFID, multi-line LiDAR, and IMU multi-sensor fusion to acquire asset information of the inventory area. The asset information is input as strong connection points of the point cloud into the mapping algorithm to complete the construction of a real-time high-precision asset point cloud map. The asset location information is directly visualized in the asset point cloud map, which improves the traditional inventory method of manually memorizing asset locations and solves problems such as unclear fixed asset locations. It can quickly respond to situations such as dynamic asset adjustments. When performing repeated asset inventory, the reference sequence of asset information assigned to the asset location is read. By comparing the asset information of the corresponding asset in the robot's recognition area with the reference sequence, a report of missed detections and false detections can be quickly given to realize asset inventory. The entire inventory process saves manual inventory steps, improves the level of intelligence, and effectively shortens the fixed asset inventory cycle. Attached Figure Description

[0032] Figure 1 This is a structural diagram of the multi-sensor fusion robot asset inventory system of the present invention;

[0033] Figure 2 This is a flowchart of the asset point cloud generation unit in the patrol and inventory module of the present invention;

[0034] Figure 3 This is a flowchart of the 3D point cloud mapping unit in the patrol and inventory module of the present invention;

[0035] Figure 4 This is a flowchart of the asset self-inspection module of the present invention. Detailed Implementation

[0036] The following is in conjunction with the appendix Figure 1 To be continued Figure 4 The present invention will be further described below.

[0037] A method for inventorying robot assets based on multi-sensor fusion includes:

[0038] An RFID tag is set on each asset device, and the UID of the RFID tag is bound to the corresponding asset device. The attribute information is registered in the JSON file of the asset entry processing module to obtain the asset information reference sequence. The asset entry processing module sends the JSON file to the computing unit of the mobile robot.

[0039] As the mobile robot passes through the work area, it perceives the environment and outputs raw laser point cloud data to its computing unit.

[0040] The IMU sensor installed on the mobile robot sends the robot's posture information to the robot's computing unit.

[0041] After the mobile robot triggers the asset patrol and inventory task instruction, the mobile robot conducts inspections and obtains RFID signals by reading the RFID patches on the asset equipment through the set RFID receiver. It then obtains the attribute information of the asset equipment corresponding to the RFID signal through the asset information reference sequence.

[0042] The patrol and inventory module consists of an asset point cloud generation unit and a 3D point cloud mapping unit. The mobile robot's computing unit sends the original laser point cloud data, RFID signals, and the mobile robot's attitude information to the patrol and inventory module to generate an asset point cloud map.

[0043] In one embodiment of the present invention, the asset self-inspection module provides intelligent reminders after generating an asset point cloud map. The asset equipment is production line equipment or daily office equipment. Attribute information includes: UID information, asset type, name, and usage time. The mobile robot traverses the work area via remote control or manual pushing. In one embodiment of the present invention, the mobile robot is equipped with a multi-line LiDAR, which perceives the environment and outputs raw LiDAR point cloud data. RFID receivers are installed on the left and right sides of the mobile robot, respectively.

[0044] In one embodiment of the present invention, the asset point cloud generation unit subscribes to the original laser point cloud published by the robot body to generate a strongly connected asset point cloud, and the point cloud mapping unit subscribes to the strongly connected asset point cloud and high-frequency IMU data to complete the construction of the asset point cloud map of the entire working area. The processing steps of the asset point cloud generation unit are as follows:

[0045] Task Anomaly Detection: After the mobile robot triggers the asset patrol and inventory task command, the mobile robot checks whether the asset information reference sequence JSON file is valid. If invalid, the asset self-check module outputs an asset information reference sequence anomaly error. If valid, the robot drives the multi-line LiDAR, IMU sensor, and RFID receiver. If the mobile robot fails to obtain the original LiDAR point cloud data and / or the mobile robot's attitude information, the asset self-check module outputs a sensor anomaly error. If the mobile robot obtains the original LiDAR point cloud data and the mobile robot's attitude information, the corresponding data queue is created by adding the original LiDAR point cloud data and the mobile robot's attitude information thin values ​​at the current timestamp.

[0046] Asset Point Cloud Generation: During the mobile robot's inspection process, the RFID receiver scans the RFID patches in the work area. If the RFID receiver does not identify relevant information within the tolerance time for the current frame of the laser point cloud, the raw laser point cloud data is directly output for subsequent mapping and positioning in the mapping module. If the RFID receiver identifies relevant information within the tolerance time for the current frame of the laser point cloud, to prevent false detections during the scanning process, such as employee cards or mobile phones with incorrect information, the identified scan information is filtered and added to the asset information reference queue. Due to the uniqueness of RFID, signals not belonging to the asset information reference queue but already registered in the current asset information sequence are filtered out. RFID signals not belonging to the asset information reference queue but belonging to the asset information reference sequence are also filtered out. If there are no valid RFID signals in the filtered scan information, the raw laser point cloud data is directly output. If there are unregistered valid RFID signals in the filtered scan information, the UID of the RFID signal is stored in the asset information reference sequence, and the RFID signal is converted to LiDAR coordinates and then processed using a formula. World coordinate system calculated In the formula The coordinate system from the RFID receiver to the robot body is as follows. This represents the coordinate system from the lidar coordinate system to the robot's body coordinate system, where T is the transpose. This is the transformation matrix from the lidar coordinate system to the world coordinate system.

[0047] world coordinate system The RFID signal is simulated as a scan point in a frame of LiDAR. This scan point is taken as a strong connection point, which has the highest priority and does not perform any filtering operation. The LiDAR point cloud with strong connection points is taken as the asset point cloud.

[0048] The tolerance time is set to 0.1s.

[0049] It also includes saving the strong connection points and laser point cloud indexes in the lidar coordinate system to the asset information reference sequence with the corresponding UID.

[0050] In one embodiment of the present invention, the processing steps of the three-dimensional point cloud mapping unit are as follows:

[0051] S01. Continuously acquire raw data from the IMU sensor and save the raw data to the IMU receive queue.

[0052] S02. This part continuously acquires raw data from the IMU sensor and saves it to the IMU receiving queue. Using the historical output pose of the 3D mapping unit at the previous moment as the IMU pre-integration observation pose, it then performs IMU inertial integration on the latest data in the IMU receiving queue based on this observation pose to calculate the robot's measured pose at the current moment. Specifically: using the formula... The original angular velocity observation value of the robot was calculated. In the formula ω t The value is the raw angular velocity measurement of the IMU sensor at time t in the robot coordinate system. Let be the random walk value of the angular velocity at time t. The white noise represents the angular velocity at time t, expressed by the formula... The original acceleration observations of the robot were calculated. R t BW Let a be the rotation matrix from the world coordinate system to the body coordinate system at time t. t Let be the initial acceleration value at time t, and g be the invariant gravitational acceleration in the world coordinate system. Let be the random walk value of acceleration at time t. The white noise represents the angular velocity at time t, expressed by the formula... The robot's velocity v at time t is calculated based on the measurements taken by the IMU sensor. t In the formula, v t-1 R is the robot's predicted velocity at time t-1 based on IMU sensor measurements, where Δt is the time difference between time t and t-1. t-1 The robot's pose information at time t-1 is obtained through the formula... The position information p predicted by the robot at time t using the IMU sensor measurements is calculated. t In the formula p t-1 The position information predicted by the robot at time t-1 using IMU sensor measurements is expressed by the formula. The robot's attitude information R, predicted by the IMU sensor measurements at time t, is calculated. t Output the robot's raw angular velocity observation value at the current time t to the 3D point cloud mapping unit. raw acceleration observations speed v t Location information p t Attitude information R t Complete the robot pose calculation.

[0053] S03. Point Cloud Matching: First, to reduce the computational burden on the key asset point cloud frame optimization, the 3D point cloud mapping unit uses motion filtering to remove part of the point cloud from the asset point cloud to obtain the optimized point cloud. Then, it calculates the robot's position information p at time t. t Attitude information R t The position information p at time t-1 in the relative point cloud optimization part t-1 Attitude information R t-1 If the asset point cloud at time t does not contain strong connections or the position and attitude increments are both less than the set position increment threshold of 0.1m and the attitude increment threshold of 5°, then step S02 is used to complete the robot's predicted pose calculation, and it does not participate in the mapping operation. If the asset point cloud at time t contains strong connections or the calculated position increment is greater than or equal to the set position increment threshold of 0.1m or the attitude increment is greater than or equal to the attitude increment threshold of 5°, then a body velocity filtering operation with a body velocity mesh size of 4cm is performed on the current asset point cloud to reduce the amount of point cloud matching calculation. Strong connections do not participate in the filtering process. The robot's position information p at time t output in step S02 is then used. t The filtered current frame asset point cloud is used as the input of the NDT matching algorithm. After scanning and matching calculation by the NDT algorithm, the relative incremental information of the current frame asset point cloud relative to the key asset point cloud frame recorded at time t-1 is obtained, and the key asset point cloud frame queue is obtained.

[0054] S04. Optimization of Key Asset Point Cloud Frames: The position increment and attitude increment values ​​of the key asset point cloud frames recorded from time t to time t-1 are added to the GTSAM factor graph as laser odometry factors. To reduce the cumulative error caused by noise in the mapping system, the SCAN CONTEXT loop closure algorithm is used to search for loops in the key asset point cloud frame queue. If the loop closure algorithm detects that the robot is in a loop environment, the loop constraint is added to the GTSAM factor graph to form a loop closure factor. The factor graph optimization operation is then performed. At this time, the saved historical key asset point cloud frame pose and the current key asset point cloud frame pose are optimized, and the cumulative error is offset. The asset point cloud information of the key asset point cloud frame at time t, the coordinate transformation from the laser radar coordinate system to the world coordinate system, and the pose information of the laser point cloud strongly connected points are saved to the key asset point cloud frame queue for use by the point cloud matching part.

[0055] S05. After the robot inspection completes the scanning of the area to be inspected, the inventory personnel need to issue a map construction completion command through the host computer. The 3D point cloud mapping unit will complete the stitching operation based on the coordinate transformation from the lidar coordinate system to the world coordinate system by the asset point cloud information recorded in the key asset point cloud frame and the pose information of the strong connection points of the laser point cloud, thus completing the construction of the 3D point cloud map.

[0056] In one embodiment of the present invention, the method by which the asset self-inspection module provides intelligent reminders after generating an asset point cloud map is as follows:

[0057] Asset Self-Check and Accounting: After the 3D point cloud mapping unit completes the mapping, all relevant information of the historical key asset point cloud frame queue is saved locally. First, the robot reads the pose information and UID information of all strongly connected points, transforms the pose of the previous connected points to the world coordinate system, and uses the UID information of all strongly connected points transformed to the world coordinate system as the key index and the pose information of the strongly connected points as the value to add to the asset information comparison sequence. Intelligent Reminder: Determine whether the asset information comparison sequence is consistent with the asset information reference sequence. If they are consistent, update the corresponding asset location information to the asset information reference sequence and complete the inventory task. If they are inconsistent and there is no asset location information in the asset information reference sequence, find the points with missed detection and perform asset rescanning. If they are inconsistent and there is asset pose information in the asset information reference sequence, find the global pose of the undetected assets based on the asset information reference sequence and provide the host computer with the route feedback value from the robot to the undetected assets based on the A* algorithm, until the asset information comparison sequence is consistent with the asset information reference sequence, and the task ends.

[0058] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for inventorying robot assets based on multi-sensor fusion, characterized in that, include: RFID tags are set on each asset device, the UID of the RFID tag is bound to the corresponding asset device, and the attribute information is registered in the JSON file of the asset entry processing module to obtain the asset information reference sequence. The asset entry processing module sends the JSON file to the mobile robot's computing unit; As the mobile robot passes through the work area, it perceives the environment and outputs raw laser point cloud data to its computing unit. The IMU sensor installed on the mobile robot sends the robot's posture information to the robot's computing unit; After the mobile robot triggers the asset patrol and inventory task instruction, the mobile robot inspects the asset equipment and obtains the RFID signal by reading the RFID patch of the asset equipment through the set RFID receiver. It then obtains the attribute information of the asset equipment corresponding to the RFID signal through the asset information reference sequence. The patrol and inventory module consists of an asset point cloud generation unit and a 3D point cloud mapping unit. The mobile robot's computing unit sends the original laser point cloud data, RFID signals, and the mobile robot's posture information to the patrol and inventory module to generate an asset point cloud map. The asset self-inspection module provides intelligent reminders after generating an asset point cloud map; The processing steps of the asset point cloud generation unit are as follows: After the mobile robot triggers the asset patrol and inventory task instruction, the mobile robot checks whether the asset information reference sequence JSON file is valid. If it is invalid, the asset self-check module outputs an asset information reference sequence error. If it is valid, the robot drives the multi-line LiDAR, IMU sensor, and RFID receiver. If the mobile robot fails to obtain the original LiDAR point cloud data and / or the mobile robot's attitude information, the asset self-check module outputs a sensor error. If the mobile robot obtains the original LiDAR point cloud data and the mobile robot's attitude information, the corresponding data queue is set to the original LiDAR point cloud data and the mobile robot's attitude information thin value under the current timestamp. During the mobile robot's inspection process, the RFID receiver scans the RFID patches in the work area. If the RFID receiver does not identify relevant information within the tolerance time for the current frame of the laser point cloud, the raw laser point cloud data is directly output. If the RFID receiver identifies relevant information within the tolerance time for the current frame of the laser point cloud, the identified scan information is filtered and added to the asset information reference queue. RFID signals that do not belong to the asset information reference queue but belong to the asset information reference sequence are filtered out. If there are no valid RFID signals in the filtered scan information, the raw laser point cloud data is directly output. If there are unregistered valid RFID signals in the filtered scan information, the UID of the RFID signal is stored in the asset information reference sequence, and the RFID signal is converted to the lidar coordinates and then processed using the formula... World coordinate system calculated In the formula The coordinate system from the RFID receiver to the robot body is as follows. From the lidar coordinate system to the robot body coordinate system, For transpose, This is the transformation matrix from the lidar coordinate system to the world coordinate system; world coordinate system The RFID signal is simulated as a scanning point in a frame of LiDAR, and this scanning point is taken as a strong connection point. The LiDAR point cloud with strong connection points is taken as the asset point cloud.

2. The robot asset inventory method based on multi-sensor fusion according to claim 1, characterized in that: The assets and equipment are either production line equipment or daily office equipment.

3. The robot asset inventory method based on multi-sensor fusion according to claim 1, characterized in that, The attribute information includes: UID information, asset type of the asset device, name, and usage time.

4. The robot asset inventory method based on multi-sensor fusion according to claim 1, characterized in that: The mobile robot moves through the work area by remote control or by hand.

5. The robot asset inventory method based on multi-sensor fusion according to claim 1, characterized in that: The mobile robot is equipped with a multi-line lidar. The mobile robot perceives the environment and outputs raw lidar point cloud data through the multi-line lidar. RFID receivers are installed on the left and right sides of the mobile robot respectively.

6. The robot asset inventory method based on multi-sensor fusion according to claim 1, characterized in that: The tolerance time is set to 0.1s.

7. The robot asset inventory method based on multi-sensor fusion according to claim 1, characterized in that: It also includes saving the strong connection points and laser point cloud indexes in the lidar coordinate system to the asset information reference sequence with the corresponding UID.

8. The robot asset inventory method based on multi-sensor fusion according to claim 1, characterized in that, The processing steps for a 3D point cloud mapping unit are as follows: S01. Continuously acquire raw data from the IMU sensor and save the raw data to the IMU receiving queue; S02. Through the formula The original angular velocity observation value of the robot was calculated. In the formula Robot coordinate system The raw angular velocity measurement value of the IMU sensor at that moment. for The random walk value of the angular velocity at time t. for The white noise of the angular velocity at time t is obtained through the formula. The original acceleration observations of the robot were calculated. , for The rotation matrix from the world coordinate system to the body coordinate system at any time. for The original acceleration value at time t. The invariant gravitational acceleration in the world coordinate system. for The random walk value of acceleration at time step. for The white noise of the angular velocity at time t is obtained through the formula. The calculations show that the robot is in Speed ​​predicted from IMU sensor measurements at all times In the formula For robots in The speed predicted from the IMU sensor measurements at all times. for Time and The time difference between moments For robots in Attitude information at any given time, expressed through a formula The calculations show that the robot is in Position information predicted from IMU sensor measurements at all times In the formula For robots in Position information predicted from IMU sensor measurements at all times, using the formula The calculations show that the robot is in Attitude information predicted from IMU sensor measurements at all times The robot outputs its current data to the 3D point cloud mapping unit. raw angular velocity observation at time t Original acceleration observations ,speed Location information Attitude information Complete the robot pose calculation; S03. The 3D point cloud mapping unit uses motion filtering to remove part of the asset point cloud, resulting in an optimized point cloud. (Computational robot...) Location information at any time Attitude information In the relative point cloud optimization section Location information at any time Attitude information Position increment and attitude increment, if If the point cloud of the time-series asset does not contain strong connections or if the position increment value and attitude increment value are both less than the set position increment threshold of 0.1m and attitude increment threshold of 5°, then step S02 is used to complete the robot's predicted pose calculation, and it does not participate in the mapping operation. If the current asset point cloud contains strong connections, or if the calculated position increment value is greater than or equal to a set position increment threshold of 0.1m or an attitude increment value is greater than or equal to an attitude increment threshold of 5°, then a body velocity filtering operation with a body velocity mesh size of 4cm is performed on the current asset point cloud. Strong connections are not included in the filtering process. The robot output from step S02 is then processed. Location information The filtered current frame asset point cloud is used as input to the NDT matching algorithm. After scanning and matching calculations by the NDT algorithm, the relative asset point cloud of the current frame is obtained. The relative incremental information of key asset point cloud frames is recorded in real time to obtain the key asset point cloud frame queue. S04. Will Moment Assets Point Cloud The position and attitude increment values ​​of the key asset point cloud frames recorded at all times are added to the GTSAM factor graph as laser odometry factors. The SCAN CONTEXT loop closure algorithm is used to search for loops in the key asset point cloud frame queue. If the algorithm detects that the robot is in a loop environment, the loop constraint is added to the GTSAM factor graph to form a loop closure factor. Factor graph optimization is then performed. The asset point cloud information, coordinate transformation from the lidar coordinate system to the world coordinate system, pose information of strongly connected points in the lidar point cloud frame, and other effective information of the key asset point cloud frame are saved to the key asset point cloud frame queue. S05. After the robot inspection completes the scanning of the area to be inspected, the 3D point cloud mapping unit stitches together the asset point cloud information recorded in the key asset point cloud frame and the pose information of the strong connection points of the laser point cloud based on the coordinate transformation from the laser radar coordinate system to the world coordinate system, thus completing the construction of the 3D point cloud map.

9. The robot asset inventory method based on multi-sensor fusion according to claim 1, characterized in that, The asset self-inspection module provides intelligent reminders after generating the asset point cloud map using the following method: The robot reads the pose information and UID information of all strong connection points, transforms the pose of the previous connection points to the world coordinate system, uses the UID information of all strong connection points transformed to the world coordinate system as the key index, and adds the pose information of the strong connection points as the value to the asset information comparison sequence. Determine if the asset information comparison sequence matches the asset information reference sequence. If they match, update the corresponding asset location information to the asset information reference sequence to complete the inventory task. If they do not match and there is no asset location information in the asset information reference sequence, find the points with missed detection anomalies and perform asset rescanning. If they do not match but there is asset pose information in the asset information reference sequence, find the global pose of the undetected asset based on the asset information reference sequence and provide the host computer with the robot's route feedback value to the undetected asset based on the A* algorithm, until the asset information comparison sequence matches the asset information reference sequence.

Images