A high-precision positioning method and system for unmanned forklifts using multi-source information

Abstract

This invention discloses a high-precision positioning method and system for unmanned forklifts using multi-source information. The method includes the following steps: configuring a dual-radar positioning system with millimeter-wave radar, lidar, and an inertial measurement unit; designing a random matching algorithm to calibrate the timestamps of the millimeter-wave radar and lidar, obtaining a main image and a secondary image; designing a keyframe extraction algorithm to extract keyframes from the main image, obtaining a primary main image; designing an information matching algorithm, utilizing a dynamic sliding window, to calculate the positioning of the unmanned forklift; designing a position diagnosis algorithm to calculate the positioning deviation and correct the positioning of the unmanned forklift; and outputting the positioning of the unmanned forklift from the dual-radar positioning system. This solution significantly improves the positioning accuracy of unmanned forklifts by acquiring multi-source information from millimeter-wave radar, lidar, and an inertial measurement unit, and establishing a multi-source fusion and dynamic correction mechanism adapted to metal processing plants.

Description

Technical Field

[0001] This invention belongs to the field of unmanned forklift positioning technology, and particularly relates to a high-precision positioning method and system for unmanned forklifts based on multi-source information. Background Technology

[0002] In the process of intelligent upgrading of the metal processing industry, unmanned forklifts have become core equipment for raw material transfer, semi-finished product turnover, and finished product storage in metal processing workshops. Their positioning accuracy and operational stability are directly related to the workshop's production rhythm, equipment safety, and personnel protection. Metal processing workshops have typical environmental characteristics that distinguish them from ordinary warehouses: densely distributed metal processing machine tools, heavy steel racks, and stacked metal billets and semi-finished products create a complex positioning environment with strong reflections and multiple obstructions. Current mainstream positioning technologies, such as lidar, are easily affected by specular reflections from metal surfaces in workshops, resulting in a large number of false point clouds or missing point clouds in key areas; GNSS signals are completely shielded in enclosed workshops and areas with dense metal structures, rendering them ineffective; while single millimeter-wave radar has strong anti-reflection capabilities, its spatial positioning accuracy is insufficient, making it difficult to meet the needs of precise parts transfer and close-range operation next to machine tools.

[0003] While existing multi-sensor fusion positioning solutions have been applied to some industrial scenarios, their adaptability to metal processing plant workshops is extremely poor, exhibiting three major pain points: First, low spatiotemporal synchronization accuracy, with timestamp misalignment between lidar and millimeter-wave radar data often exceeding 15ms, easily leading to inaccurate fused data in dynamic material handling scenarios within the workshop; second, lack of interference handling mechanisms, as spatter and machine tool vibrations generated during metal workpiece processing exacerbate point cloud interference, resulting in a point cloud loss rate of over 40% and positioning errors exceeding 60cm in traditional solutions, easily causing collisions with machine tools and material racks; third, insufficient error compensation, as continuous mechanical vibrations within the workshop accelerate the accumulation of errors in the inertial measurement unit, and existing solutions lack dynamic compensation mechanisms, causing positioning accuracy to decay to over 1m after prolonged operation. Therefore, there is an urgent need to propose a positioning method for unmanned forklifts in metal processing plants. Summary of the Invention

[0004] This invention provides a multi-source information high-precision positioning method and system for unmanned forklifts to solve the technical problems of severe positioning interference, low positioning accuracy, and lack of timely positioning compensation for unmanned forklifts in metal processing plants.

[0005] To solve the above-mentioned technical problems, the present invention provides the following technical solution: On one hand, the present invention provides a high-precision positioning method for unmanned forklifts using multi-source information, which includes: S1: The dual-radar positioning system is equipped with millimeter-wave radar, lidar and inertial measurement unit. A random matching algorithm is designed to calibrate the timestamps of millimeter-wave radar and lidar. The environmental information collected by the normalized projection lidar is used to obtain the main image, and the environmental information collected by the normalized projection millimeter-wave radar is used to obtain the auxiliary image. S2: Design a keyframe extraction algorithm to extract keyframes of the main image, initialize the loss threshold, and use the loss point cloud rate to realize the primary interference diagnosis of the keyframes of the main image. If the loss point cloud rate is less than the loss threshold, design a fuzzy replacement algorithm to fuzz the lost point cloud, calculate the fuzzy coefficient, realize the secondary interference diagnosis of the keyframes of the main image, and obtain the primary main image. S3: For the extraction of auxiliary images with the same timestamp from the primary main image, design an information matching algorithm, use a dynamic sliding window to match the radial velocity of pixels in the auxiliary image with the pixels in the primary main image, design a feature factor matching algorithm, and calculate the positioning of the unmanned forklift. S4: After obtaining the location of the unmanned forklift, design a position diagnosis algorithm, use the positioning of the inertial measurement unit and the positioning of the unmanned forklift to calculate the dynamic coefficient, initialize the dynamic threshold, and if the dynamic coefficient is greater than the dynamic threshold, calculate the positioning deviation and correct the positioning of the unmanned forklift. S5: Outputs the positioning of unmanned forklifts using a dual radar positioning system.

[0006] Preferably, step S1 includes: The unmanned forklift is equipped with a dual-radar positioning system, which includes millimeter-wave radar, lidar, and an inertial measurement unit. The millimeter-wave radar uses a 4D millimeter-wave radar with a horizontal resolution of 1° and a vertical resolution of 2°. The environmental information acquired by the millimeter-wave radar includes distance values ​​and radial velocity. Specifically, the distance value is the distance from the unmanned forklift to the reflection point, and the radial velocity is the radial velocity from the unmanned forklift to the reflection point. A mean filtering algorithm is used to remove noise points from the environmental information acquired by the millimeter-wave radar. The lidar used is a 3D lidar with 32 beams and a horizontal resolution of 0.2°. The environmental information collected by the lidar includes distance values, which are specifically the distances from the unmanned forklift to the reflection point. Each lidar point contains this distance value. The inertial measurement unit acquires the linear acceleration and angular acceleration of the unmanned forklift. Through calculation, the positioning of the unmanned forklift can be calculated from the linear acceleration and angular acceleration. The design of a random matching algorithm for calibrating the timestamps of millimeter-wave radar and lidar is as follows: (1) After the unmanned forklift is turned on, initialize the dual radar positioning system and randomly set the initial time and the predicted matching time. The initial time is the time after the unmanned forklift is turned on, and the predicted matching time is a random time within 5 seconds after the initial time. (2) Obtain the timestamp of the millimeter-wave radar at the initial moment. Timestamp of the predicted matching time Where t represents the timestamp and H represents the millimeter-wave radar, the timestamp of the lidar at the initial moment is obtained. Timestamp of the predicted matching time ,in Indicates lidar; (3) Calculate the matching coefficient The specific calculation formula is as follows: ; If the matching coefficient A value of 1 indicates that the timestamps of the millimeter-wave radar and the lidar are consistent, and recalibration is not required. If the matching coefficient... If the value is not 1, it indicates that there is an anomaly in the timestamps of the millimeter-wave radar and lidar. The lidar and millimeter-wave radar need to re-obtain the time from the dual radar system, update the timestamps of the lidar and millimeter-wave radar, and perform random matching algorithm calibration again until the calibration is successful. If the random matching algorithm calibration fails three times, an error message will be displayed. The normalized projection lidar collects environmental information to obtain the main image. The 3D environmental information collected by the lidar is projected into a 2D image. Each pixel in the main image contains a distance value. The top and bottom lidar beams are not considered in the main image. The resolution of the main image is [resolution value missing]. ; The environmental information acquired by the normalized projection millimeter-wave radar is used to obtain auxiliary images. Each pixel in the auxiliary image contains a range value and a radial velocity. The resolution of the auxiliary images is [resolution value missing]. .

[0007] Preferably, step S2 includes: The keyframe extraction algorithm is designed to extract keyframes from the main image. The specific steps are as follows: (1) Obtain the speed of the unmanned forklift at timestamp t, calculate the average speed of the unmanned forklift within 5 seconds, and calculate the number of keyframe intervals within 5 seconds. The specific calculation formula is shown below: ; in, This represents the average speed of the unmanned forklift within 5 seconds starting at time t. (2) Obtain the distance value of the pixel in column 900 of the keyframe at the start time within 5 seconds, obtain the distance value of the pixel in column 900 of the keyframe at the start time within the previous 5 seconds, and calculate the keyframe distance deviation. The specific calculation formula is shown below: ; Where i represents the sequence number. Indicates This is the distance value of the i-th pixel in the 900th column of the keyframe at the start time. Indicated by This is the distance value of the i-th pixel in the 900th column of the keyframe at the start time, where || represents the absolute value calculation; (3) Key frame confirmation: Initialize the key frame deviation threshold. If the key frame distance deviation is greater than or equal to the key frame deviation threshold, then retain the key frame starting at time t. If the keyframe distance deviation is less than the keyframe deviation threshold, then the keyframe starting at time t is similar to the previous keyframe. In this case, the keyframe starting at time t is removed, and the keyframe distance deviation between the keyframe starting at time t+5 and the keyframe starting at time t+5 is recalculated. Steps (2) and (3) are repeated. Initialize the loss threshold Calculate the missing point cloud rate of keyframes in the main image with timestamp t. The specific calculation formula is shown below: ; in, Keyframes of the main image with timestamp t The actual number of laser points received; Primary interference diagnosis of keyframes in the main image is achieved using the point cloud loss rate. If the point cloud loss rate is greater than or equal to a loss threshold, then the keyframes of the main image are considered to have interference. If the number of laser points received is qualified, no point cloud compensation is required. If the point cloud loss rate is less than the loss threshold, a fuzzy replacement algorithm is designed to blur the lost point cloud, calculate the fuzziness coefficient, realize secondary interference diagnosis of key frames of the main image, and obtain the primary main image.

[0008] Preferably, the design of the fuzzy replacement algorithm involves fuzzing the point cloud at the fuzzy loss location, calculating the fuzziness coefficient, and realizing secondary interference diagnosis of keyframes in the main image to obtain the primary main image, including: The specific steps of the fuzzy substitution algorithm are as follows: (1) Obtain keyframes of the main image For all pixels in the array, for each pixel whose distance value is NaN, count the number of pixels whose distance value to its surrounding pixels is not NaN, and obtain the set R of lost laser points, where R = { }, a , This indicates the number of pixels whose distance value is not NaN from surrounding pixels. Let A be the a-th laser point, where A represents the total number of lost laser points, and r represents the number of laser points. (2) In set R, for For pixels with a distance value less than 4, the distance value of the pixel is marked as the distance value of the nearest pixel in the main image; (3) For For pixels greater than or equal to 4, the point cloud at the blurred area is... fuzzy coefficient at The calculation formula is shown below: ; Where j represents the sequence number. express The distance value of the pixel whose distance to the j-th surrounding pixel is not NaN. for The fuzzy coefficient; Obtain the primary image.

[0009] Preferably, step S3 includes: For the primary image, extract the secondary image with the same timestamp; Design an information matching algorithm that uses a dynamic sliding window to match the radial velocity of pixels in the auxiliary image with the pixels in the primary image. The specific steps of the information matching algorithm are as follows: (1) Traverse all pixels in the auxiliary image, obtain the radial velocity of each pixel in the auxiliary image, and cluster adjacent pixels with the same radial velocity into the same dynamic sliding window to obtain the dynamic sliding window set U, where U={ }, b , Indicates radial velocity as The b-th dynamic sliding window, where B represents the total number of dynamic sliding windows, and the sliding windows are numbered from top to bottom and from left to right; (2) Establish a primary main image sub-region, with a resolution of [missing information] in the secondary image. In the primary image, the resolution is Divide the primary main image into Each subregion contains [number] subregions. Each sub-region corresponds one-to-one with the pixels in the auxiliary image; (3) Find the corresponding interval of the dynamic sliding window of the auxiliary image in the primary image, and assign the speed of the dynamic sliding window to each pixel in the corresponding interval of the primary image to achieve the matching of the radial speed of the pixel in the auxiliary image with the pixel in the primary image. After the information matching algorithm, each pixel in the primary image is matched with its corresponding radial velocity.

[0010] Preferably, the design feature factor matching algorithm for calculating the localization of the unmanned forklift includes: (1) Identify dynamic and static intervals: Initialize the static threshold, traverse the corresponding intervals of the dynamic sliding window of the auxiliary image in the first-level main image. If the speed of the corresponding interval is greater than the static threshold, then the corresponding interval is a dynamic interval. If the speed of the corresponding interval is less than or equal to the static threshold, then the corresponding interval is a static interval. (2) Initialize the feature factor threshold and calculate the feature diagnostic coefficient for the static interval in the primary image. The formula for calculating the diagnostic coefficient of the w-th feature is as follows: ; ; in, Indicates the serial number. This indicates the number of pixels in the static range. Represents the w-th static interval. The distance value or blur coefficient of each pixel Indicates the distance to the mean; (3) If the feature diagnosis coefficient of a pixel is greater than or equal to the feature factor threshold, it indicates that the region is a feature factor region; if the feature diagnosis coefficient of a pixel is less than the feature factor threshold, it indicates that the region is an unknown region. Select any feature factor region and search its neighboring regions, continuously expanding the area of ​​the feature factor region; If the adjacent region is a feature factor region, then mark it as the first first-level feature factor region; if the adjacent feature factor region is an unknown region, then mark it as the first unknown region. After traversing all first-level feature factor regions surrounding the selected feature factor region, number all connected feature factor regions traversed as p, where p is an integer greater than or equal to 1, and end the traversal. For other feature factor regions that have not been traversed, traverse and mark them again. The expansion process is repeated until all adjacent regions are marked. Traverse all unknown regions surrounding the first unknown region, number all connected unknown regions as q, where q is an integer greater than or equal to 1, and end the traversal. For other unknown regions that have not been traversed, traverse and mark them again. Repeat the expansion process until all adjacent regions are marked. Get Each primary feature factor region and In each unknown region, 10 pixels are randomly selected as primary feature factors in each primary feature factor region, and 5 pixels are randomly selected as secondary feature factors in each unknown region. The localization of the unmanned forklift is obtained by calculating the first-level and second-level feature factors in adjacent keyframes using the iterative nearest point algorithm.

[0011] On the other hand, the present invention also provides a high-precision positioning system for unmanned forklifts based on multi-source information, the system comprising: The data preprocessing module is configured with a millimeter-wave radar, a lidar and an inertial measurement unit in the dual radar positioning system. A random matching algorithm is designed to calibrate the timestamps of the millimeter-wave radar and the lidar. The environmental information collected by the normalized projection lidar is used to obtain the main image, and the environmental information collected by the normalized projection millimeter-wave radar is used to obtain the auxiliary image. The interference diagnosis module designs a keyframe extraction algorithm to extract keyframes of the main image, initializes a loss threshold, and uses the loss point cloud rate to achieve primary interference diagnosis of the keyframes of the main image. If the loss point cloud rate is less than the loss threshold, a fuzzy replacement algorithm is designed to fuzz the point cloud at the loss location, calculates the fuzziness coefficient, and achieves secondary interference diagnosis of the keyframes of the main image, thus obtaining the primary main image. The unmanned forklift positioning module extracts auxiliary images with the same timestamp from the primary main image, designs an information matching algorithm, uses a dynamic sliding window to match the radial velocity of pixels in the auxiliary image with pixels in the primary main image, and designs a feature factor matching algorithm to calculate the positioning of the unmanned forklift. The positioning correction module, after obtaining the positioning of the unmanned forklift, designs a position diagnosis algorithm, calculates dynamic coefficients using the positioning of the inertial measurement unit and the positioning of the unmanned forklift, initializes a dynamic threshold, calculates the positioning deviation if the dynamic coefficient is greater than the dynamic threshold, corrects the positioning of the unmanned forklift, and outputs the positioning of the unmanned forklift from the dual radar positioning system.

[0012] The beneficial effects of the technical solution provided by this invention include at least the following: 1. This solution constructs a dual-radar collaborative calibration system adapted to metal processing plant workshops, significantly improving the quality of positioning data sources. Metal processing plant workshops have a dense distribution of metal components, and traditional dual-radar timestamp calibration algorithms are susceptible to interference from reflected signals, with synchronization errors often exceeding 20ms, leading to data fusion failure. This solution optimizes the timestamp calibration logic through a random matching algorithm, controlling the dual-radar synchronization error to within 5ms, providing a precise spatiotemporal reference for multi-source data fusion. Simultaneously, addressing the issue of point cloud loss due to metal reflection from lidar in the workshop, normalized projection technology is used to transform lidar data into a primary image and millimeter-wave radar data into a secondary image. The strong penetration of millimeter-wave radar into metal and its resistance to reflection interference are utilized to supplement the lidar data. Testing in the heavy machine tool operating area of ​​a metal processing plant shows that this solution increases the point cloud effectiveness from 58% of the traditional solution to 93%, completely and effectively solving the problem of unreliable data sources caused by metal reflection in the workshop, laying a solid foundation for positioning safety and reliability. 2. This solution innovates a two-level interference diagnosis and repair mechanism to accurately address the complex interference scenarios in metal processing plant workshops. Metal processing plant workshops contain multiple interference factors such as machine tool vibration, metal splashes, and dynamic movement of material racks. Traditional solutions rely solely on a single threshold for interference assessment, resulting in a high point cloud drop rate and frequent positioning interruptions. This solution optimizes the interference diagnosis logic based on workshop interference characteristics: First, a keyframe extraction algorithm extracts the main image with significant changes, further reducing redundant information interference and improving the algorithm's real-time performance. The point cloud loss rate is used to complete primary interference diagnosis, quickly identifying severely interfered areas. When the point cloud loss rate is below a set threshold, a fuzzy replacement algorithm is used to repair the lost point cloud, achieving secondary interference diagnosis. In dynamic material transfer scenarios in metal processing plant stamping workshops, traditional solutions have a positioning interruption rate as high as 35%, easily leading to collisions between materials and machine tools. This solution, through a two-level diagnosis mechanism, controls the positioning interruption rate to within 2.5%, and the fuzzy replacement algorithm's repair accuracy error is less than 0.1m, ensuring that the primary main image fully reflects the workshop environment characteristics.

[0013] 3. This solution establishes a multi-source fusion and dynamic correction mechanism adapted to workshop scenarios to ensure long-term high-precision positioning in metal processing workshops. A triple precision assurance mechanism is designed for the workshop environment: First, a dynamic sliding window matching algorithm optimizes window parameters by combining fixed features such as machine tools and material racks within the workshop, increasing the matching rate between the radial velocity of auxiliary image pixels and the primary image pixels to over 96%; second, a feature factor matching algorithm extracts stable features as matching benchmarks, combining the spatial positioning accuracy of lidar with the velocity measurement advantages of millimeter-wave radar, controlling the initial positioning error within 0.12m; third, a position diagnosis system completes dynamic correction through real-time comparison of inertial measurement unit positioning data and dual-radar positioning data. Attached Figure Description

[0014] Figure 1 This is a schematic diagram of the overall execution flow of a high-precision positioning method for unmanned forklifts based on multi-source information, provided in an embodiment of the present invention.

[0015] Figure 2 This is a schematic diagram of a keyframe extraction algorithm provided in an embodiment of the present invention. Detailed Implementation

[0016] The present invention will be further described below with reference to the accompanying drawings, but this is not intended to limit the present invention in any way. Any modifications or substitutions made based on the teachings of the present invention shall fall within the protection scope of the present invention.

[0017] Example 1 This embodiment provides a high-precision positioning method for unmanned forklifts using multi-source information. This method can be implemented by electronic devices, such as... Figure 1As shown. Specifically, the method in this embodiment includes the following steps: The unmanned forklift is equipped with a dual-radar positioning system, which includes millimeter-wave radar, lidar, and an inertial measurement unit. It should be further noted that by configuring multi-source sensor information, the distortion problem of a single sensor in a metal processing plant environment can be avoided, thereby improving the positioning accuracy and safety of the unmanned forklift. The millimeter-wave radar uses a 4D millimeter-wave radar with a horizontal resolution of 1° and a vertical resolution of 2°. The environmental information collected by the millimeter-wave radar includes the distance value from the unmanned forklift to the reflection point (hereinafter referred to as distance value) and the radial velocity of the unmanned forklift to the reflection point (hereinafter referred to as radial velocity). The mean filtering algorithm is used to remove noise points from the environmental information collected by the millimeter-wave radar. The lidar used is a 3D lidar with 32 beams and a horizontal resolution of 0.2°. The environmental information collected by the lidar includes the distance value from the unmanned forklift to the reflection point (hereinafter referred to as the distance value). Therefore, each lidar point contains the distance value from the unmanned forklift to the reflection point. The inertial measurement unit acquires the linear acceleration and angular acceleration of the unmanned forklift. Through calculation, the positioning of the unmanned forklift can be calculated from the linear acceleration and angular acceleration. The pose transformation matrix is ​​set, and the positioning calculated by the millimeter-wave radar, lidar and inertial measurement unit through the pose transformation matrix is ​​the positioning of the unmanned forklift. The design of a random matching algorithm for calibrating the timestamps of millimeter-wave radar and lidar is as follows: (1) After the unmanned forklift is turned on, the dual radar positioning system is initialized and the initial time and the predicted matching time are randomly set. The initial time is the time after the unmanned forklift is turned on, and the predicted matching time is a random time within 5 seconds after the initial time. It should be further explained that the random time interval selected in this scheme is 5 seconds. In order to improve fault tolerance, this interval time can be adjusted in a timely manner according to the specific scenario and the user's purpose. For example, for scenarios with variable environment, the random time can be increased or decreased, and the random time interval can be adjusted to 2 seconds or 10 seconds. (2) Obtain the timestamp of the millimeter-wave radar at the initial moment. Timestamp of the predicted matching time Where t represents the timestamp and H represents the millimeter-wave radar, the timestamp of the lidar at the initial moment is obtained. Timestamp of the predicted matching time , where J represents lidar; (3) Calculate the matching coefficient The specific calculation formula is as follows: ; If the matching coefficient A value of 1 indicates that the timestamps of the millimeter-wave radar and the lidar are consistent, and recalibration is not required. If the matching coefficient... If the value is not 1, it indicates that there is an anomaly in the timestamps of the millimeter-wave radar and lidar. The lidar and millimeter-wave radar need to re-obtain the time from the dual radar system, update the timestamps of the lidar and millimeter-wave radar, and perform random matching algorithm calibration again until the calibration is successful. If the random matching algorithm calibration fails three times, an error message will be displayed. It should be further explained that only when the matching coefficient is 1 can it be said that the timestamp values ​​of the millimeter-wave radar and the lidar are consistent. Although the amount of data for matching only two timestamps is small, the randomness of the algorithm can improve the accuracy of diagnosis, reduce the amount of computation, and increase the accuracy of timestamp matching. Of course, to further improve the accuracy, a random matching algorithm can be performed at regular intervals. It should be noted here that since the frequency of the inertial measurement unit is high, this solution assumes that the timestamp of the inertial measurement unit is accurate and there is no need to perform additional timestamp matching between the inertial measurement unit and the millimeter-wave radar and lidar. The normalized projection lidar collects environmental information to obtain the main image. The 3D environmental information collected by the lidar is projected into a 2D image. Each pixel in the main image contains a distance value. The main image does not consider the top and bottom lidar beams; therefore, the resolution of the main image is... ; The environmental information acquired by the normalized projection millimeter-wave radar is used to obtain auxiliary images. Each pixel in the auxiliary image contains a range value and a radial velocity. The resolution of the auxiliary images is [resolution value missing]. .

[0018] After obtaining the main image and auxiliary image, the main image can be processed. Since the LiDAR collects environmental information at a high frequency, a lot of redundant information will be generated as the unmanned forklift moves. This information is meaningless for the positioning calculation of the unmanned forklift and will increase the computing power consumption of the unmanned forklift, thereby increasing the hardware cost of the unmanned forklift. Therefore, it is necessary to extract key frames. Design a keyframe extraction algorithm to extract keyframes from the main image, hereinafter referred to as keyframes, such as... Figure 2 As shown, the specific steps are as follows: (1) Obtain the speed of the unmanned forklift at timestamp t, calculate the average speed of the unmanned forklift within 5 seconds, and calculate the number of keyframe intervals within 5 seconds. The specific calculation formula is shown below: ; in, This represents the average speed of the unmanned forklift within 5 seconds starting at time t. It should be further explained that in this solution, the number of key frame intervals for extraction is set according to the operating speed of the unmanned forklift. When the average speed of the unmanned forklift is 2 meters per second within 5 seconds, the number of key frame intervals is 10. Since the acquisition frequency of the LiDAR is generally 10 frames per second, one key frame is acquired every 1 second. As the speed of the unmanned forklift increases, the number of key frame intervals decreases continuously, and the number of key frames extracted increases, which is consistent with the data acquisition logic of unmanned forklift positioning. (2) Calculate the keyframe distance deviation. Obtain the distance value of the pixel in column 900 of the keyframe at the start time within 5 seconds, and obtain the distance value of the pixel in column 900 of the keyframe at the start time within the previous 5 seconds. Calculate the keyframe distance deviation. The specific calculation formula is shown below: ; Where i represents the sequence number. Indicated by This is the distance value of the i-th pixel in the 900th column of the keyframe at the start time. Indicated by This is the distance value of the i-th pixel in the 900th column of the keyframe at the start time, where || represents the absolute value calculation; It should be further explained that the horizontal resolution of the lidar is 0.2°, so there are a total of 1800 columns. The 900th column is the middle column to avoid distortion of the edge columns and better distinguish the differences between key frames. (3) Key frame confirmation: Initialize the key frame deviation threshold. If the key frame distance deviation is greater than or equal to the key frame deviation threshold, then retain the key frame starting at time t. If the keyframe distance deviation is less than the keyframe deviation threshold, then the keyframe starting at time t is similar to the previous keyframe. In this case, the keyframe starting at time t is removed, and the keyframe distance deviation between the keyframe starting at time t+5 and the keyframe starting at time t+5 is recalculated. Steps (2) and (3) are repeated. Initialize the loss threshold Calculate the missing point cloud rate of keyframes in the main image with timestamp t. The specific calculation formula is shown below: ; in, Keyframes of the main image with timestamp t The actual number of laser points received; Primary interference diagnosis of keyframes in the main image is achieved using the point cloud loss rate. If the point cloud loss rate is greater than or equal to a loss threshold, then the keyframes of the main image are considered to have interference. If the number of laser points received is qualified, no point cloud compensation is required. If the point cloud loss rate is less than the loss threshold, a fuzzy replacement algorithm is designed to blur the lost point cloud, calculate the fuzzy coefficient, realize secondary interference diagnosis of key frames of the main image, and obtain the primary main image. The algorithm for fuzzy replacement is designed to detect point cloud data at fuzzy locations, calculate fuzziness coefficients, and perform secondary interference diagnosis on keyframes of the main image to obtain the primary main image. The specific steps of the fuzzy replacement algorithm are as follows: (1) Obtain keyframes of the main image For all pixels in the array, for each pixel whose distance value is NaN, count the number of pixels whose distance value to its surrounding pixels is not NaN, and obtain the set R of lost laser points, where R = { }, a , This indicates the number of pixels whose distance value is not NaN from surrounding pixels. Let A be the a-th laser point, where A represents the total number of lost laser points, and r represents the number of laser points. (2) In set R, for For pixels with a distance value less than 4, the distance value of the pixel is marked as the distance value of the nearest pixel in the main image; (3) For For pixels greater than or equal to 4, the point cloud at the blurred area is... fuzzy coefficient at The calculation formula is shown below: ; Where j represents the sequence number. express The distance value of the j-th non-NaN pixel in the surrounding array. for The fuzzy coefficient; After obtaining the primary image, the corresponding secondary image needs to be matched with the primary image. For the primary image, extract the secondary image with the same timestamp; Design an information matching algorithm that uses a dynamic sliding window to match the radial velocity of pixels in the auxiliary image with the pixels in the primary image. The specific steps of the information matching algorithm are as follows: (1) Traverse all pixels in the auxiliary image, obtain the radial velocity of each pixel in the auxiliary image, and cluster adjacent pixels with the same radial velocity into the same dynamic sliding window to obtain the dynamic sliding window set U, where U={ }, b , Indicates radial velocity as The b-th dynamic sliding window, where B represents the total number of dynamic sliding windows, and the sliding windows are numbered from top to bottom and from left to right; (2) Establish a primary main image sub-region, with a resolution of [missing information] in the secondary image. In the primary image, the resolution is Therefore, the primary main image can be divided into Each subregion contains [number] subregions. Each sub-region corresponds one-to-one with the pixels in the auxiliary image; (3) Find the corresponding interval of the dynamic sliding window of the auxiliary image in the primary image, and assign the speed of the dynamic sliding window to each pixel in the corresponding interval of the primary image to achieve the matching of the radial speed of the pixel in the auxiliary image with the pixel in the primary image. After the information matching algorithm, each pixel in the primary image is matched with its corresponding radial velocity.

[0019] The feature factor matching algorithm is designed to calculate the localization of the unmanned forklift. The specific steps are as follows: (1) Identify dynamic and static intervals, initialize static thresholds, traverse the corresponding intervals of the dynamic sliding window of the auxiliary image in the first-level main image, if the speed of the corresponding interval is greater than the static threshold, then the corresponding interval is a dynamic interval; if the speed of the corresponding interval is less than or equal to the static threshold, then the corresponding interval is a static interval. (2) Initialize the feature factor threshold and calculate the feature diagnostic coefficient for the static interval in the primary image. The formula for calculating the diagnostic coefficient of the w-th feature is as follows: ; ; in, Indicates the serial number. This indicates the number of pixels in the static range. Represents the w-th static interval. The distance value or blur coefficient of each pixel Indicates the distance to the mean; (3) If the feature diagnosis coefficient of a pixel is greater than or equal to the feature factor threshold, it indicates that the region is a feature factor region; if the feature diagnosis coefficient of a pixel is less than the feature factor threshold, it indicates that the region is an unknown region. Select any feature factor region and search its neighboring regions, continuously expanding the area of ​​the feature factor region; If the adjacent region is a feature factor region, then mark it as the first first-level feature factor region; if the adjacent feature factor region is an unknown region, then mark it as the first unknown region. After traversing all first-level feature factor regions surrounding the selected feature factor region, number all connected feature factor regions traversed as p, where p is an integer greater than or equal to 1, and end the traversal. For other feature factor regions that have not been traversed, traverse and mark them again. The expansion process is repeated until all adjacent regions are marked. It should be further explained that in this scheme, by continuously expanding the area of ​​adjacent feature factor regions, the area of ​​the feature factor regions can be reduced. By treating adjacent feature factor regions as the same region with similar feature factors, the amount of computation can be further reduced while ensuring positioning accuracy. Traverse all unknown regions surrounding the first unknown region, number all connected unknown regions as q, where q is an integer greater than or equal to 1, and end the traversal. For other unknown regions that have not been traversed, traverse and mark them again. Repeat the expansion process until all adjacent regions are marked. Get Each primary feature factor region and In each unknown region, 10 pixels are randomly selected as primary feature factors in each primary feature factor region, and 5 pixels are randomly selected as secondary feature factors in each unknown region. The localization of the unmanned forklift is obtained by calculating the first-level and second-level feature factors in adjacent keyframes using the iterative nearest point algorithm.

[0020] After obtaining the location of the unmanned forklift, a position diagnosis algorithm is designed to calculate the dynamic coefficients using the positioning data from the inertial measurement unit and the unmanned forklift. The specific steps are shown in the following formula: (1) For the auxiliary image corresponding to the primary image, the secondary positioning of the unmanned forklift is calculated by the iterative nearest point algorithm. For the linear acceleration and angular acceleration of the inertial measurement unit, the tertiary positioning of the unmanned forklift with the same timestamp as the primary image is calculated by integration. (2) The dynamic coefficients are calculated from the positioning, secondary positioning, and tertiary positioning of the unmanned forklift. The specific calculation formula is shown in the following formula: ; in, This represents the x-coordinate of the unmanned forklift's location. This represents the y-coordinate of the unmanned forklift's location. This represents the x-coordinate value of the secondary positioning of the unmanned forklift. This represents the secondary positioning coordinate of the unmanned forklift on the z-axis. This represents the y-coordinate value of the autonomous forklift's Level 3 positioning. This represents the z-axis coordinate value of the three-level positioning of the unmanned forklift; (3) Initialize the dynamic threshold. If the dynamic coefficient is greater than or equal to the dynamic threshold, the positioning deviation needs to be calculated and the positioning of the unmanned forklift needs to be corrected. If the dynamic coefficient is less than the dynamic threshold, the location of the unmanned forklift is output directly; Calculate positioning deviation , , ,in This indicates the positioning deviation along the x-axis. This indicates the positioning deviation along the y-axis. The positioning deviation along the z-axis is represented by the following formula: ; ; ; in, This represents the z-coordinate of the unmanned forklift's location. This represents the y-coordinate value of the secondary positioning of the unmanned forklift. This represents the x-axis coordinate value of the Level 3 positioning of the unmanned forklift; The positioning of the unmanned forklift is corrected. The positioning deviation and the sum of the unmanned forklift's positioning are the corrected positioning of the unmanned forklift, hereinafter referred to as the positioning of the unmanned forklift. The positioning of unmanned forklifts is achieved by outputting a dual radar positioning system.

[0021] Example 2 This embodiment provides a high-precision positioning system for unmanned forklifts based on multi-source information. This system includes the following modules: The data preprocessing module is configured with a millimeter-wave radar, a lidar and an inertial measurement unit in the dual radar positioning system. A random matching algorithm is designed to calibrate the timestamps of the millimeter-wave radar and the lidar. The environmental information collected by the normalized projection lidar is used to obtain the main image, and the environmental information collected by the normalized projection millimeter-wave radar is used to obtain the auxiliary image. The interference diagnosis module designs a keyframe extraction algorithm to extract keyframes of the main image, initializes a loss threshold, and uses the loss point cloud rate to achieve primary interference diagnosis of the keyframes of the main image. If the loss point cloud rate is less than the loss threshold, a fuzzy replacement algorithm is designed to fuzz the point cloud at the loss location, calculates the fuzziness coefficient, and achieves secondary interference diagnosis of the keyframes of the main image, thus obtaining the primary main image. The unmanned forklift positioning module extracts auxiliary images with the same timestamp from the primary main image, designs an information matching algorithm, uses a dynamic sliding window to match the radial velocity of pixels in the auxiliary image with pixels in the primary main image, and designs a feature factor matching algorithm to calculate the positioning of the unmanned forklift. After obtaining the location of the unmanned forklift, the positioning correction module designs a position diagnosis algorithm, calculates the dynamic coefficient using the positioning of the inertial measurement unit and the positioning of the unmanned forklift, initializes the dynamic threshold, calculates the positioning deviation if the dynamic coefficient is greater than the dynamic threshold, corrects the positioning of the unmanned forklift, and outputs the positioning of the unmanned forklift from the dual radar positioning system. As used herein, the term "preferred" is meant as an example, illustration, or illustration. Any aspect or design described herein as "preferred" need not be construed as being more advantageous than other aspects or designs. Rather, the use of the term "preferred" is intended to present the concept in a specific manner. As used in this application, the term "or" is intended to mean an inclusive "or" rather than an exclusionary "or." That is, unless otherwise specified or clear from the context, "X uses A or B" naturally includes either of the permutations. That is, if X uses A; X uses B; or X uses both A and B, then "X uses A or B" is satisfied in any of the foregoing examples.

[0022] Furthermore, although this disclosure has been shown and described with respect to one or more implementations, equivalent variations and modifications will occur to those skilled in the art based on a reading and understanding of this specification and the accompanying drawings. This disclosure includes all such modifications and variations and is limited only by the scope of the appended claims. In particular, with respect to the various functions performed by the aforementioned components (e.g., elements, etc.), the terminology used to describe such components is intended to correspond to any component (unless otherwise indicated) that performs the specified function of said component (e.g., is functionally equivalent to it), even if structurally not equivalent to the disclosed structure performing the functions in the exemplary implementations of this disclosure shown herein. Moreover, although specific features of this disclosure have been disclosed with respect to only one of several implementations, such features may be combined with one or more features of other implementations that may be desirable and advantageous for a given or particular application. Furthermore, with regard to the use of the terms “comprising,” “having,” “containing,” or variations thereof in the Detailed Description or claims, such terms are intended to be included in a manner similar to the term “including.”

[0023] The functional units in this invention embodiment can be integrated into a processing module, or each unit can exist physically separately, or multiple units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. The aforementioned devices or systems can execute the storage methods in the corresponding method embodiments.

[0024] In summary, the above embodiments are one implementation of the present invention, but the implementation of the present invention is not limited to the embodiments described above. Any changes, modifications, substitutions, combinations, or simplifications made that deviate from the spirit and principle of the present invention should be considered equivalent substitutions and are included within the protection scope of the present invention.

Claims

1. A method for high-precision positioning of multiple sources of information for unmanned forklifts, characterized in that, Includes the following steps: S1: The dual-radar positioning system is equipped with millimeter-wave radar, lidar and inertial measurement unit. A random matching algorithm is designed to calibrate the timestamps of millimeter-wave radar and lidar. The environmental information collected by the normalized projection lidar is used to obtain the main image, and the environmental information collected by the normalized projection millimeter-wave radar is used to obtain the auxiliary image. The design of a random matching algorithm for calibrating the timestamps of millimeter-wave radar and lidar is as follows: (1) After the unmanned forklift is turned on, initialize the dual radar positioning system and randomly set the initial time and the predicted matching time. The initial time is the time after the unmanned forklift is turned on, and the predicted matching time is a random time within 5 seconds after the initial time. (2) obtaining a timestamp of the millimeter wave radar at an initial time and a predicted timestamp at a matching time where t represents a timestamp, H represents a millimeter wave radar, obtaining a timestamp of the laser radar at an initial time and a predicted timestamp at a matching time where represents a laser radar; (3) Calculate the matching coefficient The specific calculation formula is as follows: ； If the matching coefficient A value of 1 indicates that the timestamps of the millimeter-wave radar and the lidar are consistent, and recalibration is not required. If the matching coefficient... If the value is not 1, it indicates that there is an anomaly in the timestamps of the millimeter-wave radar and lidar. The lidar and millimeter-wave radar need to re-obtain the time from the dual radar system, update the timestamps of the lidar and millimeter-wave radar, and perform random matching algorithm calibration again until the calibration is successful. If the random matching algorithm calibration fails three times, an error message will be displayed. S2: Design a keyframe extraction algorithm to extract keyframes of the main image, initialize the loss threshold, and use the loss point cloud rate to realize the primary interference diagnosis of the keyframes of the main image. If the loss point cloud rate is less than the loss threshold, design a fuzzy replacement algorithm to fuzz the lost point cloud, calculate the fuzzy coefficient, realize the secondary interference diagnosis of the keyframes of the main image, and obtain the primary main image. The specific steps of the fuzzy substitution algorithm are as follows: (1) Obtain keyframes of the main image For all pixels in the array, for each pixel whose distance value is NaN, count the number of pixels whose distance value to its surrounding pixels is not NaN, and obtain the set R of lost laser points, where R = { }, a , This indicates the number of pixels whose distance value is not NaN from surrounding pixels. Let A be the a-th laser point, where A represents the total number of lost laser points, and r represents the number of laser points. (2) In set R, for For pixels with a distance value less than 4, the pixel distance value is marked as the distance value of the nearest pixel in the main image. (3) For For pixels greater than or equal to 4, the point cloud at the area where blur is lost. fuzzy coefficient at The calculation formula is shown below: ； Where j represents the sequence number. express The distance value of the pixel whose distance to the j-th surrounding pixel is not NaN. for The fuzzy coefficient; S3: For the primary image, extract the secondary image with the same timestamp, design an information matching algorithm, use a dynamic sliding window to cluster adjacent pixels with the same radial velocity in the secondary image into the same dynamic sliding window, realize the matching of the radial velocity of the pixels in the secondary image with the pixels in the primary image, design a feature factor matching algorithm to calculate the positioning of the unmanned forklift. S4: After obtaining the location of the unmanned forklift, design a position diagnosis algorithm, use the positioning of the inertial measurement unit and the positioning of the unmanned forklift to calculate the dynamic coefficient, initialize the dynamic threshold, and if the dynamic coefficient is greater than the dynamic threshold, calculate the positioning deviation and correct the positioning of the unmanned forklift. S5: Outputs the positioning of unmanned forklifts using a dual radar positioning system.

2. The multi-source information high-precision positioning method for unmanned forklifts according to claim 1, characterized in that, Step S1 includes: The unmanned forklift is equipped with a dual-radar positioning system, which includes millimeter-wave radar, lidar, and an inertial measurement unit. The millimeter-wave radar uses a 4D millimeter-wave radar with a horizontal resolution of 1° and a vertical resolution of 2°. The environmental information acquired by the millimeter-wave radar includes distance values ​​and radial velocity. Specifically, the distance value is the distance from the unmanned forklift to the reflection point, and the radial velocity is the radial velocity from the unmanned forklift to the reflection point. A mean filtering algorithm is used to remove noise points from the environmental information acquired by the millimeter-wave radar. The lidar used is a 3D lidar with 32 beams and a horizontal resolution of 0.2°. The environmental information collected by the lidar includes distance values, which are specifically the distances from the unmanned forklift to the reflection point. Each lidar point contains this distance value. The inertial measurement unit acquires the linear acceleration and angular acceleration of the unmanned forklift. Through calculation, the positioning of the unmanned forklift can be calculated from the linear acceleration and angular acceleration. The normalized projection lidar collects environmental information to obtain the main image. The 3D environmental information collected by the lidar is projected into a 2D image. Each pixel in the main image contains a distance value. The top and bottom lidar beams are not considered in the main image. The resolution of the main image is [resolution value missing]. ; The environmental information acquired by the normalized projection millimeter-wave radar is used to obtain auxiliary images. Each pixel in the auxiliary image contains a range value and a radial velocity. The resolution of the auxiliary images is [resolution value missing]. .

3. The multi-source information high-precision positioning method for unmanned forklifts according to claim 1, characterized in that, Step S2 includes: The keyframe extraction algorithm is designed to extract keyframes from the main image. The specific steps are as follows: (1) Obtain the speed of the unmanned forklift at timestamp t, calculate the average speed of the unmanned forklift within 5 seconds, and calculate the number of keyframe intervals within 5 seconds. The specific calculation formula is shown below: ； in, This represents the average speed of the unmanned forklift within 5 seconds starting at time t. (2) Obtain the distance value of the pixel in column 900 of the keyframe at the start time within 5 seconds, obtain the distance value of the pixel in column 900 of the keyframe at the start time within the previous 5 seconds, and calculate the keyframe distance deviation. The specific calculation formula is shown below: ； Where i represents the sequence number. Indicates This is the distance value of the i-th pixel in the 900th column of the keyframe at the start time. Indicates This is the distance value of the i-th pixel in the 900th column of the keyframe at the start time, where || represents the absolute value calculation; (3) Key frame confirmation: Initialize the key frame deviation threshold. If the key frame distance deviation is greater than or equal to the key frame deviation threshold, then retain the key frame starting at time t. If the keyframe distance deviation is less than the keyframe deviation threshold, then the keyframe starting at time t is similar to the previous keyframe. In this case, the keyframe starting at time t is removed, and the keyframe distance deviation between the keyframe starting at time t+5 and the keyframe starting at time t+5 is recalculated. Steps (2) and (3) are repeated. Initialize the loss threshold Calculate the missing point cloud rate of keyframes in the main image with timestamp t. The specific calculation formula is shown below: ； in, Keyframes of the main image with timestamp t The actual number of laser points received; Primary interference diagnosis of keyframes in the main image is achieved using the point cloud loss rate. If the point cloud loss rate is greater than or equal to a loss threshold, then the keyframes of the main image are considered to have interference. If the number of laser points received is qualified, no point cloud compensation is required. If the point cloud loss rate is less than the loss threshold, a fuzzy replacement algorithm is designed to blur the lost point cloud, calculate the fuzziness coefficient, realize secondary interference diagnosis of key frames of the main image, and obtain the primary main image.

4. The multi-source information high-precision positioning method for unmanned forklifts according to claim 1, characterized in that, Step S3 includes: For the primary image, extract the secondary image with the same timestamp; Design an information matching algorithm that uses a dynamic sliding window to match the radial velocity of pixels in the auxiliary image with the pixels in the primary image. The specific steps of the information matching algorithm are as follows: (1) Traverse all pixels in the auxiliary image, obtain the radial velocity of each pixel in the auxiliary image, and cluster adjacent pixels with the same radial velocity into the same dynamic sliding window to obtain the dynamic sliding window set U, where U={ }, b , Indicates radial velocity as The b-th dynamic sliding window, where B represents the total number of dynamic sliding windows, and the sliding windows are numbered from top to bottom and from left to right; (2) Establish a primary main image sub-region, with a resolution of [missing information] in the secondary image. In the primary image, the resolution is Divide the primary main image into Each subregion contains [number] subregions. Each sub-region corresponds one-to-one with the pixels in the auxiliary image; (3) Find the corresponding interval of the dynamic sliding window of the auxiliary image in the primary image, and assign the speed of the dynamic sliding window to each pixel in the corresponding interval of the primary image to achieve the matching of the radial speed of the pixel in the auxiliary image with the pixel in the primary image. After the information matching algorithm, each pixel in the primary image is matched with its corresponding radial velocity.

5. The multi-source information high-precision positioning method for unmanned forklifts according to claim 4, characterized in that, The design feature factor matching algorithm calculates the localization of the unmanned forklift by including: (1) Identify dynamic and static intervals: Initialize the static threshold, traverse the corresponding intervals of the dynamic sliding window of the auxiliary image in the first-level main image. If the speed of the corresponding interval is greater than the static threshold, then the corresponding interval is a dynamic interval. If the speed of the corresponding interval is less than or equal to the static threshold, then the corresponding interval is a static interval. (2) Initialize the feature factor threshold and calculate the feature diagnostic coefficient for the static interval in the primary image. The formula for calculating the diagnostic coefficient of the w-th feature is as follows: ； ； in, Indicates the serial number. This indicates the number of pixels in the static range. Represents the w-th static interval. The distance value or blur coefficient of each pixel Indicates the distance to the mean; (3) If the feature diagnosis coefficient of a pixel is greater than or equal to the feature factor threshold, it indicates that the region is a feature factor region; if the feature diagnosis coefficient of a pixel is less than the feature factor threshold, it indicates that the region is an unknown region. Select any feature factor region and search its neighboring regions, continuously expanding the area of ​​the feature factor region; If the adjacent region is a feature factor region, then mark it as the first first-level feature factor region; if the adjacent feature factor region is an unknown region, then mark it as the first unknown region. Get Each primary feature factor region and In each unknown region, 10 pixels are randomly selected as primary feature factors in each primary feature factor region, and 5 pixels are randomly selected as secondary feature factors in each unknown region. The localization of the unmanned forklift is obtained by calculating the first-level and second-level feature factors in adjacent keyframes using the iterative nearest point algorithm.

6. A high-precision positioning system based on multi-source information for unmanned forklifts, characterized in that, include: The data preprocessing module is configured with a millimeter-wave radar, a lidar and an inertial measurement unit in the dual radar positioning system. A random matching algorithm is designed to calibrate the timestamps of the millimeter-wave radar and the lidar. The environmental information collected by the normalized projection lidar is used to obtain the main image, and the environmental information collected by the normalized projection millimeter-wave radar is used to obtain the auxiliary image. The interference diagnosis module designs a keyframe extraction algorithm to extract keyframes of the main image, initializes a loss threshold, and uses the loss point cloud rate to achieve primary interference diagnosis of the keyframes of the main image. If the loss point cloud rate is less than the loss threshold, a fuzzy replacement algorithm is designed to fuzz the point cloud at the loss location, calculates the fuzziness coefficient, and achieves secondary interference diagnosis of the keyframes of the main image, thus obtaining the primary main image. The unmanned forklift positioning module extracts auxiliary images with the same timestamp from the primary main image, designs an information matching algorithm, uses a dynamic sliding window to match the radial velocity of pixels in the auxiliary image with pixels in the primary main image, and designs a feature factor matching algorithm to calculate the positioning of the unmanned forklift. After obtaining the location of the unmanned forklift, the positioning correction module designs a position diagnosis algorithm, calculates the dynamic coefficient using the positioning of the inertial measurement unit and the positioning of the unmanned forklift, initializes the dynamic threshold, calculates the positioning deviation if the dynamic coefficient is greater than the dynamic threshold, corrects the positioning of the unmanned forklift, and outputs the positioning of the unmanned forklift from the dual radar positioning system. This implements a high-precision positioning method for unmanned forklifts based on multi-source information, as described in any one of claims 1-5.

Images