Intelligent accessory and application method thereof

Abstract

The invention relates to the field of logistics storage, and discloses an intelligent accessory and an application method thereof.The intelligent accessory comprises a forklift used for bearing a controlled module and a basic component, the controlled module comprises a clamp, a pallet fork and a sensing module, and the sensing module is installed on the clamping face of the clamp and the bearing portion of the pallet fork and used for monitoring and generating point cloud data of the surrounding environment; the control framework module is installed in the forklift and used for completing double-line control combining driving control of the forklift and action control of the controlled module; and the communication module is used for realizing high-speed data transmission and intercommunication between the controlled module and equipment including a warehouse management system, a production scheduling system and adjacent forklifts. According to the invention, multi-sensor fusion of the laser radar, the visual camera, the IMU and the millimeter-wave radar is adopted, perception and positioning of cargo positions, categories and motion states are achieved, and inventory management efficiency is improved through sensor data complementary fusion.

Description

Technical Field

[0001] The present invention relates to the technical field of logistics warehousing, and specifically to intelligent attachments and their application methods. Background Art

[0002] Under the rapid development trend of the current logistics warehousing industry, the intelligent demand for goods perception and management has become increasingly prominent. However, there are many limitations in the traditional logistics warehousing goods perception technology, which is difficult to meet the requirements of modern efficient and accurate operations.

[0003] Some warehousing facilities choose to use visual cameras as the main perception means. Visual cameras can capture rich image information, which helps to identify details such as the surface features and labels of goods. However, their performance is greatly affected by environmental lighting conditions. In the case of dim light, over-bright light or uneven lighting, the clarity and color restoration of the image will significantly decrease, making it difficult to accurately extract the features of goods, thereby affecting the recognition accuracy of goods. Moreover, when the goods are partially blocked, the field of view of the visual camera will be restricted, and complete goods information cannot be obtained, resulting in deviations in the judgment of the position and attitude of the goods.

[0004] Even if some systems attempt to combine lidar and visual cameras, they often simply superimpose the data of the two, lacking in-depth analysis and effective fusion of sensor data. In this case, the quality differences, complementarity and respective limitations of different sensor data are not fully considered. For example, in the data fusion process, the credibility of the sensors is not scientifically evaluated and reasonable weight distribution is not carried out, which may cause the fusion result to be seriously interfered by low-quality data, unable to give full play to the advantages of multi-sensor collaborative work, and difficult to achieve high-precision perception and accurate judgment of goods.

[0005] In addition, the traditional perception system is insufficient in dealing with complex and changeable warehousing operation environments. When the goods are in a fast-moving state, there are vibration interferences in the warehouse or the electromagnetic interference in the environment is strong, the data stability and accuracy of the sensors will be greatly challenged. Due to the lack of an effective adaptive adjustment mechanism, these systems cannot optimize the sensor parameters and data fusion strategies in a timely manner according to environmental changes, resulting in a significant decrease in perception accuracy, and even data loss or errors, seriously affecting the reliability and efficiency of logistics warehousing operations, increasing the risks of goods damage, loss and operation delays, and being difficult to meet the urgent needs of the modern logistics warehousing industry for intelligent and efficient operations. Summary of the Invention

[0006] Aiming at the deficiencies of the prior art, the present invention provides intelligent attachments and their application methods, which solve the problems that the intelligent attachments in the prior art lack in-depth analysis and effective fusion of sensor data and are difficult to achieve high-precision perception and accurate judgment of goods.

[0007] To achieve the above object, the present invention is realized through the following technical solutions: An intelligent attachment, comprising: a forklift for carrying a controlled module and a basic component, the controlled module including a fixture, a fork, and a sensing module, the sensing module being installed on the clamping surface of the fixture and the bearing part of the fork for monitoring and generating point cloud data of the surrounding environment; A control architecture module installed inside the forklift for performing dual-line control combining the driving control of the forklift and the action control of the controlled module; A communication module for realizing high-speed data transmission and intercommunication between the controlled module and devices including a warehouse management system, a production scheduling system, and adjacent forklifts; The sensing module includes a lidar and a vision camera; The sensing module further includes: A data processing unit for preprocessing, fusion analysis, and credibility evaluation of the data collected by the sensing module, and adaptively adjusting the parameters of each module according to the evaluation results.

[0008] Preferably, the control architecture module includes: A main controller using a high-speed operation processor chip for receiving data from the sensing module, the communication module, and other sensors; A drive controller electrically connected to the drive motor and steering mechanism of the forklift for converting the driving instructions of the main controller into motor drive signals and steering signals to control the driving speed, acceleration, steering angle, and braking of the forklift; An action controller connected to the drive devices of the fixture and the fork for receiving the action instructions of the main controller through a CAN bus or industrial Ethernet, and using a position closed-loop control algorithm and a force feedback control algorithm to control the opening and closing angle, clamping force of the fixture, and the lifting height and telescopic length of the fork.

[0009] Preferably, the communication module includes: A wireless communication unit using 5G communication technology and a high-speed Wi-Fi6 standard, equipped with a high-gain antenna and a signal booster for real-time uploading of the point cloud data collected by the sensing module, the operating status information of the forklift, and the operation data of the controlled module, and quickly receiving task instructions from the warehouse management system, inventory information updates, and operation plans from the production scheduling system; A wired communication unit using an industrial Ethernet interface and an RS485 serial port as a backup communication means for establishing a data link through a wired connection when wireless communication is interfered with or fails.

[0010] Preferably, the forklift is further configured with: A power management module including a rechargeable battery pack and a power conversion circuit; The human - machine interaction interface, including a display screen, operation buttons and a voice interaction module, is provided to the operator through a wireless connection for realizing information interaction between the operator and the intelligent attachment; The fault diagnosis and early - warning module is used for real - time monitoring of the operation status of the equipment, timely detecting potential fault hazards and sending out early - warning signals; The credibility evaluation module verifies and evaluates the data of each sensor, and dynamically allocates weights in the data fusion process according to the evaluation results; The feedback and adaptive adjustment module is used for adaptively optimizing and adjusting the sensor parameters and data fusion strategies according to the sensor data quality and the operation environment dynamics.

[0011] Preferably, the data processing unit further includes: The time synchronization sub - unit, which combines hardware synchronization and software synchronization. Hardware synchronization is achieved by connecting lidar and vision cameras to a unified clock generator based on GPS time - keeping or high - precision atomic clocks, and software synchronization is achieved by running an accurate time synchronization algorithm to mark time stamps for the data frames collected by each sensor; The space registration sub - unit uses a checkerboard calibration board or a high - precision spherical calibration object to jointly calibrate lidar and vision camera sensors; The data fusion sub - unit adopts a hierarchical fusion strategy to divide the point cloud data of lidar into three - dimensional voxel units, determines the voxel index where the point cloud data point is located according to the coordinates of the point cloud data point, and fills the image pixel values, IMU data and millimeter - wave radar data of the corresponding position of the vision camera into the corresponding voxels to form fused voxel data.

[0012] Preferably, the fault diagnosis and early - warning module includes: The sensor monitoring unit, which real - time monitors the working parameters of key components including lidar, vision camera, IMU, millimeter - wave radar, drive motor, battery pack and controller; The fault analysis unit analyzes and processes the monitored data to judge whether there are potential fault hazards in the equipment; The early - warning notification unit, after the fault analysis unit detects potential fault hazards, sends out early - warning signals to the operator and the warehouse management system through an acoustic - optical alarm device and a wireless communication unit.

[0013] Preferably, the credibility evaluation module adopts a verification method based on multi - model fusion, including geometric model verification, physical model verification and semantic model verification.

[0014] In addition, the present invention also provides an application method for the intelligent attachment, including the following steps: Install lidar, RGB-D camera, IMU and millimeter-wave radar at specific positions in the warehousing area and complete the initialization configuration to provide a basis for subsequent data collection; Combine the high-precision clock module with the GPS timing system using the PTP protocol to synchronize the time of each sensor to the microsecond level, ensure the consistency of data time, and use a checkerboard calibration board to jointly calibrate each sensor, calculate the transformation matrix, and unify the data space coordinate system; Adopt the voxel grid method to fuse the raw data of multiple sensors, divide the point cloud voxel units and fill in the feature data, and at the same time filter and denoise. After extracting features from the data of each sensor, use the PCA algorithm to fuse the feature vectors, and based on the decision results of each sensor on the goods, use Bayesian inference to obtain the final decision to achieve the perception and judgment of the goods; Conduct verification based on multi-model fusion, use multiple indicators as the input of the fuzzy logic system, calculate the credibility value of the sensor data, and accordingly adjust the weight of the measurement covariance matrix in the Kalman filter; Adjust the parameters of the lidar, vision camera, IMU and millimeter-wave radar respectively according to the data quality indicators of each sensor and the environmental conditions.

[0015] The present invention provides an intelligent attachment and its application method. It has the following beneficial effects: 1. The present invention adopts the multi-sensor fusion of lidar, vision camera, IMU and millimeter-wave radar, and combines complex data processing and analysis technical solutions to achieve high-precision perception and positioning effects of information such as the position, attitude, category and motion state of goods. Through the complementary fusion of data from multiple sensors and the fine processing of the feature layer and decision layer, it can accurately obtain detailed information of goods under various complex working conditions, improve the refinement degree of goods management in logistics warehousing operations, and enhance the accuracy and efficiency of inventory management.

[0016] 2. The present invention adopts the technical solutions of adaptive optimization of sensor parameters and adaptive adjustment of data fusion strategies to achieve the effect of automatically adjusting the working state of the system according to different logistics warehousing environmental conditions, timely adjusting the parameters of the vision camera, the frequency modulation parameters of the millimeter-wave radar and the data fusion strategy, ensuring the stable and reliable operation of the system, reducing operation interruptions and errors caused by environmental factors, improving the continuity and stability of logistics warehousing operations, and providing a solid guarantee for intelligent warehousing operations.

[0017] 3. Based on the technical solutions of goods perception, environmental adaptability, and data quality assurance, the present invention achieves the effect of providing efficient and intelligent decision-making support for logistics warehousing operations. It can intelligently plan the optimal path of handling equipment, avoid unnecessary detours and waiting, and improve the handling efficiency. At the same time, in terms of goods storage management, it can automatically arrange reasonable storage positions according to the category, quantity, and inventory space of goods, optimizing the utilization rate of warehouse space. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a display diagram of the intelligent attachment in the present invention Figure 2 It is a schematic diagram of the intelligent attachment in the present invention; Figure 3 It is a schematic diagram of the controlled module in the present invention; Figure 4 It is a schematic diagram of the control architecture module in the present invention; Figure 5 It is a schematic diagram of the communication module in the present invention; Figure 6 It is a schematic diagram of the data processing unit in the present invention; Figure 7 It is a schematic diagram of the fault diagnosis and warning module in the present invention; Figure 8 It is a schematic diagram of the method flow in the present invention.

[0019] Among them, 1. Forklift; 2. Controlled module; 3. Fixture; 4. Fork; 5. Sensing module. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0020] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0021] Please refer to the attached Figure 1-8 , the embodiments of the present invention provide an intelligent attachment. The intelligent attachment includes: a forklift 1, which is used to carry the controlled module 2 and the basic components. The controlled module 2 includes a fixture 3, a fork 4, and a sensing module 5. The sensing module 5 is installed on the clamping surface of the fixture 3 and the bearing part of the fork 4, and is used to monitor and generate point cloud data of the surrounding environment; A control architecture module, which is installed inside the forklift 1 and is used to complete the dual-line control of the driving control of the forklift 1 and the action control of the controlled module 2; A communication module that enables high-speed data transmission and interconnection between the controlled module 2 and devices including a warehouse management system, a production scheduling system, and an adjacent forklift 1.

[0022] Specifically as follows: 1. Composition and initialization of intelligent data 1.1 Sensor installation In this embodiment, lidar is installed every 4 meters at the top of the storage shelves to scan vertically downward, and lidar is installed every 8 meters on both sides of the aisle to scan downward at an angle of 45°; an RGB-D camera is installed 1.8 meters above the front of the forklift and the AGV, with a horizontal viewing angle of 80°. Four RGB-D cameras are installed in the center of the warehouse roof and are evenly distributed at 90° and tilted downward at 60°; the IMU is fixed at the root of the forklift fork 4 and the center position of the AGV chassis; millimeter-wave radar is installed every 6 meters in the middle of the aisle between the shelves, with a height of 1.5 meters.

[0023] 2. Data acquisition and preprocessing 2.1 Time synchronization Each sensor is equipped with a high-precision clock module and connected to a clock synchronization system based on GPS time service to achieve synchronous distribution of the hardware clock signal, making the clock frequencies and phases of all sensors consistent. At the same time, the PTP protocol is run for software synchronization, and timestamps are marked on the data frames collected by the sensors, synchronizing the time of each sensor within microseconds to ensure strict temporal alignment of the data.

[0024] 2.2 Spatial registration A checkerboard calibration board is used for joint calibration of the lidar, vision camera, and millimeter-wave radar. The calibration board is placed within the common field of view of the sensors, and each sensor scans or takes pictures of the calibration board from different angles to obtain multiple sets of corresponding data point sets.

[0025] According to the calibration data, the transformation matrices from the coordinate systems of each sensor to the world coordinate system are calculated, such as the transformation matrix from the lidar coordinate system to the world coordinate system , the transformation matrix from the vision camera coordinate system to the world coordinate system , the transformation matrix from the millimeter-wave radar coordinate system to the world coordinate system , and all sensor data is uniformly transformed into the world coordinate system to achieve spatial alignment.

[0026] 3. Data fusion and analysis 3.1 Data layer fusion A voxel grid-based fusion method is adopted to divide the lidar point cloud data into three-dimensional voxel units with a side length of 0.1 meters, and the voxel index where the point cloud data points are located is determined according to the point coordinates of the point cloud data.

[0027] Fill the feature visual image pixel values including color and texture at the corresponding positions, IMU data (device attitude and motion information), and millimeter-wave radar data including target distance, speed, and angle information into the corresponding voxels to form fused voxel data, realizing the preliminary integration and enrichment of the original data in space. At the same time, use the voxel grid structure for filtering and noise reduction processing.

[0028] 3.2 Feature-level fusion Extract geometric features including edges and plane features from lidar point cloud data through gradient change calculation and the RANSAC algorithm; extract color, texture, and shape features from visual image data using a pre-trained ResNet network; extract attitude features (roll angle, pitch angle, yaw angle represented by Euler angles, as well as speed and displacement features obtained through integral operations) from IMU data; extract motion features including speed, acceleration, azimuth angle, and elevation angle information from millimeter-wave radar data.

[0029] Let the feature vectors extracted by each sensor be , , , . Fuse these feature vectors through the PCA algorithm to obtain the fused feature vector , reduce the dimensionality of the feature vector, retain key information, and improve the feature discriminability.

[0030] 3.3 Decision-level fusion Fuse the decision results of different sensors regarding the position, attitude, category, etc. of the goods. Let the position estimate of the goods obtained by the lidar through the point cloud matching algorithm be , and the attitude estimate be ; the visual camera gives the category judgment of the goods as and the attitude estimate as through target detection and recognition algorithms; the IMU provides the motion state information of the device to correct the decision results of other sensors; the millimeter-wave radar provides target existence and approximate position information as a supplement.

[0031] Use Bayesian inference for decision fusion. According to Bayes' formula: ; Calculate the posterior probability in different states and select the state with the highest probability as the final decision result to achieve high-precision perception and accurate judgment of the goods. Among them, is the prior probability of the goods being in a certain state, is the joint probability of each sensor making corresponding decision results when the goods are in state , and is the joint probability of each sensor making decision results.

[0032] 4. Data Verification and Credibility Assessment 4.1 Verification Based on Multi-Model Fusion 4.1.1 Geometric Model Verification For cuboid goods, establish a geometric model (a set of plane equations) based on its length, width, and height. Use the ICP algorithm to match the lidar point cloud data with the geometric model, and calculate the distance from the point cloud data points to the corresponding planes , fitting error . Among them, represents the number of points in the lidar point cloud data, represents the i th point in the point cloud data to the distance of its corresponding geometric model plane. If is within 0.05 meters, it is considered that the lidar has high credibility in measuring the geometric shape and position of the goods; otherwise, mark the abnormal area for further analysis and processing, and there may be problems including measurement errors, occlusion, and goods deformation

[0033] 4.1.2 Physical Model Verification Consider the mass, center of gravity position, and inertia moment tensor of the goods, combine the device attitude and motion information measured by the IMU, and establish a physical model according to the Newton-Euler equations. Calculate the deviation between the actual acceleration and angular velocity measured by the IMU of the goods and the theoretical values predicted by the physical model, such as: ; Among them, , acceleration in the x-axis, y-axis, and z-axis directions. If the acceleration deviation is within 0.5 m / s² and the angular velocity deviation is within 0.1 rad / s, it means that the sensor data can accurately reflect the physical state of the goods and the data credibility is high; otherwise, correct the data or re-evaluate it, and check whether the goods are abnormal

[0034] 4.1.3 Semantic Model Verification Use the object detection based on CNN and the Faster R-CNN recognition algorithm to perform semantic analysis on the image data collected by the visual camera, and obtain the predicted results of the goods category and its probability . Compare and verify the semantic accuracy with the known goods list and inventory information, and further confirm the identity and status of the goods by combining the lidar spatial position information and the millimeter wave radar detection results. If the semantic model recognition results are mutually confirmed with other sensor data and consistent with the inventory records, it indicates that the visual data semantics has high credibility; otherwise, check the possible reasons for misidentification, such as insufficient light and the influence of occluders, and take measures including adjusting the camera parameters and re-collecting images to improve the recognition accuracy

[0035] 4.1.4. Data Consistency Detection and Anomaly Detection Consistency test index: Calculate the consistency index of the distance of the cargo position measured by the lidar and the vision camera , where represents the coordinate of the cargo position measured by the lidar in the x-axis direction, represents the coordinate in the y-axis direction, represents the coordinate in the z-axis direction, , , are the coordinate values of the cargo in the three-dimensional space measured by the vision camera. Correspondingly, is the coordinate of the cargo position measured by the vision camera in the x-axis direction, is the coordinate in the y-axis direction, is the coordinate in the z-axis direction. If it is less than 0.1 meter, it is considered that the position measurements of the two are in good agreement; Calculate the consistency index of the device attitude angles measured by the IMU and the lidar: , where are the attitude angles of the IMU and the lidar measurement devices respectively. If the deviation of each angle is within 5°, it is considered that the attitude measurements are consistent; Calculate the similarity index of the cargo features recognized by the vision image and the semantic model: ; where is the vision image feature vector, is the semantic feature vector.

[0036] If the cargo feature similarity index is greater than 0.8, it is considered that the feature extraction is reliable.

[0037] Anomaly detection algorithm: Use the clustering algorithm based on GMM to perform anomaly detection on the sensor data.

[0038] First, train the GMM model with historical data to obtain the parameters of each Gaussian distribution. For the newly collected data points, calculate their probability density function values under each Gaussian distribution and the probability of belonging to the GMM model: ; where k is the index of the Gaussian distribution in the Gaussian mixture model (GMM), representing the k-th Gaussian distribution, is the probability density function value under the k-th Gaussian distribution, is the mixing coefficient of the k-th Gaussian distribution, the covariance matrix of the k-th Gaussian distribution. If Less than 0.01, it is determined as an abnormal data point, and corresponding measures (such as data repair, sensor recalibration, alarm prompt) are taken to ensure the reliability and stability of the system.

[0039] 4.2. Credibility Assessment and Weight Allocation 4.2.1. Credibility Assessment Model Adopt the fuzzy logic reasoning method, take the geometric model fitting error, physical model prediction deviation, data consistency index, anomaly detection result, etc. as the input variables of the fuzzy logic system, define the corresponding fuzzy sets and membership functions, establish a fuzzy inference rule base based on expert experience and experimental data, and calculate the credibility value of each sensor data through fuzzy inference (the value range is from 0 to 1), the higher the credibility value, the stronger the data reliability.

[0040] 4.2.2. Dynamic Weight Allocation In the fusion algorithm based on Kalman filter, adjust its measurement covariance matrix according to the credibility of sensor data , for example, , where I is the identity matrix, is the standard deviation of the sensor measurement error, where (when ), make the sensor data with high credibility have a greater weight in the state estimation update process and have a greater impact on the fusion result; for the data with low credibility, increase the covariance matrix, reduce the weight, and reduce the interference. Through this dynamic weight allocation mechanism, adaptively adjust the fusion strategy according to the quality of sensor data to ensure that the system outputs accurate and reliable cargo status information.

[0041] 5. Feedback and Adaptive Adjustment of LiDAR 5.1. Adaptive Optimization of Sensor Parameters LiDAR parameter adjustment: According to the point cloud density of the point cloud data (count the number of points in a 1 cubic meter space, if it is less than 1000 points / cubic meter, it is insufficient density), noise level (judged by the standard deviation of the point cloud, if it is greater than 0.05 meters, the noise is high) and effective measurement range (determined according to the point cloud intensity attenuation and signal loss situation, if it is less than the expected range, it is limited), etc. Adjust the parameters according to the quality indicators. When the point cloud density is low, according to and increase the transmission power and pulse frequency ( Watt / (point / cubic meter), Hz / (point / cubic meter)), where, is the initial transmission power, is the initial pulse frequency; when the noise is high, press Increase the scanning angle overlap rate ( 0.1 / m), where is the initial overlap rate; when the effective measurement range is small, press and Increase the transmission power and adjust the wavelength ( W / m, nm / m).

[0042] Visual camera parameter adjustment: By analyzing the grayscale histogram, contrast, and sharpness characteristic indicators of the image, combined with the deep learning image quality assessment model, judge the image quality. When the image is too bright and the contrast is low, press 、 and Reduce the aperture size, shutter speed and sensitivity ( , , ), where is the initial aperture size, is the initial shutter speed; when the image is blurred, press and Increase the aperture and adjust the focus position ( , mm); according to the distance and size of the goods, press Dynamically adjust the zoom ratio ( , ), is the initial zoom ratio.

[0043] IMU parameter calibration: Regularly compare and calibrate the IMU measurement data with the lidar position data and the visual camera attitude data, and use the Kalman filter algorithm to estimate and correct the bias parameters of the IMU. When abnormal fluctuations in the IMU data are detected or inconsistent with other sensors, start the calibration program in a timely manner, recalculate and update the zero bias and scale factor parameters to ensure accurate and stable measurement data, and provide a basis for compensating other sensor data. The calibration period can be set to once per hour or immediately calibrated when the deviation exceeds the threshold, and the Kalman filter parameter estimation is performed by accumulating 10 minutes of comparison data.

[0044] Millimeter-wave radar parameter optimization: In a complex electromagnetic interference environment, according to the electromagnetic interference intensity , press Optimize frequency modulation method ( Hz / (V / m)); When the distance of the goods is far or the reflected signal is weak (judged by RSSI), according to Increase the transmission power ( W / (dBm)), is the initial transmission power; According to the movement state of the goods (speed and acceleration ), and the speed range , according to , and Adjust the target detection threshold, tracking filter parameters and ( ) in the signal processing algorithm, is the initial target detection threshold, is the initial tracking filter parameter.

[0045] 5.2. Adaptive adjustment of data fusion strategy Dynamic switching of fusion levels: When working normally, a complete three-layer data fusion architecture is adopted: data layer, feature layer, and decision layer. By monitoring the data quality indicators of each sensor (such as lidar point cloud density, visual image clarity, IMU deviation), when the data quality of a certain sensor is lower than the preset fault threshold (such as the lidar point cloud density is lower than 50% of the normal value), the fusion level switching mechanism is triggered, and it switches to a simplified fusion mode (such as feature layer and decision layer fusion), and adjusts the corresponding fusion algorithm parameters. For example, in the simplified mode, increase the feature weights of the visual camera and millimeter-wave radar, and reduce the feature weights missing due to the faulty sensor, to ensure that the system can still maintain the basic working ability when some sensors fail and avoid system paralysis.

[0046] Adaptive adjustment of fusion algorithm parameters: In the fusion algorithm based on Kalman filter, in addition to adjusting the measurement covariance matrix according to the sensor credibility, the process noise covariance matrix is also adjusted according to the dynamic changes of the environment. When the goods are quickly transported or the environmental vibration interference is large, according to increase the value ( It is the environmental acceleration to reflect the increased uncertainty of the system state and improve the accuracy of state estimation. In the voxel grid fusion method for data layer fusion, if it is found that the sensor data consistency is poor or there are many abnormal data in some areas, the voxel side length (meter / (point)) is decreased to improve the data fusion accuracy and the robustness to abnormal data. To avoid system instability caused by too frequent parameter adjustment, the adjustment step size (such as not exceeding 10% of the initial value each time) and the delay time (such as adjusting according to the latest monitoring data every 10 seconds) can be set.

[0047] Through the above steps and processes, the intelligent attachment and its application method provided by the embodiments of the present invention can efficiently and accurately sense the state of goods in the logistics warehousing environment, realize intelligent operation, and improve the management level and operation efficiency of logistics warehousing. At the same time, each step is closely connected, and the output of the previous step provides the basis and foundation for the next step, forming a complete and organic intelligent sensing system to ensure the stability, reliability and efficiency of the system.

[0048] The application method of the intelligent attachment described below can be mutually corresponding and referred to the intelligent attachment described above, specifically as follows: The application method of the intelligent attachment includes the following steps: Install lidar, RGB-D camera, IMU and millimeter wave radar at specific positions in the warehousing area and complete the initialization configuration to provide the basis for subsequent data collection; Through the combination of the high-precision clock module and the GPS time service system with the PTP protocol, synchronize the time of each sensor to the microsecond level to ensure the data time consistency, and use the checkerboard calibration board to jointly calibrate each sensor, calculate the transformation matrix, and unify the data space coordinate system; Adopt the voxel grid method to fuse the original data of multiple sensors, divide the point cloud voxel units and fill the feature data, and at the same time filter and denoise. After extracting the features from the data of each sensor, fuse the feature vectors with the PCA algorithm, and based on the decision results of each sensor on the goods, use Bayesian inference to obtain the final decision to realize the perception and judgment of the goods; Conduct verification based on multi-model fusion, use multiple indicators as the input of the fuzzy logic system, calculate the credibility value of the sensor data, and accordingly adjust the weight of the measurement covariance matrix in the Kalman filter; Adjust the parameters of lidar, vision camera, IMU and millimeter wave radar respectively according to the data quality indicators of each sensor and the environmental conditions.

[0049] The application method of this embodiment can be used to execute the above method embodiment, and its principle and technical effect are similar, so details are not described here.

[0050] Although embodiments of the present invention have been shown and described, those of ordinary skill in the art will appreciate that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. Intelligent attachment, characterized in that: include: A forklift (1) used to carry a controlled module (2) and a basic component, wherein the controlled module (2) comprises a clamp (3), a fork (4) and a sensing module (5), wherein the sensing module (5) is installed on a clamping surface of the clamp (3) and a bearing portion of the fork (4), and is used to monitor and generate point cloud data of the surrounding environment; A control architecture module, which is installed inside the forklift (1) and is used to complete a dual-line control combining the driving control of the forklift (1) and the motion control of the controlled module (2); A communication module, which realizes high-speed data transmission and communication between the controlled module (2) and equipment including a warehouse management system, a production scheduling system and an adjacent forklift (1); The perception module (5) includes a laser radar and a visual camera; The perception module (5) further includes: The data processing unit performs preprocessing, fusion analysis and credibility evaluation on the data collected by the perception module (5), and adaptively adjusts the parameters of each module according to the evaluation results.

2. The intelligent attachment according to claim 1, characterized in that: The control architecture module includes: A main controller, which uses a high-speed computing processor chip and is used to receive data from the sensing module (5), the communication module and other sensors; A drive controller electrically connected to a drive motor and a steering mechanism of the forklift (1), converting a driving instruction of the main controller into a motor drive signal and a steering signal, and controlling the driving speed, acceleration, steering angle and braking of the forklift (1); An action controller is connected to the driving devices of the clamp (3) and the fork (4), receives action instructions from the main controller via a CAN bus or industrial Ethernet, and uses a position closed-loop control algorithm and a force feedback control algorithm to control the opening and closing angle and clamping force of the clamp (3) and the lifting height and telescopic length of the fork (4).

3. The intelligent attachment according to claim 1, characterized in that: The communication module comprises: The wireless communication unit adopts 5G communication technology and high-speed Wi-Fi6 standard, is equipped with a high-gain antenna and a signal booster, and uploads point cloud data collected by the sensing module (5), the operation status information of the forklift (1), and the operation data of the controlled module (2) in real time, and quickly receives task instructions, inventory information updates, and operation plans from the warehouse management system and the production scheduling system; The wired communication unit uses the industrial Ethernet interface and RS485 serial port as backup communication means. When the wireless communication is disturbed or fails, a data link is established through a wired connection.

4. The intelligent attachment according to claim 2, characterized in that: The forklift (1) is also equipped with: A power management module, the power management module comprising a rechargeable battery pack and a power conversion circuit; The human-machine interaction interface, including a display screen, operation buttons and voice interaction module, is provided to the operator through a wireless connection to realize information interaction between the operator and the intelligent attachment; Fault diagnosis and early warning module, used to monitor the operating status of the equipment in real time, detect potential faults in time and issue early warning signals; The credibility assessment module verifies and assesses the credibility of the data from each sensor, and dynamically allocates weights in the data fusion process based on the assessment results; The feedback and adaptive adjustment module is used to adaptively optimize sensor parameters and data fusion strategies according to the sensor data quality and operating environment dynamics.

5. The intelligent attachment according to claim 1, characterized in that: The data processing unit also includes: The time synchronization subunit adopts a combination of hardware synchronization and software synchronization. The hardware synchronization connects the lidar and visual camera to a unified clock generator based on GPS timing or high-precision atomic clocks. The software synchronization timestamps the data frames collected by each sensor by running a precise time synchronization algorithm. The spatial registration subunit uses a checkerboard calibration plate or a high-precision spherical calibration object to jointly calibrate the lidar and visual camera sensors; The data fusion subunit uses a hierarchical fusion strategy to divide the lidar point cloud data into three-dimensional voxel units, determines the voxel index of the point cloud data point according to its coordinates, and fills the image pixel value, IMU data and millimeter-wave radar data of the visual camera at the corresponding position into the corresponding voxel to form fused voxel data.

6. The intelligent attachment according to claim 4, characterized in that: The fault diagnosis and early warning module includes: The sensor monitoring unit monitors the working parameters of key components including lidar, visual camera, IMU, millimeter-wave radar, drive motor, battery pack and controller in real time; The fault analysis unit analyzes and processes the monitoring data to determine whether there are hidden faults in the equipment; After the early warning notification unit and the fault analysis unit detect potential faults, they send early warning signals to the operator and the warehouse management system through the sound and light alarm device and the wireless communication unit.

7. The intelligent attachment according to claim 4, characterized in that: The credibility evaluation module adopts a verification method based on multi-model fusion, including geometric model verification, physical model verification and semantic model verification.

8. An application method of an intelligent attachment, applied to the intelligent attachment according to claim 1, characterized in that: The following steps are involved: Install LiDAR, RGB-D camera, IMU and millimeter wave radar at specific locations in the storage area and complete initial configuration to provide a basis for subsequent data collection; Through the high-precision clock module and GPS timing system combined with the PTP protocol, the time of each sensor is synchronized to the microsecond level to ensure data time consistency. The checkerboard calibration board is used to jointly calibrate each sensor, calculate the conversion matrix, and unify the data space coordinate system. The voxel grid method is used to fuse the raw data of multiple sensors, divide the point cloud voxel units and fill in the feature data, and filter and reduce noise at the same time. After extracting features from the data of each sensor, the PCA algorithm is used to fuse the feature vectors. Based on the decision results of each sensor on the goods, the final decision is made using Bayesian reasoning to achieve goods perception and judgment; Conduct verification based on multi-model fusion, use multiple indicators as input to the fuzzy logic system, calculate the sensor data credibility value, and adjust the measurement covariance matrix allocation weight in the Kalman filter accordingly; According to the data quality indicators of each sensor and the environmental conditions, the parameters of the lidar, visual camera, IMU and millimeter-wave radar are adjusted respectively.

Images