A bagged material loading control method and device based on multi-sensor fusion

Abstract

The application provides a bagged material loading control method and device based on multi-sensor fusion, comprising: determining whether the waiting vehicle meets the loading condition according to the pre-inspection result of the waiting vehicle, and allowing the waiting vehicle to enter the loading lane if yes; determining a virtual guide line and assisting the waiting vehicle to stop at the target position; re-inspecting the waiting vehicle at the target position to determine the re-inspection result; determining the loading mode based on the analysis result of the three-dimensional model according to the re-inspection result, and automatically completing the loading activation preparation; determining the operation parameters of the dust removal system according to the stages of the loading process and the equipment operation state, and adaptively adjusting; dynamically adjusting the robot loading posture according to the comparison result of the real-time three-dimensional scanning data and the preset loading path during the loading process; and determining to allow the vehicle to drive away in response to the loading completion signal. The application can realize intelligent and automatic collaborative work of the whole loading process through multi-sensor fusion and closed-loop control.

Description

Technical Field

[0001] This application relates to the field of automatic loading control technology, and more specifically, to a method and apparatus for controlling the loading of bagged materials based on multi-sensor fusion. Background Technology

[0002] In the loading of bagged materials in industries such as cement, building materials, chemicals, and grain, automated loading systems are gradually replacing traditional manual operations. However, existing automated loading systems generally suffer from the following problems: each control link is relatively independent, lacking coordination and still relying heavily on manual intervention. Specifically, there is a lack of effective compliance checks before vehicles enter the lane, and non-compliant vehicles are only discovered to be unsuitable for loading after entering the lane, causing congestion; the process of vehicles reversing into the lane relies on fixed markers and human experience, making it difficult to guarantee parking accuracy and affecting subsequent loading; the loading activation process requires manual card insertion, manual mode switching, and clicking to start, posing a risk of misoperation; the dust removal system during the loading process usually operates independently and cannot be intelligently adjusted according to real-time operating conditions, resulting in either excessive energy consumption or poor dust removal effect; the robot's loading posture is fixed and cannot be adjusted in real time according to the actual accumulation of materials, affecting palletizing quality and loading capacity.

[0003] These problems result in low overall efficiency, numerous safety hazards, and high energy consumption in existing automated loading systems, making it impossible to truly achieve efficient, safe, and energy-saving automated loading operations. Summary of the Invention

[0004] The present invention aims to overcome the shortcomings of the prior art and provide a method and device for controlling the loading of bagged materials based on multi-sensor fusion. Through multi-sensor fusion and closed-loop control, intelligent and automated collaborative operation of the entire loading process can be realized.

[0005] In a first aspect, the present invention provides a method for controlling the loading of bagged materials based on multi-sensor fusion, the method comprising: Based on the pre-inspection results of the waiting vehicles, determine whether the waiting vehicles meet the loading conditions. If so, the waiting vehicles are allowed to enter the loading lane. During the process of entering the loading lane, a virtual guide line is determined based on the three-dimensional model constructed by fusing lidar point cloud data and visual image data, as well as the relative positional relationship between the three-dimensional model and the preset ground markings, to assist the waiting vehicle in stopping at the target position. The waiting vehicles at the target location are re-inspected to determine the re-inspection results; Based on the re-inspection results, the loading mode is determined based on the analysis results of the three-dimensional model, and the loading and activation preparation is automatically completed. Based on the stages of the loading process and the operating status of the equipment, determine the operating parameters of the dust removal system and adjust them adaptively. The robot's loading posture is dynamically adjusted based on the comparison between real-time 3D scanning data during the loading process and the preset loading path. In response to the loading completion signal, the vehicle is allowed to leave.

[0006] Preferably, determining whether the waiting vehicle meets the loading conditions based on the pre-inspection results of the waiting vehicle includes: Based on the data collected by the laser scanner and vision camera installed at the entrance of the lane, the length, width, height of the waiting vehicle's cargo bed, and the reflection characteristics of the internal support structure of the cargo bed are extracted. The extracted reflection features are compared with a preset database of compatible vehicles to determine whether all parameters are within the allowable range. If yes, the gate will be opened and the waiting vehicle will be allowed to enter; if no, the waiting vehicle will be prompted to leave via the display screen.

[0007] Preferably, determining the virtual guide line and assisting the waiting vehicle to stop at the target location includes: By fusing the lidar point cloud data and the visual image data, a three-dimensional model is constructed that includes wheel positions, truck bed outline, and foreign objects inside the truck bed. Based on the relative positional relationship between the three-dimensional model and the preset ground markings, the deviation between the vehicle's current pose and the target pose is calculated in real time. Based on the deviation, the virtual guide line is dynamically generated and displayed on the vehicle terminal or on-site guide screen; When the relative position between the wheels and the preset ground markings is detected to meet the preset threshold, it is determined that the vehicle has stopped in place and the current position is locked as the target position.

[0008] Preferably, the step of re-inspecting the waiting vehicles at the target location to determine the re-inspection results includes: A second scan is performed on the waiting vehicle at the target location to obtain re-inspection three-dimensional data; The re-inspection three-dimensional data is compared with the three-dimensional data collected in the pre-inspection stage to determine whether the waiting vehicle has deformed or shifted during the entry process; If the deformation or displacement exceeds the danger threshold, the re-inspection result is determined to be unqualified, and the waiting vehicle is guided to stop again or an alarm is issued. If the deformation or displacement does not exceed the danger threshold, the re-inspection result is deemed qualified, and the process proceeds to the next activation step.

[0009] Preferably, the step of determining the loading mode based on the re-inspection results and automatically completing the loading and activation preparation includes: When the re-inspection result is qualified, the three-dimensional model is used to identify whether there are any foreign objects or supporting structures inside the truck bed that are prohibited from being loaded. If present, loading is prohibited and an alarm is triggered; If the interior of the truck bed is up to standard, then a single-machine or dual-machine loading mode will be matched based on the three-dimensional dimension analysis results of the truck bed. The radar is triggered to perform a final scan of the truck bed. Once the scan results confirm that the truck bed space matches the loading mode, the loading command is automatically initiated.

[0010] Preferably, determining the operating parameters of the dust removal system based on the stages of the loading process and the equipment operating status, and adaptively adjusting them, includes: The dust removal system includes an electric valve for adjusting the air volume and a pneumatic valve for controlling ash discharge. The loading process is divided into a standby phase, a startup phase, a stable loading phase, a wrap-up phase, and a completion phase. The equipment operating status is divided into normal, high load, low load, standby, and fault warning; Based on the combination of the current loading stage and the operating status of the equipment, a fuzzy control algorithm is used to determine the opening degree of the electric valve and the start / stop frequency of the pneumatic valve in the dust removal system.

[0011] Preferably, the dynamic adjustment of the robot's loading posture includes: During the loading process, the height and distribution of the materials already loaded in the truck bed are scanned in real time using lidar to obtain the real-time three-dimensional scanning data; The real-time 3D scanning data is compared with the preset loading path to calculate the error between the actual loading status and the target loading status. Based on the error, the robot's movement trajectory and palletizing posture are corrected in real time.

[0012] Secondly, the present invention provides a bagged material loading control device based on multi-sensor fusion, comprising: The loading condition determination module is used to determine whether the waiting vehicle meets the loading conditions based on the pre-inspection results of the waiting vehicle. If so, the waiting vehicle is allowed to enter the loading lane. The target location determination module is used to determine a virtual guide line and assist the waiting vehicle to stop at the target location during the process of entering the loading lane, based on a three-dimensional model constructed by fusing lidar point cloud data and visual image data, and the relative positional relationship between the three-dimensional model and the preset ground markings. The re-inspection result determination module is used to re-inspect the waiting vehicle at the target location to determine the re-inspection result; The loading mode determination module is used to determine the loading mode based on the re-inspection results and the analysis results of the three-dimensional model, and automatically complete the loading activation preparation. The operating parameter determination module is used to determine the operating parameters of the dust removal system based on the stage of the loading process and the operating status of the equipment, and to adjust them adaptively. The loading posture adjustment module is used to dynamically adjust the robot's loading posture based on the comparison results between real-time 3D scanning data and the preset loading path during the loading process. The signal response module is used to determine permission for the vehicle to leave in response to the loading completion signal.

[0013] Thirdly, the present invention provides a readable medium including executable instructions, which, when executed by a processor of an electronic device, cause the electronic device to perform any of the methods described in the first aspect.

[0014] Fourthly, the present invention provides an electronic device including a processor and a memory storing execution instructions, wherein when the processor executes the execution instructions stored in the memory, the processor performs the method as described in any of the first aspects.

[0015] This invention provides a method and device for controlling the loading of bagged materials based on multi-sensor fusion. By constructing a complete closed-loop control system of "pre-inspection-guidance-re-inspection-activation-dust removal-attitude adjustment-departure," it realizes intelligent collaborative operation throughout the entire loading process. It systematically solves the problems of high reliance on manual labor, fragmented links, numerous safety hazards, and high energy consumption in existing technologies. Pre-inspection at the entrance intercepts non-compliant vehicles at the source to avoid channel congestion. Intelligent guidance and re-inspection form a three-level verification mechanism to ensure parking accuracy and vehicle status stability. Automatic activation preparation eliminates the risk of human error. Adaptive dust removal control achieves a dynamic balance between energy saving and environmental protection. Real-time attitude adjustment ensures palletizing quality and loading efficiency. Ultimately, without increasing hardware costs, it significantly improves the overall efficiency, safety, and intelligence level of the loading system, achieving a comprehensive effect of cost reduction, efficiency improvement, safety, and environmental protection.

[0016] The further effects of the aforementioned non-conventional preferred method will be explained below in conjunction with specific embodiments. Attached Figure Description

[0017] To more clearly illustrate the embodiments of the present invention or the existing technical solutions, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic diagram of a bagged material loading control method based on multi-sensor fusion provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of another bagged material loading control method based on multi-sensor fusion provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the composition of a bagged material loading control device based on multi-sensor fusion, provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0020] See Figure 1 The image shows a specific embodiment of a bagged material loading control method based on multi-sensor fusion provided by the present invention. In this embodiment, the bagged material loading control method based on multi-sensor fusion includes:

[0021] Step 101: Based on the pre-inspection results of the waiting vehicles, determine whether the waiting vehicles meet the loading conditions. If so, allow the waiting vehicles to enter the loading lane. This step is used to conduct compliance checks on vehicles before they enter the loading lane, intercepting vehicles that do not meet equipment requirements at the source. The pre-inspection results refer to the initial inspection data obtained after scanning waiting vehicles using a laser scanner and a vision camera installed at the lane entrance. The laser scanner acquires 3D point cloud data of the vehicle, and the vision camera acquires image data; the fusion of these two data allows for accurate extraction of the vehicle's structural features. Loading parameters refer to the geometric dimensions and structural requirements that a vehicle must meet for safe and smooth automated loading operations. These specifically include the length, width, and height of the loading bed, as well as the presence of any supporting structures inside the bed that might affect loading (such as transverse braces or longitudinal steel pipes). The equipment-compatible vehicle database is a pre-stored set of vehicle parameter ranges that match the automated loading equipment.

[0022] In practice, data collected by laser scanners and vision cameras installed at the lane entrance is used to extract the length, width, and height of the waiting vehicle's cargo bed, as well as the reflection characteristics of the internal support structure. These extracted reflection characteristics are compared with a pre-set database of compatible vehicles to determine if all parameters are within acceptable limits. If all parameters are within limits, the vehicle is deemed to meet loading requirements, the barrier gate opens, and the waiting vehicle is allowed to enter the loading lane. If any parameter exceeds the acceptable limit, a message "Vehicle does not meet requirements, please leave" is displayed on the screen, and the barrier gate remains closed. This step, by performing the detection upfront, avoids lane congestion and efficiency losses caused by discovering non-compliant vehicles cannot be loaded only after they have entered the lane.

[0023] Step 102: During the process of entering the loading lane, based on the three-dimensional model constructed by fusing LiDAR point cloud data and visual image data, and the relative positional relationship between the three-dimensional model and the preset ground markings, determine the virtual guide line and assist the waiting vehicle to stop at the target position. Furthermore, this step provides real-time, dynamic parking guidance during the vehicle's reversing into the lane, ensuring the vehicle accurately parks at the preset loading position. The 3D model refers to the reconstructed 3D digital representation of the vehicle after fusing LiDAR point cloud data and visual image data, including details such as wheel positions, truck bed outlines, and any foreign objects inside the truck bed. Preset ground markings are positioning markers pre-painted on the loading lane floor, such as yellow borders or colored stripes, used to assist in determining whether the vehicle is parked correctly. Virtual guide lines are dynamic guide trajectories generated in real-time based on the deviation between the vehicle's current pose and the target pose, displayed on the onboard terminal or on-site guidance screen, used to instruct the driver to adjust the steering wheel angle.

[0024] In practice, LiDAR point cloud data and visual image data are integrated to construct a 3D model that includes wheel positions, truck bed outlines, and foreign objects inside the truck bed. Based on the relative positional relationship between the 3D model and preset ground markings, the deviation between the vehicle's current pose and the target pose is calculated in real time. The current pose refers to the vehicle's position and orientation at the current moment, while the target pose refers to the ideal position and orientation the vehicle should achieve when accurately parking at the loading position. Based on the calculated deviation, virtual guide lines are dynamically generated and displayed on the vehicle terminal or on-site guidance screen. When the system detects that the relative position between the wheels and the preset ground markings meets a preset threshold, it determines that the vehicle has parked correctly and locks the current position as the target position. This step, through dynamic guidance, effectively solves the problems of low parking accuracy and long adjustment times caused by traditional reliance on fixed markings and manual experience when reversing.

[0025] Step 103: Re-inspect the waiting vehicles at the target location to determine the re-inspection results; Furthermore, this step involves a secondary inspection after the vehicle has been parked to confirm whether it has undergone deformation or displacement during the entry process, ensuring that the vehicle's condition before loading is consistent with that during the pre-inspection. Specifically, the re-inspection refers to a second 3D scan of the vehicle after it has been parked at the target location, the purpose of which is to obtain actual state data of the vehicle after parking. The re-inspection result refers to the vehicle state consistency judgment result obtained by comparing the re-inspection 3D data with the pre-inspection 3D data.

[0026] In practice, the waiting vehicles at the target location are scanned a second time to obtain re-inspection 3D data. This re-inspection 3D data is compared with the 3D data collected during the pre-inspection phase to determine if the waiting vehicles have undergone deformation or displacement during entry. "Deformation" refers to the structural deformation of the vehicle bed due to bumps, shaking, or other reasons during entry; "displacement" refers to the shift in the overall position of the vehicle or the relative position of the bed. If the deformation or displacement exceeds a danger threshold, the re-inspection result is deemed unqualified, and the waiting vehicles are redirected or an alarm is issued for manual intervention. If the deformation or displacement does not exceed the danger threshold, the re-inspection result is deemed qualified, and the process proceeds to the next activation stage. This step, by comparing with the pre-inspection data, can detect subtle changes that occur during vehicle entry, preventing loading failures or equipment damage caused by changes in vehicle condition.

[0027] Step 104: Based on the re-inspection results and the analysis results of the 3D model, determine the loading mode and automatically complete the loading and activation preparation. Furthermore, this step is used to automatically determine the loading strategy and complete a series of pre-loading preparations based on the actual condition of the truck bed after the re-inspection is passed, thus automating the loading activation process. The loading mode refers to a single-machine loading mode or a dual-machine loading mode matched to the truck bed's size and structural characteristics. The single-machine mode is suitable for smaller truck beds, while the dual-machine mode is suitable for larger truck beds, with two robots working collaboratively. Loading activation preparation refers to the entire automated operation process from confirming the truck bed's qualification to initiating the loading command, including automatically reading vehicle identification information.

[0028] In practice, when the re-inspection result is qualified, the system identifies whether there are any foreign objects or supporting structures inside the truck bed that are prohibited from being loaded, based on the 3D model. "Foreign objects" refer to items that should not be present in the truck bed, such as tarpaulins, tools, and miscellaneous items. "Supporting structures" refer to inherent or temporarily added transverse braces, longitudinal steel pipes, etc., inside the truck bed that can obstruct the robot's normal stacking. If foreign objects or supporting structures are present, loading is prohibited and an alarm is triggered, prompting manual cleaning or handling. If the truck bed is qualified, the system automatically reads the vehicle's IC card information inserted into the card reader using RFID technology, or automatically identifies the license plate information using visual recognition technology, completing vehicle identity authentication. Subsequently, based on the 3D dimension analysis results of the truck bed, the system matches the single-machine or dual-machine loading mode, triggering the radar to perform a final scan of the truck bed. Once the scan confirms that the truck bed space matches the loading mode, the loading command is automatically initiated, without the need for manual card insertion, mode switching, or clicking the start button. This step, through automated activation preparation, eliminates the risk of misoperation and efficiency loss that may arise from manual operation.

[0029] Step 105: Determine the operating parameters of the dust removal system based on the stages of the loading process and the operating status of the equipment, and adjust them adaptively. Furthermore, this step is used to dynamically adjust the working parameters of the dust removal system according to real-time operating conditions during the loading process, achieving an optimal balance between dust removal efficiency and energy consumption. The loading process is divided into different stages based on the progress of the loading operation, including the standby stage (vehicle not yet arrived or loading not activated), the start-up stage (equipment starts operating after the loading command is issued), the stable loading stage (robot continuously palletizes, materials continuously fall), the finishing stage (loading is about to be completed, material falling frequency decreases), and the completion stage (loading completed, equipment stops). Equipment operating status refers to the current workload status of the loading equipment, including normal (equipment operates according to the preset process), high load (equipment operates at full load), low load (equipment operates intermittently or at low speed), standby (equipment has started but no loading action), and fault warning (equipment parameters are abnormal but the machine has not stopped). The dust removal system includes electric valves for adjusting airflow and pneumatic valves for controlling ash discharge. The electric valves control the airflow by adjusting their opening degree, and the pneumatic valves regulate the ash discharge rhythm by controlling the start-stop frequency. Fuzzy control algorithm is a rule-based intelligent control method that can dynamically determine the output value based on the combination of multiple input variables.

[0030] In practical implementation, the dust removal system includes electric valves for adjusting airflow and pneumatic valves for controlling ash discharge. The loading process is divided into standby, startup, stable loading, finishing, and completion stages. Equipment operating status is divided into normal, high load, low load, standby, and fault warning. Based on the combination of the current loading stage and equipment operating status, a fuzzy control algorithm is used to determine the opening degree of the electric valves and the start / stop frequency of the pneumatic valves in the dust removal system. For example, during the startup phase and when the equipment is under high load, the electric valve opening is set to 100%, and the pneumatic valve remains open to maximize dust removal capacity at startup. During the stable loading phase and when the equipment is in normal condition, the electric valve opening is dynamically adjusted between 60% and 90% based on real-time dust concentration, while the pneumatic valve opens intermittently. During the finishing phase and when the equipment is under low load, the electric valve opening is set to 40% to 60%, and the pneumatic valve opens intermittently. During the standby or completion phase and when the equipment is in standby mode, the electric valve opening is set to 20%, and the pneumatic valve opens intermittently to maintain negative pressure. When the equipment is in a fault warning state, regardless of the stage, the electric valve opening is adjusted to 100%, and the pneumatic valve remains open until the fault is resolved. Furthermore, the system monitors the dust removal duct pressure data in real time, predicts blockage risk based on the pressure drop trend, and automatically executes a backflushing cleaning process before triggering a formal alarm when the pressure drop rate exceeds a preset threshold. This step achieves a dynamic balance between dust removal efficiency and energy consumption by deeply integrating the dust removal system with the loading process and employing intelligent control algorithms. It also has the ability to perform fault pre-diagnosis and proactive maintenance.

[0031] Step 106: Based on the comparison results between the real-time 3D scanning data during the loading process and the preset loading path, dynamically adjust the robot's loading posture. Furthermore, this step is used to correct the robot's motion trajectory and stacking posture in real time based on the actual material accumulation in the truck bed during the loading operation, ensuring the neatness and stability of the palletizing. Real-time 3D scanning data refers to the dynamic data obtained by continuously scanning the height and distribution of the loaded material in the truck bed using LiDAR during the loading process. The preset loading path refers to the ideal robot motion trajectory pre-planned based on the truck bed size and loading mode. Dynamic adjustment refers to the process where the system calculates the correction amount of the robot's posture online and executes the adjustment in real time based on the comparison results between the real-time scanning data and the preset path. The core of this step is to establish a closed-loop feedback mechanism during the loading process, enabling the robot to adapt to random deviations during material accumulation, thereby improving palletizing quality and space utilization. The specific implementation method of this step, including the acquisition of real-time scanning data, the comparison method with the preset path, the error calculation rules, and the specific algorithm for posture correction, will be described in detail in subsequent embodiments.

[0032] Step 107: In response to the loading completion signal, determine that the vehicle is allowed to leave.

[0033] This step is used to issue a departure instruction to the driver and ensure the orderly exit of the vehicle after the loading operation is completed. The loading completion signal is the signal generated by the system after detecting that the loading operation has been completed and the equipment has stopped operating. "Permission to leave" means that the system confirms that loading is complete and the vehicle can safely leave the lane, and this is linked to the exit gate control.

[0034] In practice, once loading is complete, the system automatically stops and issues a voice prompt, "Loading complete. Please retrieve your IC card and leave the lane." The driver retrieves the vehicle IC card containing the loading record from the card reader and drives the vehicle away from the lane. Responding to the loading completion signal, the system confirms permission for the vehicle to leave, opens the exit gate, and waits for the next vehicle to enter. This step ensures a complete closed loop in the loading process, preventing vehicles from remaining in the lane for extended periods and affecting subsequent operations. Simultaneously, the loading data is recorded via the IC card, facilitating subsequent traceability and management.

[0035] As can be seen from the above technical solutions, the beneficial effects of this embodiment are: by constructing a complete closed-loop control system that includes entrance pre-inspection, intelligent guidance, docking re-inspection, automatic activation, adaptive dust removal and dynamic attitude adjustment, intelligent collaborative operation of the entire loading process is realized, systematically solving the problems of high dependence on manual labor, fragmentation of each link, many safety hazards and high energy consumption in the existing technology, and finally achieving the comprehensive effect of cost reduction, efficiency improvement and safety and environmental protection.

[0036] Figure 1 The embodiments shown are merely basic examples of the method of the present invention. Other preferred embodiments of the method can be obtained by making certain optimizations and extensions based on them.

[0037] like Figure 2 The image shows another specific embodiment of the bagged material loading control method based on multi-sensor fusion according to the present invention. This embodiment further describes the method based on the foregoing embodiments, and includes the following steps:

[0038] Step 201: During the loading process, the height and distribution of the materials loaded in the truck bed are scanned in real time using LiDAR to obtain real-time three-dimensional scanning data; Specifically, this step involves continuously collecting dynamic data on the material accumulation status within the truck bed during the loading process, providing real-time input for subsequent robot posture adjustments. LiDAR refers to a laser scanning device mounted above the loading station or on the robot itself, capable of emitting laser beams and receiving reflected signals to acquire three-dimensional spatial information of the object being measured. Real-time 3D scan data refers to a set of point cloud data reflecting the current height and distribution of the material, obtained by continuously scanning the loaded material within the truck bed using LiDAR. This data is characterized by real-time performance, high precision, and full coverage, accurately describing the actual accumulation pattern of the material within the truck bed, including information such as material height, accumulation tilt, and edge position in different areas. Unlike the scanning of empty truck beds in the pre-inspection and re-inspection stages, this step scans the material itself during the loading process, aiming to establish a real-time feedback mechanism for the loading process. The scanning frequency can be set according to the loading speed and accuracy requirements, typically several to tens of times per second, to ensure data timeliness. The acquired real-time 3D scan data will serve as the basis for comparison with the preset path in subsequent steps.

[0039] Step 202: Compare the real-time 3D scanning data with the preset loading path and calculate the error between the actual loading status and the target loading status. Furthermore, this step is used to quantitatively assess the deviation between the current loading state and the ideal state through data comparison. The preset loading path refers to the ideal robot motion trajectory and palletizing position sequence pre-planned and generated before loading, based on the three-dimensional dimensions of the truck bed, the loading mode (single or dual machines), and the material characteristics. This path includes the target placement position, stacking order, and robot posture for each bag of material within the truck bed, serving as a benchmark for the loading operation. The actual loading state refers to the actual accumulation of materials within the truck bed, as represented by real-time 3D scanning data. The target loading state refers to the ideal accumulation state of the materials within the truck bed after completing the loading operation that should be completed at the current moment according to the preset loading path. Error refers to the quantitative difference between the actual loading state and the target loading state, which can manifest in various forms such as material height deviation in local areas, shape and position deviation of the stacking contour, and offset of already stacked materials.

[0040] In practice, the system first spatially registers real-time 3D scanning data with the preset loading path, establishing a correspondence within the same coordinate system. Then, a comparison algorithm calculates the difference between the actual material stacking profile and the target stacking profile. For example, if the preset path requires five layers of stacking on the left side of the truck bed, but real-time scanning data shows the actual height in that area is only equivalent to four layers, it indicates missing stacking or material slippage. Similarly, if the preset path requires the material to be evenly distributed in the middle of the truck bed, but real-time scanning data shows excessively high local stacking in that area, it indicates material stacking deviation. The system quantifies these differences into specific error values, including parameters such as height error, position error, and volume error, providing a quantitative basis for subsequent attitude correction.

[0041] Step 203: Correct the robot's movement trajectory and palletizing posture in real time based on the error.

[0042] Furthermore, this step is used to adjust the robot's subsequent actions online based on the calculated error value, so that the actual loading state gradually approaches the target loading state. Here, the movement trajectory refers to the spatial path traversed by the robot from its current position to the next target position during the loading process, including linear motion, curved motion, and intermediate nodes. Palletizing posture refers to the angle, orientation, and action method adopted by the robot when placing materials, including the opening and closing state of the gripper, the tilt angle during placement, and the downward pressing action. Real-time correction refers to the system continuously performing closed-loop control of "collection-comparison-calculation-adjustment" during the loading process, ensuring that every robot action is optimized based on the latest material stacking state.

[0043] In practice, the system employs corresponding correction strategies based on the type and magnitude of the error. When it detects that the stacking on the left side is too high, the system can appropriately adjust the subsequent stacking positions to shift to the right to balance the overall stacking height. When it detects material slippage or displacement, it can adjust the gripper's placement angle and falling speed to increase the stability of the material after placement. When it detects gaps in a local area, it can appropriately increase the stacking density in that area in subsequent stacking to compensate. The correction amount is calculated based on the error value using a preset control algorithm, ensuring both the effectiveness of the correction and avoiding over-correction that could lead to motion disorder. This step enables the robot to dynamically adjust to actual working conditions, effectively addressing factors such as differences in material characteristics, random deviations during the stacking process, and external interference, ensuring the neatness and stability of the final loading effect, and improving the utilization rate of the truck bed space and the loading quality.

[0044] As can be seen from the above technical solutions, the beneficial effects of this embodiment are: by establishing a closed-loop feedback control mechanism of "real-time scanning - error calculation - dynamic correction", the robot can adjust its motion trajectory and stacking posture online according to the actual material stacking situation, effectively compensate for random deviations in the loading process, and ensure the neatness and stability of the stacking and maximize the utilization of the truck bed space.

[0045] This invention also provides a bagged material loading control device based on multi-sensor fusion. See also Figure 3 The image shows a specific embodiment of a bagged material loading control device based on multi-sensor fusion provided by the present invention. This embodiment of the device is used to execute... Figures 1-2 The physical apparatus of the method. Its technical solution is essentially the same as the above embodiments, and the corresponding descriptions in the above embodiments also apply to this embodiment. The apparatus includes:

[0046] The loading condition determination module 301 is configured to determine whether the waiting vehicle meets the loading conditions based on the pre-inspection results of the waiting vehicle, and if so, allow the waiting vehicle to enter the loading lane. The target location determination module 302 is configured to determine a virtual guide line and assist waiting vehicles to stop at the target location during the process of entering the loading lane, based on a three-dimensional model constructed by fusing lidar point cloud data and visual image data, and the relative positional relationship between the three-dimensional model and preset ground markings. The re-inspection result determination module 303 is configured to re-inspect the waiting vehicles at the target location to determine the re-inspection result; The loading mode determination module 304 is configured to determine the loading mode based on the re-inspection results and the analysis results of the three-dimensional model, and automatically complete the loading activation preparation. The operating parameter determination module 305 is configured to determine the operating parameters of the dust removal system based on the stage of the loading process and the operating status of the equipment, and to adjust them adaptively. The loading posture adjustment module 306 is configured to dynamically adjust the robot's loading posture based on the comparison results between real-time 3D scanning data and the preset loading path during the loading process. Signal response module 307 is configured to determine permission for the vehicle to leave in response to a loading completion signal.

[0047] Figure 4This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. The memory may include main memory, such as high-speed random-access memory (RAM), or it may also include non-volatile memory, such as at least one disk storage device. Of course, the electronic device may also include other hardware required for other services.

[0048] The processor, network interface, and memory can be interconnected via an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. Buses can be categorized as address buses, data buses, and other types. For ease of representation, Figure 4 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0049] Memory is used to store instructions for execution. Specifically, instructions for execution are computer programs that can be executed. Memory can include main memory and non-volatile memory, and it provides the processor with execution instructions and data.

[0050] In one possible implementation, the processor reads the corresponding execution instructions from non-volatile memory into main memory and then executes them. Alternatively, it can obtain the corresponding execution instructions from other devices to form a bagged material loading control device based on multi-sensor fusion at the logical level. The processor executes the execution instructions stored in the memory to implement the bagged material loading control method based on multi-sensor fusion provided in any embodiment of the present invention.

[0051] The above is as described in the present invention. Figure 3The method for controlling the loading of bagged materials based on multi-sensor fusion provided in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed through integrated logic circuits in the processor's hardware or through software instructions. The processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor can be a microprocessor or any conventional processor.

[0052] The steps of the method disclosed in the embodiments of this invention can be directly manifested as being executed by a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0053] This invention also proposes a readable medium storing execution instructions. When these instructions are executed by the processor of an electronic device, the electronic device can perform a multi-sensor fusion-based bagged material loading control method provided in any embodiment of this invention, specifically for performing actions such as... Figure 1 , Figure 2 The method shown.

[0054] The electronic devices in the foregoing embodiments may be computers.

[0055] Those skilled in the art will understand that embodiments of the present invention can be provided as methods or computer program products. Therefore, the present invention can be implemented in a completely hardware embodiment, a completely software embodiment, or a combination of software and hardware.

[0056] The various embodiments in this invention are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the apparatus embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0057] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0058] The above are merely embodiments of the present invention and are not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of the present invention should be included within the scope of the claims of the present invention.

Claims

1. A method for controlling the loading of bagged materials based on multi-sensor fusion, characterized in that, The method includes: Based on the pre-inspection results of the waiting vehicles, determine whether the waiting vehicles meet the loading conditions, including: Based on the data collected by the laser scanner and vision camera installed at the entrance of the lane, the length, width, height of the waiting vehicle's cargo bed, and the reflection characteristics of the internal support structure of the cargo bed are extracted. The extracted reflection features are compared with a preset database of compatible vehicles to determine whether all parameters are within the allowable range. If yes, the gate will be opened and the waiting vehicle will be allowed to enter the loading lane; if no, the waiting vehicle will be prompted to leave via the display screen. During the process of entering the loading lane, based on the 3D model constructed from fused lidar point cloud data and visual image data, and the relative positional relationship between the 3D model and preset ground markings, a virtual guide line is determined to assist the waiting vehicle in stopping at the target position, including: By fusing the lidar point cloud data and the visual image data, a three-dimensional model is constructed that includes wheel positions, truck bed outline, and foreign objects inside the truck bed. Based on the relative positional relationship between the three-dimensional model and the preset ground markings, the deviation between the vehicle's current pose and the target pose is calculated in real time. Based on the deviation, the virtual guide line is dynamically generated and displayed on the vehicle terminal or on-site guide screen; When the relative position between the wheels and the preset ground markings is detected to meet the preset threshold, it is determined that the vehicle has stopped in place and the current position is locked as the target position; The waiting vehicles at the target location are re-inspected to determine the re-inspection results, including: A second scan is performed on the waiting vehicle at the target location to obtain re-inspection three-dimensional data; The re-inspection three-dimensional data is compared with the three-dimensional data collected in the pre-inspection stage to determine whether the waiting vehicle has deformed or shifted during the entry process; If the deformation or displacement exceeds the danger threshold, the re-inspection result is determined to be unqualified, and the waiting vehicle is guided to stop again or an alarm is issued. If the deformation or displacement does not exceed the danger threshold, the re-inspection result is deemed qualified, and the process proceeds to the next activation stage. Based on the re-inspection results, the loading mode is determined according to the analysis results of the three-dimensional model, and the loading activation preparation is automatically completed, including: When the re-inspection result is qualified, the three-dimensional model is used to identify whether there are any foreign objects or supporting structures inside the truck bed that are prohibited from being loaded. If present, loading is prohibited and an alarm is triggered; If the interior of the truck bed is up to standard, then a single-machine or dual-machine loading mode will be matched based on the three-dimensional dimension analysis results of the truck bed. The radar is triggered to perform a final scan of the truck bed. Once the scan results confirm that the truck bed space matches the loading mode, the loading command is automatically initiated. Based on the stages of the loading process and the operating status of the equipment, determine the operating parameters of the dust removal system and adjust them adaptively, including: The dust removal system includes an electric valve for adjusting the air volume and a pneumatic valve for controlling ash discharge. The loading process is divided into a standby phase, a startup phase, a stable loading phase, a wrap-up phase, and a completion phase. The equipment operating status is divided into normal, high load, low load, standby, and fault warning; Based on the combination of the current loading stage and the operating status of the equipment, a fuzzy control algorithm is used to determine the opening degree of the electric valve and the start-stop frequency of the pneumatic valve in the dust removal system. The robot's loading posture is dynamically adjusted based on the comparison between real-time 3D scanning data during the loading process and the preset loading path. In response to the loading completion signal, the vehicle is allowed to leave.

2. The method according to claim 1, characterized in that, The dynamic adjustment of the robot's loading posture includes: During the loading process, the height and distribution of the materials already loaded in the truck bed are scanned in real time using lidar to obtain the real-time three-dimensional scanning data; The real-time 3D scanning data is compared with the preset loading path to calculate the error between the actual loading status and the target loading status. Based on the error, the robot's movement trajectory and palletizing posture are corrected in real time.

3. A bagged material loading control device based on multi-sensor fusion, used to execute the method according to any one of claims 1 to 2, characterized in that, include: The loading condition determination module is used to determine whether the waiting vehicle meets the loading conditions based on the pre-inspection results of the waiting vehicle. If so, the waiting vehicle is allowed to enter the loading lane. The target location determination module is used to determine a virtual guide line and assist the waiting vehicle to stop at the target location during the process of entering the loading lane, based on a three-dimensional model constructed by fusing lidar point cloud data and visual image data, and the relative positional relationship between the three-dimensional model and the preset ground markings. The re-inspection result determination module is used to re-inspect the waiting vehicle at the target location to determine the re-inspection result; The loading mode determination module is used to determine the loading mode based on the re-inspection results and the analysis results of the three-dimensional model, and automatically complete the loading activation preparation. The operating parameter determination module is used to determine the operating parameters of the dust removal system based on the stage of the loading process and the operating status of the equipment, and to adjust them adaptively. The loading posture adjustment module is used to dynamically adjust the robot's loading posture based on the comparison results between real-time 3D scanning data and the preset loading path during the loading process. The signal response module is used to determine permission for the vehicle to leave in response to the loading completion signal.

4. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein the program, when executed, performs the method of any one of claims 1 to 2.

5. An electronic device, characterized in that, The electronic device includes: processor; Memory used to store the processor's executable instructions; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the method described in any one of claims 1 to 2.

Images