Complex scene space construction method and system based on multi-sensor data fusion

Abstract

The invention relates to a complex scene space construction method and system based on multi-sensor data fusion, and relates to the field of artificial intelligence, and the method comprises the steps: collecting carrying information and the current position of a preset carrying device; calling an article carrying place and a carrying terminal point based on the carrying information; generating a device carrying route based on the current position of the device, the article carrying place, the carrying end point and a preset carrying site terrain; the carrying device is controlled to run along the device carrying route, and front image information is collected; and when and only when the front image information contains preset obstacle features, updating the device carrying route by using a preset route planning method, and controlling the carrying device to run along the updated device carrying route. The method has the effect of improving the luggage transportation efficiency.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to a method and system for constructing complex scene spaces based on multi-sensor data fusion. Background Art

[0002] Complex scene space construction refers to collecting environmental data through sensors, combining algorithm processing and modeling technology to build a three-dimensional space model with geometric structure, semantic information and dynamic characteristics. It is widely used in fields such as autonomous driving and intelligent robots.

[0003] In the scenario of airport luggage transportation, luggage needs to go through multiple links such as security inspection, sorting, conveyor belt transfer, and installation. From security inspection, sorting, conveyor belt transfer to installation, a large amount of manual participation is required to transport the luggage to the destination.

[0004] However, during manual transportation, luggage may be misclassified or missed due to visual fatigue or negligence, which in turn reduces the efficiency of luggage transportation and needs to be improved. Summary of the Invention

[0005] In order to improve the efficiency of luggage transportation, the present invention provides a complex scene space construction method and system based on multi-sensor data fusion.

[0006] In a first aspect, the present invention provides a method for constructing a complex scene space based on multi-sensor data fusion, which adopts the following technical solutions: A method for constructing complex scene space based on multi-sensor data fusion, comprising: S1: Collecting transport information and the current position of the preset transport device; S2: Retrieving the transport location and transport destination of the items based on the transport information; S3: generating a device transport route based on the current position of the device, the location of the transported item, the transport destination, and a preset transport site topography; S4: controlling the transport device to travel along the transport route of the device and collecting image information in front of the device; S5: When and only when the front image information contains preset obstacle features, update the device transport route using a preset route planning method, and control the transport device to travel along the updated device transport route.

[0007] By employing this technical solution, the system first collects handling information and the current location of the device, then combines it with the terrain of the handling site to generate an initial handling route, providing clear driving guidance for the handling device. As the device moves, it continuously collects image information in front of it and identifies obstacles. Once an obstacle is detected, the route planning method is immediately activated to update the path, ensuring that the handling device can avoid the obstacle in time and continue to operate efficiently. This achieves real-time perception and dynamic response to complex scenarios, thereby improving the efficiency of luggage transportation.

[0008] Optionally, the route planning method includes: S50: Obtaining the obstacle type based on the front image information and the obstacle features; S51: When the obstacle type is a preset route center obstacle, collecting the left side free space and the right side free space; S510: When both the left and right free spaces exceed the preset required action space, obtaining the moving distances on both sides according to the front image information, the left and right free spaces; S5100: Obtaining a rotation direction based on the moving distances on both sides; S5101: Update the device transport route based on the rotation direction, the moving distance on both sides, the location of the transported item, the transport destination, and the topography of the transport site.

[0009] Optionally, also include: S511: When the left side free space and the right side free space do not exceed the preset required action space, collecting the obstacle height value; S5110: When the obstacle height is lower than a preset reference height, determining whether the left clearance and the right clearance are greater than a preset device movement clearance; S5111: When the left and right clearances are greater than the device movement clearance, the moving distances on both sides are obtained based on the front image information, the left and right clearances, and S5100 to S5101 are executed, while the object is transported using a preset lifting and transporting method. S5112: When the left side free space and the right side free space are not larger than the device movement clearance, an obstacle abnormality prompt is reported.

[0010] Optionally, the lifting and transporting method includes: S51110: Collecting item image information; S51111: Scan and identify preset object features from the object image information to obtain object dimensions; S51112: Generate a lifting height value based on the object size and the obstacle height value; S51113: Control the transport device to carry out transport and lifting at the lifting height value, and after lifting, travel along the updated transport route of the device.

[0011] Optionally, also include: S52: When the obstacle type is a preset moving obstacle, collecting the obstacle moving speed and obstacle size; S520: Obtaining the obstacle disappearance time based on the obstacle movement speed and the obstacle size; S521: Collect obstacle distance; S522: Generating a device avoidance speed based on the obstacle distance, the obstacle disappearance time, and a preset safety distance; S523: Control the transport device to travel along the device transport route at the device avoidance travel speed.

[0012] Optionally, also include: S60: collecting driving image information; S61: If and only if the driving image information contains a preset steep feature, report a road steepness prompt, and scan and identify the steep feature from the driving image information to obtain a road steepness value; S610: When the road surface steepness value does not exceed a preset reference steepness value, controlling the transport device to continue traveling along the device transport route; S611: When the road surface steepness value exceeds a preset reference steepness value, performing route planning using a preset steep route planning method to obtain a steep driving route; S612: Control the transport device to travel along the steep travel route.

[0013] Optionally, the steep route planning method includes: S6110: Collect X-ray image information of items; S6111: Obtaining information about items in the box based on the X-ray image information of the items; S6112: When the items in the box include a preset fragile item, determine the filler color, filler outline, and filler width based on the X-ray image information of the item and the items in the box; S6113: Obtaining a filling density based on the filling color; S6114: Generate earthquake resistance parameters based on the contents of the box, the filler profile, the filler density, and the filler width; S6115: Obtaining a driving steepness value based on the anti-seismic parameter and a preset driving speed of the device; S6116: When the road surface steepness value does not exceed the driving steepness value, define the device transport route as a steep driving route.

[0014] Optionally, the steep route planning method further includes: S6117: When the road surface steepness value exceeds the driving steepness value, updating the driving steepness value based on the anti-seismic parameter and a preset minimum driving speed of the device; S61170: When the road surface steepness value does not exceed the updated driving steepness value, defining the device transport route as a steep driving route; S611700: Controlling the transport device to travel along the steep driving route at the device's lowest driving speed, and collecting steep driving image information; S611701: If and only if the steep driving image information does not contain the steep feature, control the transport device to travel along the steep driving route at the device driving speed; S61171: When the road surface steepness value exceeds the updated driving steepness value, a steepness abnormality prompt is reported.

[0015] Optionally, also include: S70: collecting the device travel position of the transport device; S71: When the driving position of the device is consistent with the location of the transported items, collecting regional image information of a preset luggage placement area on a preset centralized transport vehicle; S72: If and only if the regional image information does not contain a preset transport device feature, collect transport image information; S73: Scan and identify preset slope features from the transport image information to obtain a slope position; S74: generating a climbing route based on the device's driving position, the slope position, and the luggage storage area, and controlling the transport device to move toward the luggage storage area along the climbing route; S75: When the transport device reaches the luggage placement area, the transport device is controlled to clamp the items in the luggage placement area and place them on a preset transport device.

[0016] In a second aspect, the present application provides a complex scene space construction system based on multi-sensor data fusion, which adopts the following technical solutions: A complex scene space construction system based on multi-sensor data fusion, including: The acquisition module is used to collect the transport information, the current position of the device and the image information in front of it; A memory for storing a program for implementing any of the above-mentioned complex scene space construction methods based on multi-sensor data fusion; The processor is configured to load and execute the program stored in the memory.

[0017] In summary, this application includes at least one of the following beneficial technical effects: 1. By adopting the above technical solution, the system first collects handling information and the current position of the device, and combines it with the terrain of the handling site to generate an initial handling route, providing clear driving guidance for the handling device. During the driving process, the device continuously collects image information in front of it and identifies obstacles. Once an obstacle is detected, the route planning method is immediately activated to update the path, ensuring that the handling device can avoid the obstacle in time and continue to operate efficiently. This achieves real-time perception and dynamic response to complex scenarios, thereby improving the efficiency of luggage transportation. 2. The system first conducts an in-depth analysis of the image information in front of it, and combines it with the characteristics of the obstacle to accurately identify the type of obstacle. When an obstacle is detected in the center of the route, the free gaps on the left and right sides are collected synchronously, and by comparing them with the required action gaps, it is determined whether the conditions for detour are met. If both sides meet the gap requirements, the moving distance on both sides is further calculated to determine the optimal direction of rotation. In this way, when updating the device's transportation route, not only the obstacle position and moving distance are considered, but also global factors such as the location of the transported items, the transportation end point, and the terrain of the transportation site are taken into consideration to ensure that the new route meets both obstacle avoidance requirements and the overall transportation goals, thereby providing a reliable automation solution for scenarios such as airport baggage transportation, further promoting the development of intelligent logistics; 3. By first collecting X-ray image information of the items, deep learning algorithms are used to analyze the contents of the box and accurately identify fragile items. For fragile items, features such as the filler's color, outline, and width are further extracted. The filler density is then determined by combining the color-density mapping relationship, thereby constructing a multi-dimensional seismic parameter model. By combining the seismic parameters with the device's driving speed, a personalized driving steepness value is generated, which serves as the key threshold for route selection. When the road surface steepness value does not exceed the driving steepness value, a steeper route is selected. This ensures the safety of fragile items during transportation while avoiding efficiency losses caused by excessive detours, thereby improving the adaptability and reliability of the logistics system in complex scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 is a method flow chart of a complex scene space construction method based on multi-sensor data fusion in an embodiment of the present invention; Figure 2 It is a brief schematic diagram of the device transportation route in an embodiment of the present invention; Figure 3Schematic diagram of a route in which the obstacle type is a route center obstacle in an embodiment of the present invention. DETAILED DESCRIPTION

[0019] The present invention is further described in detail below with reference to the accompanying drawings and embodiments.

[0020] Reference Figure 1 and Figure 2 , the embodiment of the present application discloses a method for constructing a complex scene space based on multi-sensor data fusion, comprising the following steps: S1: Collecting transport information and the current position of a preset transport device.

[0021] Transport information refers to the relevant information required by a transport device to transport an item. This transport information can be retrieved from a pre-set transport terminal. The transport terminal stores this transport information. Whenever a transport task occurs, the relevant transport information for that task is recorded in the transport terminal for easy retrieval. A transport terminal is a terminal used to store this transport information. The transport terminal is pre-configured by those skilled in the art and will not be described in detail here. In this embodiment, the item to be transported is a suitcase.

[0022] A transport device is a mobile robot used to transport items. The current location of the device refers to the current location of the transport device. The current location of the device is acquired using a GPS positioning chip pre-installed on the transport device.

[0023] S2: Retrieve the transport location and transport destination based on the transport information.

[0024] The transport location is the location of the items being transported by the transport device. The transport destination is the location where the transport device will transport the items. The transport location and destination can be retrieved by understanding the transport information, which contains the transport location and destination.

[0025] S3: Generate a device transport route based on the current location of the device, the location of the transported items, the transport destination, and the preset transport site terrain.

[0026] The topography of the handling site refers to the topographical characteristics and layout information of the physical space environment in which the handling device performs its handling tasks. The topography of the handling site is pre-determined by those skilled in the art and is not described in detail here. The device handling route refers to the route traveled by the handling device during its handling tasks. The device handling route can be generated by inputting the current location of the device, the location of the items being handled, the destination of the handling, and the topography of the handling site into a pre-set path planning algorithm. In this embodiment, the path planning algorithm may be an A* algorithm. The A* algorithm is common knowledge in the art and is not described in detail here.

[0027] S4: Control the transport device to travel along the device transport route and collect image information in front of it.

[0028] The front image information refers to the image presented in front of the transport device during its travel. The front image information is captured by a camera on the transport device.

[0029] When controlling the transport device to move along the device transport route, it is necessary to simultaneously collect image information in front of it for subsequent steps.

[0030] S5: When and only when the image information in front contains preset obstacle features, update the device transport route using a preset route planning method, and control the transport device to travel along the updated device transport route.

[0031] Obstacle features refer to visual data features used to identify obstacles in the transport device's path. Obstacle features are pre-defined by those skilled in the art and are not described in detail here. The route planning method refers to a method for replanning the transport device's route when the image information contains obstacle features. The specific route planning method is described in detail in S50 to S5112 and S52 to S523, and is not described here in detail.

[0032] When and only when the image information in front contains obstacle features, it means that there is an obstacle blocking the transport device on the way along the device transport route. The device transport route needs to be updated using the route planning method, and the transport device needs to be controlled to travel along the updated device transport route.

[0033] Reference Figure 3 , the route planning method includes the following steps: S50: Obtain the obstacle type based on the front image information and obstacle features.

[0034] Obstacle type refers to the specific type of obstacle categorized by different characteristics and attributes. In this embodiment, obstacle types include central obstacles and mobile obstacles. Central obstacles are stationary obstacles located at the center of the device's transport path. Mobile obstacles are moving obstacles that fall within the image information in front of the device.

[0035] Because image information is continuously collected, it's possible to determine whether an obstacle is moving by observing the change in its position between two adjacent images. If the obstacle's position changes between two adjacent images, it indicates a moving obstacle. Determining whether an obstacle is moving based on consecutive images is common knowledge in the field and will not be elaborated on here.

[0036] By understanding the location of obstacle features in the image information in front of the vehicle and whether they are moving, the type of obstacle can be determined.

[0037] S51: When the obstacle type is a preset route center obstacle, the left side free space and the right side free space are collected.

[0038] The left free gap refers to the space in the area to the left of the center line of the route that is not occupied by the obstacle and can be used for the device to pass when the obstacle is located at the center of the currently planned route of the transport device.

[0039] The right free space refers to the space on the right side of the route centerline that is not occupied by obstacles and can be used for the device to pass when the obstacle is located at the center of the currently planned route of the transport device.

[0040] The left side free space and the right side free space are both obtained by scanning with an infrared scanner preset on the transport device. Infrared scanning technology is common knowledge in this field and will not be described in detail here.

[0041] If the obstacle type is a route center obstacle, it means that there is an obstacle blocking the center of the device transportation route. The left and right free spaces need to be collected for subsequent steps.

[0042] S510: When both the left and right free spaces exceed the preset required action space, the moving distances on both sides are obtained according to the front image information, the left and right free spaces.

[0043] The required action gap refers to the minimum width threshold requirement for the left and right passable spaces in order for the transport device to complete the detour operation safely and smoothly when encountering an obstacle in the center of the route. The required action gap is set in advance by those skilled in the art and will not be elaborated here. The moving distance on both sides refers to the moving distance of the transport device to the left free gap and the right free gap respectively when performing a detour operation. By understanding the image information in front of the face and the left free gap and the right free gap in the image, the distance between the transport device and the left free gap and the right free gap in the image in front can be known, and then by understanding the shooting ratio of the image information in front, the actual distance can be compared and the moving distance on both sides can be obtained. The shooting ratio of the image information in front is set in advance by those skilled in the art and will not be elaborated here.

[0044] When both the left and right free spaces exceed the required action space, it means that the transport device can choose to detour to the left or right. The moving distances on both sides must be obtained first for subsequent steps.

[0045] S5100: Obtain the rotation direction based on the moving distances on both sides.

[0046] The rotation direction refers to the direction in which the handling device needs to make a detour. This direction can be determined by understanding the relationship between the travel distances on both sides. When the left-hand clearance is smaller than the right-hand clearance, the left-hand clearance is used for detour, and the rotation direction is left. Otherwise, the rotation direction is right. When the travel distances on both sides are the same, the left side is preferred.

[0047] S5101: Update the device transport route based on the rotation direction, the moving distance on both sides, the location of the transported items, the transport destination, and the terrain of the transport site.

[0048] By inputting the rotation direction, the distance moved on both sides, the location of the transported items, the transport end point and the terrain of the transport site into the path planning algorithm, a new device transport route can be generated to update the device transport route.

[0049] The following steps are also included: S511: When the left side free space and the right side free space do not exceed the preset required action space, the obstacle height value is collected.

[0050] The obstacle height value refers to the height of the obstacle. The obstacle height value is obtained by scanning with the infrared scanner on the handling device.

[0051] When the left and right free spaces do not exceed the required action space, it means that the transport device cannot directly perform a detour on the left or right sides, and the obstacle height value must be collected first for subsequent steps.

[0052] When one of the left and right free gaps exceeds the required action gap, the direction exceeding the required action gap can be directly selected for detour.

[0053] S5110: When the obstacle height value is lower than the preset reference height value, determine whether the left side clearance and the right side clearance are greater than the preset device movement clearance.

[0054] The reference height value is the threshold used to detect whether an obstacle is too high. The device clearance refers to the clearance through which the transport device can pass. Both the reference height value and the device clearance are set by those skilled in the art and are not detailed here.

[0055] When the obstacle height value is lower than the reference height value, it means that the obstacle is not too high. You need to first determine whether the left and right clearances are greater than the device movement clearance.

[0056] S5111: When the left and right free gaps are both greater than the device movement gap, the moving distances on both sides are obtained based on the front image information, the left and right free gaps, and S5100 to S5101 are executed, while the items are transported using the preset lifting and transporting method.

[0057] The lifting and transporting method refers to a method of lifting the object to be transported and passing it through the gap for further transport. The specific lifting and transporting method is described in detail in the subsequent S51110 to S51113 and will not be described in detail here.

[0058] When both the left and right clearances are larger than the device movement clearance, it means that the clearances on both sides are large enough for the transport device to pass through. It is necessary to first obtain the moving distances on both sides and execute S5100 to S5101 to obtain a new device transport route, and then use the lifting transport method to transport items.

[0059] The method for obtaining the moving distances on both sides in this step is the same as that in the above S510 and will not be described in detail here.

[0060] S5112: When the left and right clearances are both not greater than the device's operating clearance, an obstacle anomaly prompt is reported.

[0061] When the left and right clearances are both less than the device's movement clearance, it means that the clearances on both sides are not large enough for the transport device to pass through, and an obstacle exception prompt must be reported.

[0062] When one of the left and right free gaps is larger than the device movement gap, the direction exceeding the device movement gap can be directly selected for detour.

[0063] The lifting and handling method includes the following steps: S51110: Collect object image information.

[0064] The object image information refers to an image of the object being transported by the transport device, and is captured by a camera on the transport device.

[0065] S51111: Scan and identify preset object features from the object image information to obtain the object size.

[0066] Item features refer to the physical characteristics of the item being transported. Item features are pre-determined by those skilled in the art and are not described in detail here. Item size refers to the dimensions of the item being transported. Item size can be determined by scanning and identifying item features from item image information. Image recognition technology is common knowledge in the art and is not described in detail here.

[0067] S51112: Generate a lifting height value based on the object size and the obstacle height value.

[0068] The lift height value is the height the handling device needs to lift the item. By knowing the item's dimensions, you can determine its vertical dimension. Adding the vertical dimension to the obstacle height gives you the lift height value.

[0069] S51113: Control the transport device to carry out lifting according to the lifting height value, and after lifting, travel along the updated device transport route.

[0070] The transport device is controlled to carry out lifting at the lifting height value, and after lifting, the transport device is controlled to move along the updated device transport route.

[0071] The following steps are also included: S52: When the obstacle type is a preset moving obstacle, the obstacle moving speed and obstacle size are collected.

[0072] Obstacle speed refers to the speed at which a moving obstacle moves. Obstacle size refers to the size of the moving obstacle. Obstacle speed is measured using the laser radar on the transport device. Obstacle size is measured using the infrared scanner on the transport device.

[0073] When the obstacle type is moving, it means that the obstacle is moving, and the obstacle moving speed and obstacle size need to be collected for subsequent steps.

[0074] S520: Obtaining the obstacle disappearance time based on the obstacle movement speed and obstacle size.

[0075] The obstacle disappearance duration is the time it takes for a moving obstacle to completely exit the transport device's preset detection range from the current moment. The detection range is the spatial range within which the transport device can sense and respond to obstacles in real time using the LiDAR. The detection range is pre-determined by those skilled in the art and will not be detailed here.

[0076] By understanding the obstacle size and detection range, we can determine the distance between the obstacle and the edge of the detection range. Dividing this distance by the obstacle's movement speed gives us the time it takes for the obstacle to disappear.

[0077] S521: Collect obstacle distance.

[0078] Obstacle distance refers to the distance between the handling device and the moving obstacle. Obstacle distance can be measured by LiDAR.

[0079] S522: Generate an avoidance speed for the device based on the obstacle distance, the obstacle disappearance time, and the preset safety distance.

[0080] The safety distance is the required distance between the transport device and a moving obstacle. This safety distance is set by those skilled in the art and will not be detailed here. The device avoidance speed is the reasonable speed calculated when the transport device detects a moving obstacle, ensuring it maintains a safe distance from the obstacle and passes safely within the obstacle's disappearance time. The device avoidance speed is calculated by calculating the difference between the obstacle distance and the safety distance and dividing this difference by the obstacle disappearance time.

[0081] S523: Control the transport device to travel along the device transport route at a device avoidance travel speed.

[0082] The transport device is controlled to travel along the transport route at a device avoidance speed to avoid moving obstacles. After the obstacle avoidance is completed, the transport device is controlled to return to a preset normal travel speed. The normal travel speed refers to the speed at which the transport device travels without any adjustments. The normal travel speed is pre-set by those skilled in the art and will not be described in detail here.

[0083] The following steps are also included: S60: Collecting driving image information.

[0084] The driving image information refers to the image of the transport device during driving, and is captured by a camera.

[0085] S61: If and only if the driving image information contains a preset steep feature, report a road steepness prompt, and scan and identify the steep feature from the driving image information to obtain a road steepness value.

[0086] The steepness feature refers to the visual characteristics of a steep road surface. The steepness feature is pre-set by those skilled in the art and will not be described in detail here. The road surface steepness value refers to a quantitative value used to represent the steepness of the road surface. The road surface steepness value can be obtained by geometrically measuring the steepness feature from the driving image information and converting it into an actual physical value in combination with the calibration parameters preset by the camera. The calibration parameters preset by the camera are pre-set by those skilled in the art and will not be described in detail here. Image-based geometric measurement is common knowledge in the art and will not be described in detail here.

[0087] If and only if the driving image information contains steep features, it indicates that the road surface is steep, and a road surface steepness prompt needs to be reported. The steep features are scanned and identified from the driving image information to obtain the road surface steepness value.

[0088] S610: When the road surface steepness value does not exceed the preset reference steepness value, controlling the transport device to continue traveling along the device transport route.

[0089] The reference steepness value is a reference threshold used to determine whether a road surface is too steep. The reference steepness value is set in advance by those skilled in the art and will not be described in detail here.

[0090] When the road surface steepness value does not exceed the reference steepness value, it indicates that the road surface is not too steep, and the transport device can be controlled to continue traveling along the device transport route.

[0091] S611: When the road surface steepness value exceeds a preset reference steepness value, route planning is performed using a preset steep route planning method to obtain a steep driving route.

[0092] A steep route is a route that is re-planned when the road surface becomes too steep while the transport device is traveling along the transport route. A steep route planning method can be used to obtain a steep route. The specific steep route planning method is described in detail in subsequent steps S6110 to S611711 and is not further described here.

[0093] When the road surface steepness value exceeds the benchmark steepness value, it means that the road surface is too steep and the route planning method needs to be used to obtain a steep driving route.

[0094] S612: Control the transport device to travel along the steep route.

[0095] The transport device is controlled to travel along a steep route to complete the object transport.

[0096] The steep route planning method includes the following steps: S6110: Collect X-ray image information of objects.

[0097] The X-ray image information of the object refers to an X-ray image obtained by scanning the object carried by the transport device using an X-ray imaging device (in this embodiment, the X-ray imaging device is a security inspection machine). The X-ray image information of the object can be obtained by scanning the object using the X-ray imaging device.

[0098] S6111: Obtain the status of items in the box based on the X-ray image information of the items.

[0099] The contents of a box refer to the physical properties and spatial distribution of items within a packing or luggage box. This information can be obtained by extracting and analyzing features from X-ray images of the items. Feature extraction and analysis of X-ray images is common knowledge in the art and will not be detailed here.

[0100] S6112: When the items in the box include preset fragile items, the filler color, filler outline, and filler width are determined based on the X-ray image information of the items and the items in the box.

[0101] Fragile items refer to items within a packing box or luggage that are physically fragile. Filler color refers to the color of the material used to cushion fragile items within the packing box or luggage (clothing within the box can be considered filler). Multispectral X-ray imaging technology can be used to identify the color of each item in the item's X-ray image information, thereby determining the filler color. Filler outline refers to the outline of the material used to cushion fragile items within the packing box or luggage. Image segmentation technology can be used to segment the fragile items and determine the filler outline. Filler width refers to the width of the material used to cushion fragile items within the packing box or luggage. This width can be determined by measuring the filler width from the item's X-ray image information.

[0102] Multispectral X-ray imaging technology, image segmentation technology, and width measurement from images are common knowledge in the art and will not be described in detail here.

[0103] When the contents of the box include fragile items, it means that there are fragile items in the transported packaging box and luggage. The filler color, filler outline and filler width must be obtained first for subsequent steps.

[0104] S6113: Get fill density based on fill color.

[0105] Filler density refers to the mass of filler material per unit volume. The filler density corresponding to each filler color can be found using a pre-set density comparison table. This table records the different filler densities corresponding to different filler colors. This table is compiled by those skilled in the art by sequentially recording the different filler densities corresponding to different filler colors, and is not detailed here.

[0106] S6114: Generate seismic parameters based on the contents of the box, filler profile, filler density, and filler width.

[0107] Seismic parameters are quantified values ​​for the ability of the filling material within a packaging box or luggage case to protect fragile items from earthquakes. By understanding the contents and the outline of the filling material, the volume of the fragile items and the filling wheel can be determined. A pre-set seismic comparison table can then be used to determine the seismic parameters corresponding to the volume of the fragile items, the volume of the filling wheel, the filling density, and the filling width. This table records the seismic parameters corresponding to different volumes of fragile items, volumes of filling wheels, filling densities, and filling widths. This table was created by technicians in this field through sequential testing of seismic parameters corresponding to different volumes of fragile items, volumes of filling wheels, filling densities, and filling widths, and is not detailed here.

[0108] S6115: Obtaining a driving steepness value based on the anti-seismic parameters and the preset device driving speed.

[0109] The device's travel speed refers to the speed at which the transport device travels. The device's travel speed is pre-determined by those skilled in the art and is not detailed here. The travel steepness value is a quantitative measure of the maximum road surface steepness permitted for the transport device when carrying fragile items, taking into account the seismic resistance parameters of the packaging and luggage box and the device's travel speed.

[0110] The driving steepness values ​​corresponding to the seismic resistance parameters and the device's driving speed can be found using a preset driving steepness comparison table. This comparison table records the different driving steepness values ​​corresponding to different seismic resistance parameters and device driving speeds. This driving steepness comparison table was created by those skilled in the art by sequentially testing and recording different driving steepness values ​​corresponding to different seismic resistance parameters and device driving speeds, and is not detailed here.

[0111] S6116: When the road surface steepness value does not exceed the driving steepness value, the device transport route is defined as a steep driving route.

[0112] When the road steepness value does not exceed the driving steepness value, it means that the transport device continues to travel along the device transport route at the device driving speed and will not affect the fragile items in the box. The device transport route can be directly defined as a steep driving route.

[0113] The steep route planning method further includes the following steps: S6117: When the road surface steepness value exceeds the driving steepness value, the driving steepness value is updated based on the anti-seismic parameters and the preset minimum driving speed of the device.

[0114] The minimum travel speed of the device refers to the minimum required speed of the transport device. This minimum travel speed is pre-determined by those skilled in the art and will not be detailed here. Similar to S6115 above, the travel steepness value corresponding to the seismic resistance parameters and the device's minimum travel speed can be inferred from the travel steepness comparison table, which can then be used to update the travel steepness value.

[0115] When the road steepness value exceeds the driving steepness value, it means that if the transport device continues to travel along the device transport route at the device driving speed, it will affect the fragile items in the box. The driving steepness value needs to be updated for subsequent steps.

[0116] S61170: When the road surface steepness value does not exceed the updated driving steepness value, the device transport route is defined as a steep driving route.

[0117] When the road steepness value does not exceed the updated driving steepness value, it means that the transport device continues to travel along the device transport route at the device's minimum driving speed, which will not affect the fragile items in the box. The device transport route can be directly defined as a steep driving route.

[0118] S611700: Control the transport device to travel along the steep driving route at the device's minimum driving speed and collect steep driving image information.

[0119] The steep driving image information refers to an image of the transport device driving on a steep road surface. The steep driving image information is captured by a camera.

[0120] The transport device is controlled to travel along the steep driving route at the device's lowest driving speed, while continuously collecting steep driving image information for subsequent steps.

[0121] S611701: When and only when the steep driving image information does not contain a steep feature, control the transport device to travel along the steep driving route at the device driving speed.

[0122] If and only if the steep driving image information does not contain steep features, it indicates that the transport device has left the steep section, and the transport device can be controlled to travel along the steep driving route at the device driving speed.

[0123] S61171: When the road surface steepness value exceeds the updated driving steepness value, a steepness abnormality prompt is reported.

[0124] When the road steepness value exceeds the updated driving steepness value, it means that even if the transport device continues to travel along the device transport route at the device's minimum driving speed, it will still affect the fragile items in the box, and a steepness abnormality prompt must be reported.

[0125] The following steps are also included: S70: Collect the device travel position of the transport device.

[0126] The device's travel position refers to the current location of the transport device. The device's travel position is collected by the GPS positioning chip in the transport device.

[0127] S71: When the driving position of the device is consistent with the location where the items are to be transported, regional image information of a preset luggage placement area on a preset centralized transport vehicle is collected.

[0128] A centralized transport vehicle is a vehicle used to centrally store luggage requiring transport. A luggage storage area refers to the area on a centralized transport vehicle used to store luggage. The luggage storage area is pre-defined by those skilled in the art and will not be described in detail here. Area image information refers to an image of the luggage storage area. The area image information is captured by a camera.

[0129] When the device's driving position is consistent with the location where the transported items are placed, it means that the transport device has arrived at the location where the items are placed. It is necessary to first collect regional image information for subsequent steps.

[0130] S72: Collecting transport image information if and only if the regional image information does not include the preset transport device feature.

[0131] The handling device features refer to the external contour features of the handling device. These features are pre-determined by those skilled in the art and are not described in detail here. The handling image information refers to an image that includes the characteristics of the centralized handling vehicle and the slope used to allow the handling device to enter the centralized handling vehicle. The handling image information is captured by a camera. The slope features refer to the external contour features of the slope used to allow the handling device to enter the centralized handling vehicle. The slope features are pre-determined by those skilled in the art and are not described in detail here.

[0132] If and only if the regional image information does not contain the feature of the handling device, it means that no handling device is handling the luggage on the centralized handling vehicle, and the handling image information needs to be collected first for subsequent steps.

[0133] S73: Scan and identify the preset slope features from the transport image information to obtain the slope position.

[0134] The slope position refers to the position of the slope. The slope position can be obtained by scanning and identifying the slope features in the transport image information and marking them. Image recognition technology is common knowledge in the field and will not be described in detail here.

[0135] S74: Generate a climbing route based on the device's driving position, the slope position, and the luggage storage area, and control the transport device to move toward the luggage storage area along the climbing route.

[0136] A climbing route is the path a transport unit takes as it climbs a slope onto a centralized transport vehicle. This route is generated by inputting the unit's location, the slope, and the luggage storage area into a path planning algorithm.

[0137] After the climbing route is generated, the transport device is controlled to move along the climbing route to the luggage placement area for subsequent steps.

[0138] S75: When the transport device reaches the luggage placement area, the transport device is controlled to clamp the items in the luggage placement area and place them on a preset transport device.

[0139] The transport device is a device used to transport luggage from the vehicle to the underside of the vehicle. When the transport device reaches the luggage storage area, it is controlled to clamp the items in the luggage storage area onto the transport device, thereby transporting the luggage in the luggage storage area to the underside of the vehicle.

[0140] Based on the same inventive concept, an embodiment of the present invention provides a complex scene space construction system based on multi-sensor data fusion, including: The acquisition module is used to collect handling information, the current position of the device, front image information, left clearance, right clearance, obstacle height, object image information, obstacle movement speed, obstacle size, obstacle distance, driving image information, object X-ray image information, steep driving image information, device driving position, area image information, and handling image information; A memory for storing a program for implementing a complex scene space construction method based on multi-sensor data fusion; The processor is configured to load and execute the program stored in the memory.

[0141] Those skilled in the art will clearly understand that for the sake of convenience and brevity, the division of the above-mentioned functional modules is only used as an example for illustration. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working processes of the above-mentioned systems, devices, and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0142] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiment. All technical solutions based on the concept of the present invention are within the scope of protection of the present invention. It should be noted that for those skilled in the art, various improvements and modifications that do not depart from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A complex scene space construction method based on multi-sensor data fusion, characterized in that: include: S1: Collecting transport information and the current position of the preset transport device; S2: Retrieving the transport location and transport destination of the items based on the transport information; S3: generating a device transport route based on the current position of the device, the location of the transported item, the transport destination, and a preset transport site topography; S4: controlling the transport device to travel along the transport route of the device and collecting image information in front of the device; S5: When and only when the front image information contains preset obstacle features, update the device transport route using a preset route planning method, and control the transport device to travel along the updated device transport route.

2. The method for constructing complex scene space based on multi-sensor data fusion according to claim 1, characterized in that: The route planning method comprises: S50: Obtaining the obstacle type based on the front image information and the obstacle features; S51: When the obstacle type is a preset route center obstacle, collecting the left side free space and the right side free space; S510: When both the left and right free spaces exceed the preset required action space, obtaining the moving distances on both sides according to the front image information, the left and right free spaces; S5100: Obtaining a rotation direction based on the moving distances on both sides; S5101: Update the device transport route based on the rotation direction, the moving distance on both sides, the location of the transported item, the transport destination, and the topography of the transport site.

3. The method for constructing complex scene space based on multi-sensor data fusion according to claim 2, characterized in that: Also includes: S511: When the left side free space and the right side free space do not exceed the preset required action space, collecting the obstacle height value; S5110: When the obstacle height is lower than a preset reference height, determining whether the left clearance and the right clearance are greater than a preset device movement clearance; S5111: When the left and right clearances are greater than the device movement clearance, the moving distances on both sides are obtained based on the front image information, the left and right clearances, and S5100 to S5101 are executed, while the object is transported using a preset lifting and transporting method. S5112: When the left side free space and the right side free space are not larger than the device movement clearance, an obstacle abnormality prompt is reported.

4. The method for constructing complex scene space based on multi-sensor data fusion according to claim 3, characterized in that: The lifting and transporting method comprises: S51110: Collecting item image information; S51111: Scan and identify preset object features from the object image information to obtain object dimensions; S51112: Generate a lifting height value based on the object size and the obstacle height value; S51113: Control the transport device to carry out transport and lifting at the lifting height value, and after lifting, travel along the updated transport route of the device.

5. The method for constructing complex scene space based on multi-sensor data fusion according to claim 2, characterized in that: Also includes: S52: When the obstacle type is a preset moving obstacle, collecting the obstacle moving speed and obstacle size; S520: Obtaining the obstacle disappearance time based on the obstacle movement speed and the obstacle size; S521: Collect obstacle distance; S522: Generating a device avoidance speed based on the obstacle distance, the obstacle disappearance time, and a preset safety distance; S523: Control the transport device to travel along the device transport route at the device avoidance travel speed.

6. The method for constructing complex scene space based on multi-sensor data fusion according to claim 1, characterized in that: Also includes: S60: collecting driving image information; S61: If and only if the driving image information contains a preset steep feature, report a road steepness prompt, and scan and identify the steep feature from the driving image information to obtain a road steepness value; S610: When the road surface steepness value does not exceed a preset reference steepness value, controlling the transport device to continue traveling along the device transport route; S611: When the road surface steepness value exceeds a preset reference steepness value, performing route planning using a preset steep route planning method to obtain a steep driving route; S612: Control the transport device to travel along the steep travel route.

7. The method for constructing complex scene space based on multi-sensor data fusion according to claim 6, characterized in that: The steep route planning method comprises: S6110: Collect X-ray image information of items; S6111: Obtaining information about items in the box based on the X-ray image information of the items; S6112: When the items in the box include a preset fragile item, determine the filler color, filler outline, and filler width based on the X-ray image information of the item and the items in the box; S6113: Obtaining a filling density based on the filling color; S6114: Generate earthquake resistance parameters based on the contents of the box, the filler profile, the filler density, and the filler width; S6115: Obtaining a driving steepness value based on the anti-seismic parameter and a preset driving speed of the device; S6116: When the road surface steepness value does not exceed the driving steepness value, define the device transport route as a steep driving route.

8. The method for constructing complex scene space based on multi-sensor data fusion according to claim 7, characterized in that: The steep route planning method further includes: S6117: When the road surface steepness value exceeds the driving steepness value, updating the driving steepness value based on the anti-seismic parameter and a preset minimum driving speed of the device; S61170: When the road surface steepness value does not exceed the updated driving steepness value, defining the device transport route as a steep driving route; S611700: Controlling the transport device to travel along the steep driving route at the device's lowest driving speed, and collecting steep driving image information; S611701: If and only if the steep driving image information does not contain the steep feature, control the transport device to travel along the steep driving route at the device driving speed; S61171: When the road surface steepness value exceeds the updated driving steepness value, a steepness abnormality prompt is reported.

9. The method for constructing complex scene space based on multi-sensor data fusion according to claim 1, characterized in that: Also includes: S70: collecting the device travel position of the transport device; S71: When the driving position of the device is consistent with the location of the transported items, collecting regional image information of a preset luggage placement area on a preset centralized transport vehicle; S72: If and only if the regional image information does not contain a preset transport device feature, collect transport image information; S73: Scan and identify preset slope features from the transport image information to obtain a slope position; S74: generating a climbing route based on the device's driving position, the slope position, and the luggage storage area, and controlling the transport device to move toward the luggage storage area along the climbing route; S75: When the transport device reaches the luggage placement area, the transport device is controlled to clamp the items in the luggage placement area and place them on a preset transport device.

10. A complex scene space construction system based on multi-sensor data fusion, characterized in that: include: The acquisition module is used to collect the transport information, the current position of the device and the image information in front of it; A memory for storing a program for implementing the method for constructing a complex scene space based on multi-sensor data fusion according to any one of claims 1 to 9; The processor is configured to load and execute the program stored in the memory.

Images