Robot and carriage-in-position positioning and identification method, positioning method and identification system

Abstract

The application provides a robot in-car position positioning and identification method, comprising: obtaining camera environment information and a robot position of a camera part in the environment based on a depth camera; obtaining radar environment information and a robot position of a radar part in the environment based on a laser radar; fusing the camera position and the radar position to construct an overall position of the robot in the environment information. Meanwhile, a robot in-car position positioning method based on the position positioning and identification method, a corresponding robot in-car position positioning and identification system, and a robot adopting the position positioning and identification system are provided. The application adopts a multi-sensor fusion mode, is more stable and accurate, adopts a positioning mode selected by a dynamic coordinate system, is more suitable for positioning of a loading and unloading robot in an unstructured weak light environment, and ensures consistency of robot positioning in a multi-layer goods stacking process.

Description

Technical Field

[0001] This invention relates to the field of robotics, specifically to a loading and unloading robot for use in an enclosed vehicle under unstructured low-light conditions, and a method, system, and positioning recognition system for the robot within the vehicle. Background Technology

[0002] The logistics and manufacturing industries play a crucial role in the digital transformation of industries. The demands for cargo handling, material transfer, and intelligent operation have accelerated the development of mobile robots. Position and posture localization are key challenges for mobile robots. Currently, the main localization technologies for loading and unloading robots are visual localization and SLAM (Simultaneous Localization and Mapping) mapping.

[0003] Publication No. CN116299174A proposes a global localization method for unloading robots inside vans. Based on the Malusian signal modulation criterion, it modulates a fixed light source and projects it onto the area to be localized, thereby forming a light positioning plane to obtain the real-time position information of the current point. Publication No. CN116124141A proposes a multi-feature fusion localization method for handling robots. It utilizes multi-kernel learning to perform weighted learning of environmental color features, foreground local features, and background local features, and establishes a database. It then uses a binocular camera to collect environmental data in real time and matches it with features in the feature database, achieving self-localization of the handling robot through trilateration. However, the above methods require pre-setting of the light source and the data collection dataset, making them unsuitable as general methods to cope with the dynamically changing space inside the van.

[0004] SLAM is an algorithm and technique used to enable robots to estimate their position and build environmental maps in unknown environments. Publication CN115981314A proposes an automatic obstacle avoidance method for robot navigation based on 2D LiDAR. Map construction is achieved through probabilistic filtering using visual or LiDAR ranging sensors, while Monte Carlo particle filtering is used for robot localization correction, constructing a 2D scene map to complete robot navigation and localization. Publication CN115709470A proposes a method based on laser point cloud surface fitting and robot indoor localization. This method perceives the indoor environment through laser point clouds, extracts the ground point cloud, and determines the robot's tilt angle based on the ground point cloud. Then, the principal plane of the indoor laser point cloud is extracted and matched with the indoor Building Information Model (BIM) data to determine the robot's absolute indoor location information. This method relies on prior building indoor BIM information and is suitable for localization in known environments. Publication CN109798896A proposes a mapping and localization method that improves the particle scarcity problem through resampling using a weight balancing strategy. By leveraging the concept of population evolution, a process of updating a representative particle is replicated, significantly improving the real-time performance of mapping while maintaining map accuracy. This overcomes the problems of excessive memory consumption and latency associated with traditional mapping algorithms in complex environments. Although SLAM positioning offers high accuracy and scalability, it requires pre-built maps. However, the environments within different train carriages vary, and even within the same carriage, the environment changes in real-time depending on the cargo stacking. Therefore, SLAM is not suitable for positioning requirements in such environments. Summary of the Invention

[0005] To address the aforementioned shortcomings in the prior art, this invention provides a loading and unloading robot and a method, method, and system for position and orientation recognition within the vehicle compartment.

[0006] This invention is achieved through the following technical solutions.

[0007] According to one aspect of the present invention, a method for robot pose localization and recognition inside a vehicle is provided, comprising:

[0008] The robot pose is obtained by acquiring camera environment information and camera position in the environment based on a depth camera.

[0009] The robot pose is obtained based on the radar environment information and the radar part of the environment using lidar.

[0010] By fusing the robot pose from the camera section with the robot pose from the radar section, the overall pose of the robot in the environmental information is constructed.

[0011] Preferably, the acquisition of camera environment information based on the depth camera includes: acquiring information about the uneven corrugated wall surface using the depth camera.

[0012] Preferably, the step of acquiring the robot pose of the camera portion in the environment based on the depth camera includes:

[0013] The depth information acquired by the depth camera is converted into point cloud information;

[0014] The point cloud information is then filtered.

[0015] The filtered point cloud information is segmented using normals to remove non-wall objects.

[0016] The pointed bounding box (OBB) for obtaining the camera environment information is obtained using the PCA dimensionality reduction algorithm.

[0017] The robot pose of the camera section is obtained by calculating the angle rs_theta between the robot and the wall and the vertical distances rs_l_dis and rs_r_dis from the robot to the two side walls based on the bounding box OBB.

[0018] Preferably, the step of acquiring the robot pose of the radar portion in the environment based on lidar includes:

[0019] The two-dimensional point cloud information acquired by the lidar is transformed into image information through coordinate transformation;

[0020] Divide the robot's two side regions based on the radar origin and extract information from the two side walls;

[0021] The distances lidar_l_dis and lidar_r_dis from the robot and the robot's deflection angle lidar_theta are calculated based on the geometric information relationship.

[0022] The distance obs_dis between the robot and the obstacle in front is calculated based on region filtering and image distance;

[0023] This yields the robot's pose from the radar section.

[0024] Preferably, the step of fusing the robot pose from the camera portion and the robot pose from the radar portion to construct the robot's overall pose in the environmental information includes:

[0025] Define the robot's overall pose expression as (L_dis, r_dis, obs_dis, theta), where:

[0026] l_dis is the result of dynamic weighting of the vertical distance rs_l_dis from the robot in the robot pose of the camera part and the distance lidar_l_dis from the robot in the robot pose of the radar part.

[0027] r_dis is the result of dynamic weighting of the vertical distance rs_r_dis from the robot in the robot pose of the camera section to its second side wall and the distance lidar_r_dis from the robot pose of the radar section to its second side wall.

[0028] theta is the result of dynamic weight allocation of the angle between the robot and the wall in the robot pose obtained from the camera part (rs_theta) and the robot deflection angle litar_theta in the robot pose obtained from the radar part.

[0029] According to another aspect of the present invention, a robot pose localization method in a vehicle compartment is provided, comprising: in an unstructured enclosed vehicle compartment environment, employing the robot pose localization and recognition method in a vehicle compartment as described above, dynamically selecting the origin of a reference frame based on changes in acquired environmental information, establishing the robot's position in the reference frame, and performing localization and recognition based on coordinate position information.

[0030] Preferably, the unstructured enclosed vehicle environment includes: sequentially placing and stacking goods of different sizes and quantities to be transported in enclosed spaces of different sizes of cargo boxes, wherein the number of stacking layers is not less than one layer.

[0031] Preferably, the step of dynamically selecting the origin of the reference frame based on changes in the acquired environmental information includes:

[0032] When the LiDAR scans and finds that the row in front is empty and has not been stacked with goods or that the row has been fully stacked with goods, a coordinate system is established with one side wall as the y-axis and the bottom wall or the side of the row of goods that has been fully stacked in front of the robot as the x-axis. The intersection of the y-axis, x-axis and the robot is selected as the origin of the reference system.

[0033] When a new row of goods begins to be placed, the origin of the coordinate system changes. At this time, a coordinate system is established with the wall on which the goods are placed as the y-axis and the side of the new goods that is closer to the robot as the x-axis. The intersection of the y-axis, x-axis and the robot is selected as the origin of the reference system. The origin of the coordinate system does not change until the row of goods is placed.

[0034] By analogy, the origin of the reference frame can be dynamically selected based on changes in real-time environmental information obtained by the lidar.

[0035] According to a third aspect of the present invention, a robot carriage pose localization and recognition system is provided, comprising:

[0036] The depth camera module is used to acquire information about the camera environment inside the carriage and the robot's pose within the camera area of ​​the environment.

[0037] The lidar module is used to acquire radar environment information inside the vehicle and the robot pose of the radar portion within the environment.

[0038] The industrial control computer module is used to fuse the robot pose from the camera section and the robot pose from the radar section to construct the overall pose of the robot in the environmental information.

[0039] Preferably, the depth camera module includes two depth cameras, which are respectively installed on the left and right sides of the robot.

[0040] Preferably, the lidar module is mounted in front of the robot.

[0041] Preferably, the industrial control computer module is also used to communicate with the host computer and the slave computer.

[0042] According to a fourth aspect of the present invention, a robot is provided, comprising: a robot body, and a robot carriage pose positioning and recognition system as described above, wherein the robot body moves within the carriage based on the carriage environment information and robot pose information obtained by the robot carriage pose positioning and recognition system.

[0043] By adopting the above technical solution, the present invention has at least one of the following beneficial effects compared with the prior art:

[0044] The robot and vehicle interior pose localization and recognition method, localization method, and recognition system provided by this invention employ a multi-sensor fusion approach in acquiring robot pose, which offers greater stability and accuracy compared to single-sensor pose acquisition methods. The multi-sensor data fusion method reduces the impact of single-sensor errors on measurement results; the redundancy of multiple sensors ensures that the system can continue to operate normally without being affected even if some sensors fail.

[0045] The robot and in-vehicle pose localization and recognition method, localization method, and recognition system provided by this invention employ a dynamic coordinate system selection positioning method, which is more suitable for the localization of loading and unloading robots in unstructured, low-light environments, ensuring the consistency of robot localization during multi-layer cargo stacking. Compared to localization methods based on Wi-Fi, Bluetooth, UWB, etc., this invention does not require the pre-installation of any markers and can complete the handling and unloading of goods in containers of different sizes. Compared to methods such as SLAM mapping, it does not require pre-map creation and completes localization in real time based on the current environment. Attached Figure Description

[0046] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0047] Figure 1 This is a flowchart illustrating the robot's pose localization and recognition method within a vehicle according to one embodiment of the present invention.

[0048] Figure 2 This is a schematic diagram of the components of a robot carriage pose positioning and recognition system in one embodiment of the present invention.

[0049] Figure 3 This is a schematic diagram of the hardware structure of the robot positioning and recognition system in a preferred embodiment of the present invention.

[0050] Figure 4 This is a flowchart illustrating the process of acquiring robot pose using a depth camera in a preferred embodiment of the present invention.

[0051] Figure 5 This is a flowchart illustrating the process of extracting the plane containing the wall based on a directional bounding box (OBB) in a preferred embodiment of the present invention.

[0052] Figure 6 This is a flowchart illustrating the process of radar acquiring robot pose in a preferred embodiment of the present invention.

[0053] Figure 7 This is a flowchart illustrating the sensor bias process in the data fusion section of a preferred embodiment of the present invention.

[0054] Figure 8 This is a flowchart illustrating the multi-sensor data fusion process in a preferred embodiment of the present invention.

[0055] Figure 9 This is a schematic diagram illustrating the robot pose definition in a preferred embodiment of the present invention.

[0056] Figure 10 This is a schematic diagram illustrating the selection of the origin of the robot's reference coordinate system in a preferred embodiment of the present invention.

[0057] Figure 11 This is a schematic diagram of the hardware structure of a robot that includes a positioning and recognition system, which is a specific application example of the present invention.

[0058] Figure 12 This is a flowchart illustrating the overall workflow of a robot positioning and recognition system in a specific application example of the present invention. Detailed Implementation

[0059] The embodiments of the present invention are described in detail below: These embodiments are implemented based on the technical solution of the present invention, and provide detailed implementation methods and specific operation processes. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention.

[0060] One embodiment of the present invention provides a robot pose localization and recognition method in a closed vehicle compartment. This method is a technology for robot pose localization and recognition in a closed vehicle compartment during loading and unloading, so as to determine the position and posture (pose) of the robot in the compartment in real time, thereby supporting the motion planning and control of the loading and unloading robot in the compartment.

[0061] like Figure 1 As shown, the robot carriage pose localization and recognition method provided in this embodiment may include:

[0062] S1, acquire camera environment information and robot pose of the camera portion in the environment based on the depth camera;

[0063] S2, based on lidar, acquire radar environment information and robot pose of the radar part in the environment;

[0064] S3 integrates the robot pose from the camera and the robot pose from the radar to construct the robot's overall pose in the environmental information.

[0065] In a preferred embodiment of S1, obtaining camera environment information based on a depth camera may include: using a depth camera to obtain information about the uneven corrugated wall surface.

[0066] In a preferred embodiment of S1, acquiring the robot pose of the camera portion in the environment based on the depth camera may include:

[0067] S11 converts the depth information acquired by the depth camera into point cloud information;

[0068] S12, performs filtering on the point cloud information;

[0069] S13, perform normal segmentation on the filtered point cloud information to remove non-wall objects;

[0070] S14, using the PCA dimensionality reduction algorithm to obtain the pointed bounding box (OBB) of camera environment information;

[0071] S15, calculate the angle re_theta between the robot and the wall and the vertical distance re_l_dis and rs_r_dis from the robot to its two side walls (left and right side walls) based on the bounding box OBB, thus obtaining the robot pose of the camera part.

[0072] In a preferred embodiment of S2, obtaining the robot pose of the radar portion in the environment based on the lidar may include:

[0073] S21 converts the two-dimensional point cloud information acquired by the lidar into image information through coordinate transformation.

[0074] S22, Divide the robot's two side regions according to the radar origin and extract information from the two side walls;

[0075] S23, calculate the distances lidar_l_dis and lidar_r_dis from the robot to its two side walls (left and right side walls) and the robot's deflection angle lidar_theta based on the geometric information relationship;

[0076] S24, calculate the distance obs_dis between the robot and the obstacle in front based on the region filtering and image distance;

[0077] By following the above steps, the robot pose of the radar section can be obtained.

[0078] In a preferred embodiment of S3, fusing the robot pose from the camera section and the robot pose from the radar section to construct the robot's overall pose in the environmental information may include:

[0079] S31, define the robot's overall pose expression as (l_dis, r_dis, obs_dis, theta), where:

[0080] l_dis is the result of dynamic weighting of the vertical distance rs_l_dis from the robot in the robot pose of the camera part and the distance lidar_l_dis from the robot in the robot pose of the radar part.

[0081] r_dis is the result of dynamic weight allocation of the vertical distance rs_r_dis from the robot in the robot pose of the camera part and the distance lidar_r_dis from the robot in the robot pose of the radar part.

[0082] theta is the result of dynamic weight allocation of the angle between the robot and the wall in the robot pose obtained from the camera part (rs_theta) and the robot deflection angle litar_theta in the robot pose obtained from the radar part.

[0083] The robot pose localization and recognition method provided in the above embodiments of the present invention employs a multi-sensor (depth camera and LiDAR) fusion approach in acquiring the robot's pose, which is more stable and accurate than a single-sensor pose acquisition method. The multi-sensor data fusion method reduces the impact of single-sensor errors on the measurement results; the redundancy of the multiple sensors ensures that the system can still operate normally without being affected even if some sensors fail.

[0084] An embodiment of the present invention provides a robot pose localization method inside a vehicle. This method, based on the robot pose localization and recognition method provided in the above embodiment of the present invention, may include:

[0085] In an unstructured, enclosed vehicle environment, the robot's pose localization and recognition method in the vehicle according to any one of the above-mentioned methods of the present invention is used to dynamically select the origin of the reference frame based on the changes in the acquired environmental information, establish the position of the robot in the reference frame, and perform localization and recognition based on the coordinate position information.

[0086] In a preferred embodiment of this example, the unstructured enclosed vehicle environment includes: sequentially placing and stacking goods of different sizes and quantities to be transported in enclosed spaces of different sizes of cargo boxes, wherein the number of stacking layers is not less than one layer.

[0087] In a preferred embodiment of this example, dynamically selecting the origin of the reference frame based on changes in the acquired environmental information may include:

[0088] When the LiDAR scans and finds that the row in front is empty and has not been stacked with goods or that the row has been fully stacked with goods, a coordinate system is established with one side wall as the y-axis and the bottom wall or the side of the row of goods that has been fully stacked in front of the robot as the x-axis. The intersection of the y-axis, x-axis and the robot is selected as the origin of the reference system.

[0089] When a new row of goods begins to be placed, the origin of the coordinate system changes. At this time, a coordinate system is established with the wall on which the goods are placed as the y-axis and the side of the new goods that is closer to the robot as the x-axis. The intersection of the y-axis, x-axis and the robot is selected as the origin of the reference system. The origin of the coordinate system does not change until the row of goods is placed.

[0090] By analogy, the origin of the reference frame can be dynamically selected based on changes in real-time environmental information obtained by the lidar.

[0091] The origin of the reference system is dynamically selected based on changes in the environment to establish the robot's position in the reference system; the aforementioned dynamic selection of the origin of the reference system is characterized in that the origin of the robot's reference coordinate system changes dynamically with the loading and unloading of goods in the environment.

[0092] The robot pose localization method within a cargo compartment provided in the above embodiments of the present invention employs a dynamic coordinate system selection approach, which is more suitable for the localization of loading and unloading robots in unstructured, low-light environments, ensuring consistent robot localization during multi-layer cargo stacking. Compared to localization methods based on Wi-Fi, Bluetooth, UWB, etc., this invention does not require the pre-installation of any markers and can complete cargo handling and unloading within containers of different sizes. Compared to methods such as SLAM mapping, it does not require pre-map creation and completes localization in real-time based on the current environment.

[0093] One embodiment of the present invention provides a robot pose positioning and recognition system inside a vehicle compartment.

[0094] like Figure 2 As shown, the robot carriage pose localization and recognition system provided in this embodiment may include:

[0095] The depth camera module is used to acquire information about the camera environment inside the carriage and the robot's pose within the camera area of ​​the environment.

[0096] The lidar module is used to acquire radar environment information inside the vehicle and the robot pose of the radar portion within the environment.

[0097] The industrial control computer module is used to fuse the robot pose from the camera and the robot pose from the radar to construct the robot's overall pose in the environmental information.

[0098] In a preferred embodiment of this example, the depth camera module includes two depth cameras, which are respectively mounted on the left and right sides of the robot.

[0099] In a preferred embodiment of this example, the lidar module is mounted in front of the robot.

[0100] In a preferred embodiment of this example, the industrial control computer module is also used to communicate with the host computer and the slave computer.

[0101] It should be noted that the steps in the method provided by the present invention can be implemented using corresponding devices in the system. Those skilled in the art can refer to the technical solution of the method to realize the composition of the system. That is, the embodiments in the method can be understood as preferred examples for building the system, and will not be elaborated here.

[0102] One embodiment of the present invention provides a robot.

[0103] like Figure 3 As shown, the robot provided in this embodiment includes: a robot body, and a robot carriage pose positioning and recognition system according to any of the above embodiments of the present invention, wherein the robot body moves within the carriage based on the carriage environment information and robot pose information obtained by the robot carriage pose positioning and recognition system.

[0104] It should be noted that the structure of the robot provided by the present invention can be implemented using corresponding devices in the positioning and identification system. Those skilled in the art can refer to the technical solutions in the positioning and identification system to realize the composition of the robot. That is, the embodiments in the positioning and identification system can be understood as preferred examples of building a robot, and will not be elaborated here.

[0105] The technical solutions provided by the above embodiments of the present invention will be further described in detail below with reference to the accompanying drawings and preferred embodiments. However, the scope of protection of the present invention is not limited to the following specific embodiments. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this application.

[0106] Unless otherwise defined, all technical terms used herein have the same meaning as commonly understood by those skilled in the art. The technical terms used herein are for the purpose of describing particular embodiments only and are not intended to limit the scope of the invention.

[0107] A preferred embodiment of the present invention provides a robot pose localization and recognition system inside a vehicle. In a specific application example, a schematic diagram of the localization and recognition system is shown below. Figure 3 As shown. This positioning and recognition system mainly includes: a depth camera 8, a LiDAR 7, and an industrial control computer 4. Further, it may also include auxiliary components such as a camera bracket 3. Among them:

[0108] Depth camera 8 is used to acquire information about the camera environment inside the carriage and the robot pose of the camera portion within the environment;

[0109] LiDAR 7 is used to acquire radar environment information inside the vehicle and the robot pose of the radar portion within the environment.

[0110] Industrial computer 4 is used to fuse the robot pose from the camera and the robot pose from the radar to construct the robot's overall pose in the environmental information.

[0111] Camera bracket 3 is used to fix the depth camera on the loading and unloading robot, extending from the front side of the vehicle body near the edge of the Mecanum wheel to above the Mecanum wheel.

[0112] In some embodiments of the present invention:

[0113] Combination Figure 11 The industrial computer 4 acts as the central control module, processing the data collected by the sensors and communicating with the host computer 10 and the slave computer 11. The host computer sends the robot's movement position coordinates to the central control module, which then sends movement commands to the slave computer based on the calculated pose, controlling the movement of the loading and unloading robot's chassis. Furthermore, the industrial computer 4 can be located in the rear compartment of the loading and unloading robot, communicating with the host computer and slave computer via a local area network.

[0114] The LiDAR 7 serves as a pose positioning module, acquiring the robot's radar pose and environmental information. It is fixed to the front platform support of the loading / unloading robot, ensuring the robotic arm does not interfere with the LiDAR's detection range. It is connected to the central control module via an industrial network cable 12. Alternatively, the LiDAR 7 can be located at the front of the loading / unloading robot chassis, below the loading / unloading mechanism, ensuring the mechanism does not affect the LiDAR's detection range. The LiDAR is then connected to the industrial control computer via an industrial network cable.

[0115] Depth cameras 8, serving as another pose localization module, acquire camera pose information and camera environment information from both sides of the robot. They are symmetrically distributed and mounted on camera brackets along the central axis of the vehicle body. Furthermore, depth cameras 8 can be symmetrically distributed along the central axis of the loading / unloading robot, located above the left front Mecanum wheel and the right front Mecanum wheel, close to one side of the loading / unloading robot. Each depth camera is connected and fixed to a protective plate above the Mecanum wheel via a support, and interconnected with the communication module via USB.

[0116] like Figure 11 As shown, the robot is a loading and unloading robot, and its robot body includes: an end effector 1, a robotic arm 2, Mecanum wheels 5, a platform support 6, and a mobile chassis 9.

[0117] The end effector 1 is a mechanical structure capable of grasping goods. Any structure capable of grasping goods in the prior art can be used as an end effector.

[0118] Robotic arm 2 is a linkage mechanism used to cooperate with an end effector to perform multiple degrees of freedom of movement to complete the loading and unloading of goods; any linkage mechanism in the prior art that enables the end effector to move in at least three degrees of freedom without affecting the posture detection module can be used as a robotic arm.

[0119] The mobile chassis 9 includes Mecanum wheels 5 and platform supports 6, which are used to enable the robot to move omnidirectionally within the vehicle; any chassis in the prior art that enables the robot to move omnidirectionally can be used as a mobile chassis.

[0120] like Figure 12 As shown, the overall workflow of this robot may include:

[0121] S100, activate the location recognition system;

[0122] S200, determine whether the positioning and identification system is in the working environment;

[0123] S300 receives the target location command sent by the host computer;

[0124] The S400 positioning and recognition system calculates the robot's pose and acquires current environmental information based on depth cameras and LiDAR.

[0125] S500 sends movement commands to the lower-level computer based on the target position and the robot's current pose;

[0126] In a preferred embodiment of S100, it may include:

[0127] S1001: Open the software, start all sensors, obtain sensor serial numbers and other information, and determine whether the sensor devices meet the requirements.

[0128] S1002, Initiate network connection, establish TCP / IP network connection based on the IP address and port number of the host computer and the slave computer.

[0129] In a preferred embodiment of S200, it may include:

[0130] The system determines whether it is in the working environment based on data from the LiDAR and depth camera, specifically whether the LiDAR and depth camera have entered the cargo container. When the positioning and identification system is not in the working environment, it does not send control commands to the lower-level computer. The system begins communication with both the upper-level and lower-level computers after the operator moves into the working environment.

[0131] In a preferred embodiment of S300, it may include:

[0132] The host computer sends the target position coordinates of the robot's movement via socket communication in a specified format, and the positioning and recognition system decodes the data according to the specified format to obtain the specified position coordinates.

[0133] In a preferred embodiment of S400, it may include:

[0134] The robot's current pose and distance to obstacles ahead are calculated by fusing data acquired from LiDAR and depth cameras.

[0135] In a preferred embodiment of S500, it may include:

[0136] The robot determines whether the target location is reachable based on its current pose, specifically whether it will collide with the walls, cargo, or obstacles in the cargo box during its movement. If the target location is reachable, the robot calculates the rotational speeds of its four wheels based on its current pose and the Mecanum wheel motion formula, and sends this information to the lower-level computer via socket to control the robot's movement.

[0137] Another preferred embodiment of the present invention provides a robot pose localization and recognition method inside a vehicle. This method uses the robot pose localization and recognition system inside a vehicle provided in the above embodiments of the present invention to obtain the robot's pose inside the vehicle.

[0138] The preferred embodiment of the robot's pose localization and recognition method inside the vehicle includes the following steps:

[0139] Step 1: Use depth cameras on both sides of the loading and unloading robot to acquire the robot's left and right pose information and camera environment information;

[0140] Step 2: Use the lidar on the front of the loading and unloading robot to obtain the pose information and radar environment information of the front, left and right sides of the loading and unloading robot.

[0141] Step 3: Based on Step 1 and Step 2, perform fusion processing on the redundant pose information in the environment to improve the accuracy of the pose information.

[0142] The environmental information includes: low-light environment and unstructured environment; the low-light environment is the container with one side open, and the ambient light gradually weakens in the container away from the open side; the unstructured environment includes the container containing cargo of uncertain size and quantity, and the container has corrugated walls with uncertain surface unevenness on both sides.

[0143] In a preferred embodiment of step 1, the following steps are included:

[0144] Step 1-1. According to Figure 4 As shown, the depth map captured by the depth camera is first converted into a point cloud map; then the point cloud is preprocessed by filtering, which includes time domain filtering, spatial domain filtering, pass-through filtering, and removal of discrete points.

[0145] Steps 1-2. According to Figure 4 As shown, normal segmentation is performed on the preprocessed point cloud data. The main target detected by the camera is the wall of the cargo box. The results of normal segmentation are used to remove the influence of pedestrians and non-wall objects on the detection results.

[0146] Steps 1-3. According to Figure 4 As shown, Principal Component Analysis (PCA) is used to calculate the orientation bounding box (OBB) of the wall, and the plane containing the wall is extracted. The angle rs_theta between the robot and the wall, and the perpendicular distances rs_l_dis and rs_r_dis to the left and right walls are calculated. The method for extracting the plane containing the wall is as follows: Figure 5 As shown, the OBB envelope of the generated corrugated wall is larger than the actual wall due to point cloud perturbations. Based on the center plane of the OBB envelope and prior wall thickness information, the plane containing the near-end wall (i.e., the wall closest to the robot) is calculated. Finally, based on the planar relationships and the distance from the point to the line, rs_theta, rs_l_dis, and rs_r_dis are calculated.

[0147] In a preferred embodiment of step 2, the following steps are included:

[0148] Step 2-1, according to Figure 6 The process involves converting the radar's 2D point map into an image for processing, extracting straight lines from the left and right walls of the robot. The left and right wall regions are divided with the radar origin as the center. Vertical straight lines are then selected from these regions, ensuring that only one straight line exists within each region. The minimum bounding rectangle of each straight line is calculated, and a line is fitted based on this rectangle, removing clutter and interference points. The fitted line represents the desired left and right wall lines.

[0149] Step 2-2, according to Figure 6 The pose data lidar_theta, lidar_l_dis, and lidar_r_dis are calculated based on the fitted straight line. lidar_theta is calculated from the deflection angle between the straight line and the vertical direction, and the distances lidar_l_dis and lidar_r_dis from the robot to the left and right walls are calculated from the perpendicular distance from the point to the straight line.

[0150] Steps 2-3, according to Figure 6 The diagram illustrates the calculation of the distance from the robot to an obstacle in front. Given the uncertainty of the robot's working environment, to avoid collisions, a rectangular area directly in front of the robot, with the robot's width as its base and the cargo box's depth as its length, is defined as the obstacle detection area. Within this area, isolated noise is removed through filtering. The difference between the row number of the pixel representing the obstacle and the row number of the pixel representing the radar origin in the radar image is calculated to obtain the distance between the obstacle and the robot, obs_dis.

[0151] In a preferred embodiment of step 3, the following steps are included:

[0152] Step 3-1, according to Figure 7 As shown, since the robot moves at a low speed only along the x and y directions inside the cargo box, the following two-dimensional uniform motion model of the robot is established. Here, the state variable x represents the robot's position and velocity, and ω(t) represents zero-mean white noise simulating the possible velocity changes during the robot's motion:

[0153]

[0154] Step 3-2: The predicted values ​​generated by the robot motion model and the measured values ​​generated by the sensors are input together into a Kalman filter for prediction fusion to obtain the covariance of the sensor measurements. This covariance is used as the subsequent criterion for evaluating the health of the sensors.

[0155] Step 3-3, taking a camera as an example, such as Figure 8 As shown. Through step 3-2, a set of variances for a sensor can be obtained. Based on this set of variances, a threshold range is set to remove excessively large and excessively small variances δ. min <δ Li <δ max And considering data within this range as valid, their quantity is recorded as count. L Based on the variance of all valid data, a standard deviation value δ0 is set (generally the average value). For all valid data, the absolute value of the variance relative to the standard deviation is calculated and denoted as δ0. in: Next, calculate the sum of the above variances and biases, S. L , The final definition of the camera's health level is d. camera Its definition d camera The larger the value of d, the more significant or even serious the problem is considered in the camera's measurements. Therefore, the camera's weight in the final fusion result will decrease or even drop to zero, excluding it from the fusion decision; conversely, a smaller value indicates a smaller problem. camera The smaller the value, the more accurate the camera's measurement is considered to be, and the greater its weight in the final fusion result;

[0156] Steps 3-4, the robot's pose is defined as follows: Figure 9 As shown, similarly, the radar's health expression d can be obtained according to step 3-3. lidar Finally, the distances l_dis from the robot to the left wall and r_dis from the robot to the right wall can be obtained. The robot's deflection angle is calculated using a bisection averaging method, weighting the results from camera and radar measurements. The method for obtaining obs_dis is shown in steps 2-3. The final robot pose can be represented as:

[0157] Pose=(l_dis, r_dis, obs_dis, theta).

[0158] l_dis=d camera *rs_l_dis+d lidar *lidsr_l_dis

[0159] r_dis=d camera *rs_r_dis+d lidar *lidar_r_dis

[0160]

[0161] A third preferred embodiment of the present invention provides a robot carriage pose localization method, which is based on the robot carriage pose localization and recognition method provided in the above embodiments of the present invention, and realizes localization in the working environment.

[0162] In a specific application example, the working environment is a closed cargo container with side walls made of thin, corrugated plates. The inner surface of the corrugations facing the inside of the container is flat, while the connecting parts of adjacent plate base units are corrugated bottoms. After the two corrugated bottoms are butt-welded together, they form a complete concave corrugation that slopes outward from the container. Furthermore, there is no lighting inside the cargo container, and the light conditions decrease as the container depth increases. The goods inside the cargo container are arranged in rows, and after each row of goods occupies the floor space, they are stacked in layers. Depending on the size and quantity of cargo space and the goods to be transported, an unstructured closed cargo container environment is formed.

[0163] The robot's pose localization method within the vehicle compartment under this working environment includes the following steps:

[0164] Step 1: Dynamically select the origin of the reference coordinate system based on changes in the acquired environmental information (such as the placement of goods);

[0165] Step 2: Obtain the robot's coordinate position information in this coordinate system, and use this position information for positioning and identification.

[0166] In a preferred embodiment of step 1, the method includes:

[0167] according to Figure 10 The system dynamically selects the origin of the coordinate system. Because the interior of the cargo container is an unstructured environment, and the sensors used by the positioning and identification system cannot identify completely obscured objects, a global coordinate system cannot be established for the current environment. Therefore, the system dynamically selects the origin of the coordinate system based on the dynamically changing environmental information.

[0168] When the lidar scans and detects an empty row of goods that has not yet been stacked (or a row that has already been stacked), such as Figure 10 As shown in the left image, the robot establishes a coordinate system with the left wall as the y-axis and the bottom wall (or the side of the previous row of goods placed near the robot) as the x-axis, selecting the top left corner as the origin. When the goods are arranged as shown... Figure 10 As shown in the middle image, the origin of the coordinate system changes when a new row of goods begins to be placed. The robot establishes a coordinate system with the left wall as the y-axis and the side of the new goods closest to the robot as the x-axis. This continues until the entire row of goods is placed. Figure 10 As shown in the image on the right, the origin of the coordinate system remains unchanged. Because the LiDAR can only scan one surface, and its installation position is relatively low, it can only scan the goods on the first layer. Therefore, the placement of goods on other layers in this row also depends on the placement position of the goods on the first layer.

[0169] In a preferred embodiment of step 2, the method includes:

[0170] like Figure 10As shown, the robot's coordinate position consists of x_dis and y_dis. x_dis represents the distance from the robot to the y-axis, and y_dis represents the distance from the robot to the x-axis. Referring to the definition of robot pose in the above embodiments of the present invention, where l_dis represents the distance from the robot to the left wall, therefore x_dis = l_dis in the robot coordinates.

[0171] The steps for calculating the distance y_dis from the robot to the x-axis are as follows:

[0172] Step 1: Obtain the horizontal straight line in the LiDAR scan image;

[0173] Step 2: Select the horizontal line that is farthest from the origin of the image pixel coordinate system;

[0174] Step 3: Calculate the minimum bounding rectangle of the horizontal line, and fit a straight line based on the minimum bounding rectangle;

[0175] Step 4: Calculate the distance y_dis from the robot to the horizontal line based on the summation relationship of distances from a point to a line.

[0176] The robot and vehicle-mounted pose localization and recognition method, localization method, and recognition system provided in the above embodiments of the present invention employ a multi-sensor fusion approach in acquiring robot pose, which is more stable and accurate than single-sensor pose acquisition methods. The multi-sensor data fusion method reduces the impact of single-sensor errors on measurement results; the redundancy of multiple sensors ensures that the system can still operate normally without being affected even if some sensors fail. The positioning method using a dynamic coordinate system selection is more suitable for the positioning of loading and unloading robots in unstructured, low-light environments, ensuring the consistency of robot positioning during multi-layer cargo stacking. Compared to positioning methods based on Wi-Fi, Bluetooth, UWB, etc., the present invention does not require any pre-installed markers and can complete the handling and unloading of goods in containers of different sizes. Compared to methods such as SLAM mapping, no pre-built map is required; positioning is completed in real time based on the current environment.

[0177] Any matters not covered in the above embodiments of the present invention are well-known in the art.

[0178] The above embodiments are only for illustrating the technical concept and features of the present invention, and are intended to enable those skilled in the art to understand the content of the present invention and implement it accordingly. They should not be construed as limiting the scope of protection of the present invention. All equivalent changes or modifications made in accordance with the spirit and essence of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A method for robot pose localization and recognition inside a vehicle, characterized in that, include: The robot pose is obtained by acquiring camera environment information and camera position in the environment based on a depth camera. The robot pose is obtained based on the radar environment information and the radar part of the environment using lidar. By fusing the robot pose from the camera section and the robot pose from the radar section, the overall pose of the robot in the environmental information is constructed; wherein: A two-dimensional uniform motion model of the robot is established. The predicted values ​​generated by the two-dimensional uniform motion model of the robot and the measured values ​​generated by the sensor are input into the Kalman filter for prediction fusion. The covariance of the sensor's measured values ​​is obtained and used as the evaluation standard for the subsequent sensor health. For the camera component, obtain a set of variances for one sensor. Based on this set of variances, a threshold range is set, and the variances within this threshold range are considered valid data, with the number denoted as [missing information]. Based on all valid data, set a standard deviation value. Calculate the absolute value of all valid data relative to the standard deviation. and the sum of the deviations of all valid data Define the camera's health as ; Similarly, obtain the radar's health status. ; The lower the health score, the greater its weight in the final fusion result; The acquisition of camera environment information based on the depth camera includes: using the depth camera to acquire information about the uneven corrugated wall surface; The method of obtaining the robot pose of the camera portion in the environment based on the depth camera includes: The depth information acquired by the depth camera is converted into point cloud information; The point cloud information is then filtered. The filtered point cloud information is segmented using normals to remove non-wall objects. The pointed bounding box (OBB) for obtaining the camera environment information is obtained using the PCA dimensionality reduction algorithm. The angle between the robot and the wall is calculated based on the bounding box OBB. and the vertical distance from the robot to its two side walls and This means obtaining the robot's pose from the camera's position.

2. The robot carriage pose localization and recognition method according to claim 1, characterized in that, The method of acquiring the robot pose based on the radar portion of the environment using lidar includes: The two-dimensional point cloud information acquired by the lidar is transformed into image information through coordinate transformation; Divide the robot's two side regions based on the radar origin and extract information from the two side walls; Calculate the distances from the robot to its two side walls based on geometric information. and and the robot's deflection angle ; Based on region filtering and image distance calculation, the distance between the robot and the obstacle in front is determined. ; This yields the robot's pose from the radar section.

3. The robot carriage pose localization and recognition method according to claim 2, characterized in that, The process of fusing the robot pose from the camera section and the robot pose from the radar section to construct the robot's overall pose in the environmental information includes: Define the robot's overall pose expression as follows: ,in: The vertical distance from the robot to its first side wall in the robot pose obtained from the camera section. The distance from the robot to its first side wall in the robot pose of the radar section. The result obtained through dynamic weight allocation; The vertical distance from the robot to its second side wall in the robot pose obtained from the camera section. The distance from the robot to its second side wall in the robot pose of the radar section. The result obtained through dynamic weight allocation; The angle between the robot and the wall in the robot pose obtained from the camera section. Robot deflection angle in the robot pose and radar section The result obtained through dynamic weight allocation.

4. A method for robot pose localization inside a vehicle, characterized in that, include: In an unstructured, enclosed vehicle environment, the robot's pose localization and recognition method in the vehicle as described in any one of claims 1-3 is used to dynamically select the origin of the reference frame based on changes in the acquired environmental information, establish the robot's position in the reference frame, and perform localization and recognition based on the coordinate position information.

5. The robot carriage pose localization method according to claim 4, characterized in that, It also includes any one or more of the following: - The unstructured enclosed car environment includes: placing and stacking goods of different sizes and quantities to be transported in the enclosed space of cargo boxes of different sizes, wherein the number of stacking layers is not less than 1 layer; - The step of dynamically selecting the origin of the reference frame based on changes in the acquired environmental information includes: When the LiDAR scans and finds that the row in front is empty and has not been stacked with goods or that the row has been fully stacked with goods, a coordinate system is established with one side wall as the y-axis and the bottom wall or the side of the row of goods that has been fully stacked in front of the robot as the x-axis. The intersection of the y-axis, x-axis and the robot is selected as the origin of the reference system. When a new row of goods begins to be placed, the origin of the coordinate system changes. At this time, a coordinate system is established with the wall on which the goods are placed as the y-axis and the side of the new goods that is closer to the robot as the x-axis. The intersection of the y-axis, x-axis and the robot is selected as the origin of the reference system. The origin of the coordinate system does not change until the row of goods is placed. By analogy, the origin of the reference frame can be dynamically selected based on changes in real-time environmental information obtained by the lidar.

6. A robot carriage pose positioning and recognition system, characterized in that, include: A depth camera module is used to acquire camera environment information inside the carriage and the robot pose of the camera portion within the environment. Acquiring the camera environment information inside the carriage includes: using the depth camera to acquire information about the uneven corrugated wall surface; acquiring the robot pose of the camera portion within the environment includes: converting the depth information acquired by the depth camera into point cloud information; filtering the point cloud information; performing normal segmentation on the filtered point cloud information to remove non-wall objects; using the PCA dimensionality reduction algorithm to obtain the pointed bounding box (OBB) of the camera environment information; and calculating the angle between the robot and the wall based on the OBB. and the vertical distance from the robot to its two side walls and That is, to obtain the robot pose of the camera portion; The lidar module is used to acquire radar environment information inside the vehicle and the robot pose of the radar portion within the environment. An industrial control computer module is used to fuse the robot pose from the camera section and the robot pose from the radar section to construct the overall pose of the robot in the environmental information; wherein: A two-dimensional uniform motion model of the robot is established. The predicted values ​​generated by the two-dimensional uniform motion model of the robot and the measured values ​​generated by the sensor are input into the Kalman filter for prediction fusion. The covariance of the sensor's measured values ​​is obtained and used as the evaluation standard for the subsequent sensor health. For the camera component, obtain a set of variances for one sensor. Based on this set of variances, a threshold range is set, and the variances within this threshold range are considered valid data, with the number denoted as [missing information]. Based on all valid data, set a standard deviation value. Calculate the absolute value of all valid data relative to the standard deviation. and the sum of the deviations of all valid data Define the camera's health as ; Similarly, obtain the radar's health status. ; The lower the health score, the greater its weight in the final fusion result.

7. The robot carriage pose positioning and recognition system according to claim 6, characterized in that, It also includes any one or more of the following: - The depth camera module includes two depth cameras, which are respectively installed on the left and right sides of the robot; - The lidar module is installed at the front of the robot; - The industrial control computer module is also used to communicate with the host computer and the slave computer.

8. A robot, comprising: The robot body is characterized by further comprising the robot carriage pose positioning and recognition system as described in claim 6 or 7, wherein the robot body moves within the carriage based on the carriage environment information and robot pose information obtained by the robot carriage pose positioning and recognition system.

Images