Pose detection method, device and equipment of movable material rack and storage medium

Abstract

The invention provides a position and posture detection method, device and equipment of a mobile material rack and a storage medium, and relates to the technical field of computers, the method is applied to an automatic guided vehicle, and the method comprises the following steps: determining whether the mobile material rack exists in a target station area according to first point cloud data of the target station area; the first point cloud data of the target station area is obtained by detecting the target station area by using a detection device on the automated guided vehicle; if the moving material frame exists in the target station area, the initial pose of the moving material frame is determined according to the first point cloud data; extracting the column leg map of the mobile material rack to obtain second point cloud data of the current frame column leg map; the second point cloud data of the column leg map of the current frame is the point cloud corresponding to the edge of the column leg of the movable material rack seen from the view angle of the automatic guided vehicle; and matching the second point cloud data of the current frame column leg map with the first point cloud data to obtain a target pose of the mobile material rack. According to the invention, the accuracy of pose detection of the movable material rack can be improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method, device, equipment and storage medium for detecting the posture of a mobile material rack. Background Art

[0002] The backpack Automated Guided Vehicle (AGV) is an unmanned transport vehicle widely used in the fields of industrial automation and intelligent logistics. It uses various navigation technologies to achieve automatic path planning and driving to complete the material handling work. When the AGV transports a mobile rack (such as a structure with four legs and wheels), the position of the mobile rack is not accurate after it is placed manually. If the AGV only relies on global positioning to blindly pick it up, it may be overloaded or even hit the legs. Therefore, it is necessary to detect the position of the mobile rack.

[0003] In the prior art, two-dimensional (2D) radar is used to scan the legs of the mobile rack to obtain the point cloud of the current frame, and then the point cloud of the current frame is aligned with the map of the legs by iterative closest point (ICP) and other point cloud registration. However, this method has the following disadvantages: when there are neighboring objects around the mobile rack, the point cloud of the neighboring objects will participate in the point cloud matching, which can easily lead to the wrong posture of the matched mobile rack. Summary of the invention

[0004] The present invention provides a method, device, equipment and storage medium for detecting the posture of a mobile material rack, so as to solve the defect in the prior art that the posture of a matched mobile material rack is easily caused to be wrong, and improve the accuracy of the posture detection of the mobile material rack.

[0005] In a first aspect, the present invention provides a method for detecting the posture of a mobile material rack, which is applied to an automatic guided vehicle, and the method comprises the following steps.

[0006] Determine whether there is a mobile rack in the target workstation area according to the first point cloud data of the target workstation area; the first point cloud data of the target workstation area is obtained by detecting the target workstation area using the detection equipment on the automatic guided vehicle; In the case where it is determined that there is a mobile rack in the target workstation area, determining an initial position and posture of the mobile rack according to the first point cloud data; Extracting the column leg map of the mobile rack to obtain second point cloud data of the column leg map of the current frame; the second point cloud data of the column leg map of the current frame is a point cloud corresponding to the edge of the column leg of the mobile rack seen from the perspective of the automatic guided vehicle; The second point cloud data of the column leg map of the current frame is matched with the first point cloud data to obtain the target posture of the mobile rack.

[0007] According to a method for detecting the posture of a mobile material rack provided by the present invention, determining whether there is a mobile material rack in the target workstation area according to first point cloud data of the target workstation area includes: Perform column leg target recognition according to the first point cloud data to obtain at least two potential column leg targets; Clustering the first point cloud data according to a distance threshold to obtain at least one point cloud set; Determining at least two potential column leg targets based on each of the point cloud sets; Combining each of the potential column-leg targets to obtain at least one group of column-leg target combinations; It is determined whether there is a mobile material rack in the target workstation area based on the distance between each of the column leg targets in each group of the column leg target combinations and the distance between each of the column leg targets in the column leg map of the mobile material rack.

[0008] According to a method for detecting the posture of a mobile material rack provided by the present invention, determining whether there is a mobile material rack in the target workstation area according to first point cloud data of the target workstation area includes: According to the first point cloud data, the mobile rack is detected by using a target detection model to determine whether there is a mobile rack in the target workstation area; Among them, the target detection model is a deep learning network, which is obtained after initial network training using multiple groups of third point cloud data and mobile rack detection results corresponding to each of the third point cloud data, and the mobile rack detection results are regression boxes of the mobile racks corresponding to each of the second point cloud data.

[0009] According to a method for detecting the posture of a mobile rack provided by the present invention, when it is determined that there is a mobile rack in the target workstation area, before determining the initial posture of the mobile rack according to the first point cloud data, it also includes: According to the first point cloud data and the column leg map, a point cloud of a left column leg of the mobile rack and a point cloud of a right column leg of the mobile rack from the perspective of the automatic guided vehicle are determined; a reflective strip is provided on the left column leg of the mobile rack; When it is determined that the laser reflection intensity of the left column leg point cloud is greater than the laser reflection intensity of the right column leg point cloud, the distance between the left column leg and the right column leg in the first point cloud data is compared with the distance between the left column leg and the right column leg in the column leg map to obtain a consistency comparison result; When the consistency comparison result is that the distance between the left column leg and the right column leg in the first point cloud data is consistent with the distance between the left column leg and the right column leg in the column leg map, it is determined that there is a mobile material rack in the target workstation area.

[0010] According to a method for detecting the posture of a mobile rack provided by the present invention, the second point cloud data of the column leg map of the current frame contains at least two source point clouds, and the second point cloud data of the column leg map of the current frame is matched with the first point cloud data to obtain the target posture of the mobile rack, including: For any of the source point clouds, find a target point cloud whose first point cloud data is closest to the source point cloud; The source point cloud is transformed using a transformation matrix to obtain a source point cloud with an updated position; the transformation matrix is ​​used to minimize the distance from the source point cloud to the target point cloud; Obtaining a target position of the source point cloud according to the source point cloud after the updated position and a preset number of iterations; According to the target position of each source point cloud, the target position and posture of the mobile rack are obtained.

[0011] According to a method for detecting the posture of a mobile rack provided by the present invention, the method further comprises: Planning a moving path according to the target position of the mobile rack and the current position of the automatic guided vehicle; According to the moving path, move to the bottom of the moving rack.

[0012] In a second aspect, the present invention further provides a posture detection device for a mobile rack, which is applied to an automatic guided vehicle, and the device includes the following modules: A determination module, used to determine whether there is a mobile material rack in the target workstation area according to the first point cloud data of the target workstation area; the first point cloud data of the target workstation area is obtained by detecting the target workstation area using the detection equipment on the automatic guided vehicle; A posture detection module, for determining an initial posture of the mobile material rack according to the first point cloud data when it is determined that there is a mobile material rack in the target workstation area; Extracting the column leg map of the mobile rack to obtain second point cloud data of the column leg map of the current frame; the second point cloud data of the column leg map of the current frame is a point cloud corresponding to the edge of the column leg of the mobile rack seen from the perspective of the automatic guided vehicle; The second point cloud data of the column leg map of the current frame is matched with the first point cloud data to obtain the target posture of the mobile rack.

[0013] In a third aspect, the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, a method for detecting the posture of a mobile material rack as described above is implemented.

[0014] In a fourth aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a method for detecting a posture of a mobile material rack as described in any one of the above.

[0015] In a fifth aspect, the present invention further provides a computer program product, comprising a computer program, which, when executed by a processor, implements a method for detecting a posture of a mobile material rack as described in any one of the above-mentioned methods.

[0016] The present invention provides a method, device, equipment and storage medium for detecting the posture of a mobile material rack. The method is applied to an automatic guided vehicle. The method first determines whether there is a mobile material rack in the target workstation area based on first point cloud data of the target workstation area, wherein the first point cloud data of the target workstation area is obtained by detecting the target workstation area using a detection device on the automatic guided vehicle; then, when it is determined that there is a mobile material rack in the target workstation area, the initial posture of the mobile material rack is determined based on the first point cloud data; then, the column leg map of the mobile material rack is extracted to obtain second point cloud data of the column leg map of the current frame, wherein the second point cloud data of the column leg map of the current frame is the point cloud corresponding to the edge of the column leg of the mobile material rack seen from the perspective of the automatic guided vehicle; then, the second point cloud data of the column leg map of the current frame is matched with the first point cloud data to obtain the target posture of the mobile material rack.

[0017] In the present invention, it is first determined whether there is a mobile material rack in the target workstation area. When it is determined that there is a mobile material rack, the column leg map of the mobile material rack is extracted to obtain the second point cloud data of the current frame column leg map, that is, the point cloud corresponding to the edge of the column leg of the mobile material rack seen from the perspective of the automatic guided vehicle is obtained, and then the second point cloud data of the current frame column leg map is reversely matched with the first point cloud data to obtain the target posture of the mobile material rack. This can avoid matching with adjacent objects and improve the accuracy of the mobile material rack posture detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1It is one of the flow charts of the method for detecting the posture of the mobile material rack provided by the present invention.

[0020] Figure 2a It is one of the plan schematic diagrams of the mobile material rack provided by the present invention.

[0021] Figure 2b This is the second plan view of the mobile rack provided by the present invention.

[0022] Figure 3 It is a schematic diagram of the principle of the method for detecting the position and posture of a mobile material rack provided by the present invention.

[0023] Figure 4 This is the second flow chart of the method for detecting the posture of a mobile material rack provided by the present invention.

[0024] Figure 5 It is a structural schematic diagram of the position and posture detection device of the mobile material rack provided by the present invention.

[0025] Figure 6 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] Combine the following Figure 1-Figure 6 The present invention describes a method, device, equipment and storage medium for detecting the position and posture of a mobile rack.

[0028] Figure 1 It is one of the flow diagrams of the method for detecting the posture of a mobile rack provided by the present invention, and the method is applied to an automatic guided vehicle, such as Figure 1 As shown, the method includes the following: Step 101: Determine whether there is a mobile material rack in the target workstation area according to the first point cloud data of the target workstation area; the first point cloud data of the target workstation area is obtained by detecting the target workstation area using a detection device on an automatic guided vehicle; Specifically, the execution subject of this embodiment is an automated guided vehicle (AGV), which is an unmanned transport vehicle widely used in the fields of industrial automation and intelligent logistics. It uses various navigation technologies to achieve automatic path planning and driving to complete the material handling work. In this embodiment, the posture detection of the mobile material rack is realized to improve the accuracy of the posture detection.

[0029] First, the detection equipment on the automatic guided vehicle is used to detect the target workstation area to obtain the first point cloud data. The detection equipment on the AGV, such as a two-dimensional 2D radar, uses the 2D radar to scan the target workstation area. It can scan a certain area near the target workstation (for example, the maximum allowable material rack deviation is ±50cm / ±30°) to obtain the current frame point cloud, that is, the first point cloud data.

[0030] Furthermore, it is possible to determine whether there is a mobile rack in the target workstation area based on the first point cloud data, for example, by using a Euclidean clustering method or a deep learning method.

[0031] Step 102: When it is determined that there is a mobile rack in the target workstation area, determine the initial position of the mobile rack according to the first point cloud data; Specifically, when it is determined that there is a mobile rack in the target workstation area, the position and posture of the mobile rack can be further detected.

[0032] After detecting the presence of a mobile rack in the target workstation area in the previous step, the structural features of the potential column legs can be obtained based on the first point cloud data, and the rough initial posture of the mobile rack can be obtained through geometric structure matching.

[0033] Step 103, extracting the column leg map of the mobile rack to obtain the second point cloud data of the column leg map of the current frame; the second point cloud data of the column leg map of the current frame is the point cloud corresponding to the edge of the column leg of the mobile rack seen from the perspective of the automatic guided vehicle; Specifically, the column leg map of the mobile rack refers to the map drawn in advance according to the structural features of the mobile rack. First, four rectangles of the mobile rack (corresponding to the column legs) are drawn, and the point cloud or line segment features of the column leg map can be generated based on the four rectangles. Figure 2a It is one of the planar schematic diagrams of the mobile material rack provided by the present invention. Figure 2b This is the second plan view of the mobile rack provided by the present invention. Figure 2a As shown, the four rectangles at the four corners correspond to the table legs, and the lines between the table legs form line segments, such as Figure 2b As shown, corresponding to the rack type, rack length, rack width and target offset, the target offset refers to the offset of the AGV relative to the center position of the rack.

[0034] If the point cloud (first point cloud data) collected in the current frame is directly matched with the column leg map of the mobile rack to determine the target posture of the mobile rack, the point cloud of the neighboring objects will cause interference and pose matching errors. Therefore, in this embodiment, the column leg map is used to perform reverse matching with the collected first point cloud data to avoid interference from neighboring objects. Therefore, the present invention extracts the column leg map, and the purpose of the extraction is to obtain the point cloud corresponding to the edge of the column leg of the mobile rack seen from the perspective of the automatic guided vehicle, that is, to extract the second point cloud data.

[0035] Step 104: Match the second point cloud data of the column leg map of the current frame with the first point cloud data to obtain the target posture of the mobile rack.

[0036] Specifically, after extracting the second point cloud data, the point cloud of the map is reversely aligned with the point cloud of the current frame radar scan (the first point cloud data) to avoid matching with neighboring objects. Then, the target position of the mobile rack can be obtained based on the aligned point cloud.

[0037] The method provided in this embodiment is applied to an automatic guided vehicle, which first determines whether there is a mobile material rack in the target workstation area based on the first point cloud data of the target workstation area, wherein the first point cloud data of the target workstation area is obtained by detecting the target workstation area using the detection equipment on the automatic guided vehicle; then, when it is determined that there is a mobile material rack in the target workstation area, the initial posture of the mobile material rack is determined according to the first point cloud data; then, the column leg map of the mobile material rack is extracted to obtain the second point cloud data of the column leg map of the current frame, wherein the second point cloud data of the column leg map of the current frame is the point cloud corresponding to the edge of the column leg of the mobile material rack seen from the perspective of the automatic guided vehicle; then, the second point cloud data of the column leg map of the current frame is matched with the first point cloud data to obtain the target posture of the mobile material rack.

[0038] In the present invention, it is first determined whether there is a mobile material rack in the target workstation area. When it is determined that there is a mobile material rack, the column leg map of the mobile material rack is extracted to obtain the second point cloud data of the current frame column leg map, that is, the point cloud corresponding to the edge of the column leg of the mobile material rack seen from the perspective of the automatic guided vehicle is obtained, and then the second point cloud data of the current frame column leg map is reversely matched with the first point cloud data to obtain the target posture of the mobile material rack. This can avoid matching with adjacent objects and improve the accuracy of the mobile material rack posture detection.

[0039] According to a method for detecting the posture of a mobile rack provided by the present invention, determining whether there is a mobile rack in a target workstation area according to first point cloud data of the target workstation area includes: Perform column leg target recognition according to the first point cloud data to obtain at least two potential column leg targets; Clustering the first point cloud data according to a distance threshold to obtain at least one point cloud set; Determine at least two potential column leg targets based on each point cloud set; Combining each potential column-leg target to obtain at least one set of column-leg target combination; Determine whether there is a mobile material rack in the target workstation area based on the distance between each column leg target in each group of column leg target combinations and the distance between each column leg target in the column leg map of the mobile material rack.

[0040] Specifically, in some embodiments, the process of determining whether there is a mobile rack in the target workstation area according to the collected first point cloud data in step 101 can be implemented by the following steps, including: For example, the Euclidean clustering method can be used to identify multiple potential column leg targets.

[0041] First, column leg targets are identified based on the first point cloud data to obtain at least two potential column leg targets. Then, the first point cloud data is clustered based on a distance threshold to obtain at least one point cloud set. For example, point clouds in which the Euclidean distance of adjacent points is less than a certain threshold are taken as a set to obtain multiple point cloud sets. Further, at least two potential column leg targets are determined based on each point cloud set, and each group of point cloud sets is determined as a point cloud of a potential column leg target. The several point cloud sets obtained by classification correspond to several potential column leg targets.

[0042] Furthermore, after obtaining the potential column leg targets, it is possible to calculate whether the column leg spacings of different combinations are consistent with the column leg spacings and layout of the column leg map.

[0043] Step a, first combine each potential column-leg target to obtain at least one group of column-leg target combinations; Step b, determine whether there is a mobile material rack in the target station area according to the spacing between each column leg target in each group of column leg target combination and the spacing between each column leg target in the column leg map of the mobile material rack. If the spacing between each column leg target in each group of column leg target combination and the spacing between each column leg target in the column leg map of the mobile material rack are consistent, it is considered that there is a mobile material rack in the target station area, and the detection of the presence or absence of the mobile material rack is realized.

[0044] The method provided in this embodiment firstly identifies column leg targets according to the first point cloud data to obtain at least two potential column leg targets, clusters the first point cloud data according to the distance threshold to obtain at least one point cloud set, and then determines at least two potential column leg targets according to each point cloud set; then, combines each potential column leg target to obtain at least one group of column leg target combinations; determines whether there is a mobile material rack in the target workstation area according to the spacing between each column leg target in each group of column leg target combinations and the spacing between each column leg target in the column leg map of the mobile material rack. In this embodiment, the European clustering method is used to determine whether there is a mobile material rack in the target workstation area, so as to facilitate the subsequent further posture detection according to the initial posture of the mobile material rack, thereby improving the detection accuracy of the posture detection.

[0045] According to a method for detecting the posture of a mobile rack provided by the present invention, determining whether there is a mobile rack in a target workstation area according to first point cloud data of the target workstation area includes: According to the first point cloud data, the mobile rack is detected using the target detection model to determine whether there is a mobile rack in the target workstation area; Among them, the target detection model is a deep learning network, which is obtained after the initial network is trained using multiple groups of third point cloud data and the mobile rack detection results corresponding to each third point cloud data. The mobile rack detection result is the regression box of the mobile rack corresponding to each second point cloud data.

[0046] Specifically, in some embodiments, the process of determining whether there is a mobile rack in the target workstation area according to the collected first point cloud data in step 101 can be implemented by the following steps, including: According to the first point cloud data, the mobile rack is detected using the target detection model to determine whether there is a mobile rack in the target workstation area.

[0047] Among them, the target detection model is a deep learning network, and the deep learning network is obtained by training the initial network using multiple sets of third point cloud data and the mobile rack detection results corresponding to each third point cloud data, and the mobile rack detection result is the regression frame of the mobile rack corresponding to each second point cloud data. Because the target detection is obtained by training it using a large number of training samples, the accuracy of target detection is high. Commonly used target detection models can be used in this embodiment, for example, 1. R-CNN series: including R-CNN, Fast R-CNN, Faster R-CNN and Mask R-CNN. These models generate candidate regions through the Region Proposal Network (RPN), and then classify and regress these regions. 2. YOLO series: including YOLOv1, YOLOv2, YOLOv3, YOLOv4 and YOLOv5. The YOLO (YouOnly Look Once) model is known for its fast speed and easy implementation, and is suitable for real-time target detection tasks.

[0048] The method provided in this embodiment uses a deep learning method to detect the mobile material rack and determine whether there is a mobile material rack in the target workstation area. The detection accuracy is high, which facilitates further posture detection based on the initial posture of the mobile material rack, thereby improving the detection accuracy of the posture detection.

[0049] According to a method for detecting the posture of a mobile rack provided by the present invention, when it is determined that there is a mobile rack in a target workstation area, before determining the initial posture of the mobile rack according to the first point cloud data, the method further includes: According to the first point cloud data and the column leg map, the point cloud of the left column leg of the mobile rack and the point cloud of the right column leg of the mobile rack under the perspective of the automatic guided vehicle are determined; a reflective strip is provided on the left column leg of the mobile rack; When it is determined that the laser reflection intensity of the left column leg point cloud is greater than the laser reflection intensity of the right column leg point cloud, the distance between the left column leg and the right column leg in the first point cloud data is compared with the distance between the left column leg and the right column leg in the column leg map to obtain a consistency comparison result; When the consistency comparison result is that the distance between the left column leg and the right column leg in the first point cloud data is consistent with the distance between the left column leg and the right column leg in the column leg map, it is determined that there is a mobile material rack in the target workstation area.

[0050] Specifically, in some embodiments, before determining the initial position of the mobile rack according to the first point cloud data, it also includes: using some technical means to further determine whether the mobile rack actually exists in the target workstation area. The process is exemplified as follows: It is understandable that usually when detecting the mobile rack, it is at the door of the workstation, so it can be assumed that the two outermost column legs must be seen to consider that there is a real rack. There will still be extreme cases based on the above method to determine whether there is a rack, which is prone to false alarms. For example, there is no rack at the target station, but there are racks at the left and right stations, and the distance between the left and right racks is equal to the distance on the column leg map. In this case, it is easy to have a false alarm. The solution provided by the present invention to this extreme case is: a reflective strip is provided on one side of the mobile rack, such as the left column leg of the mobile rack, so that only when the left leg has a high reflection intensity and the right leg has a low reflection intensity, and the distance between the left and right legs is equal to the map, it is considered that there is a real rack.

[0051] Specifically, first, based on the first point cloud data and the column leg map, the point cloud of the left column leg and the right column leg of the mobile rack are determined from the perspective of the automatic guided vehicle. That is, it is determined that the two outermost column legs can be seen from the perspective of the AGV before it is considered that there is a real rack.

[0052] Furthermore, the laser reflection intensity of the left column leg point cloud is compared with the laser reflection intensity of the right column leg point cloud. When it is determined that the laser reflection intensity of the left column leg point cloud is greater than that of the right column leg point cloud, the distance between the left column leg and the right column leg in the first point cloud data is further compared with the distance between the left column leg and the right column leg in the column leg map to obtain a consistency comparison result. When the consistency comparison result is that the distance between the left column leg and the right column leg in the first point cloud data is consistent with the distance between the left column leg and the right column leg in the column leg map, it is determined that there is a mobile material rack in the target workstation area.

[0053] The method provided in this embodiment first determines the point cloud of the left column leg of the mobile material rack and the point cloud of the right column leg of the mobile material rack from the perspective of the automatic guided vehicle according to the first point cloud data and the column leg map, wherein a reflective strip is provided on the left column leg of the mobile material rack; then, when it is determined that the laser reflection intensity of the left column leg point cloud is greater than the laser reflection intensity of the right column leg point cloud, the spacing between the left column leg and the right column leg in the first point cloud data is compared with the spacing between the left column leg and the right column leg in the column leg map to obtain a consistency comparison result; further, when the consistency comparison result is that the spacing between the left column leg and the right column leg in the first point cloud data is consistent with the spacing between the left column leg and the right column leg in the column leg map, it is determined that the target station area has a mobile material rack. The detection accuracy of the present invention in detecting whether there is a material rack in the target station area is high, and it is not easy to miss or misreport the existence of the material rack.

[0054] According to a method for detecting the posture of a mobile rack provided by the present invention, the second point cloud data of the current frame column leg map contains at least two source point clouds, and the second point cloud data of the current frame column leg map is matched with the first point cloud data to obtain the target posture of the mobile rack, including: For any source point cloud, find the target point cloud whose first point cloud data is closest to the source point cloud; The source point cloud is transformed using a transformation matrix to obtain a source point cloud with updated position; the transformation matrix is ​​used to minimize the distance from the source point cloud to the target point cloud; According to the source point cloud after the updated position and the preset number of iterations, the target position of the source point cloud is obtained; According to the target position of each source point cloud, the target position and posture of the mobile rack are obtained.

[0055] Specifically, in some embodiments, the process of matching the second point cloud data of the column leg map of the current frame with the first point cloud data to obtain the target posture of the mobile rack can be achieved by the following steps, including: For example, the Iterative Closest Point (ICP) algorithm can be used for point cloud matching. The ICP algorithm is an algorithm widely used in computer vision, robotics, and 3D graphics. It is mainly used to achieve point cloud data registration, that is, to find a transformation (including rotation and translation) so that one point cloud data is spatially aligned with another point cloud data. The specific example of the process of using the ICP algorithm to match the second point cloud data of the current frame column leg map with the first point cloud data is as follows: It can be understood that the second point cloud data of the current frame column leg map contains at least two source point clouds, that is, there are multiple point clouds in the map. The point cloud in the map can be used as the source point cloud for reverse matching with the first point cloud data perceived by the lidar, so as to avoid matching with adjacent objects.

[0056] First, step 1, for any source point cloud in the second point cloud data of the column leg map of the current frame, find the target point cloud that is closest to the first point cloud data and the source point cloud. This process can be understood as an initialization process.

[0057] Furthermore, in step 2, the source point cloud is transformed using a transformation matrix to obtain a source point cloud after the position is updated. The transformation matrix is ​​used to minimize the distance from the source point cloud to the target point cloud. First, find the nearest point: that is, for each point in the source point cloud, find the closest point in the target point cloud. Then, minimize the error: calculate a transformation so that the sum of the distances from the points in the source point cloud to the corresponding nearest points in the target point cloud is minimized. This is usually achieved by the least squares method. Then, apply the transformation matrix: apply the calculated transformation to the source point cloud so that the position of the source point cloud is updated to obtain the source point cloud after the position is updated.

[0058] Furthermore, the target position of the source point cloud can be obtained according to the source point cloud after the updated position and the preset number of iterations. For example, the steps in step 2 are repeated until a certain stop condition is met, such as the change amount of the transformation is less than a certain threshold, or the preset number of iterations is reached. The position corresponding to the source point cloud after the updated position at the end of the iteration is determined as the target position of the source point cloud.

[0059] Furthermore, the target position of the mobile rack can be obtained according to the target position of each source point cloud.

[0060] For example, Figure 3 Schematic diagram of the principle of the method for detecting the posture of a mobile rack provided by the present invention, showing the process of calculating the target posture of the mobile rack by point cloud matching, such as Figure 3 As shown in the figure, under the perspective of the automatic guided vehicle, the column leg map is first extracted. According to the current posture of the AGV and the initial posture of the material rack, it is possible to obtain which side of the column leg can be seen under the current AGV perspective, and the edge of the column leg that may be seen is used as the point cloud of the current frame map, that is, the second point cloud data of the column leg map of the current frame. Then, the second point cloud data of the column leg map of the current frame is used to reversely perform ICP matching with the point cloud scanned by the current frame radar, so as to avoid matching to adjacent objects.

[0061] The method provided in this embodiment includes at least two source point clouds in the second point cloud data of the current frame column leg map, and reversely matches the second point cloud data of the current frame column leg map with the first point cloud data to determine the target posture of the mobile rack. This can avoid matching with adjacent objects during matching, thereby improving the accuracy of the target posture calculation of the mobile rack.

[0062] According to a method for detecting the posture of a mobile rack provided by the present invention, the method further comprises: According to the target posture of the mobile rack and the current posture of the automatic guided vehicle, the moving path is planned; According to the moving path, move to the bottom of the mobile rack.

[0063] Specifically, in some embodiments, after identifying the target posture of the mobile rack, the method further includes: planning a moving path for the automatic guided vehicle to move to the mobile rack. The specific implementation process of the automatic guided vehicle planning the path is as follows: First, according to the target position of the mobile rack and the current position of the automatic guided vehicle (AGV), it moves to the bottom of the mobile rack according to the moving path.

[0064] Specifically, planning a mobile path involves multiple aspects, including environmental perception, path planning algorithms, and dynamic obstacle avoidance. The following is an example of a path planning method: 1. Environmental modeling and sensor application: First, the working area of ​​the AGV needs to be modeled, which can be achieved through grid method, topology method, geometric feature method or hybrid representation method, so that the AGV can understand its working environment. At the same time, the AGV needs to be equipped with appropriate sensors, such as lidar and visual sensors, for environmental perception and obstacle detection.

[0065] 2. Path planning algorithm: Path planning algorithms can be divided into global path planning and local path planning. Global path planning relies on a comprehensive understanding of the environment, while local path planning focuses on real-time obstacle avoidance and path adjustment. In terms of algorithm selection, there are a variety of algorithms to choose from, including classical algorithms (such as Dijkstra algorithm), bionic algorithms (such as genetic algorithms, ant colony algorithms) and artificial intelligence algorithms (such as neural network algorithms and fuzzy logic). This embodiment does not specifically limit the specific path planning algorithm to be selected.

[0066] 3. Application of intelligent optimization algorithms: In path planning, intelligent optimization algorithms such as the improved A* algorithm can effectively reduce the number of turns and combine time windows and priority strategies to achieve dynamic collision-free path planning for multiple AGVs.

[0067] 4. Real-time response and dynamic obstacle avoidance: AGV needs to respond to environmental changes in real time and avoid obstacles dynamically during movement. This can be achieved through algorithms such as the improved dynamic window method, which can dynamically adjust weight parameters based on LiDAR data to improve obstacle avoidance.

[0068] In summary, when planning the moving path of AGV, it is necessary to comprehensively consider multiple aspects such as environmental modeling, sensor application, path planning algorithm selection, multi-AGV coordination, intelligent optimization algorithm and real-time response to ensure the accuracy and efficiency of path planning.

[0069] The method provided in this embodiment calculates the target posture of the mobile rack with high accuracy, and then moves to the bottom of the mobile rack according to the target posture of the mobile rack and the current posture of the automatic guided vehicle (AGV) according to the moving path, thereby ensuring the accuracy and efficiency of path planning.

[0070] Figure 4 This is a second flow chart of the method for detecting the posture of a mobile rack provided by the present invention, such as Figure 4 As shown, the posture detection method of the mobile rack includes the following steps: Step 401: Determine whether there is a mobile rack in the target workstation area.

[0071] Step 402: perform posture detection on the mobile rack.

[0072] Step 403: perform path planning according to the target posture of the mobile rack.

[0073] The position and posture detection device of the mobile material rack provided by the present invention is described below. The position and posture detection device of the mobile material rack described below and the position and posture detection method of the mobile material rack described above can be referred to each other.

[0074] Figure 5 Schematic diagram of the structure of the mobile rack posture detection device provided by the present invention, the device is applied to an automatic guided vehicle, such as Figure 5 As shown, the posture detection device 500 of the mobile rack includes the following modules: A determination module 510 is used to determine whether there is a mobile rack in the target workstation area according to the first point cloud data of the target workstation area; the first point cloud data of the target workstation area is obtained by detecting the target workstation area using the detection equipment on the automatic guided vehicle; A posture detection module 520 is used to determine the initial posture of the mobile rack according to the first point cloud data when it is determined that there is a mobile rack in the target workstation area; Extracting the column leg map of the mobile rack to obtain second point cloud data of the column leg map of the current frame; the second point cloud data of the column leg map of the current frame is a point cloud corresponding to the edge of the column leg of the mobile rack seen from the perspective of the automatic guided vehicle; The second point cloud data of the column leg map of the current frame is matched with the first point cloud data to obtain the target posture of the mobile rack.

[0075] The device provided in this embodiment is applied to an automatic guided vehicle, and the device includes a determination module 510 and a posture detection module 520. First, the determination module 510 determines whether there is a mobile material rack in the target workstation area based on the first point cloud data of the target workstation area, wherein the first point cloud data of the target workstation area is obtained by detecting the target workstation area using the detection equipment on the automatic guided vehicle; then, when it is determined that there is a mobile material rack in the target workstation area, the posture detection module 520 determines the initial posture of the mobile material rack based on the first point cloud data; then, the column leg map of the mobile material rack is extracted to obtain the second point cloud data of the column leg map of the current frame, wherein the second point cloud data of the column leg map of the current frame is the point cloud corresponding to the edge of the column leg of the mobile material rack seen from the perspective of the automatic guided vehicle; then, the second point cloud data of the column leg map of the current frame is matched with the first point cloud data to obtain the target posture of the mobile material rack.

[0076] In the present invention, it is first determined whether there is a mobile material rack in the target workstation area. When it is determined that there is a mobile material rack, the column leg map of the mobile material rack is extracted to obtain the second point cloud data of the current frame column leg map, that is, the point cloud corresponding to the edge of the column leg of the mobile material rack seen from the perspective of the automatic guided vehicle is obtained, and then the second point cloud data of the current frame column leg map is reversely matched with the first point cloud data to obtain the target posture of the mobile material rack. This can avoid matching with adjacent objects and improve the accuracy of the mobile material rack posture detection.

[0077] According to a mobile rack posture detection device 500 provided by the present invention, the determination module 510 is specifically used for: Perform column leg target recognition according to the first point cloud data to obtain at least two potential column leg targets; Clustering the first point cloud data according to a distance threshold to obtain at least one point cloud set; Determining at least two potential column leg targets based on each of the point cloud sets; Combining each of the potential column-leg targets to obtain at least one group of column-leg target combinations; It is determined whether there is a mobile material rack in the target workstation area based on the distance between each of the column leg targets in each group of the column leg target combinations and the distance between each of the column leg targets in the column leg map of the mobile material rack.

[0078] According to a posture detection device 500 for a mobile rack provided by the present invention, the determination module 510 is further used for: According to the first point cloud data, the mobile rack is detected by using a target detection model to determine whether there is a mobile rack in the target workstation area; Among them, the target detection model is a deep learning network, which is obtained after initial network training using multiple groups of third point cloud data and mobile rack detection results corresponding to each of the third point cloud data, and the mobile rack detection results are regression boxes of the mobile racks corresponding to each of the second point cloud data.

[0079] According to a posture detection device 500 for a mobile rack provided by the present invention, the determination module 510 is further used for: According to the first point cloud data and the column leg map, a point cloud of a left column leg of the mobile rack and a point cloud of a right column leg of the mobile rack from the perspective of the automatic guided vehicle are determined; a reflective strip is provided on the left column leg of the mobile rack; When it is determined that the laser reflection intensity of the left column leg point cloud is greater than the laser reflection intensity of the right column leg point cloud, the distance between the left column leg and the right column leg in the first point cloud data is compared with the distance between the left column leg and the right column leg in the column leg map to obtain a consistency comparison result; When the consistency comparison result is that the distance between the left column leg and the right column leg in the first point cloud data is consistent with the distance between the left column leg and the right column leg in the column leg map, it is determined that there is a mobile material rack in the target workstation area.

[0080] According to a mobile rack posture detection device 500 provided by the present invention, the second point cloud data of the column leg map in the current frame includes at least two source point clouds; The posture detection module 520 is specifically used for: For any of the source point clouds, find a target point cloud whose first point cloud data is closest to the source point cloud; The source point cloud is transformed using a transformation matrix to obtain a source point cloud with an updated position; the transformation matrix is ​​used to minimize the distance from the source point cloud to the target point cloud; Obtaining a target position of the source point cloud according to the source point cloud after the updated position and a preset number of iterations; According to the target position of each source point cloud, the target position and posture of the mobile rack are obtained.

[0081] According to a mobile rack posture detection device 500 provided by the present invention, the device also includes a path planning module; The path planning module is used to: Planning a moving path according to the target position of the mobile rack and the current position of the automatic guided vehicle; According to the moving path, move to the bottom of the moving rack.

[0082] Figure 6 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 6 As shown, the electronic device may include: a processor 610, a communication interface 620, a memory 630 and a communication bus 640, wherein the processor 610, the communication interface 620 and the memory 630 communicate with each other through the communication bus 640. The processor 610 may call the logic instructions in the memory 630 to execute the posture detection method of the mobile rack, and the method includes: Determine whether there is a mobile rack in the target workstation area according to the first point cloud data of the target workstation area; the first point cloud data of the target workstation area is obtained by detecting the target workstation area using the detection equipment on the automatic guided vehicle; In the case where it is determined that there is a mobile rack in the target workstation area, determining an initial position and posture of the mobile rack according to the first point cloud data; Extracting the column leg map of the mobile rack to obtain second point cloud data of the column leg map of the current frame; the second point cloud data of the column leg map of the current frame is a point cloud corresponding to the edge of the column leg of the mobile rack seen from the perspective of the automatic guided vehicle; The second point cloud data of the column leg map of the current frame is matched with the first point cloud data to obtain the target posture of the mobile rack.

[0083] In addition, the logic instructions in the above-mentioned memory 630 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0084] On the other hand, the present invention further provides a computer program product, the computer program product includes a computer program, the computer program can be stored in a non-transitory computer-readable storage medium, when the computer program is executed by a processor, the computer can execute the posture detection method of the mobile rack provided by the above methods, the method includes: Determine whether there is a mobile rack in the target workstation area according to the first point cloud data of the target workstation area; the first point cloud data of the target workstation area is obtained by detecting the target workstation area using the detection equipment on the automatic guided vehicle; In the case where it is determined that there is a mobile rack in the target workstation area, determining an initial position and posture of the mobile rack according to the first point cloud data; Extracting the column leg map of the mobile rack to obtain second point cloud data of the column leg map of the current frame; the second point cloud data of the column leg map of the current frame is a point cloud corresponding to the edge of the column leg of the mobile rack seen from the perspective of the automatic guided vehicle; The second point cloud data of the column leg map of the current frame is matched with the first point cloud data to obtain the target posture of the mobile rack.

[0085] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to execute the posture detection method of the mobile rack provided by the above methods, the method comprising: Determine whether there is a mobile rack in the target workstation area according to the first point cloud data of the target workstation area; the first point cloud data of the target workstation area is obtained by detecting the target workstation area using the detection equipment on the automatic guided vehicle; In the case where it is determined that there is a mobile rack in the target workstation area, determining an initial position and posture of the mobile rack according to the first point cloud data; Extracting the column leg map of the mobile rack to obtain second point cloud data of the column leg map of the current frame; the second point cloud data of the column leg map of the current frame is a point cloud corresponding to the edge of the column leg of the mobile rack seen from the perspective of the automatic guided vehicle; The second point cloud data of the column leg map of the current frame is matched with the first point cloud data to obtain the target posture of the mobile rack.

[0086] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0087] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0088] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting the posture of a mobile rack, characterized in that: Applied to an automated guided vehicle, the method comprises: Determine whether there is a mobile material rack in the target workstation area according to the first point cloud data of the target workstation area; the first point cloud data of the target workstation area is obtained by detecting the target workstation area using the detection equipment on the automatic guided vehicle; In the case where it is determined that there is a mobile rack in the target workstation area, determining an initial position and posture of the mobile rack according to the first point cloud data; Extracting the column leg map of the mobile rack to obtain second point cloud data of the column leg map of the current frame; the second point cloud data of the column leg map of the current frame is a point cloud corresponding to the edge of the column leg of the mobile rack seen from the perspective of the automatic guided vehicle; The second point cloud data of the column leg map of the current frame is matched with the first point cloud data to obtain the target posture of the mobile rack.

2. The method for detecting the position and posture of a mobile rack according to claim 1, characterized in that: The determining, based on the first point cloud data of the target workstation area, whether there is a mobile rack in the target workstation area comprises: Perform column leg target recognition according to the first point cloud data to obtain at least two potential column leg targets; Clustering the first point cloud data according to a distance threshold to obtain at least one point cloud set; Determining at least two potential column leg targets based on each of the point cloud sets; Combining each of the potential column-leg targets to obtain at least one group of column-leg target combinations; It is determined whether there is a mobile material rack in the target workstation area based on the distance between each of the column leg targets in each group of the column leg target combinations and the distance between each of the column leg targets in the column leg map of the mobile material rack.

3. The method for detecting the position and posture of a mobile rack according to claim 1, characterized in that: The determining, based on the first point cloud data of the target workstation area, whether there is a mobile rack in the target workstation area comprises: According to the first point cloud data, the mobile rack is detected by using a target detection model to determine whether there is a mobile rack in the target workstation area; Among them, the target detection model is a deep learning network, which is obtained after initial network training using multiple groups of third point cloud data and mobile rack detection results corresponding to each of the third point cloud data, and the mobile rack detection results are regression boxes of the mobile racks corresponding to each of the second point cloud data.

4. The method for detecting the position and posture of a mobile rack according to claim 1, characterized in that: In the case where it is determined that there is a mobile rack in the target workstation area, before determining the initial position of the mobile rack according to the first point cloud data, the method further includes: According to the first point cloud data and the column leg map, a point cloud of a left column leg of the mobile rack and a point cloud of a right column leg of the mobile rack from the perspective of the automatic guided vehicle are determined; a reflective strip is provided on the left column leg of the mobile rack; When it is determined that the laser reflection intensity of the left column leg point cloud is greater than the laser reflection intensity of the right column leg point cloud, the distance between the left column leg and the right column leg in the first point cloud data is compared with the distance between the left column leg and the right column leg in the column leg map to obtain a consistency comparison result; When the consistency comparison result is that the distance between the left column leg and the right column leg in the first point cloud data is consistent with the distance between the left column leg and the right column leg in the column leg map, it is determined that there is a mobile material rack in the target workstation area.

5. The method for detecting the position and posture of a mobile rack according to any one of claims 1 to 4, characterized in that: The second point cloud data of the column leg map of the current frame includes at least two source point clouds, and the second point cloud data of the column leg map of the current frame is matched with the first point cloud data to obtain the target posture of the mobile rack, including: For any of the source point clouds, find a target point cloud whose first point cloud data is closest to the source point cloud; The source point cloud is transformed using a transformation matrix to obtain a source point cloud with an updated position; the transformation matrix is ​​used to minimize the distance from the source point cloud to the target point cloud; Obtaining a target position of the source point cloud according to the source point cloud after the updated position and a preset number of iterations; According to the target position of each source point cloud, the target position and posture of the mobile rack are obtained.

6. The method for detecting the position and posture of a mobile rack according to any one of claims 1 to 4, characterized in that: The method further comprises: Planning a moving path according to the target position of the mobile rack and the current position of the automatic guided vehicle; According to the moving path, move to the bottom of the moving rack.

7. A posture detection device for a mobile rack, characterized in that: Applied to an automatic guided vehicle, the device comprises: A determination module, used to determine whether there is a mobile material rack in the target workstation area according to the first point cloud data of the target workstation area; the first point cloud data of the target workstation area is obtained by detecting the target workstation area using the detection equipment on the automatic guided vehicle; A posture detection module, for determining an initial posture of the mobile material rack according to the first point cloud data when it is determined that there is a mobile material rack in the target workstation area; Extracting the column leg map of the mobile rack to obtain second point cloud data of the column leg map of the current frame; the second point cloud data of the column leg map of the current frame is a point cloud corresponding to the edge of the column leg of the mobile rack seen from the perspective of the automatic guided vehicle; The second point cloud data of the column leg map of the current frame is matched with the first point cloud data to obtain the target posture of the mobile rack.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the posture detection method of the mobile material rack as described in any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for detecting the posture of a mobile material rack as described in any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the posture detection method of the mobile material rack as described in any one of claims 1 to 6 is implemented.

Images