Goods falling detection method based on multi-line radar

Abstract

The invention provides a cargo falling detection method based on a multi-line radar, and the method comprises the steps: S1, collecting point cloud data, carrying out the filtering operation of the point cloud data, and enabling the point cloud data to serve as reference data; s2, executing point cloud gridding operation based on the point cloud data, obtaining three-dimensional statistical characteristics and constructing a baseline statistical map; s3, collecting real-time point cloud data, executing the step S1 on the real-time point cloud data, obtaining current statistical feature data based on the step S2, and constructing a current statistical map; and S4, grid-level multi-dimensional difference judgment is executed, candidate abnormal grids are obtained based on the grid-level multi-dimensional difference judgment, judgment operation is executed based on the candidate abnormal grids, a judgment result is obtained based on the judgment operation, and whether the goods fall or are normal can be judged through the judgment result. And timely parking and reporting a result to a scheduling system.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of AGV system design, in particular to a method for detecting cargo falling based on multi-line radar. BACKGROUND

[0002] In the scenarios of automated logistics, warehouse transportation, etc., the running stability and cargo transportation safety of the automatic guided vehicle (AGV) as the core transportation equipment directly affect the logistics flow efficiency and operation cost. In the actual operation process of the AGV, it often faces complex working conditions such as uneven road surface, vehicle body vibration, and unstable cargo fixation, which easily leads to partial or complete falling of the cargo from the bearing device. If such an abnormality cannot be found in time, it not only may cause damage and loss of the cargo, but also may cause collision risks of the AGV and surrounding equipment and personnel, seriously affecting the operation safety and continuity. Therefore, real-time and accurate detection of the cargo falling state in the AGV transportation process, timely parking and scheduling reporting in abnormal conditions, become key technical requirements in the design of the AGV system.

[0003] The present application provides a method capable of timely sensing the cargo falling condition, so that the AGV can timely park and report to the scheduling system. SUMMARY

[0004] In view of the above-mentioned problems, in combination with the first aspect of the present application, the embodiments of the present application provide a method for detecting cargo falling based on multi-line radar, which comprises:

[0005] S1: collecting point cloud data and performing filtering operation on the point cloud data, taking the point cloud data as reference data;

[0006] S2: performing point cloud gridding operation based on the point cloud data, acquiring three-dimensional statistical features and constructing a baseline statistical map;

[0007] S3: collecting real-time point cloud data, performing step S1 on the real-time point cloud data, and acquiring current statistical feature data and constructing a current statistical map based on step S2;

[0008] S4: performing grid-level multi-dimensional difference judgment, acquiring candidate abnormal grids based on the grid-level multi-dimensional difference judgment, performing judgment operation based on the candidate abnormal grids, and acquiring a judgment result based on the judgment operation, the judgment result comprising a cargo falling result and a normal cargo result.

[0009] As a further scheme of the present application, the collecting point cloud data and performing filtering operation on the point cloud data, taking the point cloud data as reference data, comprises:

[0010] When the AGV takes the goods, point cloud data is collected based on the multi-line radar, and the point cloud data is taken as reference data and the ROI range is configured;

[0011] The point cloud data is filtered by using a pass-through filtering algorithm, and the point cloud data in the x, y and z axes is filtered according to the ROI range, so that only the point cloud data containing the goods is reserved.

[0012] As a further scheme of the present application, the point cloud data is subjected to a point cloud gridding operation, three-dimensional statistical features are obtained, and a baseline statistical map is constructed, including:

[0013] The grid resolution is set, and the point cloud data is divided into MxN grids in the y-z plane based on the grid resolution, where M is the number of grids in the y-axis direction, and N is the number of grids in the z-axis direction.

[0014] The three-dimensional statistical features of each grid are calculated, and a baseline statistical map is formed.

[0015] As a further scheme of the present application, the baseline statistical map is formed, including:

[0016] A first formula set is used for calculation to obtain a point cloud corresponding grid, and the baseline statistical map is constructed based on the point cloud corresponding grid and the three-dimensional statistical features;

[0017] The first formula set is specifically:

[0018] ;

[0019] ;

[0020] wherein, is the grid resolution, is the minimum value of the y direction of the baseline statistical map, is the minimum value of the z direction of the baseline statistical map, is the y coordinate of the point cloud coordinate, is the z coordinate of the point cloud coordinate, m is the corresponding grid in the y-axis direction, and n is the corresponding grid in the z-axis direction.

[0021] As a further scheme of the present application, the method further includes:

[0022] The three-dimensional statistical features include a minimum x value, an average x value and a point cloud number;

[0023] wherein, the minimum x value is the minimum value of the x coordinate of all points in the grid, reflecting the nearest position of the goods surface to the sensor;

[0024] wherein, the average x value is the average value of the x coordinate of all points in the grid, assisting in verifying the overall offset of the goods surface.

[0025] The point cloud count represents the total number of point clouds within the grid, reflecting the coverage density of goods in that area.

[0026] As a further aspect of the present invention, the step of performing grid-level multi-dimensional difference determination, obtaining candidate abnormal grids based on the grid-level multi-dimensional difference determination, performing a judgment operation based on the candidate abnormal grids, and obtaining a judgment result based on the judgment operation, wherein the judgment result includes cargo drop result and cargo normal result, including:

[0027] Compare the statistical characteristics of the baseline statistical map with the current statistical map grid by grid. If at least two of the first, second, and third conditions are met, the corresponding grid is marked as a candidate abnormal grid.

[0028] For each candidate anomalous grid, a 3×3 neighborhood verification is used to obtain the anomalous percentage data. The anomalous percentage data is compared with the neighborhood verification threshold. If the anomalous percentage is greater than the neighborhood verification threshold, the grid is identified as a valid anomalous grid; otherwise, the grid is removed.

[0029] Count all valid abnormal grids. If the number of valid abnormal grids is greater than the abnormal grid count threshold, it is determined that the cargo has fallen; otherwise, it is determined that the cargo is normal.

[0030] As a further aspect of the present invention, the method further includes:

[0031] The first condition indicates the x-value offset condition;

[0032] The second condition is expressed as the point decay condition;

[0033] The third condition is represented as the average x-value offset condition.

[0034] As a further aspect of the present invention, the step of using 3×3 neighborhood verification to obtain anomaly percentage data includes:

[0035] The number of candidate anomalous grids in the eight neighboring grids of the current grid is counted, and the anomalous percentage data is obtained based on the number of anomalous grids in the neighborhood divided by 8.

[0036] Compared with the prior art, the present invention has the following beneficial effects:

[0037] First, point cloud data is collected and filtered, and used as reference data. Then, point cloud gridding is performed based on the point cloud data to obtain three-dimensional statistical features and construct a baseline statistical map. Next, real-time point cloud data is collected, and step S1 is performed on the real-time point cloud data. Based on step S2, current statistical feature data is obtained and the current statistical map is constructed. Finally, grid-level multi-dimensional difference judgment is performed, and candidate abnormal grids are obtained based on the grid-level multi-dimensional difference judgment. A judgment operation is performed based on the candidate abnormal grids, and the judgment result is obtained based on the judgment operation. The judgment result can determine whether the goods have fallen or are normal. Through the above method, the AGV can perceive the situation of goods falling in real time during the operation, so that the AGV can stop in time and report the result to the scheduling system. Attached Figure Description

[0038] Figure 1 This is a flowchart of the steps of a method for detecting falling cargo based on multi-line radar according to the present invention;

[0039] Figure 2 This is a schematic diagram of the principle of a cargo drop detection method based on multi-line radar according to the present invention. Detailed Implementation

[0040] The present invention will now be described in detail with reference to the accompanying drawings. Figure 1 This is a schematic diagram illustrating the execution flow of a cargo drop detection method based on multi-line radar according to an embodiment of the present invention. Figure 2 This is a block diagram illustrating the principle of a cargo drop detection method based on multi-line radar according to the present invention. The following is a detailed description of this cargo drop detection method based on multi-line radar.

[0041] Step S1: Collect point cloud data and perform filtering operations on the point cloud data, using the point cloud data as reference data.

[0042] In this embodiment, step S1 specifically includes:

[0043] Step S1-1: After the AGV picks up the goods, it collects point cloud data based on multi-line radar, uses the point cloud data as reference data, and configures the ROI range.

[0044] Step S1-2: The point cloud data is filtered using a pass-through filtering algorithm. Based on the ROI range, the point cloud along the x, y, and z axes is filtered to retain only the point cloud data containing the goods.

[0045] Specifically, when the AGV receives the instruction and picks up the goods, it collects point cloud data of the goods based on the multi-line radar set on the AGV, and uses this point cloud data as initial data, i.e., reference data, and configures the ROI range for subsequent filtering operations. Then, a pass-through filtering algorithm is used to filter out point clouds that exceed the detection area on the x, y, and z axes according to the ROI range set in step S1-1, retaining only the effective point cloud containing the goods, thus reducing the amount of computation.

[0046] Understandably, the pass-through filtering algorithm is a basic filtering method based on the spatial location of point cloud data. Its core principle is to directly retain point cloud data within a preset three-dimensional coordinate threshold range (such as the maximum and minimum values ​​of the X, Y, and Z axes) and remove outliers or irrelevant areas of the point cloud that are outside the range.

[0047] Step S2: Perform point cloud meshing operation based on point cloud data to obtain three-dimensional statistical features and construct a baseline statistical map.

[0048] In this embodiment, step S2 specifically includes:

[0049] Step S2-1: Set the grid resolution. Based on the grid resolution, divide the point cloud data into M×N grids in the yz plane, where M is the number of grids in the y-axis direction and N is the number of grids in the z-axis direction.

[0050] Understandably, the baseline point cloud is divided into N×N grids in the yz plane according to the set grid resolution.

[0051] Step S2-2: Calculate the three-dimensional statistical characteristics of each grid and form a baseline statistical map.

[0052] Among them, the first set of formulas is used to calculate and obtain the grid corresponding to the point cloud, and a baseline statistical map is constructed based on the grid corresponding to the point cloud and the three-dimensional statistical features.

[0053] The first formula group is specifically as follows:

[0054] ;

[0055] ;

[0056] in, Represented as grid resolution, This is represented as the minimum value in the y-direction of the baseline statistical map. Represented as the minimum value in the z-direction of the baseline statistical map. Represented as the y-coordinate of the point cloud. The z-coordinate is represented as the z-coordinate of the point cloud, m represents the grid corresponding to the y-axis direction, and n represents the grid corresponding to the z-axis direction.

[0057] Specifically, the three-dimensional statistical characteristics of each grid are calculated to form a baseline statistical map. Assuming the coordinates of point cloud P are (x0, y0, z0), the grid (m, n) in which the point is located is calculated according to the following formula.

[0058] Furthermore, the three-dimensional statistical features include the minimum x-value, the average x-value, and the number of point clouds;

[0059] The minimum x-value represents the minimum x-coordinate of all points within the grid, reflecting the closest position of the cargo surface to the sensor;

[0060] The average x-value represents the average x-coordinate of all points within the grid, which helps to verify the overall offset of the cargo surface.

[0061] The point cloud count represents the total number of point clouds within the grid, reflecting the coverage density of goods in that area.

[0062] Step S3: Collect real-time point cloud data, execute step S1 on the real-time point cloud data, and obtain current statistical feature data and construct the current statistical map based on step S2.

[0063] Step S4: Perform grid-level multi-dimensional difference determination, obtain candidate abnormal grids based on grid-level multi-dimensional difference determination, perform judgment operation based on candidate abnormal grids, and obtain judgment result based on the judgment operation. The judgment result includes cargo drop result and cargo normal result.

[0064] Specifically, the statistical characteristics of the baseline statistical map are compared with those of the current statistical map grid by grid. If at least two of the first, second, and third conditions are met, the corresponding grid is marked as a candidate abnormal grid.

[0065] Furthermore, the first condition represents the x-value offset condition, which is that the absolute value of the difference between the current grid min_x and the baseline grid min_x is greater than the x-value change threshold, indicating that the cargo surface is farther from the sensor (due to a drop causing a decrease in height). The second condition represents the point count attenuation condition, which is that the ratio of the current grid count to the baseline grid count is less than the point count attenuation ratio threshold, indicating that the cargo is missing in this area and the number of point clouds is reduced. The third condition represents the average x-value offset condition, which is that the absolute value of the difference between the current grid avg_x and the baseline grid avg_x is greater than the x-value change threshold / 2, which helps to verify the overall positional offset of the cargo surface.

[0066] Next, for each candidate abnormal grid, a 3×3 neighborhood verification is used to obtain the abnormality percentage data. The abnormality percentage data is compared with the neighborhood verification threshold. If the abnormality percentage is greater than the neighborhood verification threshold, the grid is identified as a valid abnormal grid; otherwise, the grid is removed.

[0067] Specifically, the number of candidate anomalous grids in the eight neighboring grids around the current grid is counted, and the anomalous percentage data is obtained based on the number of anomalous grids in the neighborhood divided by 8.

[0068] Then, count all valid abnormal grids. If the number of valid abnormal grids is greater than the abnormal grid count threshold, it is determined that the cargo has fallen; otherwise, it is determined that the cargo is normal.

[0069] Understandably, the above method can detect the situation of goods falling in real time, enabling the AGV to stop in time and report to the dispatch system.

[0070] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via limited means (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0071] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0072] It should be understood that, in the embodiments of the present invention, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0073] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for cargo drop detection based on multi-line radar, characterized in that, The method comprises the following steps: S1: collecting point cloud data and performing filtering operation on the point cloud data, taking the point cloud data as reference data; S2: performing point cloud gridding operation based on the point cloud data, obtaining three-dimensional statistical features and constructing a baseline statistical map; S3: collecting real-time point cloud data, performing step S1 on the real-time point cloud data, and obtaining current statistical feature data and constructing a current statistical map based on step S2; S4: performing grid-level multi-dimensional difference judgment, obtaining candidate abnormal grids based on the grid-level multi-dimensional difference judgment, performing judgment operation based on the candidate abnormal grids, and obtaining a judgment result based on the judgment operation, wherein the judgment result comprises a goods falling result and a goods normal result.

2. The method of cargo fall detection based on multi-line radar according to claim 1, characterized in that, The method comprises the following steps: When the AGV takes the goods, the point cloud data is collected based on the multi-line radar, and the point cloud data is taken as the reference data and the ROI range is configured; The point cloud data is filtered by using a straight-through filtering algorithm, and the x, y and z axis point clouds in the point cloud data are filtered according to the ROI range, so as to retain only the point cloud data containing the goods.

3. The method of cargo fall detection based on multi-line radar according to claim 1, wherein, The method comprises the following steps: The grid resolution is set, and the point cloud data is divided into M×N grids in the y-z plane based on the grid resolution, wherein M is the number of grids in the y-axis direction, and N is the number of grids in the z-axis direction; The three-dimensional statistical features of each grid are calculated, and a baseline statistical map is formed.

4. The method of cargo fall detection based on multi-line radar according to claim 3, characterized in that, The method comprises the following steps: A first formula group is used for calculation to obtain a point cloud corresponding grid, and a baseline statistical map is constructed based on the point cloud corresponding grid and the three-dimensional statistical features; The first formula group is specifically as follows: ； ； wherein, is represented as a grid resolution, is represented as a y-direction minimum value of the baseline statistical map, is represented as a z-direction minimum value of the baseline statistical map, is represented as a y-coordinate of the point cloud coordinate, is represented as a z-coordinate of the point cloud coordinate, m is represented as a corresponding grid in the y-axis direction, and n is represented as a corresponding grid in the z-axis direction.

5. The method of cargo fall detection based on multi-line radar according to claim 3, wherein, The method further comprises the following steps: The three-dimensional statistical features comprise a minimum x value, an average x value and a point cloud number; The minimum x value represents the minimum x coordinate value of all points in the grid, and reflects the nearest position of the goods surface to the sensor; The average x value represents the average x coordinate value of all points in the grid, and assists in verifying the overall shift of the goods surface; The point cloud number represents the total number of point clouds in the grid, and reflects the coverage density of the goods in the area.

6. The method of cargo fall detection based on multi-line radar according to claim 1, wherein, The method further comprises the following steps: The statistical features of the baseline statistical map and the current statistical map are compared grid by grid, and if at least two of the first condition, the second condition and the third condition are met, the corresponding grid is marked as a candidate abnormal grid; For each candidate abnormal grid, a 3×3 neighborhood verification is performed to obtain abnormal proportion data, and the abnormal proportion data is compared with a neighborhood verification threshold value, and if the abnormal proportion is greater than the neighborhood verification threshold value, the grid is identified as an effective abnormal grid, otherwise the grid is excluded; Count all valid anomaly grids, if the number of valid anomaly grids is greater than the anomaly grid count threshold, it is determined that the goods fall result, otherwise it is determined that the goods are normal.

7. The method of cargo fall detection based on multi-line radar according to claim 6, characterized in that, The method further comprises: The first condition is represented as an x value offset condition; The second condition is represented as a point decay condition; The third condition is represented as an average x value offset condition.

8. The method of cargo fall detection based on multi-line radar according to claim 6, wherein, The 3x3 neighborhood verification is used to obtain anomaly proportion data, including: Count the number of candidate anomaly grids in the 8 adjacent grids around the grid, and calculate the anomaly proportion data based on the number of adjacent anomaly grids / 8.

Images