Method for identifying defects in container

Abstract

The invention provides a method for identifying defects in a container, which comprises the following steps of: enabling a mobile device loaded with a laser radar to enter the container through a back door of the container, and enabling the laser radar to perform circumferential scanning in a direction vertical to the advancing direction in the advancing process; filtering and coordinate conversion are carried out on the collected data, then equations of the four walls are determined, distance measurement is carried out on the data obtained after coordinate conversion and a linear equation, abnormal points are judged, and modeling display is carried out on the abnormal points. According to the method for identifying the defects in the container, the 3D data of the inner wall of the container can be formed through scanning identification and processing, then the normal points and the abnormal points are judged by comparing the collected data of each point with the measured value of the 3D data, then identification and display are carried out, and reliable reference can be provided for stacking operation during loading.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent loading equipment, and in particular to a method for identifying defects inside a container. Background Art

[0002] The interior of a container is usually in the shape of a standard cuboid. However, with the use of the container, it is inevitable that the outer wall will be subject to forces, collisions, abrasions, etc., resulting in inward depressions on the side walls of the container to varying degrees after a period of use.

[0003] For conventional container loading, these defects can be avoided during manual loading. However, with the development of industrial automated loading technology, more and more unmanned loading is adopted. In the case of unmanned loading, it is necessary to identify and mark the inner wall of the container and the defects thereon, so as to avoid collisions and friction between the goods and the depressed parts during the automated loading process, resulting in damage to the goods and abnormal operation of the loading.

[0004] Conventional techniques mainly use external scanning to scan and judge each outer surface of the container. However, external scanning can only identify defects on the outer surface, and defects on the outer surface do not necessarily lead to defects on the inner surface. Therefore, it is not possible to directly judge the defects on the inner surface based on the defects on the outer surface.

[0005] At the same time, the external scanning method requires at least two scans to complete. Considering the large volume and inconvenient transportation of the container, in order to save scanning time, two sets of independent scanning devices are required to complete the scanning, which also leads to high costs. Summary of the Invention

[0006] In order to solve the problems existing in the prior art, the present invention provides a method for identifying defects inside a container, which solves the problem that external scanning cannot directly determine the defects inside the container.

[0007] To achieve the above object, the following technical solutions are provided:

[0008] A method for identifying defects inside a container, comprising:

[0009] Step A: Enter a mobile device carrying a lidar into the container through the rear door of the container;

[0010] Step B: Identify the distance between the two side walls through the sensors on both sides of the mobile device, and then determine the midline of the container bottom plate;

[0011] Step C: Move the mobile device forward along the center line inside the container, and during the forward movement, make the lidar perform circumferential scanning in a direction perpendicular to the forward direction to obtain a scanning data set D = {θ kl , r kl , z kl};

[0012] Step D: Filter each data r kl obtained in Step C, and then convert the scanning data set from the polar coordinate system to the rectangular coordinate system;

[0013] Step E: Determine four polar coordinate angle regions 0° - 90°, 90° - 180°, 180° - 270°, 270° - 360° according to the polar coordinate angle data θ kl in Step C, and none of the above four regions include the endpoint values; take the maximum value of r kl in each scanning period according to the data r l after filtering in Step D, and establish an angle index corresponding to the maximum value for each of the four corners; establish the starting index and ending index of the four walls of the carriage according to the angle indexes of the four corners, and further determine the 3D data of the four box walls in this scanning period;

[0014] Step F: Establish a straight-line equation according to the 3D data of the four box walls respectively, then judge the distance between the acquisition data corresponding to each box wall converted to the rectangular coordinate system in Step D and the straight-line equation, and set a threshold to judge normal points and abnormal points;

[0015] Step G: Uniformly mark the normal points and abnormal points respectively, and then perform defect display.

[0016] In Step C, D = {θ kl , r kl , z kl}, where k ∈ {0, 1,..., K - 1}, K is the number of circumferences of 2D lidar scanning; l ∈ {0, 1,..., L - 1}, L is the number of scanning points per week of 2D lidar, corresponding to 360° in one week; θ kl is the angle corresponding to the l-th scanning point in the k-th week, r kl is the measured distance corresponding to the l-th scanning point in the k-th week, and z kl is the walking distance corresponding to the l-th scanning point in the k-th week.

[0017] The lidar in Step A is a 2D lidar.

[0018] In Step E, the method of taking the maximum value of the data r kl is to establish an index for each region:

[0019] The method of establishing the angle index corresponding to the maximum value for the four corners in Step E is:

[0020] The angular index in the lower right corner is

[0021] The angular index in the upper right corner is

[0022] The angular index in the upper left corner is

[0023] The angular index in the lower left corner is

[0024] The method for establishing the start index and end index of the four walls of the carriage based on the angular indices of the four corners in step E, and then determining the 3D data of the four box walls within this scanning period is as follows:

[0025] The start index and end index of the right wall of the carriage are respectively and

[0026] The start index and end index of the upper wall of the carriage are respectively and

[0027] The start index and end index of the left wall of the carriage are respectively and

[0028] The start index and end index of the lower wall of the carriage are respectively and

[0029] Thus, the 3D data for determining the scan data of the four walls of the carriage are respectively:

[0030]

[0031] The method for establishing the straight-line equation in step F is as follows:

[0032] Based on the data of one of the box walls Extract the data at both ends as or For the extracted data D ir Adopt the least squares method for linear fitting, that is:

[0033] Assume the straight-line equation is y = ax + b, and obtain the equation parameters according to the least squares method as:

[0034]

[0035] where, (x i , y i ) ∈ D ir , and n is the number of data points in D ir .

[0036] The method for uniformly marking normal points and abnormal points in step G is as follows:

[0037] Based on the marked abnormal scan points, an abnormal marking image I(k, l) is generated, where k ∈ {0, 1, …, K - 1}, and K is the number of scans of the 2D lidar; l ∈ {0, 1, …, L - 1}, and L is the number of points scanned by the 2D lidar per scan. For the points marked as normal, I(k, l) = 0; for the points marked as abnormal, I(k, l) = 1. In this way, a binary marking image is generated, where the area with a value of 0 is the normal area and the area with a value of 1 is the abnormal area.

[0038] The method for defect display in step G is as follows:

[0039] Through the conversion formula from polar coordinates to rectangular coordinates, the scan data (θ kl , r kl , z kl ) is converted to (x kl , y kl , z kl ), and its color value is set to the normal color; the scan data (θ kl , r kl , z kl ) determined by the coordinates (k, l) of the abnormal connected region is converted to (x kl , y kl , z kl ), and its color value is set to the abnormal color; the central coordinates (k j0 , l j0 ) and the circumscribed rectangle {(k j1 , l j1 ), (k j2 , l j2 ), (k j3 , l j3 ), (k j4 , l j4 )} of the connected region determined to be defective are calculated. According to the (θ kl , r kl , z kl ) determined by the coordinates (k, l), it is converted to (x kl , y kl , z kl ), and the corresponding position and region are set to the prompt color.

[0040] Compared with the prior art, the advantages of the present invention are as follows:

[0041] The method for identifying defects inside a container provided by the present invention can form 3D data of the inner wall of the container through scanning, recognition, and processing, and then compare the data collected at each point with the measured values of the 3D data to determine normal points and abnormal points, and then perform marking and display, which can provide a reliable reference for the stacking operation during loading. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is a schematic diagram of modeling and image display of the method for identifying defects inside a container according to the present invention;

[0043] Figure 2 It is a schematic diagram of the scanning imaging of a lidar;

[0044] Figure 3 It is a schematic diagram of lidar scanning modeling. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0045] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. Usually, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0046] Embodiment

[0047] Combined with Figures 1-3 As shown, the method for identifying defects inside a container provided by the present invention includes:

[0048] A mobile device carrying a 2D lidar enters the container through the rear door of the container; the mobile device can adopt the mobile track disclosed in the applicant's previous patent for a loading machine. Distance sensors are provided on both sides of the mobile track to detect the distance between it and the two side walls of the container. Its 2D lidar can be arranged on a rail forklift, and the rail forklift can move along the mobile track.

[0049] Identify the distance between the two side walls through the sensors on both sides of the mobile device, and then determine the midline of the container bottom plate;

[0050] The 2D lidar is vertically installed at the front end of the loading machine. During operation, the 2D lidar is located at the midline position of the carriage body, and its scanning plane is parallel to the end face of the carriage. The starting position of the 2D lidar scanning is vertically downward. Each time the 2D lidar scans a circle, the distances measured at different angles from 0 to 360° are obtained. Control the loading device to travel from one end of the carriage to the other end at a certain speed. At the same time, start the 2D lidar to continuously scan to obtain the carriage scan data set as D = {θ kl , r kl , zkl}, where \(k\in\{0,1,\ldots,K - 1\}\), \(K\) is the number of revolutions of the 2D lidar scan; \(l\in\{0,1,\ldots,L - 1\}\), \(L\) is the number of points scanned per revolution of the 2D lidar, corresponding to 360° per revolution; \(\theta\) kl is the angle corresponding to the \(l\)-th scan point in the \(k\)-th revolution, \(r\) kl is the measured distance corresponding to the \(l\)-th scan point in the \(k\)-th revolution, \(z\) kl is the traveled distance corresponding to the \(l\)-th scan point in the \(k\)-th revolution.

[0051] Filtering process:

[0052] For each measurement data \(r\) kl , take its neighborhood data set \(Nb=\{r\) k(l-N) , \(r\) k(l-N+1) , \(\ldots\), \(r\) kl , \(\ldots\), \(r\) k(l+N-1) , \(r\) k(l+N) \}, where \(2N + 1\) is the size of the neighborhood. Sort the data set \(Nb\) from largest to smallest to get Let where \(M\lt N\) is the size of the mean filter window. Then denote as the filtering result of the measurement data \(r\) kl .

[0053] Coordinate transformation:

[0054] For the \(l\)-th scan point in each revolution, convert its scan data from polar coordinates \((\theta\) kl , \(r\) kl ) to Cartesian coordinates \((x\) kl , \(y\) kl ), as follows.

[0055] \(x\) kl \(=r\) kl \cos(\theta\) kl ), \(y\) kl \(=r\) kl \sin(\theta\) kl ).

[0056] Determination of the compartment wall:

[0057] The starting position of the 2D lidar scan is vertically downward, corresponding to a scan angle of 0°. The 2D lidar is located at the middle position of the compartment body. It can be analyzed that the angle data \(\theta\) kl of the four corners of the compartment should be in the ranges of polar coordinate angles 0° - 90°, 90° - 180°, 180° - 270°, and 270° - 360° respectively. And the measured distances of the four corners of the compartment should be the largest. Therefore, take the positions with the largest measured distances within the four angle ranges in the same acquisition period as the positions of the four corners of the compartment, as follows.

[0058] The angular index in the lower right corner is

[0059] The angular index in the upper right corner is

[0060] The angular index in the upper left corner is

[0061] The angular index in the lower left corner is

[0062] Since there is noise in the measurement data of the 2D lidar and the four corners of the carriage are not very regular, it is not very stable and accurate to determine the four walls of the carriage simply based on the above indexes. In order to obtain stable four-wall scan data, the start and end indexes of the four walls of the carriage are determined based on the above indexes, as follows.

[0063] The start index and end index of the right wall of the carriage are respectively and

[0064] The start index and end index of the upper wall of the carriage are respectively and

[0065] The start index and end index of the left wall of the carriage are respectively and

[0066] The start index and end index of the lower wall of the carriage are respectively and

[0067] Thus, the 3D data of the four-wall scan data of the carriage are respectively:

[0068]

[0069] Scanning abnormal point detection:

[0070] For the data of a certain wall in the four walls of the carriage The data (x kl , y kl , z kl ) should be on a straight line. Considering that z kl changes little within a scanning period, it can be considered unchanged. In this way, the straight line fitting problem in the three-dimensional space is simplified to the straight line fitting problem in the two-dimensional space, that is, it is considered that (x kl , y kl ) is the point on a certain wall of the four walls of the carriage in the k-th week. Such points in the data of a certain wall should be on the same straight line, and this straight line is a certain wall of the carriage at the k-th time. Therefore, by performing straight line fitting on this data, a certain wall in the carriage is determined.

[0071] The purpose of the box condition detection is to detect the protrusions or depressions of the box body, which can be judged by calculating the degree of deviation of the coordinate points on the box wall from the straight line of the box wall. The determination of the box wall straight line has a great influence on the detection of abnormal points. The result of a poor determination of the box wall straight line may cause misdetection of abnormal points. If all the data on a certain wall is used for fitting the box wall straight line, since there may be some abnormal points among them, the fitting of the box wall straight line will be inaccurate, affecting the detection of abnormal points. Considering that the defects of the carriage usually occur at the positions of the four walls of the carriage far from the four corners, only the data at both ends of the data on a certain wall is used for straight line fitting here to avoid the influence of abnormal points on the defective data, as follows. According to the data on a certain wall Extract the data at both ends of it as or For the extracted data D ir Perform straight line fitting, and here the least squares method is used. That is:

[0072] Let the straight line equation be y = ax + b, and the equation parameters are obtained according to the least squares method as:

[0073]

[0074] where, (x i ,y i ) ∈ D ir , and n is the number of data points in D ir .

[0075] After determining the straight line equation of a certain wall, calculate the distance from the data (x kl ,y kl ) on a certain wall to this straight line, as follows.

[0076]

[0077] If d(x lk ,y lk ) > d T , then it is judged as an abnormal point, and the scanning point (θ kl ,r kl ,z kl ) is marked as abnormal.

[0078] Box wall defect detection:

[0079] According to the marked abnormal scanning points, generate an abnormal marking image I(k, l), where k ∈ {0, 1, …, K - 1}, K is the number of weeks of 2D lidar scanning; l ∈ {0, 1, …, L - 1}, L is the number of points scanned by the 2D lidar per week. For the points marked as normal, I(k, l) = 0; for the points marked as abnormal, I(k, l) = 1. In this way, a binary marking image is generated, and the area with a value of 0 is the normal area, and the area with a value of 1 is the abnormal area.

[0080] Perform connected component labeling on the abnormal marked image I(k, l), and extract the connected components A = {A j , 0 ≤ j ≤ J - 1}, where J is the number of detected connected components. Perform conditional judgment on the extracted connected components, including area, aspect ratio, etc., and those that meet the conditions are determined to be wall defects.

[0081] Defect display:

[0082] Through the conversion formula from polar coordinates to rectangular coordinates, convert the scan data (θ kl , r kl , z kl ) to (x kl , y kl , z kl ), and set its color value to the normal color; convert the scan data (θ kl , r kl , z kl ) determined by the coordinates (k, l) of the abnormal connected component to (x kl , y kl , z kl ), and set its color value to the abnormal color; calculate the center coordinates (k j0 , l j0 ) and the circumscribed rectangle {(k j1 , l j1 ), (k j2 , l j2 ), (k j3 , l j3 ), (k j4 , l j4 )} for the connected component determined to be a defect. According to the (θ kl , r kl , z kl ) determined by the coordinates (k, l), convert it to (x kl , y kl , z kl ), and set the corresponding position and area to the prompt color. The generated 3D data is displayed on the interface, and the defect area data is also displayed.

[0083] Note that the above is only the preferred embodiment of the present invention and the applied technical principles. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described here, and various obvious changes, re-adjustments, and substitutions can be made by those skilled in the art without departing from the protection scope of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments. Without departing from the concept of the present invention, it can also include more other equivalent embodiments, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A method for identifying defects in a container, comprising: Step A: The mobile device carrying the laser radar enters the container through the back door of the container; Step B: using sensors on both sides of the mobile device to identify the distance between the two side walls, and then determine the center line of the container bottom plate; Step C: Make the mobile device move forward along the center line in the container, and make the laser radar perform circumferential scanning in a direction perpendicular to the moving direction during the moving process, and obtain the scanning data set D = {θ kl ,r kl ,z kl }; Step D: for each data r obtained in step C kl Perform filtering and then convert the scanned data set from a polar coordinate system to a rectangular coordinate system; Step E: According to the polar coordinate angle data θ in step C kl Determine four polar coordinate angle regions 0°~90°, 90°~180°, 180°~270°, 270°~360°, none of which contain endpoint values; according to the data r after filtering in step D kl Take r in each scanning cycle l The maximum value of the image is obtained, and the angle indexes corresponding to the maximum values ​​of the four corners are established; the starting index and the ending index of the four walls of the carriage are established according to the angle indexes of the four corners, and then the 3D data of the four walls of the carriage within the scanning cycle are determined; Step F, establishing straight line equations according to the 3D data of the four box walls, then determining the distance between the collected data corresponding to each box wall converted into a rectangular coordinate system in step D and the straight line equation, and setting a threshold to determine normal points and abnormal points; Step G: Mark normal points and abnormal points uniformly, and then display defects.

2. The method according to claim 1, characterized in that In step C, D = {θ kl ,r kl ,z kl }, where k∈{0,1,…,K-1}, K is the number of cycles scanned by the 2D LiDAR; l∈{0,1,…,L-1}, L is the number of points scanned by the 2D LiDAR per week, corresponding to 360° per week; θ kl is the angle corresponding to the lth scanning point in the kth week, r kl is the measured distance corresponding to the lth scanning point in the kth week, z kl is the walking distance corresponding to the lth scanning point in the kth week.

3. The method according to claim 1, characterized in that The laser radar in step A is a 2D laser radar.

4. The method according to claim 1, characterized in that: The method for establishing the angle index corresponding to the maximum value of the four angles in step E is: The angle index of the lower right corner is The angle index of the upper right corner is The angle index of the upper left corner is The angle index of the lower left corner is 5. The method according to claim 1, characterized in that In step E, the starting index and the ending index of the four walls of the carriage are established according to the angle indexes of the four corners, and the method for determining the 3D data of the four walls in the scanning cycle is as follows: The starting index and ending index of the right wall of the carriage are and The starting index and ending index of the upper wall of the carriage are and The starting index and ending index of the left wall of the carriage are and The starting index and ending index of the lower wall of the carriage are and Therefore, the 3D data of the four walls of the carriage are determined as follows:

6. The method according to claim 1, characterized in that The method for establishing the equation of the line in step F is: According to one of the box wall data D i ={x kl ,y kl ,z kl }, Extract the data at both ends as D ir ={x kl ,y kl }, or For the extracted data D ir The least squares straight line fitting is adopted, that is: Assume the equation of the line is y=ax+b, and the equation parameters are obtained by the least squares method: Among them, (x i ,y i )∈D ir , n is D ir The number of data points in .

7. The method according to claim 1, characterized in that The method for uniformly marking normal points and abnormal points in step G is: According to the marked abnormal scanning points, an abnormal marked image I(k,l) is generated, where k∈{0,1,…,K-1}, K is the number of weeks scanned by the 2D laser radar; l∈{0,1,…,L-1}, L is the number of points scanned by the 2D laser radar per week. For points marked as normal, I(k,l)=0; for points marked as abnormal, I(k,l)=1. In this way, a binary marked image is generated, where the area with a value of 0 is the normal area and the area with a value of 1 is the abnormal area.

8. The method according to claim 1, characterized in that The method for displaying defects in step G is: Through the conversion formula from polar coordinates to rectangular coordinates, the scan data (θ kl ,r kl ,z kl ) is converted to (x kl ,y kl ,z kl ), whose color value is set to normal color; the scan data (θ kl ,r kl ,z kl ) is converted to (x kl ,y kl ,z kl ), and its color value is set to the abnormal color; the center coordinates (k j0 ,l j0 ) and the circumscribed rectangle {(k j1 ,l j1 ),(k j2 ,l j2 ),(k j3 ,l j3 ),(k j4 ,l j4 )}, according to the coordinates (k, l) determined by (θ kl ,r kl ,z kl ) is converted to (x kl ,y kl ,z kl ), the corresponding position and area are set as prompt colors.

Images