3D intelligent inventory checking method and device

Abstract

The invention relates to the technical field of electric digital data processing, and discloses a 3D intelligent stocktaking method and device, and the device comprises a 4D millimeter wave radar disposed at the top of a stock bin, S1, obtaining the data of a corresponding coverage area detected by the 4D millimeter wave radar; s2, preprocessing the acquired data; S3, processing the preprocessed data by a point cloud data grid filtering algorithm to obtain point cloud data on grids on a global grid map; s4, checking the integrity of the data, and if blank grids exist in the global grid map, filling the grids of the blank data in the global grid map by using a point cloud smooth interpolation algorithm to form a final smooth point cloud; and S5, reconstructing the surface of the material, and constructing a three-dimensional diagram of the surface of the material according to the final smooth point cloud determined in the step S4. Through the 4D millimeter wave radar and various algorithms, the real-time and all-around monitoring of the material bin is realized, and accurate three-dimensional material level imaging is provided.

Description

Technical Field

[0001] The present invention relates to the technical field of electrical digital data processing, and in particular, to a 3D intelligent warehouse inventory method and device. Background Art

[0002] Granular or bulk materials are stored in a silo. The problem of material inventory has always existed in large power plants and grain-related enterprises. Facing the mountain of materials, most enterprises still conduct inventory and statistics of materials through manual inventory. Manual inventory means that relevant technicians use detection instruments to measure and calculate from multiple azimuth angles. This method of inventory takes a long time and has a low inventory accuracy. Summary of the Invention

[0003] To solve the technical problems in the background art, the present invention proposes a 3D intelligent warehouse inventory method and device.

[0004] A 3D intelligent warehouse inventory method proposed by the present invention is used for inventory of cylindrical or conical silos. A cover plate is placed on the top of the silo, and c 4D millimeter-wave radars are arranged on the cover plate. The coverage area of each 4D millimeter-wave radar is determined according to the specific model of the radar, and can be rectangular or circular. c is a positive integer. The coverage areas of the c 4D millimeter-wave radars have an overlapping part, and there is an uncovered area of the silo for the c 4D millimeter-wave radars;

[0005] S1. Obtain the data of the corresponding coverage area detected by the 4D millimeter-wave radar;

[0006] S2. Preprocess the obtained data;

[0007] Remove data points:

[0008] Remove the maximum distance. Determine the maximum distance detected by the 4D millimeter-wave radar according to the height, diameter and shape of the silo, and remove the data greater than the maximum distance;

[0009] Due to engineering errors and the adhesion of materials on the inner wall of general silos, remove the points that are too close according to the height of the silo. Calculate the minimum distance using the formula fmax(0.2, fmin(0.5, silo height * 0.05)), and remove the data less than the minimum distance. The unit of the silo height in the above formula is meters;

[0010] Remove the data with a small signal-to-noise ratio;

[0011] Remove the data reflected by the 4D millimeter-wave radar from the silo: Since the silo is generally made of metal materials, the reflection intensity of the 4D millimeter-wave radar for different materials is different. Remove the data obtained within the reflection intensity range of the 4D millimeter-wave radar for the silo material;

[0012] S3. The point cloud data raster filtering algorithm processes the preprocessed data to obtain the point cloud data on the raster of the global raster map. The raster without data is defined as a blank raster, and the raster with data is defined as a valid raster;

[0013] S4. Check the data integrity. If there are blank rasters in the global raster map, use the point cloud smoothing interpolation algorithm to fill the rasters with missing data in the global raster map to form the final smoothed point cloud;

[0014] If each raster in the global raster map has data, the data is complete, indicating that there is an error in the operation of step S2 or S3, and remind the staff to repair;

[0015] S5. Reconstruct the surface of the material, and construct a three-dimensional map of the material surface according to the final smoothed point cloud determined in step S4.

[0016] Specifically in S5, confirm the relationship between each point and the nearby points, generate triangular patches, and finally form a three-dimensional map of the material surface.

[0017] Preferably, specifically in step S3:

[0018] S30. Convert the preprocessed radar data into point cloud data and map it to the silo coordinates;

[0019] S31. Fuse the point cloud data located in the same silo;

[0020] S32. The fused point cloud data is processed by a raster algorithm for dimensionality reduction, and the maximum value is taken on each raster, thereby obtaining the point cloud data located on the raster.

[0021] Specifically, step S32 is:

[0022] S320. Use the maximum diameter of the silo + a as the side length to produce a large rectangular surface, where a is 5% - 12% of the silo diameter;

[0023] S321. Generate continuous rasters of squares with a side length of one-thousandth of the silo diameter in the rectangular surface to form a global raster map;

[0024] S322. Project the fused point cloud data to the raster position, and retain the point cloud data with the maximum height in the same raster according to the height, and remove other data in the same raster.

[0025] Preferably, the smoothing interpolation algorithm in step S4 includes a local smoothing interpolation algorithm and a global smoothing interpolation algorithm. The local smoothing interpolation algorithm and / or the global smoothing interpolation algorithm can be selected to fill the rasters with missing data in the global raster map.

[0026] Preferably, the specific steps for filling the raster with missing data using the local smoothing interpolation algorithm are as follows:

[0027] Local smoothing interpolation: Define a local neighborhood for each point in the point cloud, and calculate the weighted average position of the points within this neighborhood. The weights are based on distance, with closer points having higher weights:

[0028] The specific steps are as follows:

[0029] Generate a filtering kernel kernel with a side length of 3 grids as a moving window, and define the size of the filtering kernel kernel with a side length of 3 grids as 3*3;

[0030] Starting from the upper left corner of the global grid map (usually the [0,0] coordinates of this grid), first traverse the rows and then the columns; A grid without data in a moving window is defined as a blank grid, a grid with data in a moving window is defined as a valid grid, and the other grids outside a grid in a moving window are the neighborhood of this grid;

[0031] Preferably, during the process of moving the moving window, if the neighborhood of any grid in the moving window is a blank grid, expand the filtering kernel kernel in the order of 3*3 - 5*5 - 7*7 -... - (2b + 1)*(2b + 1), 3 < n < 7; Ensure that the grids in the moving window have valid grids during the movement, and ensure that the neighborhood of each grid has valid grids;

[0032] Update the height of the data in each grid in the moving window during the movement of the moving window:

[0033] Calculate the distance d(i,j) between the blank grid and other valid grids:

[0034]

[0035] where: (x i , y i ) are the coordinates of the blank grid, and (x j , y j ) are the coordinates of the valid grid in the neighborhood of the blank grid;

[0036] Process the distance d(i,j) using the reciprocal method, and calculate the weight coefficient smooth coefficient according to the smoothing coefficient n:

[0037]

[0038] where n is the smoothing coefficient, n is a constant, n can be adjusted according to the smoothing degree, and smoothcoefficient j is (x j , yj ) Weight coefficient of the effective grid;

[0039] Assume that there are m effective grids relative to the blank grid within a moving window, and the height of this blank grid is updated to h i :

[0040]

[0041] Where: h j is the height of the effective grid with coordinates (x j , y j ).

[0042] Preferably, the method for filling the data of the vacant data grid by using the global smoothing interpolation algorithm is specifically as follows:

[0043] Calculate the planar distance d(i,j) between the blank grid and the effective grid:

[0044]

[0045] Where: (x i , y i ) are the coordinates of the blank grid, and (x j , y j ) are the coordinates of the effective grid in the neighborhood of the blank grid;

[0046] Calculate the weight W(i,j) according to the planar distance between the effective grid and a blank grid (center point) by using the selected radial basis function;

[0047] Calculate the data (height) of the blank grid by weighted average through the data (height) of the effective grid and the corresponding weight:

[0048]

[0049] m is the number of effective grids.

[0050] It should be noted that when using the global smoothing interpolation algorithm and the local smoothing interpolation algorithm to fill the grid data of the vacant data in the global grid map, the following can be adopted:

[0051] When the proportion of the blank grids in the global grid map is greater than or equal to 20%, first complete the data of the blank grids through the local smoothing interpolation algorithm;

[0052] Then update the data of the blank grids through the global smoothing interpolation algorithm. When using the global smoothing interpolation algorithm to update the data of the blank grids, the input includes the data of other blank grids filled by using the local smoothing interpolation algorithm.

[0053] Preferably, when the proportion of blank grids in the global grid map is less than 20%, the data in the grids is filled and updated simultaneously by the local smoothing interpolation algorithm and the global smoothing interpolation algorithm;

[0054] Specifically, the filter kernel kernel of the local smoothing interpolation algorithm is complemented from the upper left corner of the global grid map, and the global smoothing interpolation algorithm starts from the lower right corner of the global grid map to update the data of each grid. Among them, the grid data updated by the local smoothing interpolation algorithm is the input when the global smoothing interpolation algorithm updates the data later, and the data updated by the global smoothing interpolation algorithm is the input when the local smoothing interpolation algorithm updates the data later.

[0055] A 3D intelligent inventory device includes a 4D millimeter-wave radar, an edge processor, a data processor, and a display disposed on the top of the bin. The 4D millimeter-wave radar transmits the detected data to the edge processor, and the edge processor preprocesses the data detected by the 4D millimeter-wave radar. The data processor is embedded with software of a point cloud data grid filtering algorithm, a point cloud smoothing interpolation algorithm, and a material surface reconstruction algorithm. The display is connected to the data processor for displaying the three-dimensional map of the material surface formed by the data processor, or the cloud is connected to the data processor to facilitate users to view the remaining situation of the materials in the bin through the network.

[0056] Preferably, a selection module is further embedded in the data processor. When the proportion of blank grids in the global grid map is greater than or equal to 10%, the grid data is first complemented by the local smoothing interpolation algorithm, and then the data in the grids is updated by the global smoothing interpolation algorithm; when the proportion of blank grids in the global grid map is less than 10%, the data in the grids is filled and updated simultaneously by the local smoothing interpolation algorithm and the global smoothing interpolation algorithm.

[0057] A 3D intelligent inventory method and device proposed by the present invention realize real-time and all-round monitoring of the bin through a 4D millimeter-wave radar and various algorithms, provide accurate three-dimensional level imaging, and promote the intelligentization and safety upgrade of warehouse management.

[0058] Additional aspects and advantages of the present invention will be given in part in the following description, become apparent in part from the following description, or be learned through the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 Distribution of the 4D millimeter-wave radar of the present invention on the top of the bin;

[0060] Figure 2 Flowchart of this method;

[0061] Figure 3 Flowchart of filling blank grids by the point cloud smoothing interpolation algorithm;

[0062] Figure 4 Schematic diagram of the moving window moving in the global raster map;

[0063] Figure 5 3D map of the material surface formed in the embodiment of the present invention. Detailed implementation manners

[0064] The embodiments of the present invention will be described in detail below. Examples of the embodiments are shown in the drawings, where the same or similar symbols represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the drawings are exemplary and are only used to explain the present invention and should not be construed as a limitation to the present invention.

[0065] As Figure 1 shown, a 3D intelligent inventory management method:

[0066] As Figure 2 shown, a cover plate is placed on the top of a cylindrical or conical silo. Three 4D millimeter-wave radars are arranged on the cover plate. The coverage area of each 4D millimeter-wave radar is rectangular. As Figure 1 shown, there is an overlapping part in the coverage areas of the three 4D millimeter-wave radars, and there is an uncovered area of the silo by the three 4D millimeter-wave radars;

[0067] S1. Obtain the data of the corresponding coverage area detected by the 4D millimeter-wave radar;

[0068] S2. Preprocess the obtained data:

[0069] Remove data points:

[0070] Remove the data greater than the maximum distance determined according to the height, diameter and shape of the silo. Remove the data greater than this maximum distance;

[0071] Due to engineering errors and the adhesion of materials on the inner wall of general silos, remove the points that are too close according to the height of the silo. Calculate the minimum distance using the formula fmax(0.2, fmin(0.5, silo height * 0.05)), and remove the data less than this minimum distance, where the unit of the silo height in the above formula is meters;

[0072] Remove the data with low signal-to-noise ratio;

[0073] Remove the data reflected by the 4D millimeter-wave radar from the silo: Since the silo is generally made of metal materials, and the reflection intensities of the 4D millimeter-wave radar for different materials are different, remove the data obtained within the reflection intensity range of the 4D millimeter-wave radar for the silo material;

[0074] S3. The point cloud data raster filtering algorithm processes the preprocessed data to obtain the point cloud data on the raster of the global raster map. The raster without data is defined as a blank raster, and the raster with data is defined as a valid raster. Specifically:

[0075] S30. Convert the preprocessed radar data into point cloud data and map it to the silo coordinates.

[0076] S31. Integrate the point cloud data located in the same silo.

[0077] S32. The integrated point cloud data is processed by a raster algorithm for dimensionality reduction, and the maximum value is taken for each raster, thereby obtaining the point cloud data located on the raster. Specifically:

[0078] S320. Use the maximum diameter of the silo + a as the side length to generate a large rectangular surface, where a is about 10% of the silo diameter. For the silos storing coal in some large power plants, the diameter is generally about 100m. In some embodiments, a is taken as 10m.

[0079] S321. Generate continuous rasters with a side length of one-thousandth of the silo diameter in the rectangular surface to form a global raster map. In this embodiment, continuous rasters with a side length of 0.1m are generated in the rectangular surface.

[0080] S322. Project the integrated point cloud data to the raster position, retain the point cloud data with the maximum height in the same raster according to the height, and remove other data in the same raster.

[0081] As Figure 3 shown: S4. Check the integrity of the data. If each raster in the global raster map has data, the data is complete, indicating that there is an error in the operation of step S2 or S3, and the staff is reminded to repair it. If there are blank rasters in the global raster map, use the point cloud smoothing interpolation algorithm to fill the rasters with missing data in the global raster map.

[0082] Specifically, the smoothing interpolation algorithm includes a local smoothing interpolation algorithm and a global smoothing interpolation algorithm. The local smoothing interpolation algorithm and / or the global smoothing interpolation algorithm can be selected to fill the rasters with missing data in the global raster map.

[0083] The specific steps to fill the rasters with missing data using the local smoothing interpolation algorithm are as follows:

[0084] Local smoothing interpolation: Define a local neighborhood for each point in the point cloud, calculate the weighted average position of the points in this neighborhood, and the weight is based on the distance. Closer points have higher weights. As Figure 3 shown, the specific steps are as follows:

[0085] Generate a filtering kernel (kernel) with a side length of 3 grids as a moving window, and define the size of the filtering kernel (kernel) with a side length of 3 grids as 3*3;

[0086] Start from the upper left corner of the global grid map (the [0,0] coordinates of this grid), traverse the rows first, and then the columns; the other grids outside one grid in a moving window are the neighborhood of this grid;

[0087] During the process of moving the moving window, if the neighborhood of any grid in the moving window is a blank grid, expand the filtering kernel (kernel) in the order of 3*3 - 5*5 - 7*7 -... - (2b+1)*(2b+1), 3 < b < 7; ensure that the grids in the moving window during the movement have valid grids, and ensure that the neighborhood of each grid has valid grids;

[0088] Update the height of the data in each grid in the moving window during the movement of the moving window:

[0089] The height of the blank grid in the moving window is calculated as:

[0090] Calculate the weight coefficient of the valid grid:

[0091] Calculate the distance d(i,j) between the blank grid and other valid grids:

[0092]

[0093] where: (x i ,y i ) are the coordinates of the blank grid, and (x j ,y j ) are the coordinates of the valid grid in the neighborhood of the blank grid;

[0094] Process the distance d(i,j) by using the reciprocal method, and calculate the weight coefficient smooth coefficient of the valid grid according to the smoothing coefficient n:

[0095]

[0096] where n is the smoothing coefficient, n is a constant, and n can be adjusted according to the smoothing degree;

[0097] Assume that there are m valid grids relative to the blank grid in a moving window, and the height of this blank grid is updated to h i :

[0098]

[0099] where: h j is the coordinate (xj , y j ) Height of the valid grid.

[0100] The method for filling the data of the missing data grid using the global smoothing interpolation algorithm is as follows:

[0101] Calculate the planar distance d(i, j) between the blank grid and the valid grid:

[0102]

[0103] Where: (x i , y i ) is the coordinate of the blank grid, and (x j , y j ) is the coordinate of the valid grid in the neighborhood of the blank grid;

[0104] Calculate the weight W(i, j) according to the planar distance between the valid grid and a blank grid (center point) and select the radial basis function. The radial odd function in this embodiment is the Gaussian function;

[0105] Calculate the data (height) of the blank grid by weighted average through the data (height) of the valid grid and the corresponding weight:

[0106]

[0107] m is the number of valid grids.

[0108] It should be noted that when using the global smoothing interpolation algorithm and the local smoothing interpolation algorithm to fill the grid data of the missing data in the global grid map, the following can be adopted:

[0109] When the proportion of the number of blank grids in the global grid map is greater than or equal to 20%, first complete the data of the blank grids through the local smoothing interpolation algorithm;

[0110] Then update the data of the blank grids through the global smoothing interpolation algorithm. The input when using the global smoothing interpolation algorithm to update the data of the blank grids includes the data of other blank grids filled by the local smoothing interpolation algorithm.

[0111] In some embodiments, when the proportion of the number of blank grids in the global grid map is less than 20%, fill and update the data in the grid through the local smoothing interpolation algorithm and the global smoothing interpolation algorithm at the same time;

[0112] Specifically, the filtering kernel of the local smoothing interpolation algorithm is completed from the upper left corner of the global grid map, and the global smoothing interpolation algorithm starts from the lower right corner of the global grid map to update the data of each grid. Among them, the grid data updated by the local smoothing interpolation algorithm is the input when the global smoothing interpolation algorithm updates the data later, and the data updated by the global smoothing interpolation algorithm is the input when the local smoothing interpolation algorithm updates the data later.

[0113] Filter according to the maximum and minimum values of the grid height and the points outside the silo to form the final smoothed point cloud. S5. Reconstruct the material surface, and construct a three-dimensional map of the material surface according to the final smoothed point cloud determined in step S4;

[0114] Confirm the relationship between each point and the nearby points, generate triangular patches, and finally form a three-dimensional map of the material surface as shown in Figure 5 Figure

[0115] A 3D intelligent inventory device for silos includes a 4D millimeter-wave radar, an edge processor, a data processor, and a display, which are arranged on the top of the silo. The 4D millimeter-wave radar transmits the detected data to the edge processor, and the edge processor preprocesses the data detected by the 4D millimeter-wave radar. The data processor is embedded with software of a point cloud data grid filtering algorithm, a point cloud smoothing interpolation algorithm, and a material surface reconstruction algorithm. The display is connected to the data processor to display the three-dimensional map of the material surface formed by the data processor, or the cloud is connected to the data processor to facilitate users to view the remaining material in the silo through the network.

[0116] Preferably, a selection module is also embedded in the data processor. When the proportion of blank grids in the global grid map is greater than or equal to 10%, the grid data is first completed by the local smoothing interpolation algorithm, and then the data in the grid is updated by the global smoothing interpolation algorithm; when the proportion of blank grids in the global grid map is less than 10%, the data in the grid is filled and updated by the local smoothing interpolation algorithm and the global smoothing interpolation algorithm at the same time.

[0117] The above is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention shall cover equivalent replacements or changes made according to the technical solution and inventive concept of the present invention within the protection scope of the present invention.

Claims

1. A 3D intelligent disk library method, characterized in that, Used for inventory checking of cylindrical or conical silos. A cover plate is placed on the top of the silo, and c 4D millimeter-wave radars are set on the cover plate, where c is a positive integer. The coverage areas of the c 4D millimeter-wave radars have overlapping parts, and there are uncovered areas of the silo in the coverage of the c 4D millimeter-wave radars; S1. Obtain the data of the corresponding coverage area detected by the 4D millimeter-wave radar; S2. Preprocess the obtained data; S3. Use the point cloud data grid filtering algorithm to process the preprocessed data to obtain the point cloud data on the grid of the global grid map. The grid without data is defined as a blank grid, and the grid with data is defined as a valid grid; S4. Check the integrity of the data. If there are blank grids in the global grid map, use the point cloud smoothing interpolation algorithm to fill the grids with missing data in the global grid map to form the final smooth point cloud; S5. Reconstruct the material surface and construct a three-dimensional map of the material surface according to the final smooth point cloud determined in step S4.

2. The 3D intelligent disc library method according to claim 1, characterized in that Step S2 specifically includes: Removing data points: Determine the maximum detection distance of the 4D millimeter-wave radar according to the height, diameter and shape of the silo, and remove the data greater than the maximum detection distance; Remove the points that are too close according to the height of the silo. Use the formula fmax(0.2, fmin(0.5, silo height * 0.05)) to calculate the minimum distance, and remove the data less than the minimum distance. In the above formula, the unit of the silo height is meters; Remove the data with low signal-to-noise ratio; Remove the data reflected by the 4D millimeter-wave radar on the silo.

3. The 3D intelligent disk library method according to claim 1, wherein Specifically in step S3: S30. Convert the preprocessed radar data into point cloud data and map it to the silo coordinates; S31. Fuse the point cloud data located in the same silo; S32. The fused point cloud data is processed by the grid algorithm for dimensionality reduction, and the maximum value is taken on each grid, so as to obtain the point cloud data located on the grid.

4. The 3D intelligent disc library method according to claim 3, wherein Step S32 is specifically: S320. Use the maximum diameter of the silo + a as the side length to generate a rectangular surface, where a is 5%-12% of the silo diameter; S321. Generate continuous grids of squares with a side length of one-thousandth of the silo diameter in the rectangular surface to form a global grid map; S322. Project the fused point cloud data to the grid position, retain the point cloud data with the maximum height in the same grid according to the height, and remove the other data in the same grid.

5. The 3D intelligent disc library method according to claim 1, wherein, In step S4, the point cloud smoothing interpolation algorithm is a local smoothing interpolation algorithm and / or a global smoothing interpolation algorithm.

6. The 3D intelligent disc library method according to claim 5, characterized in that, The specific steps of the local smoothing interpolation algorithm are as follows: Generate a filtering kernel kernel with a side length of 2b + 1 grids as a moving window, and define the size of the filtering kernel kernel with a side length of 2b + 1 grids as (2b + 1)*(2b + 1), where b is a positive integer; Start from the upper left corner of the global grid map, first traverse the rows, and then traverse the columns; the other grids outside one grid in a moving window are the neighborhood of this grid; Update the height of the data in each grid in the moving window during the movement of the moving window; Calculate the distance d(i,j) between the blank grid in the neighborhood and other valid grids; Where: (x i , y i ) are the coordinates of the blank grid, and (x j , y j ) are the coordinates of the valid grid in the blank grid area; Process the distance d(i,j) by taking the reciprocal, and calculate the weight coefficient smoothcoefficient of the effective grid according to the smoothing coefficient n: Among them, n is the smoothing coefficient, n is a constant, smoothcoefficient j is the weight coefficient of the effective grid of (x j , y j ); Assume that there are m valid grids relative to the blank grid within a moving window, and the height of the blank grid is updated to h i : Where: h j is the height of the effective grid with coordinates (x j , y j ).

7. The 3D intelligent disk library method according to claim 5, characterized in that The method for filling the data of the vacant data grid by using the global smoothing interpolation algorithm is as follows: Calculate the planar distance d(i,j) between the blank grid and the effective grid: where: (x i , y i ) are the coordinates of the blank grid, and (x j , y j ) are the coordinates of the valid grid in the blank grid area; Substitute the planar distance between the effective grid and a blank grid into the radial basis function to calculate the weight W(i,j); Calculate the data of the blank grid by weighted averaging the data of the effective grid and the corresponding weights: m is the number of effective grids.

8. The 3D intelligent disc library method according to claim 5, characterized in that, When the proportion of the number of blank grids in the global grid map is greater than or equal to 20%, first complete the data of the blank grids by using the local smoothing interpolation algorithm.

9. The 3D intelligent disc library method according to claim 5, wherein When the proportion of the number of blank grids in the global grid map is less than 20%, fill and update the data in the grid by using the local smoothing interpolation algorithm and the global smoothing interpolation algorithm at the same time.

10. A 3D intelligent disk library device, characterized in that, For implementing the method according to any one of claims 1-9, including: a 4D millimeter-wave radar arranged at the top of the silo, and further including an edge processor, a data processor and a display; The 4D millimeter-wave radar transmits the detected data to the edge processor, the edge processor preprocesses the data detected by the 4D millimeter-wave radar, the data processor is embedded with software of a point cloud data grid filtering algorithm, a point cloud smoothing interpolation algorithm and a material surface reconstruction algorithm, and the display is connected to the data processor for displaying the three-dimensional map of the material surface formed by the data processor.

Images