DeepLabV3 +-based granary grain storage height quantitative detection method and device

Abstract

The invention relates to a DeepLabV < 3 + >-based granary grain storage height quantitative detection method, which comprises the following steps: carrying out segmentation processing on a picture shot by a camera in a granary to obtain coordinate values of a grain surface edge and upper and lower edges of a ventilation window, and carrying out linear fitting on the coordinate points of the grain surface edge and the upper and lower edges of the ventilation window by utilizing a least square method to obtain a specific equation of each edge; and calculating the actual height corresponding to the unit pixel by combining the grain surface of the granary, the ventilating window and the actual height from the lower edge of the ventilating window to the ground and taking the ventilating window as a reference object, thereby calculating the height of the grain surface. And the grain quantity of the granary can be calculated by combining the grain surface height with the granary size and the grain density. The method is high in data accuracy, low in error, high in reliability and capable of effectively avoiding data counterfeiting and the like, and the grain storage height of the granary can be quickly calculated, so that the specific grain storage quantity of the granary is obtained.

Description

Technical Field

[0001] The present invention relates to the field of grain storage management, and in particular to a method and device for quantitatively detecting the height of stored grain in a grain storage bin based on DeepLabV3+. Background Art

[0002] Food is the basis of human survival, food security is the foundation of national security, and the amount of grain stored in granaries is an important part of food security. Due to my country's vast territory and large population, grain depots are scattered across the country, and traditional methods of monitoring the amount of grain in granaries are mostly manual weighing methods, including direct weighing and random bag inspection. The weighing method is difficult to operate, costly, and may be affected by human factors, resulting in data falsification and other phenomena, making it more difficult to supervise and count the amount of grain, and illegal and irregular phenomena such as unauthorized use of reserve grain, "circling grain", and rotation beyond the idle period are emerging in an endless stream. Summary of the invention

[0003] In order to solve the problems of high cost, heavy workload, and easy data falsification in the existing methods for counting the amount of grain stored in grain warehouses in the prior art, the present invention provides the following technical solutions:

[0004] A method for quantitatively detecting the height of stored grain in a granary based on DeepLabV3+ comprises the following steps:

[0005] S1, build a granary image dataset;

[0006] S2. Input the data set into the DeepLabV3+ network model for training, and use the trained model to segment the grain surface and ventilation windows in the granary image;

[0007] S3. Obtain pixel values ​​of the grain surface edge and the upper and lower edges of the ventilation window by calculation, and calculate the actual height of the grain stored in the granary based on the height of the upper and lower edges of the ventilation window from the ground. Combined with basic data such as the length, width and grain density of the granary, the actual amount of grain stored in the granary is obtained.

[0008] The step S1 comprises:

[0009] S11, collecting multiple pictures inside the granary through the granary monitoring camera;

[0010] S12, screen out images with uniform light, clear images, and no noise or light spot interference factors;

[0011] S13, expanding the data set by translating the image, changing the brightness, and adding noise;

[0012] S14: Adjust the image resolution for manual annotation.

[0013] Furthermore, the resolution size of step S14 is 512×512 pixels, and the LabelMe image labeling tool is used for manual labeling to label the grain surface and ventilation windows in the image, so that the data set only includes three semantic categories: grain surface, ventilation window and background.

[0014] Furthermore, the step S2 comprises:

[0015] S21, establishing a rectangular coordinate system;

[0016] S22, constructing the grain surface and ventilation window coordinate point set;

[0017] S23, using a box plot to eliminate outliers in each coordinate point set, fitting different types of curves using the least squares method, and calculating the coefficient of determination and specific equation of each curve;

[0018] S24, obtaining specific equations for the upper and lower edges of the ventilation window and the edge of the grain surface, and using the average value of the integral to calculate the pixel height of the ventilation window and the pixel height from the lower edge to the grain surface;

[0019] S25. Calculate the conversion coefficient between unit pixel and actual height according to the actual size data of the ventilation window, finally calculate the grain storage height of the granary, and calculate the quantitative error in combination with the actual grain storage height value.

[0020] Furthermore, the step S22 includes identifying the grain surface as a white area and the ventilation window as a black area in the prediction result image, obtaining the coordinate values ​​of the last white pixel, the last and the first black pixel in each column of pixels along the positive direction of the vertical axis, and forming several sets of coordinate points of the upper and lower edges of the grain surface and the ventilation window.

[0021] Further, the step S23 includes:

[0022] After the outliers are removed from the box plot, the pixel points on the upper and lower edges of the grain surface and the ventilation window form several points on the coordinate system UV. For each edge, a curve is set, and the curve equation is obtained using the least squares method.

[0023] Further, the step S24 includes:

[0024] After using least squares to obtain the specific equations of each edge fitting curve, the integral average value of each equation within the value range on the horizontal axis of each ventilation window is used to calculate the difference between the integral average values ​​of the upper and lower edge curves of the ventilation window, and then the average value of the pixel heights of all ventilation windows is calculated as the pixel height h1 of the ventilation window in the image.

[0025]

[0026] Where n represents n ventilation windows, [a i ,b i] is the value range of the i-th ventilation window on the horizontal axis, W 1i (x) is the fitting curve of the lower edge of the i-th ventilation window, W 2i (x) is the fitting curve of the upper edge of the i-th ventilation window;

[0027] Calculate the difference between the integral average of the lower edge curve of the ventilation window and the grain surface curve, and then calculate the average of the heights of the lower edges of all ventilation windows from the grain surface as the pixel height h2 of the lower edge of the ventilation window from the grain surface in the image;

[0028]

[0029] Among them, G(x) is the grain edge fitting curve.

[0030] Furthermore, the step S25 includes

[0031] According to the ratio of the ventilation window pixel height h1 to the actual height H1, combined with the pixel distance h2 from the lower edge of the ventilation window to the grain surface, the distance H2 from the lower edge of the ventilation window to the grain surface is calculated.

[0032]

[0033] The grain surface height X is calculated using the actual height H' of the lower edge of the ventilation window from the ground.

[0034] X=HH'-H2

[0035] Combined with the actual grain storage height H, calculate the grain surface height error δ

[0036]

[0037] The present invention also provides a device for quantitatively detecting the height of stored grain in a granary based on DeepLabV3+, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processing process implemented by the processor when executing the computer program comprises the following steps:

[0038] S1, build a granary image dataset;

[0039] S2. Input the data set into the DeepLabV3+ network model for training, and use the trained model to segment the grain surface and ventilation windows in the granary image;

[0040] S3. Obtain pixel values ​​of the grain surface edge and the upper and lower edges of the ventilation window by calculation, and calculate the actual height of the grain stored in the granary based on the height of the upper and lower edges of the ventilation window from the ground. Combined with basic data such as the length, width and grain density of the granary, the actual amount of grain stored in the granary is obtained.

[0041] The beneficial effects of the present invention are: the quantitative detection of grain storage height in a granary based on DeepLabV3+ has high data accuracy, low error, avoids data falsification, has high reliability, and high accuracy of detection results, and can quickly calculate the storage height of the granary, and then obtain the specific amount of grain stored in the granary; the present invention collects images through existing surveillance cameras in existing granaries, thereby reducing the cost of installing other granary grain quantity monitoring equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 Flowchart of the present invention;

[0043] Figure 2 Cartesian coordinate system diagram of granary image;

[0044] Figure 3 Schematic diagram of the calculation variables for grain height; DETAILED DESCRIPTION

[0045] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0046] Reference Figure 1 The experiment is divided into two parts: model training and model reasoning. Model training includes taking pictures of granaries, making data sets, expanding data sets using data enhancement methods, adjusting picture formats, labeling pictures, training models, and obtaining models. Model reasoning includes selecting test pictures, collecting granary data, putting pictures into the obtained model for prediction and obtaining segmentation results, and judging whether the segmentation results are accurate. If not, adjust the model parameters and then conduct training. If accurate, complete the entire experiment by obtaining the edge coordinates of the grain surface and ventilation windows, eliminating outliers, fitting regression curves, and calculating grain height and errors.

[0047] The dataset is constructed by manually screening the pictures taken randomly by granary monitoring equipment across the country. Due to shooting environment problems such as equipment failure, strong or dark light, it is necessary to screen out images with uniform light, no noise, blur, light spots and other interference factors. Due to experimental needs, pictures with smooth and clear edges of grain surface and ventilation window and no excessive debris on the grain surface are selected in the screening process to be added to the dataset. In order to achieve better image segmentation effect, most of the datasets select images with the camera shooting direction facing the ventilation window or with a small oblique angle to avoid the ventilation window being too small in the picture, which affects subsequent picture annotation and other work. The dataset is expanded by translating, changing the brightness and adding noise to the pictures through data enhancement tools, and finally there are 4980 pictures as the dataset of this experiment. The dataset is divided according to a ratio of about 8:1:1, that is, there are 4032 pictures in the training set, 499 pictures in the validation set, and 449 pictures in the test set. In order to carry out the subsequent experiments smoothly, it is necessary to adjust the resolution of all images in the dataset to 512×512 pixels, and then use the LabelMe image labeling tool for manual labeling to mark the grain surface and ventilation windows in the image, so that the dataset only includes three semantic categories, namely grain surface, ventilation windows and background. At this point, the granary image dataset is constructed and can then be input into the network model for training to obtain semantic information.

[0048] Since the subsequent experiments need to obtain pixel coordinate values, it is necessary to establish a rectangular coordinate system in the image. Figure 2 , with the lower left corner of the image as the origin, the pixel as the unit, the horizontal axis to the right, and the vertical axis upward as the positive direction to establish the UV rectangular coordinate system, and the value range of u and v is (0, 512). In the image prediction result, the grain surface is identified as a white area, and the ventilation window is identified as a black area. The coordinate values ​​of the last white pixel, the last and the first black pixel in each column of pixels are obtained along the positive direction of the vertical axis to form several grain surface and ventilation window upper and lower edge coordinate point sets.

[0049] The box plot is a statistical graph that reflects the distribution characteristics of discrete data. It is often used for data outlier statistics, data distribution skewness, and tail weight judgment. The least squares method requires the establishment of a continuous curve in a discrete point set so that the curve is as close to all data points as possible. The principle of minimizing the sum of squared errors from known data points to the curve is usually used to control the fitting accuracy. The grain surface and the upper and lower edges of the ventilation window are fitted with curves. It is observed that the shapes of the grain surface and the ventilation window edges are relatively simple and close to straight lines, so only linear fitting is considered. After the outliers are removed from the box plot, the pixel points of the upper and lower edges of the grain surface and the ventilation window form several points P on the coordinate system. i (x i ,y i ), (i=1,2,…,m), the process of calculating each edge fitting curve using the least squares method is as follows:

[0050]

[0051] Where: φ(x) is the fitting curve of each edge; Q is the sum of square errors from each discrete point to the curve. Solving the equations in the formula one by one, we can get the specific equation of the curve.

[0052] Reference Figure 3 , let the actual height of the ventilation window of the granary be H1, the actual height of the lower edge of the ventilation window from the ground be H', and the actual height of the grain stored in the granary be H. Let the pixel height of the ventilation window in the image be h1, and the pixel height from the lower edge of the ventilation window to the grain surface be h2. By calculation, the height from the lower edge of the ventilation window to the grain surface is H2, and the grain surface height is X. There are n ventilation windows in the image.

[0053] After using least squares to obtain the specific equations of each edge fitting curve, the integral average value of each equation within the value range on the horizontal axis of each ventilation window is used to calculate the difference between the integral average values ​​of the upper and lower edge curves of the ventilation window, and then the average value of the pixel heights of all ventilation windows is calculated as the pixel height h1 of the ventilation window in the image.

[0054]

[0055] Where n represents n ventilation windows, [a i ,b i ] is the value range of the i-th ventilation window on the horizontal axis, W 1i (x) is the fitting curve of the lower edge of the i-th ventilation window, W 2i (x) is the fitting curve of the upper edge of the i-th ventilation window;

[0056] Calculate the difference between the integral average of the lower edge curve of the ventilation window and the grain surface curve, and then calculate the average of the heights of the lower edges of all ventilation windows from the grain surface as the pixel height h2 of the lower edge of the ventilation window from the grain surface in the image;

[0057]

[0058] Among them, G(x) is the grain edge fitting curve.

[0059] According to the ratio of the ventilation window pixel height h1 to the actual height H1, combined with the pixel distance h2 from the lower edge of the ventilation window to the grain surface, the distance H2 from the lower edge of the ventilation window to the grain surface is calculated.

[0060]

[0061] The grain surface height X is calculated using the actual height H' of the lower edge of the ventilation window from the ground.

[0062] X=H'-H2

[0063] Combined with the actual grain storage height H, calculate the grain surface height error δ

[0064]

[0065] The above-mentioned method for quantitatively detecting the height of stored grain in a grain silo can also be used as a computer program, which is set in the memory of the device for quantitatively detecting the height of stored grain in a grain silo and executed by the processor in the device for quantitatively detecting the height of stored grain in a grain silo.

[0066] Specific implementation methods are given above, but the present invention is not limited to the described implementation methods. The basic idea of ​​the present invention lies in the above basic scheme. For ordinary technicians in this field, it does not take creative work to design various deformed models, formulas, and parameters according to the teachings of the present invention. Changes, modifications, substitutions, and variations of the implementation methods without departing from the principles and spirit of the present invention still fall within the scope of protection of the present invention.

Claims

1. A method for quantitative detection of grain storage height in granaries based on DeepLabV3+, characterized in that: The steps include: S1, build a granary image dataset; S2. Input the data set into the DeepLabV3+ network model for training, and use the trained model to segment the grain surface and ventilation windows in the granary image; S3. Obtain pixel values ​​of the grain surface edge and the upper and lower edges of the ventilation window by calculation, and calculate the actual height of the grain stored in the granary based on the height of the upper and lower edges of the ventilation window from the ground. Combined with basic data such as the length, width and grain density of the granary, the actual amount of grain stored in the granary is obtained.

2. A method for quantitatively detecting the height of stored grain in a granary based on DeepLabV3+ as claimed in claim 1, characterized in that: The step S1 comprises: S11, collecting multiple pictures inside the granary through the granary monitoring camera; S12, screen out images with uniform light, clear images, and no noise or light spot interference factors; S13, expanding the data set by translating the image, changing the brightness, and adding noise; S14: Adjust the image resolution for manual annotation.

3. A method for quantitatively detecting the height of stored grain in a granary based on DeepLabV3+ as claimed in claim 1, characterized in that: The resolution size of step S14 is 512×512 pixels, and the LabelMe image labeling tool is used for manual labeling to label the grain surface and ventilation windows in the image, so that the data set only includes three semantic categories: grain surface, ventilation windows and background.

4. A method for quantitatively detecting the height of stored grain in a granary based on DeepLabV3+ as claimed in claim 1, characterized in that: The step S2 comprises: S21, establishing a rectangular coordinate system; S22, constructing the grain surface and ventilation window coordinate point set; S23, using a box plot to eliminate outliers in each coordinate point set, fitting different types of curves using the least squares method, and calculating the coefficient of determination and specific equation of each curve; S24, obtaining specific equations for the upper and lower edges of the ventilation window and the edge of the grain surface, and using the average value of the integral to calculate the pixel height of the ventilation window and the pixel height from the lower edge to the grain surface; S25. Calculate the conversion coefficient between unit pixel and actual height according to the actual size data of the ventilation window, finally calculate the grain storage height of the granary, and calculate the quantitative error in combination with the actual grain storage height value.

5. A method for quantitatively detecting the height of stored grain in a granary based on DeepLabV3+ as claimed in claim 4, characterized in that: The step S22 includes identifying the grain surface as a white area and the ventilation window as a black area in the prediction result graph, obtaining the coordinate values ​​of the last white pixel point, the last black pixel point and the first black pixel point in each column of pixels along the positive direction of the longitudinal axis, and forming a plurality of grain surface and ventilation window upper and lower edge coordinate point sets.

6. A method for quantitatively detecting the height of stored grain in a granary based on DeepLabV3+ as claimed in claim 4, characterized in that: The step S23 comprises: After the outliers are removed from the box plot, the pixel points on the upper and lower edges of the grain surface and ventilation windows form several points on the coordinate system UV. For each edge, a curve is set, and the curve equation is obtained using the least squares method.

7. A method for quantitatively detecting the height of stored grain in a granary based on DeepLabV3+ as claimed in claim 4, characterized in that: The step S24 comprises: After using least squares to obtain the specific equations of each edge fitting curve, the integral average value of each equation within the value range on the horizontal axis of each ventilation window is used to calculate the difference between the integral average values ​​of the upper and lower edge curves of the ventilation window, and then the average value of the pixel heights of all ventilation windows is calculated as the pixel height h1 of the ventilation window in the image. Where n represents n ventilation windows, [a i ,b i ] is the value range of the i-th ventilation window on the horizontal axis, W 1i (x) is the fitting curve of the lower edge of the i-th ventilation window, W 2i (x) is the fitting curve of the upper edge of the i-th ventilation window; Calculate the difference between the integral average of the lower edge curve of the ventilation window and the grain surface curve, and then calculate the average of the heights of the lower edges of all ventilation windows from the grain surface as the pixel height h2 of the lower edge of the ventilation window from the grain surface in the image; Among them, G(x) is the grain edge fitting curve.

8. The method for quantitatively detecting the height of stored grain in a granary based on DeepLabV3+ according to claim 1, characterized in that: The step S25 includes According to the ratio of the ventilation window pixel height h1 to the actual height H1, combined with the pixel distance h2 from the lower edge of the ventilation window to the grain surface, the distance H2 from the lower edge of the ventilation window to the grain surface is calculated. The grain surface height X is calculated using the actual height H' of the lower edge of the ventilation window from the ground. X=H′-H2 Combined with the actual grain storage height H, calculate the grain surface height error δ 9. A quantitative detection device for grain storage height in a granary based on DeepLabV3+, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: The processing process implemented by the processor when executing the computer program includes the following steps: S1, build a granary image dataset; S2. Input the data set into the DeepLabV3+ network model for training, and use the trained model to segment the grain surface and ventilation windows in the granary image; S3. Obtain pixel values ​​of the grain surface edge and the upper and lower edges of the ventilation window by calculation, and calculate the actual height of the grain stored in the granary based on the height of the upper and lower edges of the ventilation window from the ground. Combined with basic data such as the length, width and grain density of the granary, the actual amount of grain stored in the granary is obtained.

Images