Automatic grain loading and unloading detection system based on computer vision

Abstract

The invention discloses an automatic grain loading and unloading detection system based on computer vision, and belongs to the field of computer vision. The system comprises a container area identification module, a grain area segmentation module and a full state monitoring module. The container area identification module realizes accurate positioning of a container area through detection and correction of a specific identifier, and ensures the working stability at different angles and in dynamic environments; the grain region segmentation module extracts significant features of a grain region through combination of color filtering and edge detection, and adopts morphological closed operation and connectivity analysis; and the full state monitoring module is used for distinguishing a full area and a non-full area in real time according to the relationship between the grain edge and the warning line, and controlling the grain unloading process, so that the condition of non-uniform loading or overflow is avoided. The system can accurately detect the distribution condition of grains in the cargo compartment and send out an adjustment prompt, so that the uniformity of grain loading is ensured, and the unbalance risk in the transportation process is reduced.

Description

Technical Field

[0001] The invention belongs to the field of computer vision, and in particular relates to an automatic grain loading and unloading detection system based on computer vision. Background Art

[0002] Grain loading and unloading is a key step in agricultural logistics, and its operation efficiency and accuracy directly affect the subsequent production and transportation links. Traditional manual operations have problems of low efficiency and poor accuracy.

[0003] At present, grain loading and unloading relies on manual identification. Long-term loading and unloading monitoring requires high physical strength and attention from operators, which can easily cause fatigue, especially during high-intensity continuous operations. This method is heavily dependent on manpower and has a low level of automation standardization.

[0004] Especially when unloading grain on the move, there are great defects. Unloading grain on the move requires cooperation from multiple parties (such as drivers and observers), and the manual identification and synchronous adjustment process is complicated. Poor communication or collaboration errors may lead to loading and unloading deviations. When the working conditions are complex (such as uneven terrain and bumpy vehicles), operators handling multiple tasks at the same time can easily lead to untimely judgments or monitoring blind spots.

[0005] At the same time, some domestic and foreign grain storage management companies have realized the monitoring of grain pile status in the grain storage by combining cameras with AI algorithms. However, grain loading and unloading, especially loading and unloading on the move, is dynamic. If the existing pre-trained model of image segmentation is directly used and fine-tuned, it will be relatively bloated. When processing video streams, it is easy to reduce real-time performance due to the large amount of calculation. This is not suitable for grain loading and unloading, which requires high real-time performance. If both are taken into account, the hardware requirements will be high, and the cost cannot meet the needs of ordinary farmers.

[0006] Moreover, deep learning algorithms rely on large amounts of labeled data, while grain loading and unloading detection requires diverse scenarios and high-quality pixel-level annotations, which places high demands on time and labor costs. Summary of the invention

[0007] The purpose of the present invention is to solve the problems of low efficiency and poor precision in traditional manual grain loading and unloading operations, and to provide an automated grain loading and unloading detection system based on computer vision.

[0008] To achieve the above purpose, the technical solution adopted by the present invention is as follows:

[0009] An automated grain loading and unloading detection system based on computer vision, the system comprising a cargo box area recognition module, a grain area segmentation module and a filling status monitoring module;

[0010] The cargo box area identification module realizes accurate positioning of the cargo box area through detection and correction of specific identifiers, ensuring stability in operation at different angles and in dynamic environments;

[0011] The grain area segmentation module extracts the significant features of the grain area by combining color filtering and edge detection.

[0012] Morphological closing operations and connectivity analysis are used to ensure the integrity and accuracy of the segmentation results;

[0013] The filling status monitoring module distinguishes between full and unfilled areas in real time according to the relationship between the grain edge and the warning line, and controls the grain unloading process to avoid uneven loading or overflow.

[0014] The beneficial effects of the present invention compared to the prior art are as follows: the system can analyze the loading conditions of different areas in the cargo compartment in real time, quickly discover and prompt the full or underloaded status of local areas, and avoid the lag and response delay problems of manual monitoring. The system can accurately detect the distribution of grain in the cargo compartment and issue adjustment prompts to ensure the uniformity of grain loading and reduce the risk of imbalance during transportation. It avoids misjudgment caused by manual fatigue or visual interference, significantly reduces the waste and economic losses caused by grain overflow or partial underloading, and reduces the burden on workers. It solves the defect that AI detection relies on a large amount of labeled data and is costly. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 A complete process rendering of grain area segmentation;

[0016] Figure 2 Run a flowchart for the system;

[0017] Figure 3 This is a general test chart;

[0018] Figure 4 This is a test image under shadow;

[0019] Figure 5 This is a test image blocked by a thin railing. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0021] The present invention is an automated grain loading and unloading detection system based on computer vision. The system includes three core modules: cargo box area recognition, grain area segmentation, and filling status monitoring. It can achieve automatic recognition, precise control, and efficient operation, significantly improving the automation level and reliability of the grain loading and unloading process.

[0022] The cargo box area recognition module detects and corrects specific identifiers to accurately locate the cargo box area, ensuring stability in operation at different angles and in dynamic environments. The grain area segmentation module extracts the significant features of the grain area through a combination of color filtering and edge detection, and uses morphological closing operations and connectivity analysis to ensure the integrity and accuracy of the segmentation results. The filling status monitoring module distinguishes between full and unfilled areas in real time based on the relationship between the grain edge and the warning line, and controls the grain unloading process to avoid uneven loading or overflow.

[0023] The present invention will study the segmentation of grain area and cargo box area, and elaborate on the design and implementation of the system from three modules: cargo box area identification, grain area segmentation, and filling status monitoring. It will analyze the technical details and performance of each module, and explore the application effect of the system in complex practical scenarios and future optimization directions.

[0024] In this invention, an automated grain loading and unloading detection system based on computer vision and image processing technology is proposed and implemented to solve the efficiency and accuracy problems in grain loading and unloading. The system effectively improves the automation level and operation accuracy of the loading and unloading process by combining accurate identification of cargo box areas, segmentation and positioning of grain areas, and dynamic distinction between filled and unfilled areas.

[0025] First, the system uses identifiers of specific colors and shapes to identify the cargo box area. This technology ensures that the cargo box area can be accurately located under different environmental conditions and provides a reliable basis for subsequent grain area identification. Through the real-time correction mechanism, the system can automatically adapt to the dynamic changes of the cargo box area, further improving the stability and robustness of the system.

[0026] Secondly, the grain area recognition effectively extracts the grain area boundary through image segmentation and convex hull detection technology, and removes noise and unnecessary details through morphological operations, making the grain area outline clearer and more accurate. This module provides support for the accurate judgment of grain areas during loading and unloading, and provides sufficient information for subsequent status judgment.

[0027] Finally, the module for distinguishing between full and unfilled areas realizes dynamic detection of grain loading status by setting warning lines and determining the intersection of the grain area and the warning lines. When the grain area reaches the preset warning line, the system can automatically determine that the area is full and stop unloading grain, avoiding overflow or uneven loading and unloading, and ensuring the accuracy and safety of the unloading process.

[0028] In general, the grain loading and unloading detection system of the present invention solves the problems of large labor demand and low efficiency in traditional manual operation, and has a high level of automation. The system has shown good stability and adaptability in practical applications, can effectively improve the efficiency and accuracy of grain loading and unloading operations, and has broad application prospects. With the continuous development and optimization of technology, the system can also be combined with more intelligent algorithms and sensors in the future to further improve its degree of automation and intelligence, and provide stronger technical support for agricultural logistics and automated operations.

[0029] The detection system of the present invention is mainly composed of the following three modules:

[0030] 1. Cargo box area interception module

[0031] The cargo box area interception module uses computer vision technology to detect and locate the identifiers by attaching specific color and shape identifiers to the four corners of the cargo box. The system uses the four detected identifiers as vertices and intercepts the resulting quadrilateral area as the cargo box working area.

[0032] Preprocessing stage: Before the detection starts, complete the accurate identification of identifiers and establish the initial cargo box area model.

[0033] Real-time correction stage: During the detection process, the system tracks and updates the identifiers of consecutive frames, dynamically adjusts the positioning of the cargo area, and ensures that deviations caused by environmental changes or equipment jitter are corrected in real time.

[0034] 2. Grain area identification module

[0035] This module performs segmentation and recognition based on the difference in characteristics between the cargo box area and the grain area, extracts the grain area and accurately locates it. In the cargo box working area, the grain area is segmented from the background using color, texture or shape features to generate a binary image. The convex hull detection algorithm is applied to the binary image to calculate and depict the boundary of the grain area to obtain a complete area outline.

[0036] 3. Module for distinguishing between filled and unfilled areas

[0037] The module for distinguishing between full and underfilled areas determines the grain loading status by setting the warning line ratio parameter k and provides dynamic guidance for underfilled areas. Taking the long side of the cargo box as a reference, two warning lines parallel to the long side are drawn in the cargo box area according to the ratio parameter k to demarcate the boundary between the "full area" and the "underfilled area". The positional relationship between the convex hull boundary and the warning line of the grain area in the "underfilled area" is detected. If the convex hull intersects with any warning line, it is judged that the grain area has reached the warning line and the part is marked as the "full area".

[0038] The system dynamically calculates the boundary range of the full area and the unfilled area in real time, and feeds it back to the unloading device through marking. Then, the unloading operation is stopped for the full area, and the unloading pipe is guided to move to the unfilled area for operation. If all unfilled areas are marked as full, the system determines that the cargo box is completely full and stops the unloading operation.

[0039] The overall system process is as follows Figure 2 As shown, the general test is as follows Figure 3 As shown, the test under full shadow is as follows Figure 4 As shown, the test under the small railing is as follows Figure 5 shown.

[0040] Embodiment 1:

[0041] An automated grain loading and unloading detection system based on computer vision, the system comprising a cargo box area recognition module, a grain area segmentation module and a filling status monitoring module;

[0042] The cargo box area identification module realizes accurate positioning of the cargo box area through detection and correction of specific identifiers, ensuring stability in operation at different angles and in dynamic environments;

[0043] The grain area segmentation module extracts the significant features of the grain area by combining color filtering and edge detection, and uses morphological closing operations and connectivity analysis to ensure the integrity and accuracy of the segmentation results;

[0044] The filling status monitoring module distinguishes between full and unfilled areas in real time according to the relationship between the grain edge and the warning line, and controls the grain unloading process to avoid uneven loading or overflow.

[0045] The specific process of image brightness balancing for grain area and cargo box area segmentation is as follows:

[0046] Step 1: Image reading and color space conversion: Read the original image and convert it to the HSV color space. The V channel in the HSV space contains brightness information and can be used alone for brightness adjustment.

[0047] Step 2: Segmentation of bright and dark areas: Binarize the brightness channel using the Otsu threshold method to generate masks of bright and dark areas. Process the masks through morphological closing operations to ensure that the dark areas are connected areas.

[0048] Step 3: Calculate the average brightness of the bright area: Use the mask to calculate the average brightness V of the bright area blight , as the target brightness for adjusting the brightness of the dark area;

[0049] Step 4: Dark area connected region analysis and brightness ratio improvement: Use connected region analysis to extract each independent dark area connected region, and calculate the average brightness of each dark area. And determine the scaling factor k (i) ;

[0050] Step 5: Scale the brightness of all pixels in the dark area by the proportional factor k (i) Raise it so that the dark area is the same brightness as the light area while maintaining the contrast between light and dark inside.

[0051] The image brightness balance processing in the grain area and cargo box area segmentation is as follows:

[0052] In natural conditions, there will be shadow areas caused by the angle of sunlight, which will affect the accuracy of subsequent edge detection. Therefore, it is necessary to eliminate the shadow part of the image or increase the brightness of the image. This algorithm proportionally increases the brightness of the dark area in the image, making the average brightness of the dark area close to that of the bright area, while maintaining the light-dark ratio within the dark area. This method is suitable for images containing multiple independent dark areas. By automatically identifying and processing each dark area separately, the overall brightness of the image is more balanced.

[0053] The algorithm is based on the brightness information of the V channel of the HSV color space. Through binarization and connected area analysis, the image is divided into bright areas and dark areas, and the brightness of the dark areas is increased proportionally.

[0054] Assume that the average brightness of the bright area in the image brightness channel is The average brightness of the i-th dark area is The brightness increase factor k (i) for:

[0055]

[0056] The adjustment of the pixel brightness in each dark area is calculated as:

[0057]

[0058] Among them, V max Make sure that the adjusted brightness does not exceed the upper threshold of the image brightness range; (x, y) refers to the pixel point.

[0059] In the grain area segmentation module, the specific segmentation steps are:

[0060] Step 1: x and y direction gradient calculation: Use the Sobel operator to calculate the gradient of the image in the x and y directions;

[0061] Step 2: Gradient amplitude calculation: Calculate the gradient amplitude to represent the grayscale change amplitude of the image at each pixel position;

[0062] Step 3: Binarization of gradient amplitude: Use the threshold interval to binarize the gradient amplitude and highlight the area with amplitude within the threshold range.

[0063] In the grain area segmentation module, the gradient calculation of the Sobel operator is as follows:

[0064] It can be observed that the edge density of images of grain piles of field crops such as corn, wheat, soybeans, and rice is very high, that is, the image gradient changes frequently, and in most cases is much higher than that of cargo box images.

[0065] Calculate the gradient amplitude of the image and binarize it to highlight the edge area where the grayscale changes significantly in the image. This method uses the Sobel operator to calculate the gradient in the x and y directions respectively, obtains the gradient amplitude map, and then performs binarization processing according to the set range. It is suitable for edge detection and enhancement applications;

[0066] Use the Sobel operator to calculate the gradient of the image in the x and y directions respectively. The gradient amplitude can represent the degree of grayscale change of the image at that position. Suppose the components of the image gradient in the x and y directions are G x and G y , then the gradient amplitude G is expressed as the square root of the sum of the two squares;

[0067] In actual calculations, the gradient map often contains high-frequency noise. At the same time, since the edge of the cargo box is often a strong edge in the actual scene, in order to remove the interference between the noise and the edge of the cargo box, this module uses a binarization operation to only retain pixels with amplitudes within the specified range. The binarization operation is defined as:

[0068]

[0069] Among them, G binary (x, y) is the binary gradient map.

[0070] In the grain area segmentation module, the specific process of color filtering is as follows:

[0071] The colors of field crops are mostly yellow-brown and green, so the grain area can be found by color filtering. The study uses a color-based detection method to accurately identify the grain in the grain transport vehicle by extracting the narrow yellow-brown area in the image.

[0072] Step 1: Convert the original image from BGR space to HSV space to enhance the description ability of the target color features;

[0073] Step 2: Set the HSV value range of yellow-brown and extract the pixel area that meets the color characteristics;

[0074] Step 3: Generate a mask according to the set color range, and superimpose the mask to form a complete yellow-brown target area;

[0075] Step 4: Use the mask to generate a binary image, where the target area is white and the rest of the area is black;

[0076] Step 5: Output the grayscale binary image. The white part indicates that the detected grain is a yellow-brown area within a limited HSV range, which is resistant to environmental interference, but also reduces the grain area recognition rate. The final output result shows that the grain area is finely fragmented.

[0077] The specific steps of the morphology and connectivity constraints of the dual-channel graph are as follows:

[0078] Step 1: Perform AND operation on the two binary images after edge detection and color filtering;

[0079] Step 2: Close and then open the binary image to connect the grain edges, remove the interference of environmental cracks, and restore the grain area;

[0080] Step 3: By analyzing the spatial connectivity of pixels, identify the connected regions in the image, calculate the area of ​​all connected regions, and filter out the connected regions larger than the threshold as the final target region;

[0081] Step 4: Output the processed single-channel grayscale image, where the white part is the grain area.

[0082] Morphology and connectivity constraints of dual-channel graphs in grain area segmentation module

[0083] The two binary images after edge detection and color filtering are logically ANDed pixel by pixel to obtain a binary image, in which the white area is the yellow-brown edge in the original image; the binary image is further processed to extract the final target area through morphological closing operation, morphological opening operation and connectivity analysis; the morphological closing operation connects scattered edges and fills small holes to make the target area more complete. The closing operation refers to dilating the image first and then eroding it. By selecting a suitable convolution kernel to connect scattered areas and eliminate small holes, for input I and structural element B, the closing operation is:

[0084]

[0085] The opening operation is to corrode first and then expand to remove the thin lines on the main connected areas. In real scenarios, these may be some small yellow-brown vertical cracks on the inner wall of the cargo box. The opening operation is as follows:

[0086]

[0087] Then, connectivity analysis is used to intercept connected areas that are smaller than the set threshold to reduce background interference.

Claims

1. An automated grain loading and unloading detection system based on computer vision, characterized in that: The system includes a cargo box area identification module, a grain area segmentation module and a filling status monitoring module; The cargo box area identification module realizes accurate positioning of the cargo box area through detection and correction of specific identifiers, ensuring stability in operation at different angles and in dynamic environments; The grain area segmentation module extracts the significant features of the grain area by combining color filtering and edge detection, and uses morphological closing operations and connectivity analysis to ensure the integrity and accuracy of the segmentation results; The filling status monitoring module distinguishes between full and unfilled areas in real time according to the relationship between the grain edge and the warning line, and controls the grain unloading process to avoid uneven loading or overflow.

2. The computer vision-based automated grain loading and unloading detection system according to claim 1, characterized in that: The specific process of image brightness balancing for grain area and cargo box area segmentation is as follows: Step 1: Image reading and color space conversion: Read the original image and convert it to the HSV color space. The V channel in the HSV space contains brightness information and can be used alone for brightness adjustment. Step 2: Segmentation of bright and dark areas: Binarize the brightness channel using the Otsu threshold method to generate masks of bright and dark areas. Process the masks through morphological closing operations to ensure that the dark areas are connected areas. Step 3: Calculate the average brightness of the bright area: Use the mask to calculate the average brightness of the bright area As the target brightness for adjusting the brightness of dark areas; Step 4: Dark area connected region analysis and brightness ratio improvement: Use connected region analysis to extract each independent dark area connected region, and calculate the average brightness of each dark area. And determine the scaling factor k (i) ; Step 5: Scale the brightness of all pixels in the dark area by the proportional factor k (i) Raise it so that the dark area is the same brightness as the light area while maintaining the contrast between light and dark inside.

3. The computer vision-based automated grain loading and unloading detection system according to claim 1 or 2, characterized in that: The image brightness balance processing in the grain area and cargo box area segmentation is as follows: The algorithm is based on the brightness information of the V channel of the HSV color space. Through binarization and connected area analysis, the image is divided into bright areas and dark areas, and the brightness of the dark areas is increased proportionally. Assume that the average brightness of the bright area in the image brightness channel is The average brightness of the i-th dark area is The brightness increase factor k (i) for: The adjustment of the pixel brightness in each dark area is calculated as: Among them, V max Make sure that the adjusted brightness does not exceed the upper threshold of the image brightness range; (x, y) refers to the pixel point.

4. The computer vision-based automated grain loading and unloading detection system according to claim 1, characterized in that: In the grain area segmentation module, the specific segmentation steps are: Step 1: x and y direction gradient calculation: Use the Sobel operator to calculate the gradient of the image in the x and y directions; Step 2: Gradient amplitude calculation: Calculate the gradient amplitude to represent the grayscale change amplitude of the image at each pixel position; Step 3: Binarization of gradient amplitude: Use the threshold interval to binarize the gradient amplitude and highlight the area with amplitude within the threshold range.

5. The computer vision-based automated grain loading and unloading detection system according to claim 1 or 4, characterized in that: In the grain area segmentation module, the gradient calculation of the Sobel operator is as follows: Assume that the components of the image gradient in the x and y directions are G x and G y , then the gradient amplitude G is expressed as the square root of the sum of the two squares; The binarization operation is used to retain only the pixels whose amplitude is within the specified range. The binarization operation is defined as: Among them, G binary (x, y) is the binary gradient map.

6. The computer vision-based automated grain loading and unloading detection system according to claim 1, characterized in that: In the grain area segmentation module, color filtering The specific process is as follows: Step 1: Convert the original image from BGR space to HSV space to enhance the description ability of the target color features; Step 2: Set the HSV value range of yellow-brown and extract the pixel area that meets the color characteristics; Step 3: Generate a mask according to the set color range, and superimpose the mask to form a complete yellow-brown target area; Step 4: Use the mask to generate a binary image, where the target area is white and the rest of the area is black; Step 5: Output the grayscale binary image. The white part indicates that the detected grain is a yellow-brown area within the limited HSV interval. The output result shows that the grain area is finely fragmented.

7. The computer vision-based automated grain loading and unloading detection system according to claim 1 or 6, characterized in that: The specific steps of the morphology and connectivity constraints of the dual-channel graph are as follows: Step 1: Perform AND operation on the two binary images after edge detection and color filtering; Step 2: Close and then open the binary image to connect the grain edges, remove the interference of environmental cracks, and restore the grain area; Step 3: By analyzing the spatial connectivity of pixels, identify the connected regions in the image, calculate the area of ​​all connected regions, and filter out the connected regions larger than the threshold as the final target region; Step 4: Output the processed single-channel grayscale image, where the white part is the grain area.

8. The computer vision-based automated grain loading and unloading detection system according to claim 1, 6 or 7, characterized in that: Morphology and connectivity constraints of dual-channel graphs in grain area segmentation module The two binary images after edge detection and color filtering are logically ANDed pixel by pixel to obtain a binary image, in which the white area is the yellow-brown edge in the original image; the binary image is further processed to extract the final target area through morphological closing operation, morphological opening operation and connectivity analysis; the morphological closing operation connects scattered edges and fills small holes to make the target area more complete. The closing operation refers to dilating the image first and then eroding it. By selecting a suitable convolution kernel to connect scattered areas and eliminate small holes, for input I and structural element B, the closing operation is: The opening operation is to corrode first and then expand to remove the thin lines on the main connected areas. In real scenarios, these may be some small yellow-brown vertical cracks on the inner wall of the cargo box. The opening operation is as follows: Then, connectivity analysis is used to intercept connected areas that are smaller than the set threshold to reduce background interference.

Images