Cargo falling detection method based on edge calculation

Abstract

The invention discloses a goods falling detection method based on edge calculation, relates to the technical field of image processing and edge calculation, and aims to realize real-time monitoring of goods falling behaviors in a fixed area by performing real-time processing and analysis on image data at a data source by utilizing an edge calculation technology. According to the method, the real-time performance can be achieved, data are locally processed and analyzed through the edge computing technology, the real-time monitoring of the goods falling behavior is achieved, and the response time is greatly shortened; and due to no need of a large number of sensor devices, the method based on edge calculation can be flexibly arranged, is suitable for monitoring requirements of various different fixed areas, and is simple, effective and easy to use.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing and edge computing, and specifically is a cargo drop detection method based on edge computing. Background Art

[0002] In the production, warehousing and logistics industries, real-time monitoring and detection of cargo drop events are crucial. Traditional cargo drop detection methods mainly rely on manual inspections or sensor equipment, which have the following problems and challenges:

[0003] Reliance on manual inspection: Traditional cargo drop detection usually requires manual inspection on a regular basis, which consumes manpower and material resources and is prone to omissions and delays;

[0004] Sensor limitations: Traditional sensor equipment usually requires a lot of wiring and equipment installation, which is not only costly but also inflexible in deployment and cannot meet the real-time monitoring needs of specific areas;

[0005] Insufficient data processing capabilities: Traditional monitoring equipment often transmits large amounts of data to central servers for processing, resulting in heavy network transmission pressure and high latency, which cannot meet the requirements of real-time monitoring.

[0006] Based on the above defects, the present application needs to provide a cargo drop detection method based on edge computing. By utilizing edge computing technology, image data can be processed and analyzed in real time at the data source, so as to realize real-time monitoring of cargo drop behavior in a fixed area. Summary of the invention

[0007] The present invention aims to solve at least one of the technical problems existing in the prior art; to this end, the present invention proposes a cargo drop detection method based on edge computing, which specifically includes the following steps:

[0008] Set an initial background image, set a region of interest ROI on the background image, and initialize edge detection of the background image and ROI; obtain real-time images through a camera;

[0009] Put the real-time image and initialized edge detection data into the queue;

[0010] Take out the earliest historical image from the queue storing historical images and compare it with the real-time image;

[0011] The image after the background image update strategy is compared with the real-time image again, and edge detection is performed; finally, the target contour is output according to the detection results.

[0012] Compared with the prior art, the present invention has the following beneficial effects:

[0013] Compared with traditional methods, the present invention has the following advantages: real-time performance: edge computing technology enables data to be processed and analyzed locally, realizing real-time monitoring of cargo falling behavior and greatly shortening the response time; flexibility: since a large number of sensor devices are not required, the edge computing-based method can be flexibly arranged and is suitable for various fixed area monitoring needs;

[0014] Moreover, edge computing technology reduces dependence on central servers, reduces network transmission costs, and reduces system deployment and maintenance costs; the present invention is simple, effective, and easy to use. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a schematic diagram of the overall system flow of the present invention;

[0016] Figure 2 This is a schematic diagram of the background image update strategy flow of the present invention;

[0017] Figure 3 It is a schematic diagram of the target contour calculation process of the present invention;

[0018] Figure 4 It is the effect diagram of the edge detection algorithm of the present invention. DETAILED DESCRIPTION

[0019] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0020] See also Figure 1-Figure 4 , the present application provides a cargo drop detection method based on edge computing, the method specifically comprises the following steps:

[0021] S1, set the initial background image and obtain the real-time image through the camera;

[0022] S2, setting the region of interest, i.e., ROI, on the background image and initializing edge detection;

[0023] The specific method of setting the area of ​​interest on the background image is:

[0024] The region of interest can be determined by user setting or automatic identification; when the user sets the region of interest, the coordinates and bounding box size of the region of interest can be provided to manually determine the region of interest. When automatically identifying, a hybrid method combining color features, shape features and deep learning target detection algorithms is used to automatically identify the region of interest. The formula for determining the region of interest is:

[0025]

[0026] Among them, ROI represents the region of interest; R represents the candidate region; Color score (R), Shape score (R), DL score (R) represents the scores of color features, shape features and deep learning target detection respectively; α, β, γ are the weights of each feature score respectively.

[0027] S3, putting the real-time image and the initialized edge detection data into a queue;

[0028] Each time a real-time image is processed, the real-time image is stored in a historical image queue so that the historical image can be used for background update and comparison with the current image later;

[0029] At the same time, the edge detection data is initialized by taking out the earliest picture in the historical picture queue for contour comparison with the current image;

[0030] When initializing edge detection data, by calculating the difference value of each pixel, it is possible to clearly detect the changed area that may be the cargo falling or other objects moving, which helps to improve the accuracy of detection. It is used to calculate the absolute difference between the current real-time image and the background image to identify the changed area in the image. The formula is as follows:

[0031] diff_img(x,y)=|gray_background(x,y)-gray_current(x,y)|

[0032] Among them, gray_background(x,y) is the gray value of the background image at position (x,y), and gray_current(x,y) is the gray value of the current real-time image at position (x,y).

[0033] The calculated difference image is converted into a binary image for further processing. By significantly marking the changed area, the accuracy and efficiency of edge detection and contour extraction are improved, making subsequent target detection more accurate. The threshold processing formula is as follows:

[0034]

[0035] When diff_img(x,y) is greater than the set threshold (such as 30), thresh(x,y) is assigned a value of 255, indicating that there may be a target at this location; otherwise, thresh(x,y) is assigned a value of 0, indicating that this location is the background.

[0036] S4, taking out the earliest historical picture from the queue storing historical pictures, comparing it with the real-time picture and updating the background picture;

[0037] like Figure 2 As shown, the background image update strategy is:

[0038] (1) Background image fine-tuning:

[0039] When the detected target contour is empty, or the similarity between the target contours in the current image and the historical image is low, where low means the similarity is lower than 0.3, it indicates that the current background is similar to the real-time image, so the background image is updated to the current image.

[0040] (2) Image background replacement:

[0041] When the target contour similarity between the historical image and the real-time image is high, the high similarity is represented by a similarity exceeding 0.7, indicating that the background image has been changed for a period of time, so the background image is updated to the current image.

[0042] S5, the image after the background image update strategy is compared with the real-time image again, and edge detection is performed;

[0043] The image after the background image update strategy is compared with the real-time image again, and edge detection is performed;

[0044] After executing the background image update strategy, the updated background image is further compared with the real-time image. This comparison process is designed to ensure that the updated background image is as consistent as possible with the current scene, thereby improving the accuracy of target detection. Specifically, the comparison process includes the following steps:

[0045] (1) Image comparison: The updated background image is compared with the real-time image at the pixel level. By comparing the differences between the two images, any inconsistencies between the background and the real-time scene can be detected.

[0046] The calculation formula for the mean square error (MSE) in image comparison is:

[0047]

[0048] Among them, I and K represent two images, m and n represent the height and width of the image respectively;

[0049] (2) Edge detection: During the comparison process, edge detection is usually performed on the two images to accurately capture the contour information of the target object. Edge detection can help determine the position and shape of the target object, providing a basis for subsequent target recognition and tracking.

[0050] The calculation steps of edge detection algorithm are:

[0051] First convert the color image into a grayscale image to simplify subsequent processing. The formula is as follows:

[0052] gray(x,y)=0.299×R(x,y)+0.587×G(x,y)+0.114

[0053] ×B(x,y)

[0054] Among them, R, G and B represent the pixel values ​​of the red, green and blue channels in the color image; gray represents the pixel value of the grayscale image;

[0055] Gaussian filtering is performed on the grayscale image to smooth the image and reduce noise, as shown below:

[0056]

[0057] Where blur(x,y) represents the grayscale value after Gaussian filtering at position (x,y). σ represents the standard deviation of Gaussian filtering, which controls the degree of blur.

[0058] Then calculate the gradient magnitude G mag (x,y) and the gradient direction G dir (x,y), used to describe the strength and direction of edges in an image.

[0059]

[0060] Among them, G x and G y Respectively represented as the gradient of the image in the horizontal and vertical directions;

[0061] Then, non-maximum suppression (NMS) is performed on the gradient magnitude image to retain the local maximum of the edge and suppress non-edge points. The gradient magnitude is checked in the gradient direction to see if it is the local maximum. If so, it is retained, otherwise it is suppressed to zero. The formula is as follows:

[0062]

[0063] Finally, the NMS image is subjected to double threshold detection, which is divided into two steps: high threshold and low threshold, to determine the final edge image. Pixels above the high threshold (100) are considered strong edges, and pixels below the low threshold (50) are considered non-edges. Pixels between the high threshold and the low threshold are determined to be edges by checking their connectivity. The formula is as follows:

[0064]

[0065] Among them, edge represents the final edge image pixel value.

[0066] S6, outputting the target contour according to the detection result and issuing an alarm to the driver;

[0067] A method for drawing the contours of objects and their related features in an image is provided, such as Figure 3 As shown, Figure 3 ,First, an original test image is read and edge detection is performed on it to obtain the ,contour list and binary image; then, different drawing operations are performed for each contour; for example, the ,boundary rectangle, minimum enclosing rectangle and minimum enclosing circle of the contour are drawn on the ,image in blue, green and red, respectively; then, the approximation polygon and convex hull polygon are drawn to ,highlight the shape of the target contour; finally, the images with different types of contours are ,displayed for observing and analyzing the features of the target contour; the whole process aims to provide a ,simple and effective method for calculating and visualizing the target contour information in ,images, providing a basis for subsequent target recognition and ,tracking;

[0068] The minimum bounding rectangle refers to the smallest rectangle that completely contains the target contour. Using the edge detection results, all boundary points of the target contour are found, and the smallest rectangle containing these boundary points is calculated, whose sides are parallel to the coordinate axes of the image;

[0069] The minimum circumscribed circle refers to the smallest circle that completely contains the target contour. The edge points of the target object are obtained through the edge detection algorithm. The contour of the target object is extracted using the detected edge points, and the minimum circle containing all contour points is calculated. The radius of this circle is the smallest, and the distance from the center of the circle to all contour points is not less than the radius.

[0070] Blurring is used to smooth images, reduce noise, and make edges more coherent. The image is convolved using a convolution kernel to calculate the weighted average of each pixel and its neighboring pixels to generate a blurred image and reduce high-frequency noise. The formula is as follows:

[0071]

[0072] Where thresh(x,y) represents the pixel value after threshold processing at the position (x,y), k is the size of the convolution kernel, usually an odd number (for example, 3, 5, 7), i and j are used to traverse the index variables of the convolution kernel, and their value range is from 0 to k-1;

[0073] Polygonal approximation simplifies the contour representation by fitting complex contour shapes with polygons with a small number of vertices, reducing the number of contour points and simplifying the contour shape representation. It is suitable for scenes where complex shapes need to be simplified. For each contour point (x i ,y i ), the result of polygonal approximation is (x′ j ,y′ j), the formula is as follows:

[0074]

[0075] in,

[0076] represents the summation operation of all approximated vertices, where N is the number of polygon vertices;

[0077] ||(x i ,y i )-(x′ j ,y′ j )|| represents the original contour point (x i ,y i ) and the approximated polygon vertex (x′ j ,y′ j ) between ;

[0078] Represents the vertex set that minimizes the objective function;

[0079] The convex hull polygon refers to the smallest convex polygon that contains all target points. It can quickly calculate the smallest convex polygon that contains all contour points and is suitable for objects with irregular shapes but tend to be convex overall.

[0080] The alarm mechanism includes a sound alarm function, which emits a specific sound signal to remind the operator or relevant personnel that an abnormal situation has occurred; the characteristics of the sound alarm can be customized according to actual needs, such as setting different alarm audio files or distinguishing different alarm types through parameters such as sound frequency and volume; by introducing the sound alarm function, the practicality and application scope of the system can be further improved, making it more suitable for various scenarios and needs.

[0081] The above embodiments are only used to illustrate the technical method of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present invention.

Claims

1. A cargo drop detection method based on edge computing, It is characterized in that The specific steps include: Set the initial background image, set the region of interest (ROI) on the background image, and initialize the edge detection of the background image and ROI; obtain the real-time image through the camera; Put the real-time image and initialized edge detection data into the queue; Take out the earliest historical image from the queue storing historical images and compare it with the real-time image; The image after the background image update strategy is compared with the real-time image again and edge detection is performed; finally, the changed target contour is output according to the detection result, that is, the cargo that has fallen or is about to fall.

2. According to the method for detecting cargo drop based on edge computing in claim 1, It is characterized in that The specific method of setting the area of ​​interest on the background image is: The coordinates or bounding box size of the region of interest are set by the user.

3. According to the method for detecting cargo drop based on edge computing in claim 1, It is characterized in that The specific method of setting the area of ​​interest on the background image is: The region of interest can be determined by user setting or automatic identification. When the user sets the region of interest, the coordinates and bounding box size of the region of interest can be provided to manually determine the region of interest. When the region of interest is automatically identified, a hybrid method combining color features, shape features and deep learning target detection algorithms is used to automatically identify the region of interest. The formula for determining the region of interest is: Among them, ROI represents the region of interest; R represents the candidate region; Color score (R), Shape score (R), DL score (R) represents the scores of color features, shape features and deep learning target detection respectively; α, β, γ are the weights of each feature score respectively.

4. According to the method for detecting cargo drop based on edge computing in claim 1, It is characterized in that Put the real-time image and initialized edge detection data into the queue process: Each time a real-time image is processed, the real-time image is stored in a historical image queue for subsequent use of the historical image for background update and comparison with the current image; At the same time, the earliest picture stored in the historical picture queue is taken out, and then the edge detection data is initialized for contour comparison with the current image; When initializing edge detection data, by calculating the difference value of each pixel, it is possible to clearly detect the changed area that may be the cargo falling or other objects moving, which helps to improve the accuracy of detection. It is used to calculate the absolute difference between the current real-time image and the background image to identify the changed area in the image. The formula is as follows: diff_img(x,y)=|gray_background(x,y)-gray_current(x,y)| Among them, gray_background(x,y) is the gray value of the background image at position (x,y), and gray_current(x,y) is the gray value of the current real-time image at position (x,y); The calculated difference image is converted into a binary image for further processing. By significantly marking the changed area, the accuracy and efficiency of edge detection and contour extraction are improved, making subsequent target detection more accurate. The threshold processing formula is as follows: Among them, when diff_img(x,y) is greater than the set threshold (such as 30), thresh(x,y) is assigned a value of 255, indicating that there may be a target at this location; otherwise, thresh(x,y) is assigned a value of 0, indicating that this location is the background.

5. According to the method for detecting cargo drop based on edge computing in claim 1, It is characterized in that The background image update strategy is: Background image fine-tuning: when the detected target outline is empty, or the similarity between the target outline in the current image and the historical image is less than 0.3, it indicates that the current background is similar to the real-time image, so the background image is updated to the current image; Image background replacement: when the target contour similarity between the historical image and the real-time image is higher than 0.7, the background image is updated to the current image.

6. According to the method for detecting cargo drop based on edge computing in claim 1, It is characterized in that The image after the background image update strategy is compared with the real-time image again, the specific method is as follows: Compare the updated background image with the real-time image at the pixel level, and detect any inconsistency between the background and the real-time scene by comparing the differences between the two images; During the comparison process, edge detection is performed on the two images to accurately capture the contour information of the target object; Edge detection is used to determine the position and shape of target objects.

7. The cargo drop detection method based on edge computing according to claim 6, It is characterized in that When the image after the background image update strategy is compared with the real-time image again, the mean square error (MSE) is used to calculate the average of the sum of squares of the differences between the two images at each pixel position to evaluate the overall similarity between the background image and the real-time image, so as to determine whether the background image needs to be updated and the matching degree between the updated background image and the current scene. The calculation formula for mean square error (MSE) is: Among them, I and K represent two images respectively, and m and n represent the height and width of the image respectively.

8. According to the method for detecting cargo drop based on edge computing of claim 7, It is characterized in that The calculation steps of edge detection algorithm are: First convert the color image into a grayscale image to simplify subsequent processing. The formula is as follows: gray(x,y)=0.299×R(x,y)+0.587×G(x,y)+0.114 ×B(x,y) Among them, R, G and B represent the pixel values ​​of the red, green and blue channels in the color image; gray represents the pixel value of the grayscale image; Gaussian filtering is performed on the grayscale image to smooth the image and reduce noise, as shown below: Where blur(x,y) represents the gray value after Gaussian filtering at the position (x,y); σ represents the standard deviation of Gaussian filtering, which controls the degree of blur; Then calculate the gradient magnitude G mag (x,y) and the gradient direction G dir (x,y), used to describe the strength and direction of edges in an image; Among them, G x and G y Respectively represented as the gradient of the image in the horizontal and vertical directions; Then, non-maximum suppression (NMS) is performed on the gradient magnitude image to retain the local maximum of the edge and suppress non-edge points. The gradient magnitude is checked in the gradient direction to see if it is the local maximum. If so, it is retained, otherwise it is suppressed to zero. The formula is as follows: Finally, the NMS image is subjected to double threshold detection, which is divided into two steps: high threshold and low threshold, to determine the final edge image. Pixels above the high threshold (100) are considered strong edges, and pixels below the low threshold (50) are considered non-edges. Pixels between the high threshold and the low threshold are determined to be edges by checking their connectivity. The formula is as follows: Among them, edge represents the final edge image pixel value.

9. The cargo drop detection method based on edge computing according to claim 1, It is characterized in that The specific method of drawing the target contour in the image is: Use any of the minimum enclosing rectangle, minimum enclosing circle, fuzzy processing, polygon approximation, and convex hull polygon: The minimum bounding rectangle refers to the smallest rectangle that completely contains the target contour. Using the edge detection results, all boundary points of the target contour are found, and the smallest rectangle containing these boundary points is calculated, whose sides are parallel to the coordinate axes of the image; The minimum circumscribed circle refers to the smallest circle that completely contains the target contour. The edge points of the target object are obtained through the edge detection algorithm. The contour of the target object is extracted using the detected edge points, and the minimum circle containing all contour points is calculated. The radius of this circle is the smallest, and the distance from the center of the circle to all contour points is not less than the radius. Blurring is used to smooth images, reduce noise, and make edges more coherent. The image is convolved using a convolution kernel to calculate the weighted average of each pixel and its neighboring pixels to generate a blurred image and reduce high-frequency noise. The formula is as follows: Where thresh(x,y) represents the pixel value after threshold processing at the position (x,y), k is the size of the convolution kernel, usually an odd number (for example, 3, 5, 7), i and j are used to traverse the index variables of the convolution kernel, and their value range is from 0 to k-1; Polygonal approximation simplifies the contour representation by fitting complex contour shapes with polygons with a small number of vertices, reducing the number of contour points and simplifying the contour shape representation. It is suitable for scenes where complex shapes need to be simplified. For each contour point (x i ,y i ), the result of polygonal approximation is (x′ j ,y′ j ), the formula is as follows: in, represents the summation operation of all approximated vertices, where N is the number of polygon vertices; ||(x i ,y i )-(x′ j ,y′ j )|| represents the original contour point (x i ,y i ) and the approximated polygon vertex (x′ j ,y′ j ) between ; Represents the vertex set that minimizes the objective function; The convex hull polygon refers to the smallest convex polygon that contains all target points. It can quickly calculate the smallest convex polygon that contains all contour points and is suitable for objects with irregular shapes but tend to be convex overall.

10. The cargo drop detection method based on edge computing according to claim 1, It is characterized in that The alarm mechanism for alerting the driver includes an audible alarm function that alerts the operator by emitting a specific sound signal; When sound alarm occurs, different alarm audio files will be set or different alarm types will be distinguished by changing the sound frequency and volume.

Images