A method for detecting residues without interference from flying insects

By obtaining the edge of the object from the video image and removing the interference of flying insects, using the Kalman filter and the Hungarian matching algorithm for target tracking, the problem of low detection accuracy of legacy in complex scenarios is solved, and efficient and real-time legacy detection is achieved.

CN112634299BActive Publication Date: 2024-11-05CRSC COMM & INFORMATION GRP CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202011528041.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-12-22
Publication Date
2024-11-05
Estimated Expiration
2040-12-22

AI Technical Summary

Technical Problem

Existing legacy detection methods are susceptible to interference from flying insects and light changes in complex scenarios, resulting in low detection accuracy, high error detection rate, and complex calculations of deep learning algorithms, making it difficult to meet real-time processing requirements.

Method used

By obtaining the edge of the object from the video image, tracking the moving target and determining whether it is a flying insect, eliminating the flying insect interference, using the Kalman filter and the Hungarian matching algorithm for target tracking, combining the edge mask and time accumulation, classifying suspected residual targets isolates. Box, calculate the score to confirm the remains.

Benefits of technology

It improves detection accuracy, reduces false positives, enhances adaptability to complex scenarios, reduces the amount of calculation, and ensures real-time and accuracy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112634299B_ABST
    Figure CN112634299B_ABST
Patent Text Reader

Abstract

The present invention belongs to the field of video image processing, and relates to a method for detecting relicts excluding the interference of flying insects, comprising the following steps: S1 obtains the edge of an object from a video image; S2 detects a moving object from a video image, and determines whether it is a flying insect, if not, directly proceeds to the next step, if so, removes the edge of the moving object from the edge of the object obtained in step S1, and proceeds to the next step; S3 classifies the edge of the object processed by step S2, and determines a target frame of suspected relicts according to the classification result; S4 calculates the score of the object in the target frame of the suspected relict, if the score is greater than a threshold, confirms that the object in the target frame is a relic, if the score is less than the threshold, the object in the target frame is not a relic. It can effectively improve the detection accuracy while also ensuring its real-time performance and adaptability to complex scenes, and can effectively save manpower, material resources and time costs.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to a method for detecting residues by eliminating interference from flying insects, and belongs to the technical field of video image processing. Background Art

[0002] Relic detection can effectively detect unknown relics such as explosives and toxic substances that threaten public safety, and has broad application space in crowded public areas such as train stations, airports, and subway stations. Although most relics can be found through video surveillance, relics left on the track are usually not easy to be found. With the continuous increase in the speed of rail transit, relics left on the track can easily pose a hidden danger to the safety of passengers and vehicles, so real-time monitoring of relics is required.

[0003] At present, the methods of debris detection can be mainly divided into two categories. One is the method based on target tracking, which detects independent moving targets in each frame by analyzing the video sequence images, thereby determining the specific position of the target in each frame, which can not only provide the target's motion trajectory, but also perform motion detection and recognition based on the target's tracking information. The key issue of target tracking is the corresponding matching problem between consecutive image frames. The method based on target tracking needs to solve the problem of how to efficiently select matching features, overcome scene interference, and adapt to the changes of the target itself. The process is relatively complex and the algorithm complexity is relatively large. The other is the method based on target detection, which refers to detecting and extracting the foreground area with relative motion with the reference image in the video sequence, analyzing its grayscale, texture and edge features, and then segmenting it into several independent targets based on this. At present, the methods of target detection are mainly divided into optical flow method, inter-frame difference method, background difference method, etc. The principle and calculation of the method based on target detection are relatively simple, the detection speed is fast, and the development and application cost is low, but the adaptability in complex environment applications with frequent occlusion and light changes needs to be improved.

[0004] Although there have been many studies on the detection of objects left behind, there are still many problems in detecting objects left behind in complex scenes. For example, in complex scenes, it is very easy to be disturbed by other objects, such as stationary pedestrians, fast-flying insects at night, and other factors. It is also affected by changes in lighting, resulting in low detection accuracy and high false detection rate. Using deep learning algorithms to filter non-target objects such as pedestrians and animals requires a large number of labeled samples to train the model in different scenes. Once other interfering objects appear in the scene, the model needs to be retrained, otherwise the detection accuracy will be greatly reduced. At the same time, the computational complexity and amount of deep learning algorithms are very large, and it is difficult to meet the real-time processing requirements of intelligent video surveillance systems. Summary of the invention

[0005] In view of the above problems, the purpose of the present invention is to provide a method for detecting leftovers that eliminates the interference of flying insects, which is not only applicable to ordinary train stations, shopping malls and other scenes, but also can be used to check whether obstacles are maliciously placed on train tracks. While effectively improving the detection accuracy, it can also ensure its real-time performance and adaptability to complex scenes, and can effectively save manpower, material resources and time costs.

[0006] To achieve the above-mentioned purpose, the present invention adopts the following technical scheme: a method for detecting relics that eliminates the interference of flying insects, comprising the following steps: S1 obtains the edge of an object from a video image; S2 tracks a moving target from a video image and determines whether it is a flying insect, if not, directly proceeds to the next step, if so, removes the edge of the moving object from the edge of the object obtained in step S1, and then proceeds to the next step; S3 classifies the edge of the object processed by step S2, and determines a target frame of suspected relics based on the classification result; S4 calculates the score of the object in the target frame of the suspected relics, if the score is greater than a threshold, it is confirmed that the object in the target frame is a relic, if the score is less than the threshold, the object in the target frame is not a relic.

[0007] Further, the method for obtaining the edge of an object from a video image in step S1 is as follows: S1.1 calculates the gradient values ​​of the video image in the horizontal and vertical directions, and calculates the gradient values ​​of the background image in the horizontal and vertical directions; S1.2 calculates the gradient difference between the video image and the background image in the horizontal direction, and calculates the gradient difference between the video image and the background image in the vertical direction; S1.3 obtains a binary foreground edge mask based on the gradient difference in the horizontal and vertical directions; S1.4 obtains a stable edge mask based on the foreground edge mask and the time accumulation of each edge pixel.

[0008] Furthermore, the gradient value of the background image in the horizontal direction is:

[0009] B x,t (x,y)=B x,t-1 (x,y)+αD x,t (x,y)

[0010] The vertical gradient value of the background image is:

[0011] B y,t (x,y)=B y,t-1 (x,y)+αD y,t (x,y)

[0012] Among them, B x,t (x, y) is the gradient of the background model in the horizontal direction at time t, B x,t (x, y) is the gradient of the background model in the horizontal direction at time t-1; B y,t(x, y) is the gradient of the background model in the vertical direction at time t, B y,t (x, y) is the gradient of the background model in the vertical direction at time t-1; α is the learning rate; D x,t (x, y) is the gradient difference between the video image and the background image in the horizontal direction at time t; D y,t (x, y) is the gradient difference between the video image and the background image in the vertical direction at time t.

[0013] Furthermore, the calculation formula of the stable edge mask is:

[0014] SEMask t (x,y)=hyst(ACC t (x,y),AO time / 2,AO time )

[0015] Among them, SEMask t (x, y) represents the edge mask, hyst() is the hysteresis threshold function; AO time Threshold of the time that the legacy is left behind; ACC t (x, y) is the mask of the stable edge pixel with an initial value of 0 at time t, which is accumulated over time. The formula is:

[0016]

[0017] ACC t-1 (x, y) is the mask of the stable edge pixel with an initial value of 0 at time t-1, i represents the number of frames of the detection video, and i%10=0 means it is updated every 10 frames; F t (x,y) is the binary foreground edge mask, and its formula is:

[0018] F x,t (x,y)=hyst(|D x,t |,T low ,T high )

[0019] F y,t (x,y)=hyst(|D y,t |,T low ,T hig )

[0020] Among them, D x,t is the gradient difference between the video image and the background image in the horizontal direction at time t; D y,t is the gradient difference between the video image and the background image in the vertical direction at time t, which is higher than T high The pixel value is set to 1, and the pixel value below T lowThe pixel value is set to 0. At the same time, if one of the eight adjacent pixels has a value greater than T hig , then in T low and T hig The pixel values ​​between are set to 1, otherwise they are set to 0. low is the minimum threshold of gradient difference, T high is the maximum threshold of gradient difference.

[0021] Further, the method for tracking the moving target from the video image in step S2 is: S2.1 calculating feature information of the moving target in the video image; S2.2 initializing the Kalman filter using the feature information;

[0022] S2.3 uses the Kalman filter to predict the corresponding target area in the next frame. When the next frame arrives, the image in the target area is matched with the prediction result; S2.4 updates the Kalman filter if the match is successful.

[0023] Furthermore, the matching method is the Hungarian matching algorithm, which matches the prediction result and the detected image by minimizing the sum of the Euclidean distances between the centroid of the prediction result obtained by Kalman filtering and the centroid of the image actually detected in the next frame.

[0024] Further, the method for judging whether it is a flying insect in step S2 is: judging whether the distance moved by the center of mass of the object in adjacent frames is greater than 1 / 2 of the shorter edge of its target frame, and the moving direction is upward; or, whether the movement direction changes three or more times in ten adjacent frames. If so, it is a flying insect, otherwise it is not a flying insect.

[0025] Furthermore, in step S3, the method for classifying the edges of the object processed in step S2 is: placing each edge of the object in an edge target frame, calculating the distance between each edge target frame and other edge target frames, and dividing the edges into four categories based on the distance: above, below, left, and right of the suspected remains, thereby forming a target frame of the suspected remains.

[0026] Furthermore, the score of the object in the target frame of the suspected remains in step S4 includes an object score and a static score. Only when both the object score and the static score are greater than the threshold in step S4 is the object in the target frame confirmed to be a remains.

[0027] Furthermore, the object score calculation formula S b for:

[0028]

[0029] Among them, T reg , B reg , R regand L reg are the sum of the lengths of the edges that meet the convexity condition in the four categories above, below, left, and right of the suspected remains, BB W and BB L are the length and width of the target box of the suspected remains, respectively, and λ is a constant;

[0030] Static score C b for:

[0031]

[0032] in, Refers to the set of edge connections of the i-th edge group of an object. An edge group is a group of edges of a certain type.

[0033] Since the present invention adopts the above technical scheme, it has the following advantages: 1. The present application uses edges to detect and analyze candidate suspected remains, which can effectively reduce false alarms in complex scenes. 2. The present application uses the movement speed and direction of flying insects when flying, which can effectively suppress the interference caused by them in target detection. It has strong robustness to scene changes and can accurately eliminate false alarms caused by stationary people. At the same time, it can also reduce the impact of flying insects on subsequent intrusion detection and other system functions at night when there are many flying insects outdoors. 3. The present invention ensures that a relatively stable scene edge is obtained by performing multi-frame detection on the foreground edge. 4. The present invention simply and effectively identifies flying insects that interfere with the picture in outdoor scenes at night, and excludes them from the targets of intrusion detection, effectively improving the accuracy of the system's detection in complex environments and effectively reducing the amount of calculation. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a flow chart of a method for detecting residues that eliminates interference from flying insects in one embodiment of the present invention. DETAILED DESCRIPTION

[0035] In order to make those skilled in the art better understand the technical direction of the present invention, the present invention is described in detail through specific embodiments. However, it should be understood that the provision of specific embodiments is only for a better understanding of the present invention, and they should not be understood as limitations of the present invention. In the description of the present invention, it should be understood that the terms used are only for the purpose of description and cannot be understood as indicating or suggesting relative importance.

[0036] The present invention discloses a method for detecting relicts excluding the interference of flying insects, which ensures that a relatively stable scene edge is obtained by performing multi-frame detection on the foreground edge. Then, a clustering algorithm is used to group these edges, and a rectangular frame containing suspected relicts is obtained. Next, in order to classify the target of the suspected relict, the position, direction and static score of its edge are used to effectively judge whether the suspected relict is really a relict. At the same time, during foreground detection, the moving object can be framed and clustered, and the ratio of the length of its motion trajectory to the length and width of its target frame in each frame is judged to be too large, whether the direction of motion is upward and whether the direction of motion changes rapidly in a short period of time, so as to simply and effectively identify the flying insects that interfere with the picture in the outdoor scene at night, and exclude them from the target, effectively improve the accuracy of the system detection in a complex environment, and effectively reduce the amount of calculation. The scheme of the present invention is described in detail below through specific embodiments.

[0037] This embodiment discloses a method for detecting residues without interference from flying insects, comprising the following steps:

[0038] S1 obtains the edges of objects from video images.

[0039] The edge detection of objects is to combine the moving edge detection technology with the time accumulation process to generate a stable edge mask. The edge is used instead of the pixel intensity because the edge is insensitive to the change of illumination and does not need to remove the shadow. At the same time, it can better describe the scene information and avoid the influence of illumination change on the image and detection results.

[0040] The method for obtaining the edge of the object from the video image in step S1 is:

[0041] S1.1 calculates the gradient values ​​of the video image in the horizontal direction and the vertical direction, and calculates the gradient values ​​of the background image in the horizontal direction and the vertical direction.

[0042] Apply the Sobel operator to the input video image to calculate the first-order derivatives of the input video image in the horizontal and vertical directions, that is, the gradient values ​​of the video image in the horizontal and vertical directions. Then, use a continuous averaging algorithm on each pixel over time to calculate the gradient values ​​of the background image in the horizontal and vertical directions. The formula is:

[0043] The horizontal gradient value of the background image is:

[0044] B x,t (x,y)=B x,t-1 (x,y)+αD x,t (x,y)

[0045] The vertical gradient value of the background image is:

[0046] B y,t (x,y)=B y,t-1 (x,y)+αD y,t (x,y)

[0047] Among them, B x,t (x, y) is the gradient of the background model in the X direction at time t, B x,t (x, y) is the gradient of the background model in the X direction at time t-1; B y,t (x, y) is the gradient of the background model in the Y direction at time t, B y,t (x,y) is the gradient of the background model in the Y direction at time t-1; α is the learning rate; D x,t (x, y) is the gradient difference between the video image and the background image in the X direction at time t; D y,t (x, y) is the gradient difference between the video image and the background image in the Y direction at time t, where the X direction is the horizontal direction and the Y direction is the vertical direction.

[0048] S1.2 calculating the gradient difference between the video image and the background image in the horizontal direction, and calculating the gradient difference between the video image and the background image in the vertical direction;

[0049] The gradient difference of the background image in the horizontal direction is:

[0050] D x,t (x,y)=G x,t -B x,t (x,y)

[0051] The vertical gradient value of the background image is:

[0052] D y,t (x,y)=G y,t -B y,t (x,y)

[0053] Among them, G x,t (x, y) is the gradient value of the video image in the horizontal direction, G x,t (x,y) is the gradient value of the video image in the horizontal direction.

[0054] S1.3 obtains a binary foreground edge mask according to the gradient differences in the horizontal and vertical directions.

[0055] F t (x,y) is the binary foreground edge mask, and its formula is:

[0056] F x,t (x,y)=hyst(|D x,t |,T low ,T high )

[0057] F y,t (x,y)=hyst(|D y,t |,T low ,T high )

[0058] Where hyst() is the hysteresis threshold function, D x,t is the gradient difference between the video image and the background image in the X direction at time t; D y,t is the gradient difference between the video image and the background image in the Y direction at time t, T low is the minimum threshold of gradient difference, T high is the highest threshold of gradient difference. x,t (x,y) and F y,t (x,y) is calculated using a hysteresis threshold function. The horizontal or vertical gradient difference is higher than T high The pixel value is set to 1, and the gradient difference in the horizontal or vertical direction is lower than T low The pixel value is set to 0. At the same time, if the horizontal or vertical gradient difference of one pixel value among the 8 adjacent pixels is greater than T high , then in T low and T high The pixel values ​​between are set to 1, otherwise the 8 adjacent pixels are set to 0. x,t (x,y) and F y,t The operation on (x, y) obtains the binary foreground edge mask F.

[0059] S1.4 obtains a stable edge mask based on the foreground edge mask and the time accumulation of each edge pixel.

[0060] Through the binary foreground mask F t And the time accumulation of each edge pixel to obtain a stable edge pixel, where ACC t (x, y) is the mask of the stable edge pixel with an initial value of 0 at time t, and its formula is:

[0061]

[0062] ACC t-1 (x, y) is the mask of the stable edge pixel with an initial value of 0 at time t-1, i represents the number of frames of the detected video, and i%10=0 means updating every 10 frames. This setting can avoid considering temporarily stationary and slow-motion objects. ACC t (x,y) will only be detected when the edge pixel is F t Accumulation is performed only when (x,y)=1.

[0063] The calculation formula for the stable edge mask is:

[0064] SEMask t (x,y)=hyst(ACC t (x,y),AO time / 2,AO time )

[0065] Among them, SEMask t (x, y) represents the edge mask, hyst() is the hysteresis threshold function; AO time Threshold for how long the leftovers are left behind. Using a hysteresis threshold instead of a single threshold helps overcome the effects of occluded objects and slow-moving objects, and prevents static edge pixels from being accumulated in the ACC mask.

[0066] You can use the stable edge mask SEMask t (x,y) binarization estimates the stable gradient in the horizontal and vertical directions. The stable gradient Sg in the horizontal direction x,t and the vertical stable gradient Sg y,t Non-maximum suppression (NMS) is performed to obtain stable scene fine edges in the horizontal and vertical directions respectively:

[0067]

[0068]

[0069] S2 detects a moving object from the video image and determines whether it is a flying insect. If not, the process proceeds directly to the next step. If it is, the process proceeds to the next step after removing the edge of the moving object from the edge of the object obtained in step S1.

[0070] The detected stable back edge and binary foreground edge mask are used to calculate the input video image to obtain the center coordinates and target frame of the moving target. The connected domain is detected through blob analysis to obtain the corresponding moving target. Then, the Kalman filter and Hungarian matching algorithm are used to establish the relationship between the target frames between frames, so as to achieve the tracking and matching of multiple moving objects.

[0071] The method for tracking and matching multiple moving objects in the video image in step S2 is:

[0072] S2.1 calculates feature information of a moving target in a video image, including a target frame of the moving target and the center coordinates of the target frame;

[0073] S2.2 uses the characteristic information to initialize the Kalman filter;

[0074] S2.3 uses a Kalman filter to predict the corresponding target area in the next frame, and when the next frame arrives, matches the image in the target area with the prediction result;

[0075] S2.4 If the match is successful, the Kalman filter is updated.

[0076] Among them, the state equation of the Kalman filter is:

[0077] x k =Ax k-1 +Bu k-1 +q k-1

[0078] The observation equation of the Kalman filter is:

[0079] y k =Hx k +r k

[0080] In the above two equations, x k is the system state at time k, u k is the control quantity of the system at time k. A is the transmission parameter, B is the control parameter, A and B are both system parameters. For multi-model systems, A and B are matrices. k is the measurement value at time k, H is the parameter of the measurement system, and for a multi-measurement system, H is the state transfer matrix. k and r k They represent the state noise and measurement noise at time k respectively, and both are assumed to be Gaussian white noise.

[0081] The matching method is the Hungarian matching algorithm, which matches the prediction result with the detected image by minimizing the sum of the Euclidean distances between the center of mass of the prediction result obtained by the Kalman filter and the center of mass of the image actually detected in the next frame. The specific method is: calculate the loss matrix of the Hungarian matching algorithm, whose size is [MN], where M is the number of trajectories and N is the number of moving objects detected, and solve the loss matrix.

[0082] Once a moving object is tracked, the Euclidean distance of its center of mass between adjacent frames is found, and the moving direction and speed of the moving object can be obtained. By observing the flying of mosquitoes near cameras near multiple railroad tracks, the law of mosquitoes flying is simply summarized, and the following conclusion is obtained: when the object moves very fast, the distance moved by the center of mass in adjacent frames is greater than 1 / 2 of the shorter edge of its target frame and when the moving direction is upward or the moving direction has more than or equal to three changes in the adjacent ten frames. Therefore, the method for judging whether the moving target is a flying insect in step S2 is: judging whether the moving distance of the center of mass of the object in adjacent frames is greater than 1 / 2 of the shorter edge of its target frame, and the moving direction is upward; or, whether the moving direction has more than or equal to three changes in the adjacent ten frames, if so, it is a flying insect, otherwise it is judged not to be a flying insect.

[0083] S3 classifies the edges of the object processed in step S2, and determines the target frame of the suspected remains according to the classification result;

[0084] In step S3, the method for classifying the edges of the object processed in step S2 is as follows: each edge of the object is placed in an edge target frame, the distance between each edge target frame and other edge target frames is calculated, and the edges are divided into four categories: above, below, left and right of the suspected remains according to the distance, thereby forming a target frame of the suspected remains. The stability of the edge is detected in real time as the temporal distance of the clustering. The distance between the rectangular frames is not measured by the distance between the centroids but by the minimum distance between the four corners of the rectangular frame and the corners of other rectangular frames. The temporal distance and spatial distance of an edge target frame with other edge target frames are calculated. If the spatial distance of the first edge target frame of another edge target frame is less than the distance threshold D th , the time distance is less than the time threshold T th , the edge in the edge target box is considered to be of the same category as the edge in the first edge target box. This embodiment adopts a clustering method based on recursive labeling, which recursively traverses the unlabeled edge target box in the form of a graph to find edges of the same category as the edges in the previous edge target box.

[0085] S4 calculates the score of the object in the target frame suspected of being a relic. If the score is greater than a threshold, the object in the target frame is confirmed to be a relic. If the score is less than the threshold, the object in the target frame is not a relic.

[0086] In the prior art, a large number of scores are usually used to determine whether the suspected relic is caused by a ghost, a stationary person, or a sudden change in lighting. In this embodiment, it is only necessary to check whether the target frame of the suspected relic contains a whole object to determine whether the suspected relic is really the object left behind. Based on the principle of category-independent target detection, before target detection, a target frame with a high probability of containing the relic is generated to check all possible positions and the range of the sliding window.

[0087] The score of the object in the target frame suspected of being a relic includes an object score and a static score. Only when both the object score and the static score are greater than the threshold in step S4, the object in the target frame is confirmed to be a relic.

[0088] Subject score:

[0089] For the human eye, objects with clear closed boundaries are likely to be objects. It is assumed that the abandoned objects generally have simple convex edges, and the objective score of the object is defined accordingly.

[0090] In this embodiment, the width and length of the target frame of the suspected remains are BB W and BB L ,The four bounding boxes are divided into four regions: top, bottom, left, and right. Then, the convexity of the enclosed object in each region is calculated using the edge type to obtain the object score of the object. In this implementation, the edge group and the average gradient direction of the edge group are used for calculation. The edge group is formed by dividing the connected edges into groups according to their categories, and calculating the gradient direction difference between the two edges in the group. The gradient direction differences of all the edges in the group are added together, and the result does not exceed 90°.

[0091]

[0092]

[0093]

[0094]

[0095] T reg and B reg The length of the target box BB L For comparison, R reg and L reg The width of the target box BB W By comparison, we can get the calculation formula S of the object score b for:

[0096]

[0097] Among them, θi and segLength i They refer to the average direction and average length of the ith edge group in each region, T reg , B reg , R reg and L reg They are the sum of the lengths of the edges that satisfy the convexity condition in the four categories: above, below, left, and right of the suspected remains. as well as are the initial edges that meet the convexity condition in the four categories of above, below, left and right of the suspected remains. That is, if the condition |sin(θ i ) 2 -1|<δ or |sin(θ i ) 2 |<δ, then the initial edge plus segLength i , until all edges that meet the convexity condition are traversed, BB W and BB L are the length and width of the target box of the suspected relic, respectively, and λ is a constant.

[0098] Static score

[0099] A stationary person is likely to pose a challenge to the detection of objects left behind. However, when a person is stationary, there are usually small internal movements, such as shrugging shoulders. Therefore, this embodiment defines a static score to measure the stability of the target contour. The connectivity of the object contour is measured by verifying the association between each edge group and its adjacent edge groups to obtain a score C. b If there is an object, each edge group must have at least two connections with adjacent edge groups. If there are less than two, points will be deducted. This score helps to exclude distracting items such as stationary people.

[0100] Static score C b for:

[0101]

[0102] in, Refers to the set of edge connections of the i-th edge group of an object.

[0103] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit them. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation of the present invention can still be modified or replaced by equivalents, and any modification or equivalent replacement that does not deviate from the spirit and scope of the present invention should be included in the protection scope of the claims of the present invention. The above content is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or replacements within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A method for detecting residues without interference from flying insects. It is characterized in that The following steps are involved: S1 obtains the edge of the object from the video image; The method for obtaining the edge of the object from the video image in step S1 is: S1.1 calculates the gradient values ​​of the video image in the horizontal direction and the vertical direction, and calculates the gradient values ​​of the background image in the horizontal direction and the vertical direction; S1.2 calculating the gradient difference between the video image and the background image in the horizontal direction, and calculating the gradient difference between the video image and the background image in the vertical direction; S1.3 obtains a binary foreground edge mask according to the gradient difference in the horizontal direction and the vertical direction; S1.4 obtaining a stable edge mask according to the foreground edge mask and the time accumulation of each edge pixel; S2 tracks the moving target from the video image and determines whether it is a flying insect. If not, proceed directly to the next step. If yes, remove the edge of the moving object from the edge of the object obtained in step S1 and proceed to the next step. The method for tracking the moving target from the video image in step S2 is: S2.1 calculates feature information of moving targets in video images; S2.2 uses the characteristic information to initialize the Kalman filter; S2.3 uses the Kalman filter to predict the corresponding target area in the next frame, and when the next frame arrives, matches the image in the target area with the prediction result; S2.4 If the match is successful, the Kalman filter is updated; The matching method is a Hungarian matching algorithm, which matches the prediction result with the detected image by minimizing the sum of the Euclidean distances between the centroid of the prediction result obtained by Kalman filtering and the centroid of the image actually detected in the next frame; The method for judging whether it is a flying insect in step S2 is: judging whether the distance of the center of mass of the object moving in adjacent frames is greater than 1 / 2 of the shorter edge of its target frame, and the moving direction is upward; or, whether the moving direction changes three or more times in ten adjacent frames, if so, it is a flying insect, otherwise it is not a flying insect; S3 classifies the edges of the object processed in step S2, and determines the target frame of the suspected remains according to the classification result; S4 calculates the score of the object in the target frame of the suspected remains. If the score is greater than a threshold, the object in the target frame is confirmed to be a remains. If the score is less than the threshold, the object in the target frame is not a remains.

2. The method for detecting residues without interference from flying insects as claimed in claim 1, It is characterized in that The gradient value of the background image in the horizontal direction is: The gradient value of the background image in the vertical direction is: in, is the gradient of the background model in the horizontal direction at time t, is the gradient of the background model in the horizontal direction at time t-1; is the gradient of the background model in the vertical direction at time t, is the gradient of the background model in the vertical direction at time t-1; is the learning rate; is the gradient difference between the video image and the background image in the horizontal direction at time t; It is the gradient difference between the video image and the background image in the vertical direction at time t.

3. The method for detecting residues eliminating the interference of flying insects as claimed in claim 2, It is characterized in that The calculation formula of the stable edge mask is: in, represents the edge mask, hyst() is the hysteresis threshold function; The threshold of how long the legacy is left behind; It is the mask of the stable edge pixel with an initial value of 0 at time t under time accumulation, and its formula is: is the mask of the stable edge pixel with an initial value of 0 at time t-1, i represents the number of frames of the detected video, Indicates update every 10 frames; is the binary foreground edge mask, and its formula is: in, is the gradient difference between the video image and the background image in the horizontal direction at time t; is the gradient difference between the video image and the background image in the vertical direction at time t, The horizontal or vertical gradient difference is higher than The pixel value is set to 1, and the horizontal or vertical gradient difference is lower than The pixel value is set to 0.

4. The method for detecting residues without interference from flying insects according to any one of claims 1 to 3, It is characterized in that In step S3, the method for classifying the edges of the object processed in step S2 is: placing each edge of the object in an edge target frame, calculating the distance between each edge target frame and other edge target frames, and dividing the edges into four categories: above, below, left and right of the suspected remains according to the distance, thereby forming a target frame of the suspected remains.

5. The method for detecting residues without interference from flying insects as claimed in claim 4, It is characterized in that The score of the object in the target frame of the suspected remains in step S4 includes an object score and a static score. Only when both the object score and the static score are greater than the threshold in step S4, the object in the target frame is confirmed to be a remains.

6. The method for detecting residues without interference from flying insects as claimed in claim 5, It is characterized in that The calculation formula of the object score is for: in, , , as well as are the sum of the lengths of the edges that meet the convexity condition in the four categories above, below, left and right of the suspected remains, and are the length and width of the target frame of the suspected remains, is a constant; The static score for: in, Refers to the set of edge connections of the i-th edge group of an object. An edge group is a group of edges of a certain type.