A kitchen harmful organism living body detection method based on image processing technology

Abstract

The application discloses a kitchen harmful organism living body detection method based on image processing technology, and relates to the field of image processing technology, and comprises the following steps: motion target detection, modeling the background; sending a new video frame into the model for prediction, separating out the mask of the motion target; tracking all targets; frame by frame recording the information of each target; logically analyzing and judging the disappeared target. The application utilizes background modeling for motion target detection, then utilizes multi-frame time sequence information for trajectory tracking of the detected motion target in each frame, finally smoothes the multi-frame results, logically analyzes the target motion trajectory, and judges whether the trajectory conforms to the harmful organism living body behavior, compared with the prior art, the application utilizes multi-frame time sequence information, improves the accuracy, analyzes the motion trajectory, and judges whether the trajectory conforms to the harmful organism living body behavior through specific logic, so that the detection effect is greatly improved.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method for detecting live harmful organisms in the kitchen based on image processing technology. Background Technology

[0002] Kitchen hygiene is a key area of ​​focus in food safety. Unidentified live organisms such as rats and cockroaches are important sources of disease transmission and food contamination in kitchens. It is imperative to improve the efficiency of detection and early warning of live pests in kitchens using current methods.

[0003] For example, in the patent with publication number CN209376444U and titled "Rat Detection and Monitoring System", pressure sensors and other devices are used to capture rat activity signals. However, this requires hardware deployment, is expensive, and is difficult to implement on a large scale. At the same time, it is easily affected by interference, which affects its accuracy.

[0004] For example, in the patent with publication number CN116453043A and titled "A Kitchen Mouse Detection Method Based on Background Modeling and Target Detection," the idea of ​​using target detection algorithms and background modeling is used. However, for target detection, the detection effect is poor due to factors such as scarce samples and insufficient lighting. As for background modeling, the results cannot distinguish whether the target is a living object and are easily interfered with by other moving objects. Since no temporal information is combined, even if target detection and background modeling are combined, other targets cannot be effectively filtered out.

[0005] For example, in the patent with publication number CN111462169B and titled "A Mouse Trajectory Tracking Method Based on Background Modeling", although the trajectory of the target is generated, no further trajectory judgment is performed, so it is also easily interfered with by other moving targets.

[0006] To improve detection results, we propose a method for detecting live pests in the kitchen based on image processing technology. Summary of the Invention

[0007] The purpose of this invention is to provide a method for detecting live harmful organisms in the kitchen based on image processing technology, so as to solve the problems mentioned in the background art.

[0008] To achieve the above objectives, the present invention provides the following technical solution: a method for detecting live pests in the kitchen based on image processing technology, comprising the following steps:

[0009] Step 1: Moving target detection and background modeling;

[0010] Step 2: Extract information corresponding to the moving targets in the foreground, feed the new video frames into the model for prediction, separate the mask of the moving targets and extract information;

[0011] Step 3: Track all targets;

[0012] Step 4: Record information for each target frame by frame;

[0013] Step 5: Perform logical analysis on the disappeared target and determine whether the target is a living harmful organism.

[0014] Preferably, the background modeling method in step one includes using OpenCV's createBackgroundSubtractorKNN or createBackgroundSubtractorMOG2.

[0015] Preferably, the information of the mask in step two includes the minimum bounding box of the mask being the bounding box and the centroid of the mask being the centroid.

[0016] Preferably, the method for tracking the target in step three can be to assign an ID to the target.

[0017] Preferably, the information for target analysis in step four includes the mask, bounding box, and centroid coordinates.

[0018] Preferably, the logical analysis process in step five includes border size filtering, trajectory range determination, and instantaneous speed determination.

[0019] Preferably, the specific method for filtering the border size is as follows:

[0020] Collect size data of common harmful organisms in the kitchen and size data of normal targets in the target scenario;

[0021] A classifier is trained using a neural network algorithm to determine whether a bounding box belongs to a harmful organism.

[0022] Preferably, the specific method for determining the trajectory range is as follows:

[0023] Collect activity trajectories of common harmful organisms in kitchens in real-world scenarios;

[0024] The collected data is subjected to grid projection, the grid range of all trajectories is statistically analyzed, and the mean, median, number of trees, variance and standard deviation are calculated.

[0025] Based on the statistical characteristics of the data, fit a probability distribution and estimate the parameters;

[0026] The distribution and parameters are used to determine whether the data conforms to the distribution and whether the target is a harmful living organism.

[0027] Preferably, the specific method for determining the instantaneous velocity is as follows:

[0028] Collect activity trajectories of common harmful organisms in kitchens based on real-world scenarios;

[0029] Convert the activity trajectory into instantaneous velocity between frames;

[0030] Calculate instantaneous velocity, sort the instantaneous velocities from largest to smallest, take the top K largest values ​​as topK, and perform statistical analysis to calculate their mean, median, mode, variance and standard deviation statistics;

[0031] Based on the statistical characteristics of the data, fit a probability distribution and estimate the parameters;

[0032] The distribution and parameters are used to determine whether new data conforms to the distribution, thereby determining whether the target is a harmful living organism.

[0033] Preferably, in the step of determining whether a bounding box belongs to a pest, a voting process is required. The voting rule is to obtain a judgment result sequence after judging the size of each bounding box in the tracking sequence of a target. Each element in the sequence is divided into conforming and non-conforming. If the number of conforming elements is greater than the number of non-conforming elements, it is determined to be a pest. If the number of conforming elements is less than the number of non-conforming elements, it is not determined to be a pest.

[0034] The technical effects and advantages of this invention are as follows:

[0035] This invention utilizes background modeling for moving target detection, then uses multi-frame temporal information to track the trajectory of the detected moving target in each frame. Finally, it smooths the multi-frame results and performs logical analysis on the target's trajectory to determine whether it conforms to the behavior of a live harmful organism. Compared to existing methods, this invention utilizes multi-frame temporal information, which improves accuracy through a smoothing strategy and analyzes the trajectory to determine whether it conforms to the behavior of a live harmful organism, thereby greatly improving the detection effect. Attached Figure Description

[0036] Figure 1 This is a schematic diagram of the detection method of the present invention.

[0037] Figure 2 This is a schematic diagram of the moving target's activity trajectory grid according to the present invention. Detailed Implementation

[0038] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0039] This invention provides, for example Figure 1-2 The method for detecting live pests in the kitchen based on image processing technology, as shown, includes the following steps:

[0040] Step 1: Moving target detection and background modeling;

[0041] Step 2: Extract information corresponding to the moving target in the foreground, feed the new video frame into the model for prediction, separate the mask of the moving target and extract information. The mask is called a mask.

[0042] Step 3: Track all targets;

[0043] Step 4: Record information for each target frame by frame;

[0044] Step 5: Perform logical analysis on the disappeared target and determine whether the target is a living harmful organism. The logical analysis process includes border size filtering, trajectory range determination, and instantaneous speed determination.

[0045] This invention utilizes background modeling for moving target detection, then uses multi-frame temporal information to track the trajectory of the detected moving target in each frame. Finally, it smooths the multi-frame results and performs logical analysis on the target's trajectory to determine whether it conforms to the behavior of a live harmful organism. Compared to existing methods, this invention utilizes multi-frame temporal information, which improves accuracy through a smoothing strategy and analyzes the trajectory to determine whether it conforms to the behavior of a live harmful organism, thereby greatly improving the detection effect.

[0046] In step one, background modeling can be achieved using either `createBackgroundSubtractorKNN` or `createBackgroundSubtractorMOG2` from OpenCV. OpenCV is an open-source computer vision and machine learning library, built upon a function library written in optimized C / C++ code. It can run on various operating systems and provides over 2500 optimized algorithms commonly used in image processing and computer vision. `createBackgroundSubtractorKNN` is a function in OpenCV used for background segmentation. This function performs background subtraction based on the nearest neighbor algorithm. Specifically, it "learns" from each frame of the video, compares and calculates shadows, and excludes shadow areas from the detected image, thereby improving the results of motion analysis. `createBackgroundSubtractorMOG2` is a function in the OpenCV library used to create a background subtractor, which is implemented based on a Gaussian mixture model.

[0047] In step two, the information of the mask includes the minimum bounding box of the mask as the bounding box and the centroid of the mask as the centroid.

[0048] In step three, the target tracking method can be to assign an ID to the target, and the method can be the SORT method. SORT is a simple, online and real-time multi-target tracking method. The core idea of ​​this method is to track targets by jointly detecting targets and predicting motion.

[0049] The implementation of the SORT algorithm mainly includes the following steps:

[0050] Object detection: In each frame, the object detector is used to detect the position and bounding box of the target object.

[0051] Motion prediction: The Kalman filter is used to model and predict the motion of the target. The Kalman filter combines the target's current state with the motion model to estimate the target's position and velocity in the next frame.

[0052] Data Association: The Hungarian algorithm is used to establish associations between targets in the current frame and previous frames. This method determines the target's identity and trajectory by minimizing the association cost to match targets in the current frame with tracked targets in previous frames.

[0053] The advantages of the SORT algorithm lie in its simplicity and real-time performance, enabling efficient target tracking even with limited computing resources.

[0054] In step four, the information for target analysis includes the mask, bounding box, and centroid coordinates.

[0055] The specific method for border size filtering is as follows: collect the size data of common harmful organisms in the kitchen and the size data of normal targets in the target scene. Harmful organisms include rats, cockroaches, etc., and normal targets include light spots, pedestrians, etc.; use a neural network algorithm to train a classifier to determine whether the border belongs to a harmful organism. The classifier here is denoted as C1.

[0056] Training a neural network classifier is a process designed to teach the neural network model how to classify data. In this process, the neural network continuously adjusts its weights and biases using methods such as backpropagation and gradient descent to minimize the error between the predicted and actual results. A neural network classifier consists of an input layer, hidden layers, and an output layer. The input layer receives the raw data, the hidden layers perform progressively complex calculations and processing on the data, and the output layer provides the classification result. The training process involves two steps: forward propagation and backpropagation. Forward propagation involves processing the input data through the network to obtain the output result; backpropagation, on the other hand, calculates the gradient in the network based on the error between the output result and the actual label, and updates the weights and biases. During training, a large amount of labeled sample data is needed to train the neural network model. As training progresses, the model gradually learns how to extract useful features from the input data and map them to the corresponding output category. After training, the classifier can classify new, unknown data.

[0057] Border size filtering can limit the size of a target to a certain range, thereby eliminating interference from other targets.

[0058] The specific method for determining the trajectory range is as follows: collect the activity trajectories of common harmful organisms in the kitchen in actual scenarios; perform grid projection on the collected data, count the grid range of all trajectories, and perform statistical analysis to calculate their mean, median, variance, and standard deviation statistics; fit a probability distribution based on the statistical characteristics of the data, using a normal distribution, and perform parameter estimation; use this distribution and parameters to determine whether the data conforms to the distribution, and determine whether the target is a harmful living organism. The data part here is denoted as C2.

[0059] Trajectory range determination can eliminate the influence of local disturbances, large-area illumination, and non-specific targets.

[0060] The specific method for determining instantaneous velocity is as follows: collect the activity trajectories of common harmful organisms in the kitchen based on actual scenarios; convert the activity trajectories into inter-frame instantaneous velocities, i.e., trajectory coordinates → distance → instantaneous velocity; calculate the instantaneous velocities, sort them from largest to smallest, take the top K maximum values ​​as topK, and perform statistical analysis to calculate their mean, median, mode, variance, and standard deviation statistics; fit a probability distribution based on the statistical characteristics of the data and perform parameter estimation; use this distribution and parameters to determine whether new data conforms to the distribution, thereby determining whether the target is a harmful living organism.

[0061] Instantaneous velocity determination can eliminate interference caused by slow changes in lighting and non-specific targets.

[0062] In the step of determining whether a bounding box belongs to a pest, a voting process is required. The voting rule is to determine the size of each bounding box in the tracking sequence of a target and obtain a result sequence. Each element in this sequence is divided into conforming and non-conforming. If the number of conforming elements is greater than the number of non-conforming elements, it is determined to be a pest. If the number of conforming elements is less than the number of non-conforming elements, it is not determined to be a pest. The data part here is denoted as C3.

[0063] In practice, this invention is implemented as follows:

[0064] 1) The pre-processing method uses background modeling to detect moving targets. For foreground moving targets, a target mask can be obtained, and then the boundingRect method in OpenCV can be used to calculate the bounding box (bbox).

[0065] 2) Find the center point of the bounding box, and model the target as a point using this centroid;

[0066] 3) Then, using multi-target tracking algorithms, such as SORT, each target point can be tracked, and a tracking ID can be assigned to each target;

[0067] 4) By continuously tracking and recording the coordinates P and bounding box bbox (containing four values: x1, y1, x2, y2, representing the top corner respectively) of the same ID until the target disappears, each ID can obtain a set of corresponding coordinate points PS_i and bounding box set BS_i (i is the tracking ID);

[0068] 5) Finally, analyze BS_i and PS_i separately:

[0069] 5.1) For each tracking ID, analyze its bounding box set BS_i. For moving targets whose width (x2-x1) and height (y2-y1) of the bounding boxes pass the C1 criterion, consider them as suspected live harmful organisms. Then, through voting, determine whether the ID belongs to a live harmful organism. If it does, set the flag_b to True. This logic can eliminate interference from targets that are too large or too small. The voting rule is as follows: assuming we have a target tracking sequence BS_i, after judging the size of each bounding box, we obtain a judgment result sequence. Each element in this sequence is True / False, representing compliance and non-compliance, respectively. The voting method is to count whether there are more True or False values; the one with more True values ​​is the final voting result.

[0070] 5.2) For each tracking ID, analyze its coordinate point set PS_i, and map all the coordinates of PS_i to a grid screen. The overall size of the grid screen is equal to the size of the original video screen, and it is divided into several equally sized grids. Then, count how many grids the entire motion trajectory falls into, and determine the target's activity range based on this. If the activity range passes the C2 judgment, it is considered a live harmful organism, and the flag_p1 is set to True. This part of the logic can eliminate the influence of local disturbances and disturbances such as large-area illumination.

[0071] 5.3) For each tracking ID, analyze its coordinate point set PS_i, treat it as a time series, and calculate the difference to obtain the difference sequence PSD_i. Each element in PSD_i can be regarded as the inter-frame moving pixel distance, and then divided by the frame interval time (the frame interval time is obtained according to the FPS of the video stream and is equal to 1 / FPS) to obtain the inter-frame instantaneous velocity sequence PSV_i. Calculate topK for PSV_i. If it passes the C3 judgment, set the flag_p2 to True. This part of the logic can eliminate interference caused by slow lighting changes.

[0072] 6) Based on the three flags flag_b, flag_p1, and flag_p2 obtained in step 5, if all three are TRUE, the corresponding target will be regarded as a living harmful organism, and subsequent operations such as message reporting or warning will be performed.

[0073] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for detecting live pests in the kitchen based on image processing technology, characterized in that, Includes the following steps: Step 1: Moving target detection and background modeling; Step 2: Extract information corresponding to the moving targets in the foreground, feed the new video frames into the model for prediction, separate the mask of the moving targets and extract information; Step 3: Track all targets; Step 4: Record information for each target frame by frame; Step 5: Perform logical analysis on the disappeared target and determine whether the target is a living harmful organism.

2. The method for detecting live pests in the kitchen based on image processing technology according to claim 1, characterized in that, The background modeling method in step one includes using OpenCV's createBackgroundSubtractorKNN or createBackgroundSubtractorMOG2.

3. The method for detecting live pests in the kitchen based on image processing technology according to claim 1, characterized in that, In step two, the information of the mask includes the minimum bounding box of the mask being the bounding box, and the centroid of the mask being the centroid.

4. The method for detecting live pests in the kitchen based on image processing technology according to claim 1, characterized in that, The third step in tracking the target can be done by assigning an ID to the target.

5. The method for detecting live pests in the kitchen based on image processing technology according to claim 1, characterized in that, The information for target analysis in step four includes the mask, bounding box, and centroid coordinates.

6. The method for detecting live pests in the kitchen based on image processing technology according to claim 1, characterized in that, The logical analysis process in step five includes border size filtering, trajectory range determination, and instantaneous speed determination.

7. The method for detecting live pests in the kitchen based on image processing technology according to claim 6, characterized in that, The specific method for filtering the border size is as follows: Collect size data of common harmful organisms in the kitchen and size data of normal targets in the target scenario; A classifier is trained using a neural network algorithm to determine whether a bounding box belongs to a harmful organism.

8. A method for detecting live pests in the kitchen based on image processing technology according to claim 6, characterized in that, The specific method for determining the trajectory range is as follows: Collect activity trajectories of common harmful organisms in kitchens in real-world scenarios; The collected data is subjected to grid projection, the grid range of all trajectories is statistically analyzed, and the mean, median, number of trees, variance and standard deviation are calculated. Based on the statistical characteristics of the data, fit a probability distribution and estimate the parameters; The distribution and parameters are used to determine whether the data conforms to the distribution and whether the target is a harmful living organism.

9. A method for detecting live pests in the kitchen based on image processing technology according to claim 6, characterized in that, The specific method for determining the instantaneous velocity is as follows: Collect activity trajectories of common harmful organisms in kitchens based on real-world scenarios; Convert the activity trajectory into instantaneous velocity between frames; Calculate instantaneous velocity, sort the instantaneous velocities from largest to smallest, take the top K largest values ​​as topK, and perform statistical analysis to calculate their mean, median, mode, variance and standard deviation statistics; Based on the statistical characteristics of the data, fit a probability distribution and estimate the parameters; The distribution and parameters are used to determine whether new data conforms to the distribution, thereby determining whether the target is a harmful living organism.

10. A method for detecting live pests in the kitchen based on image processing technology according to claim 7, characterized in that, In the step of determining whether a bounding box belongs to a pest, a voting process is required. The voting rule is to determine the size of each bounding box in the tracking sequence of a target to obtain a determination result sequence. Each element in the sequence is divided into conforming and non-conforming. If the number of conforming elements is greater than the number of non-conforming elements, it is determined to be a pest. If the number of conforming elements is less than the number of non-conforming elements, it is not determined to be a pest.

Images