Food material storage equipment, pest detection method and device thereof and electronic equipment

Abstract

The invention discloses a food material storage device, a pest detection method and device thereof and an electronic device. The method comprises the steps that a current frame image, collected by an image collection device, of the interior of the food material storage device is obtained; performing target detection on the current frame image by using a target detection model to obtain a detection result; if the detection result indicates that the pest target exists in the current frame image, tracking the pest target to obtain a movement track of the pest target; and if the detection result indicates that the food type target exists in the current frame image, recording the position of the food type target. By means of the food material storage equipment, the technical problems that existing food material storage equipment does not have a pest detection function and cannot detect the pest condition in time are solved, the pests in the food material storage equipment are detected, tracked and counted in real time, and then a user is helped to know the pest condition in time and accurately so as to take corresponding measures in time.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a food storage device and a pest detection method, device and electronic equipment thereof. Background Art

[0002] Refrigerators, rice buckets and other food storage equipment are one of the essential appliances in people's lives. However, during use, if food is stored for too long or is not used hygienically, pests such as ants, cockroaches, and moths are likely to breed. These pests not only affect the hygiene and safety of food, but may also pose a threat to people's health. Currently, products on the market generally do not have pest detection functions. Users often have to observe or open the device to judge the pest situation. This is not only impossible to detect abnormalities in time, but also uncertain which food has pests when there are too many ingredients. Therefore, real-time detection of pests in food storage equipment is an urgent problem to be solved. Summary of the invention

[0003] The embodiments of the present invention provide a food storage device and a pest detection method, device, storage medium and electronic device thereof, so as to solve the technical problem that the existing food storage device has no pest detection function and cannot detect the pest situation in time, and realize real-time detection, tracking and statistics of pests in the food storage device, thereby helping users to understand the pest situation in time and accurately, so as to take corresponding measures in time.

[0004] In order to solve the above-mentioned technical problems, the present invention is achieved as follows:

[0005] In a first aspect, the present application provides a method for detecting pests in a food storage device, the method comprising:

[0006] Acquire a current frame image of the interior of the food storage device acquired by an image acquisition device;

[0007] Performing target detection on the current frame image using a target detection model to obtain a detection result;

[0008] If the detection result indicates that there is a pest target in the current frame image, the pest target is tracked to obtain a moving trajectory of the pest target;

[0009] If the detection result indicates that a food object exists in the current frame image, the position of the food object is recorded.

[0010] Furthermore, if the detection result indicates that there are a plurality of pest targets in the current frame image, tracking the pest targets to obtain the movement trajectory of the pest targets includes:

[0011] Acquire a predicted movement trajectory of a first target among the plurality of pest targets, wherein the predicted movement trajectory is composed of predicted positions of the first target in a current frame and previous images predicted by a Kalman filter;

[0012] Determine the actual positions of the plurality of pest targets in the current frame image using the target detection model;

[0013] A position matching the predicted movement trajectory is selected from the actual positions of the plurality of pest targets, and the position matching the predicted movement trajectory is added to the actual movement trajectory of the first target, wherein the actual movement trajectory is composed of the position matching the predicted movement trajectory and the actual position of the first target in the image before the current frame.

[0014] Furthermore, after obtaining the moving trajectory of the pest target, the pest detection method further includes:

[0015] Determining whether the moving trajectory passes through a target range centered on the position of the food target;

[0016] If the moving track passes through the target range centered on the position of the food object, prompt information for the food object is output.

[0017] Furthermore, the performing target detection on the current frame image using the target detection model includes:

[0018] Obtaining the size of the target in the current frame image;

[0019] Target detection is performed using a detection head of a scale corresponding to the size of the target in the target detection model, wherein detection heads of different scales are provided in the target detection model.

[0020] Furthermore, if the detection result indicates that there are pest targets in the current frame image, the pest detection method further includes:

[0021] Obtain at least one of the following information:

[0022] Obtaining the number of the pest targets;

[0023] Obtaining the type of the pest target;

[0024] The occurrence frequency of the pest target at the target position inside the food storage device is obtained.

[0025] Further, after obtaining at least one of the following information, the pest detection method further includes:

[0026] Outputting warning information of the food storage device, wherein the warning information is used to indicate that the pest target exists inside the food storage device.

[0027] In a second aspect, the present application provides a pest detection device for a food storage device, the pest detection device comprising:

[0028] A first acquisition unit, configured to acquire a current frame image inside the food storage device acquired by the image acquisition device;

[0029] A detection unit, configured to perform target detection on the current frame image using a target detection model to obtain a detection result;

[0030] A tracking unit, configured to track the pest target if the detection result indicates that there is a pest target in the current frame image, so as to obtain a moving trajectory of the pest target;

[0031] A recording unit is used to record the position of the food object if the detection result indicates that there is a food object in the current frame image.

[0032] In a third aspect, the present application provides a food storage device, wherein an image acquisition device and a processor are provided inside the food storage device, the image acquisition device is used to acquire images inside the food storage device, and the processor is used to execute the pest detection method described in the first aspect above.

[0033] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the pest detection method described in the first aspect above.

[0034] In a fifth aspect, the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the pest detection method described in the first aspect through the computer program.

[0035] The present application provides a method for detecting pests in a food storage device, the method comprising: obtaining a current frame image of the interior of the food storage device acquired by an image acquisition device; performing target detection on the current frame image using a target detection model to obtain a detection result; if the detection result indicates that there is a pest target in the current frame image, then tracking the pest target to obtain a moving trajectory of the pest target; if the detection result indicates that there is a food target in the current frame image, then recording the position of the food target. In an embodiment of the present application, the pests in the refrigerator are detected, tracked and counted in real time through image processing technology, which solves the technical problem that the existing food storage device does not have a pest detection function and cannot detect the pest situation in time, and realizes real-time detection, tracking and statistics of pests in the food storage device, thereby helping users to understand the pest situation in a timely and accurate manner so as to take corresponding measures in time. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0037] Figure 1 It is a flowchart of a pest detection method for food storage equipment according to an embodiment of the present application;

[0038] Figure 2 It is a flow chart of a camera-based pest detection and early warning method in a refrigerator according to a preferred embodiment of the present application;

[0039] Figure 3 It is a flowchart of a target detection method according to a preferred embodiment of the present application;

[0040] Figure 4 It is a flowchart of the target tracking statistical method of the preferred embodiment of the present application;

[0041] Figure 5 It is a structural schematic diagram of a pest detection device for food storage equipment according to one embodiment of the present application. DETAILED DESCRIPTION

[0042] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0043] The present invention provides a pest detection method for food storage equipment, which can effectively detect the pest situation inside the food storage equipment (such as a refrigerator, a freezer, etc.), and analyze and compile statistics on the data to help users understand the pest situation so as to take corresponding measures in time.

[0044] Figure 1 FIG. 1 is a flow chart of a pest detection method for food storage equipment according to an embodiment of the present application. Figure 1 As shown, the method includes:

[0045] Step S101, obtaining a current frame image inside a food storage device captured by an image capture device;

[0046] Step S102, performing target detection on the current frame image using a target detection model to obtain a detection result;

[0047] Step S103, if the detection result indicates that there is a pest target in the current frame image, the pest target is tracked to obtain a moving trajectory of the pest target;

[0048] Step S104: if the detection result indicates that a food object exists in the current frame image, the position of the food object is recorded.

[0049] In the above step S101, an image acquisition device, such as a camera, is provided inside the food storage device, and the image acquisition device can be used to obtain images inside the food storage device. In the present application, a camera is used to monitor the situation inside the food storage device in real time and obtain image data. The camera should be installed inside the food storage device so that the best field of view can be obtained. The camera should be high-definition to ensure that details and subtle changes in pests can be captured. In addition, the camera should be able to monitor the situation inside the food storage device in real time so that pest problems can be dealt with in a timely manner.

[0050] Optionally, after the image acquisition device acquires the current frame image inside the food storage device, the present application may preprocess the current frame image to improve the image quality and clarity so as to better detect pests. The preprocessing may include removing noise and smoothing the image, as well as increasing the image contrast and brightness. The specific method is as follows:

[0051] Histogram equalization redistributes the pixel values ​​of an image to make the brightness distribution of the image more uniform, thereby enhancing the contrast and clarity of the image.

[0052] Sharpening enhances the image's clarity and contrast by enhancing the high-frequency information of the image and making the edges of the image clearer.

[0053] Denoising, by removing noise from the image through filtering and other methods, thereby enhancing the clarity and quality of the image.

[0054] After the current frame image is preprocessed, the target detection model may be used to perform target detection on the current frame image in step S102 to obtain a detection result.

[0055] Optionally, performing target detection on the current frame image using a target detection model may include:

[0056] Obtaining the size of the target in the current frame image;

[0057] Target detection is performed using a detection head of a scale corresponding to the size of the target in the target detection model, wherein detection heads of different scales are provided in the target detection model.

[0058] The target detection model described in the present application can be a trained Yolov8 model. The trained Yolov8 model is designed with a multi-scale detection head, which uses feature maps of different scales for detection of targets of different sizes. This design can effectively improve the accuracy and robustness of small target detection.

[0059] It should be noted that step S102 is completed using computer vision and machine learning technologies, which can help the system automatically detect pests and determine their location, shape and type. Pest detection is performed using the Yolov8 target detection method. Since pests are generally relatively small, a small target detection method can be used, that is, adding a layer of shallow features to the output layer, which can allow the network to pay more attention to the detection of small targets and improve the detection effect. Specifically, this can be achieved through the following processes:

[0060] 1) Feature fusion: In the Yolov8 architecture, feature pyramid networks (FPN) or path aggregation networks (PANet) can be used to fuse feature maps at different levels, which can combine low-level detail information and high-level semantic information to enhance the detection effect.

[0061] 2) Modify the network structure: You can modify the network structure of Yolov8, for example, add an upsampling, concatenate and C2f module to the Neck part, splice the P2 of the originally discarded backbone, and increase the features of small targets. Such modifications help retain more detailed information and improve the detection accuracy of small targets.

[0062] 3) Use multi-scale detection head: Yolov8 designed a multi-scale detection head, which uses feature maps of different scales for detection of targets of different sizes. This design can effectively improve the accuracy and robustness of small target detection.

[0063] 4) Introducing a new detection layer: In order to more effectively utilize high-resolution spatial information, a fourth detection layer can be introduced in Yolov8. This new detection layer is specifically used to capture the detailed features of small targets, thereby improving the detection accuracy.

[0064] 5) Optimize loss function: Improve the loss function and IoU (Intersection over Union) calculation method to improve the regression accuracy and detection performance of the model. For example, Powerful-IoU (PIoU) is introduced as an alternative to CIoU, focusing on medium-quality anchor boxes and adding penalties based on the difference between the predicted box corners and the actual box corners. This method simplifies the calculation, speeds up convergence, and improves detection accuracy.

[0065] Through the above methods, the Yolov8 model can more effectively detect small targets in images and improve the accuracy and robustness of target detection.

[0066] After adjusting the Yolov8 model, you can train the Yolov8 model. First, you need to collect data. The data includes two types: pests and food such as fruits and vegetables stored in general food storage equipment. Pests mainly include flies, cockroaches, mosquitoes, fleas, termites, lice, aphids, etc. Food includes fruits, vegetables, meat, etc. Then use the image data to train the model.

[0067] After the Yolov8 model is trained, it can be applied. The trained model is transplanted into the processor of the food storage device, and combined with the image data transmitted by the image acquisition device for real-time detection, real-time detection of pest targets and food targets, as well as their specific types and locations.

[0068] After using the trained Yolov8 model to perform target detection on the current frame image, if the detection result indicates that there is a pest target in the current frame image, the pest target is tracked to obtain the movement trajectory of the pest target; if the detection result indicates that there is a food target in the current frame image, the position of the food target is recorded.

[0069] Optionally, if the detection result indicates that there are a plurality of pest targets in the current frame image, tracking the pest targets to obtain the movement trajectory of the pest targets may include:

[0070] Acquire a predicted movement trajectory of a first target among the plurality of pest targets, wherein the predicted movement trajectory is composed of predicted positions of the first target in a current frame and previous images predicted by a Kalman filter;

[0071] Determine the actual positions of the plurality of pest targets in the current frame image using the target detection model;

[0072] A position matching the predicted movement trajectory is selected from the actual positions of the plurality of pest targets, and the position matching the predicted movement trajectory is added to the actual movement trajectory of the first target, wherein the actual movement trajectory is composed of the position matching the predicted movement trajectory and the actual position of the first target in the image before the current frame.

[0073] It should be noted that the detailed process of pest target tracking using the Yolov8 model combined with the Kalman filter algorithm can be divided into the following steps:

[0074] Target detection: First, use the Yolov8 model to detect pests in video frames or images to obtain the location (bounding box) of the pests and other related information.

[0075] Initialize the Kalman filter: Create a Kalman filter instance for each detected target. Initialize the state vector (containing information such as the position and velocity of the target) and the covariance matrix (describing the uncertainty of the state estimate).

[0076] Prediction step: In each frame, a prediction step using a Kalman filter is used to estimate the position of the target in the next frame, which typically involves a state transfer matrix and process noise.

[0077] Update step: When a new frame is detected, the prediction is corrected using the Kalman filter’s update step, which involves calculating the Kalman gain, a coefficient that weighs the uncertainty of the prediction and observation.

[0078] Measurement update: The target position detected by the Yolov8 model is input into the Kalman filter as an observation. The state vector and covariance matrix are updated using the observation to obtain a more accurate estimate of the target position.

[0079] Data Association: In multi-target tracking, it is necessary to match detected targets with existing tracks. This can be achieved by comparing the predicted trajectories with the observed target positions.

[0080] Track maintenance: For each tracked object, a unique identifier is maintained and its state is updated between video frames. If the object leaves the field of view or becomes occluded, the Kalman filter can continue to predict its position.

[0081] Track Recording: Record the trajectory of each target over time, which can be achieved by storing the position of the target in each frame.

[0082] Performance optimization: Adjust the parameters of the Kalman filter, such as the covariance of process noise and observation noise, according to actual conditions to optimize tracking performance.

[0083] Through the above steps, the combination of Yolov8 model and Kalman filter algorithm can realize real-time tracking of pest targets and record their movement trajectory in the video.

[0084] Optionally, after obtaining the movement trajectory of the pest target, the pest detection method further includes:

[0085] Determining whether the moving trajectory passes through a target range centered on the position of the food target;

[0086] If the moving track passes through the target range centered on the position of the food object, prompt information for the food object is output.

[0087] This application records the food targets that pests have passed through, and then transmits the information to the user to remind the user to throw away or wash the food several times. The Yolov8 model can detect food targets (type and location), and if the movement trajectory of the pest passes through the food location target box, the information of the food (type, location and image) is recorded.

[0088] Optionally, if the detection result indicates that there is a pest target in the current frame image, the pest detection method further includes:

[0089] Obtain at least one of the following information:

[0090] Obtaining the number of the pest targets;

[0091] Obtaining the type of the pest target;

[0092] The occurrence frequency of the pest target at the target position inside the food storage device is obtained.

[0093] Further, after obtaining at least one of the following information, warning information of the food storage device is output, wherein the warning information is used to indicate that the pest target exists inside the food storage device.

[0094] This application can help the system identify areas where pests breed by statistically analyzing the number, type, and distribution of pests, and provide feedback to users to take appropriate measures to control pests. By sending alerts and statistical information to users, including: the number, type, distribution, and associated food (food where pests have stayed) information, users are promptly reminded to take appropriate measures to clean up pests, which can prevent pests from breeding and spreading.

[0095] The present application provides a preferred embodiment, which relates to an image-based pest detection and early warning system, which includes a camera and a smart refrigerator. The camera is installed inside the smart refrigerator, and uses image processing technology to detect, track and count pests in the refrigerator in real time. According to the statistical results, the pest problem in the refrigerator is warned and processed to avoid the breeding and spread of pests.

[0096] Figure 2 is a flow chart of a camera-based pest detection and early warning method in a refrigerator according to a preferred embodiment of the present application. Figure 2 As shown, the specific implementation steps are as follows:

[0097] S201, acquiring image data.

[0098] The camera monitors the situation inside the refrigerator in real time and obtains image data. The camera should be installed inside the refrigerator to obtain the best view. The camera should be high-definition to ensure that details and subtle changes of pests can be captured. In addition, the camera should be able to monitor the situation inside the refrigerator in real time so that pest problems can be dealt with in a timely manner.

[0099] S202: Preprocess the image data acquired by the camera.

[0100] Preprocess the image data acquired by the camera. This step aims to improve the quality and clarity of the image for better detection of pests. Preprocessing can include removing noise and smoothing the image, as well as increasing the contrast and brightness of the image. The specific methods are as follows:

[0101] Histogram equalization redistributes the pixel values ​​of an image to make the brightness distribution of the image more uniform, thereby enhancing the contrast and clarity of the image.

[0102] Sharpening enhances the image's clarity and contrast by enhancing the high-frequency information of the image and making the edges of the image clearer.

[0103] Denoising, by removing noise from the image through filtering and other methods, thereby enhancing the clarity and quality of the image.

[0104] S203, performing target detection on the preprocessed image data to identify pest targets in the refrigerator.

[0105] This step is done using computer vision and machine learning techniques. These techniques help the system automatically detect pests and determine their location, shape, and type. Figure 3As shown, the target detection method uses the Yolov8 model to perform target detection on the preprocessed image, and obtains the information contained in each detected target, including the category and position. The category may include pests and food, and the position may include the center point coordinates and the width and height of the target box.

[0106] Use target detection methods for pest detection. Since pests are generally relatively small, using a small target detection method, that is, adding a layer of shallow features to the output layer, can allow the network to pay more attention to the detection of small targets and improve the detection effect.

[0107] Specifically, this can be achieved through the following steps:

[0108] 1) Feature fusion: In the Yolov8 architecture, feature pyramid networks (FPN) or path aggregation networks (PANet) can be used to fuse feature maps at different levels. This can combine low-level detail information and high-level semantic information to enhance the detection effect.

[0109] 2) Modify the network structure: You can modify the network structure of Yolov8, for example, add an upsampling, concatenate and C2f module to the Neck part, splice the P2 of the originally discarded backbone, and increase the features of small targets. Such modifications help retain more detailed information and improve the detection accuracy of small targets.

[0110] 3) Use multi-scale detection head: Yolov8 designed a multi-scale detection head, which uses feature maps of different scales for detection of targets of different sizes. This design can effectively improve the accuracy and robustness of small target detection.

[0111] 4) Introducing a new detection layer: In order to more effectively utilize high-resolution spatial information, a fourth detection layer can be introduced in Yolov8. This new detection layer is specifically used to capture the detailed features of small targets, thereby improving detection accuracy.

[0112] 5) Optimize loss function: Improve the loss function and IoU (Intersection over Union) calculation method to improve the regression accuracy and detection performance of the model. For example, Powerful-IoU (PIoU) is introduced as an alternative to CIoU, focusing on medium-quality anchor boxes and adding penalties based on the difference between the predicted box corners and the actual box corners. This method simplifies the calculation, speeds up convergence, and improves detection accuracy.

[0113] Through the above methods, the Yolov8 model can more effectively detect small targets in images and improve the accuracy and robustness of target detection.

[0114] To train the Yolov8 model, you first need to collect data. The data includes two types: pests and food such as fruits and vegetables stored in general refrigerators. Pests mainly include flies, cockroaches, mosquitoes, fleas, termites, lice, aphids, etc. Food includes fruits, vegetables, meat, etc. Then use the image data to train the model.

[0115] The Yolov8 model is applied to transplant the trained model into the refrigerator processing chip, and real-time detection is performed in combination with the image data sent by the camera. The detection results are the category and location of the pests.

[0116] S204, tracking the detected pest targets, and recording their locations in the refrigerator and the food that the pests have visited.

[0117] The Yolov8 model can detect pests and food, as well as their types and locations in real time, and then use the Kalman filter algorithm to track the pests, thereby recording the trajectory of the pests. The specific process of the target tracking statistical method is as follows Figure 4 As shown in the figure, if the detected target is a pest, it will track the target, determine the target trajectory, and perform statistical analysis on the number, type, and frequency of pests at different locations; if it is food, it will mark the video location and determine the food that the pest has passed by. Finally, the statistical analysis results will be fed back to the user.

[0118] The detailed process of pest target tracking using the Yolov8 model combined with the Kalman filter algorithm can be divided into the following steps:

[0119] Target detection: First, use the Yolov8 model to detect pests in video frames or images to obtain the location (bounding box) of the pests and other related information.

[0120] Initialize the Kalman filter: Create a Kalman filter instance for each detected target. Initialize the state vector (containing information such as the position and velocity of the target) and the covariance matrix (describing the uncertainty of the state estimate).

[0121] Prediction step: In each frame, a prediction step using a Kalman filter is used to estimate the position of the target in the next frame. This typically involves a state transfer matrix and process noise.

[0122] Update step: When a new frame is detected, the prediction is corrected using the Kalman filter update step. This involves calculating the Kalman gain, which is a coefficient that weighs the uncertainty of the prediction and observation.

[0123] Measurement update: The target position detected by Yolov8 is input into the Kalman filter as an observation. The state vector and covariance matrix are updated using the observation to obtain a more accurate estimate of the target position.

[0124] Data Association: In multi-target tracking, it is necessary to match detected targets with existing tracks. This can be achieved by comparing the predicted trajectories with the observed target positions.

[0125] Track maintenance: For each tracked object, a unique identifier is maintained and its state is updated between video frames. If the object leaves the field of view or becomes occluded, the Kalman filter can continue to predict its position.

[0126] Track Recording: Record the trajectory of each target over time, which can be achieved by storing the position of the target in each frame.

[0127] Performance optimization: Adjust the parameters of the Kalman filter, such as the covariance of process noise and observation noise, according to actual conditions to optimize tracking performance.

[0128] Through the above steps, the combination of Yolov8 model and Kalman filter algorithm can realize real-time tracking of pest targets and record their movement trajectory in the video. And count the number of pests at different time points and the total statistics over a period of time (the number of pest targets tracked).

[0129] The food targets that pests have passed by are recorded, and the information is subsequently transmitted to the user to remind the user to throw away or wash the food several times. The Yolov8 model can detect food targets (type and location), and if the location of the pest (target box) is within the location of the food (target box), the information of the food (type, location and image) is recorded.

[0130] S205, performing statistical analysis on the tracked pest targets.

[0131] Statistical analysis includes information such as quantity, type, and distribution. The distribution information counts the frequency of pests appearing in specific locations of the refrigerator based on the trajectory information of the tracking target. Statistical analysis can help the system identify areas where pests breed and provide feedback to users to take appropriate measures to control pests.

[0132] S206, based on the statistical results, early warning and treatment of pest problems in the refrigerator.

[0133] Output the result information, conduct early warning and treatment to avoid the breeding and spread of pests. By sending alarms and statistical information to users: the number, type, distribution and related food (food where pests have stayed) of pests, users are reminded to take appropriate measures to clean up pests in a timely manner.

[0134] The embodiments of the present invention also provide a pest detection device for food storage equipment. The device is used to implement the above embodiments and preferred embodiments, and the descriptions that have been made will not be repeated. As used below, the terms "module", "unit", "subunit", etc. can implement a combination of software and / or hardware for a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0135] Figure 5 Schematic diagram of a pest detection device for food storage equipment according to an embodiment of the present application. Figure 5 As shown, the pest detection device comprises:

[0136] A first acquisition unit 51 is used to acquire a current frame image inside the food storage device acquired by the image acquisition device;

[0137] A detection unit 52, configured to perform target detection on the current frame image using a target detection model to obtain a detection result;

[0138] A tracking unit 53, configured to track the pest target to obtain a moving trajectory of the pest target if the detection result indicates that the pest target exists in the current frame image;

[0139] The recording unit 54 is configured to record the location of a food object if the detection result indicates that a food object exists in the current frame image.

[0140] Furthermore, if the detection result indicates that there are multiple pest targets in the current frame image, the tracking unit 53 includes:

[0141] A first acquisition module, used to acquire a predicted movement trajectory of a first target among the plurality of pest targets, wherein the predicted movement trajectory is composed of predicted positions of the first target in a current frame and previous images predicted by a Kalman filter;

[0142] A determination module, used for determining the actual positions of the plurality of pest targets in the current frame image by using the target detection model;

[0143] A screening module is used to screen out positions matching the predicted movement trajectory from the actual positions of the multiple pest targets, and add the positions matching the predicted movement trajectory to the actual movement trajectory of the first target, wherein the actual movement trajectory is composed of the positions matching the predicted movement trajectory and the actual position of the first target in the image before the current frame.

[0144] Furthermore, the pest detection device further comprises:

[0145] A judging unit, configured to judge whether the moving trajectory of the pest target passes through a target range centered on the position of the food target after the moving trajectory of the pest target is obtained;

[0146] The first output unit is configured to output prompt information for the food target if the moving track passes through the target range centered on the position of the food target.

[0147] Furthermore, the detection unit 52 includes:

[0148] A second acquisition module, used to acquire the size of the target in the current frame image;

[0149] The detection module is used to perform target detection using a detection head of a scale corresponding to the size of the target in the target detection model, wherein detection heads of different scales are provided in the target detection model.

[0150] Furthermore, the pest detection device further comprises:

[0151] The second acquisition unit is configured to acquire at least one of the following information if the detection result indicates that there is a pest target in the current frame image:

[0152] Obtaining the number of the pest targets;

[0153] Obtaining the type of the pest target;

[0154] The occurrence frequency of the pest target at the target position inside the food storage device is obtained.

[0155] Furthermore, the pest detection device further comprises:

[0156] The second output unit is used to output the warning information of the food storage device after obtaining at least one of the following information, wherein the warning information is used to indicate that the pest target exists inside the food storage device.

[0157] It should be noted that the above modules can be functional modules or program modules, and can be implemented by software or hardware. For modules implemented by hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0158] An embodiment of the present invention further provides a food storage device, wherein an image acquisition device and a processor are disposed inside the food storage device, wherein the image acquisition device is used to acquire images inside the food storage device, and the processor is used to execute any pest detection method described in the above method embodiments.

[0159] An embodiment of the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes any pest detection method described in the above method embodiments through the computer program.

[0160] The embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, each process of the above method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0161] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0162] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0163] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0164] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0165] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0166] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0167] Computer-readable media include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, modules of programs or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. According to the definition in this article, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0168] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0169] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0170] The above are only embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of the claims of the present invention.

Claims

1. A method for detecting pests in food storage equipment, characterized in that: The pest detection method comprises: Acquire a current frame image of the interior of the food storage device acquired by an image acquisition device; Performing target detection on the current frame image using a target detection model to obtain a detection result; If the detection result indicates that there is a pest target in the current frame image, the pest target is tracked to obtain a moving trajectory of the pest target; If the detection result indicates that a food object exists in the current frame image, the position of the food object is recorded.

2. The pest detection method according to claim 1, characterized in that: If the detection result indicates that there are a plurality of pest targets in the current frame image, tracking the pest targets to obtain the movement trajectory of the pest targets includes: Acquire a predicted movement trajectory of a first target among the plurality of pest targets, wherein the predicted movement trajectory is composed of predicted positions of the first target in a current frame and previous images predicted by a Kalman filter; Determine the actual positions of the plurality of pest targets in the current frame image using the target detection model; A position matching the predicted movement trajectory is selected from the actual positions of the plurality of pest targets, and the position matching the predicted movement trajectory is added to the actual movement trajectory of the first target, wherein the actual movement trajectory is composed of the position matching the predicted movement trajectory and the actual position of the first target in the image before the current frame.

3. The pest detection method according to claim 1, characterized in that: After obtaining the moving trajectory of the pest target, the pest detection method further includes: Determining whether the moving trajectory passes through a target range centered on the position of the food target; If the moving track passes through the target range centered on the position of the food object, prompt information for the food object is output.

4. The pest detection method according to claim 1, characterized in that: The performing target detection on the current frame image by using the target detection model comprises: Obtaining the size of the target in the current frame image; Target detection is performed using a detection head of a scale corresponding to the size of the target in the target detection model, wherein detection heads of different scales are provided in the target detection model.

5. The pest detection method according to any one of claims 1 to 4, characterized in that: If the detection result indicates that there are pest targets in the current frame image, the pest detection method further includes: Obtain at least one of the following information: Obtaining the number of the pest targets; Obtaining the type of the pest target; The occurrence frequency of the pest target at the target position inside the food storage device is obtained.

6. The pest detection method according to claim 5, characterized in that: After obtaining at least one of the following information, the pest detection method further includes: Outputting warning information of the food storage device, wherein the warning information is used to indicate that the pest target exists inside the food storage device.

7. A pest detection device for food storage equipment, characterized in that: The pest detection device comprises: A first acquisition unit, configured to acquire a current frame image inside the food storage device acquired by the image acquisition device; A detection unit, configured to perform target detection on the current frame image using a target detection model to obtain a detection result; A tracking unit, configured to track the pest target if the detection result indicates that there is a pest target in the current frame image, so as to obtain a moving trajectory of the pest target; A recording unit is used to record the position of the food object if the detection result indicates that there is a food object in the current frame image.

8. A food storage device, characterized in that: An image acquisition device and a processor are provided inside the food storage device, wherein the image acquisition device is used to acquire images inside the food storage device, and the processor is used to execute the pest detection method described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the pest detection method described in any one of claims 1 to 6 is executed.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: The processor executes the pest detection method described in any one of claims 1 to 6 through the computer program.

Images