Living mite detection method, device and equipment and storage medium

Abstract

The invention relates to the technical field of image recognition, in particular to a living mite detection method, device and equipment and a storage medium, and the method comprises the steps: capturing static images of mites at different time points from a mite sample through a large-field-of-view technology, so as to obtain a continuous mite image sequence; the specific position of each frame of acarid in the image can be accurately identified by using the multi-target tracking model, the moving path, speed and behavior mode of the acarid can be observed, and the acarid activity identification precision is improved; according to the method, image texture and shape features are described by calculating a gradient direction histogram of a local region of an image, and main components and edge information in the image are extracted, so that a system can understand internal features of the mite image more deeply, and evaluation of similarity of the mite image is facilitated; by comparing the similarities between different frames, the system can judge whether the mites are the same individual or have obvious morphological changes, and judge whether the mites are living bodies or not according to a similarity result.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and in particular to a method, device, equipment and storage medium for detecting live mites. Background Art

[0002] The number of live mites is an important indicator for measuring the anti-mite performance of daily necessities. The traditional method of detecting live mites is usually implemented by manual counting. The mites in the sample are imaged locally through a microscope, and then the live mites are identified by human eyes to see whether the mites are moving. This method mainly relies on manual operation and has great human risks, including the movement of the microscopic imaging area, the adjustment of the focal length, and the identification and counting of the naked eye. In addition, when people work for a long time and are in a state of fatigue, the probability of miscounting increases greatly, affecting the accuracy of the test results.

[0003] It can be seen that the existing technology still needs to be improved and enhanced. Summary of the invention

[0004] In order to overcome the shortcomings of the prior art, the purpose of the present invention is to provide a method, device, equipment and storage medium for detecting live mites, which obtain images that continuously cover the entire mite sample based on large field of view imaging, and judge the liveness of mites based on the image features of each image to improve the recognition accuracy of live mites.

[0005] A first aspect of the present invention provides a method for detecting live mites, comprising: acquiring a continuous sequence of mite images based on a large field of view imaging technology; acquiring bbox detection frame information of each frame image of the mite image sequence using a multi-target tracking model; acquiring a mite motion image set according to the bbox detection frame information and the mite image sequence, and determining whether the mite motion image set is in a continuous state; if the mite motion image set is in a continuous state, normalizing each frame of the mite image in the mite motion image set to obtain a mite detection image sample set; performing feature extraction on the mite detection image sample set to obtain PCA feature information and HOG feature information; performing inter-frame similarity calculation based on the mite detection image sample set, the PCA feature information and the HOG feature information to obtain a first structural similarity result, a second structural similarity result and a third structural similarity result; performing a mite liveness score based on the first structural similarity result, the second structural similarity result and the third structural similarity result to obtain a live mite detection result.

[0006] Optionally, in a first implementation method of the first aspect of the present invention, the use of a multi-target tracking model to obtain detection frame information of each frame image in the mite image sequence includes: forward reasoning each frame image in the mite image sequence through a YOLOV5 model to obtain first detection frame information; constructing a multi-target tracking model based on a SORT algorithm; calling a Kalman filter algorithm from the multi-target tracking model, and performing motion modeling on each frame image in the mite image sequence based on the Kalman filter algorithm to obtain a target mite motion model; predicting the coordinates of the next frame of the target mite according to the target mite motion model to obtain second detection frame information; using a Hungarian algorithm to calculate an intersection-and-union coefficient between the first detection frame information and the second detection frame information, and comparing the intersection-and-union coefficient with a preset matching threshold; if the intersection-and-union coefficient is greater than the preset matching threshold, combining the first detection frame information and the second detection frame information to obtain bbox detection frame information.

[0007] Optionally, in a second implementation method of the first aspect of the present invention, the method of obtaining a mite motion image set based on the bbox detection frame information and the mite image sequence, and judging whether the mite motion image set is in a continuous state, includes: obtaining a mite image of each mite from each frame image of the mite image sequence according to a preset number of continuous detection frames and the bbox detection frame information, and associating and matching each mite portrait to form a mite motion image set; obtaining the image coordinates of each frame of the mite image in the mite motion image set, and judging whether the mite motion image set is in a continuous state based on the continuity of the coordinates of each frame image.

[0008] Optionally, in a third implementation method of the first aspect of the present invention, after obtaining the image coordinates of each frame of the mite image in the mite motion image set and judging whether the mite motion image set is in a continuous state based on the continuity of the image coordinates of each frame, it also includes: if the mite motion image set is in a discontinuous state, obtaining the number of broken frames of each frame of the image coordinates, and comparing the number of broken frames with a preset broken frame threshold; if the number of broken frames is less than or equal to the preset broken frame threshold, using the interpolation method to supplement the mite motion image set to obtain the supplemented mite motion image set; if the number of broken frames is greater than the preset broken frame threshold, exporting the mite motion image set to a preset recycling pool.

[0009] Optionally, in a fourth implementation of the first aspect of the present invention, the feature extraction of the mite detection image sample set to obtain PCA feature information and HOG feature information includes: obtaining a mite sample set vector matrix based on the detection image sample set, and calculating a mite covariance matrix based on the mite sample set vector matrix; performing eigenvalue decomposition on the mite covariance matrix, and sorting the eigenvalues ​​in order from large to small values ​​to obtain the top N eigenvalues; obtaining corresponding eigenvectors based on the N eigenvalues, and standardizing each eigenvector to form a eigenvector matrix; obtaining PCA feature information based on the eigenvector matrix and the mite sample set vector matrix; graying and gamma correcting the detection image sample set, and calculating the pixel gradient of the processed detection image sample set according to a preset operator; counting the pixel gradients within a preset pixel area to form a gradient histogram; obtaining the eigenvector of the gradient histogram to form HOG feature information.

[0010] Optionally, in a fifth implementation manner of the first aspect of the present invention, the inter-frame similarity calculation is performed based on the mite detection image sample set, PCA feature information and HOG feature information to obtain a first structural similarity result, a second structural similarity result and a third structural similarity result, including: adding a sliding detection window to each image in the mite detection image sample set, and obtaining brightness data, contrast data and structural data of each image through the sliding detection window; calculating the structural similarity between each adjacent image based on the brightness data, contrast data and structural data, and performing mean calculation based on each structural similarity to obtain a first structural similarity result; using a cosine similarity algorithm to calculate the cosine similarity of the PCA feature information, and forming a second structural similarity result based on each cosine similarity; using a cosine similarity algorithm to calculate the cosine similarity of the HOG feature information, and forming a third structural similarity result based on each cosine similarity.

[0011] Optionally, in a sixth implementation manner of the first aspect of the present invention, the performing mite liveness scoring according to the first structural similarity result, the second structural similarity result, and the third structural similarity result to obtain a live mite detection result includes: determining whether the first structural similarity result is greater than a first preset similarity threshold to obtain a first comparison result; determining whether each cosine similarity in the second structural similarity result is greater than a second preset similarity threshold, and counting the number of cosine similarities greater than the second preset similarity threshold to obtain a second comparison result; determining whether each cosine similarity in the third structural similarity result is greater than a third preset similarity threshold, and counting the number of cosine similarities greater than the third preset similarity threshold to obtain a third comparison result; obtaining a live mite score value according to the first comparison result, the second comparison result, and the third comparison result; and comparing the live mite score value with a preset live reference value to obtain a live mite detection result.

[0012] The second aspect of the present invention provides a living mite detection device, comprising: an imaging module, used to obtain a continuous mite image sequence based on a large field of view imaging technology; a tracking module, used to obtain the bbox detection frame information of each frame image of the mite image sequence using a multi-target tracking model; a continuous module, used to obtain a mite motion image set according to the bbox detection frame information and the mite image sequence, and determine whether the mite motion image set is in a continuous state; a normalization module, used to normalize each frame of the mite image in the mite motion image set if the mite motion image set is in a continuous state , to obtain a sample set of mite detection images; a feature module, used to extract features from the sample set of mite detection images, to obtain PCA feature information and HOG feature information; a calculation module, used to perform inter-frame similarity calculation based on the sample set of mite detection images, the PCA feature information and the HOG feature information, to obtain a first structural similarity result, a second structural similarity result and a third structural similarity result; a scoring module, used to perform a mite liveness score based on the first structural similarity result, the second structural similarity result and the third structural similarity result, to obtain a live mite detection result.

[0013] A third aspect of the present invention provides a live mite detection device, which includes: a memory and at least one processor, wherein instructions are stored in the memory; at least one of the processors calls the instructions in the memory so that the live mite detection device performs each step of any of the live mite detection methods described above.

[0014] A fourth aspect of the present invention provides a computer-readable storage medium, on which instructions are stored, and when the instructions are executed by a processor, each step of any of the above-mentioned methods for detecting live mites is implemented.

[0015] In the technical solution of the present invention, a high-resolution, large-field-of-view camera is used to capture static images of mites at different time points of a mite sample, and their dynamic behaviors are recorded to obtain a continuous sequence of mite images. The movement of each mite and its legs can be distinguished through the mite image sequence; the multi-target tracking model can be used to accurately identify the specific position of the mite in each frame of the image, and the movement path, speed and behavior pattern of the mite can be observed by analyzing continuous frame images, thereby improving the accuracy of identifying the activity of the mite; the image texture and shape features are described by calculating the gradient direction histogram of the local area of ​​the image, and the main components and edge information in the image are extracted, so that the system can have a deeper understanding of the intrinsic characteristics of the mite image, which is helpful to evaluate the similarity of the mite images; by comparing the similarity between different frames, the system can determine whether the mites are the same individual, or whether there are significant morphological changes, and determine whether the mites are alive based on the similarity results. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A first flow chart of a method for detecting live mites provided in an embodiment of the present invention; Figure 2 A second flow chart of the live mite detection method provided by an embodiment of the present invention; Figure 3 A third flow chart of the live mite detection method provided by an embodiment of the present invention; Figure 4 A fourth flow chart of the live mite detection method provided by an embodiment of the present invention; Figure 5 A fifth flow chart of the live mite detection method provided in an embodiment of the present invention; Figure 6 A sixth flow chart of the live mite detection method provided by an embodiment of the present invention; Figure 7 A schematic diagram of the structure of a live mite detection device provided in an embodiment of the present invention; Figure 8 A schematic diagram of the structure of a live mite detection device provided in an embodiment of the present invention; Fig. 9 This is an example of optical imaging of a mite sample; Fig.10 An example diagram of a process for normalizing a mite image in a method for detecting live mites provided in an embodiment of the present invention; Fig.11 An example diagram of the process of performing liveness scoring on a mite image in a live mite detection method provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0017] The present invention provides a method, device, equipment and storage medium for detecting live mites. The present invention uses a high-resolution, large-field-of-view camera to capture static images of mites at different time points for mite samples, and records their dynamic behaviors to obtain a continuous mite image sequence. The movement of each mite and its legs can be distinguished through the mite image sequence; a multi-target tracking model can be used to accurately identify the specific position of mites in each frame of the image, and by analyzing continuous frame images, the movement path, speed and behavior pattern of the mites can be observed, thereby improving the recognition accuracy of mite activity; the image texture and shape features are described by calculating the gradient direction histogram of the local area of ​​the image, and by extracting the main components and edge information in the image, the system can have a deeper understanding of the intrinsic features of the mite image, which is helpful to evaluate the similarity of the mite images; by comparing the similarity between different frames, the system can determine whether the mites are the same individual, or whether there are significant morphological changes, and determine whether the mites are alive based on the similarity results.

[0018] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0019] For ease of understanding, the specific process of the embodiment of the present invention is described below. Figure 1 , an embodiment of the method for detecting live mites in the embodiment of the present invention includes: 101. Obtaining continuous mite image sequences based on large field of view imaging technology; 102. Using a multi-target tracking model, obtain bbox detection frame information of each frame image of the mite image sequence; 103. Obtain a mite motion image set according to the bbox detection frame information and the mite image sequence, and determine whether the mite motion image set is in a continuous state; In this embodiment, the mite sample is imaged by a high-resolution, large-field-of-view camera to obtain a continuous sequence of mite images, and the integrity and continuity of the sample are ensured by the mite image sequence. This process not only captures the static images of mites at different time points, but also records their dynamic behavior, and can distinguish the movement of each mite and its legs; the multi-target tracking model is used to obtain the bbox detection frame information of each frame image, and the precise positioning of multiple mites on the same image is achieved. The multi-target tracking model can accurately identify the specific position of each mite in the image, thereby providing a data basis for subsequent motion analysis and behavior pattern recognition; the acquisition of the mite motion image set and the judgment of its continuity are helpful to analyze the movement trajectory and behavior characteristics of the mite. By analyzing the continuous frame images, the system can observe the movement path, speed and behavior pattern of the mite, which is of great significance for understanding the activity of the mite.

[0020] like Fig. 9 As shown, the image in the figure is a mite image obtained after optical imaging of the mite sample by a high-resolution, large-field-of-view camera.

[0021] 104. If the mite motion image set is in a continuous state, normalizing each frame of the mite image in the mite motion image set to obtain a mite detection image sample set; In this embodiment, a set of continuous mite motion images is normalized to improve the accuracy and efficiency of subsequent processing; since the image target sizes obtained by target tracking are inconsistent, that is, the size of the detection frame of the same target in each frame will change, and the similarity calculation of the image information needs to ensure that the two vector dimensions are the same, a set of continuous mite images needs to be processed to have a uniform size.

[0022] like Fig.10 As shown, the specific steps are as follows: traverse the information of the mite motion image set, count the length and width of each frame, record the maximum length and width respectively, use this maximum value as a template to fill each image, and finally obtain a mite detection image sample set.

[0023] 105. Perform feature extraction on the mite detection image sample set to obtain PCA feature information and HOG feature information; 106. Perform inter-frame similarity calculation based on the mite detection image sample set, PCA feature information, and HOG feature information to obtain a first structural similarity result, a second structural similarity result, and a third structural similarity result; 107. Perform a live mite score according to the first structure similarity result, the second structure similarity result, and the third structure similarity result to obtain a live mite detection result.

[0024] In this embodiment, PCA feature information and HOG feature information are obtained based on the feature extraction of the mite detection image sample set. The PCA feature can reflect the main contour information of the image and retain the main structural features of the image through dimensionality reduction. The HOG feature can reflect the edge direction information of the image and describe the image texture and shape characteristics by calculating the gradient direction histogram of the local area of ​​the image. By extracting the main components and edge information in the image, the system can have a deeper understanding of the intrinsic characteristics of the mite image.

[0025] The inter-frame similarity calculation obtains the structural similarity result, which helps to evaluate the similarity of the mite images. By comparing the similarity between different frames, the system can determine whether the mite is the same individual or whether there are significant morphological changes, which is crucial for liveness detection. Based on the similarity results, it is determined whether the mite is alive, and finally the live mite detection result is output. Combining the data and analysis results obtained in the above steps, the system can accurately score whether the mite is alive, thereby providing reliable detection results for the dynamic statistics of live mites.

[0026] In an embodiment of the present invention, a high-resolution, large-field-of-view camera is used to capture static images of mites at different time points in a mite sample, and their dynamic behaviors are recorded to obtain a continuous sequence of mite images. The movement of each mite and its legs can be distinguished through the mite image sequence; the multi-target tracking model can be used to accurately identify the specific position of the mite in each frame of the image, and the movement path, speed and behavior pattern of the mite can be observed by analyzing continuous frame images, thereby improving the accuracy of identifying the activity of the mite; the image texture and shape features are described by calculating the gradient direction histogram of the local area of ​​the image, and the main components and edge information in the image are extracted, so that the system can have a deeper understanding of the intrinsic characteristics of the mite image, which is helpful to evaluate the similarity of the mite images; by comparing the similarity between different frames, the system can determine whether the mites are the same individual, or whether there are significant morphological changes, and determine whether the mites are alive based on the similarity results.

[0027] See also Figure 2 , a second embodiment of the method for detecting live mites in the embodiment of the present invention includes: 201. Perform forward reasoning on each frame image in the mite image sequence through the YOLOV5 model to obtain first detection frame information; 202. Construct a multi-target tracking model based on SORT algorithm; 203. Calling a Kalman filter algorithm from the multi-target tracking model, and performing motion modeling on each frame image in the mite image sequence based on the Kalman filter algorithm to obtain a target mite motion model; 204. Predicting the coordinates of the target mite in the next frame according to the target mite motion model to obtain second detection frame information; 205. Calculate the intersection-over-union ratio coefficient between the first detection frame information and the second detection frame information by using the Hungarian algorithm, and compare the intersection-over-union ratio coefficient with a preset matching threshold; 206. If the intersection-over-union ratio coefficient is greater than a preset matching threshold, the first detection box information and the second detection box information are combined to obtain bbox detection box information.

[0028] In this embodiment, the YOLOV5 model is used to perform forward reasoning on the mite image sequence, and the preliminary detection frame information of the mite in each frame image can be obtained. This process improves the accuracy of detection, ensures the processing speed, and uses the first detection frame information as a benchmark reference. The multi-target tracking model built based on the SORT algorithm calls the Kalman filter algorithm to build a target mite motion model, and predicts the coordinates of the target mite in the next frame based on the model, thereby obtaining the second detection frame information. This process not only enhances the accuracy of prediction, but also provides reliable data support for the subsequent fusion of detection frame information. After obtaining the first and second detection frame information, the Hungarian algorithm is used to calculate the intersection-and-union ratio coefficient between the two, and compare it with the preset matching threshold. The matching degree of the detection frame information is ensured by the above method, so as to effectively screen out high-quality detection results. Finally, when the intersection-and-union ratio coefficient is greater than the preset matching threshold, the system combines the first detection frame information with the second detection frame information to obtain the final bbox detection frame information. This comprehensive process improves the accuracy and stability of the detection frame information, and improves the robustness of the data by combining the predicted detection frame information. The whole process can remain stable under different environmental conditions, which improves the recognition effect of the model in complex scenes. Whether it is light changes, background interference or the rapid movement of mites, the model can adapt to and accurately identify the target mites. This robustness enables researchers to conduct effective monitoring under a variety of experimental conditions, providing a solid foundation for subsequent research and application.

[0029] To facilitate understanding, the following explanation is provided to assist understanding: First, the YOLOV5 model is used as the detector, and each image in the mite image sequence is input into the YOLOV5 model. Then, the images are uniformly processed into pictures of the same size before forward reasoning. At the same time, the category information and detection frame information of the target are output, that is, the first detection frame information. Then, non-maximum suppression is performed on the first detection frame information to remove redundant detection frames superimposed on the same target.

[0030] After obtaining the detection results of the multi-target tracking model, the multi-target tracking model uses Kalman filtering to predict the mathematical model of the target motion and obtain the predicted value. Then the Hungarian algorithm associates the predicted value obtained by the Kalman filter with the observed value obtained by the detector to find the optimal estimated value. The specific process is as follows: Now we need to obtain the detection box information of a certain target at the tth frame, recorded as bbox t optimal (It contains the width w and height h of the target rectangle, as well as the coordinates x and y of the center of the rectangle in the entire image). The currently known information is the detection box information bbox of the 1st to t-1st frames. 1 ~t-1Next, we use two methods to get the detection box information of the tth frame. The first is to use Kalman filter mathematical modeling, based on the bbox of the 1st to t-1st frames. 1~t-1 Predict the detection box information of the tth frame, recorded as bbox t prediction The second method is to directly detect the target detection box information of the tth frame through the detector, which is recorded as bbox t detection Finally, the Hungarian algorithm is used to calculate the bbox of the t-1 frame. 1~t-1 and bbox t detection To perform association matching, we actually calculate the intersection-and-union ratio coefficient of the bbox in the two frames. If the intersection-and-union ratio coefficient is greater than the preset threshold, it means that the bbox is matched successfully. Finally, we combine the bbox t prediction and bbox t detection Get a more accurate bbox t optimal , the bbox t optimal That is the bbox detection box information.

[0031] See also Figure 3 , a third embodiment of the method for detecting live mites in the embodiment of the present invention includes: 301. Obtaining a mite image of each mite from each frame image of the mite image sequence according to a preset number of continuous detection frames and bbox detection frame information, and associating and matching each mite head portrait to form a mite motion image set; 302. Obtain the image coordinates of each frame of the mite image in the mite motion image set, and determine whether the mite motion image set is in a continuous state according to the continuity of the image coordinates of each frame; 303. If the mite motion image set is in a discontinuous state, the number of broken frames of each frame image coordinate is obtained, and the number of broken frames is compared with a preset broken frame threshold; 304. If the number of broken frames is less than or equal to a preset broken frame threshold, interpolation is used to perform frame interpolation processing on the mite motion image set to obtain a frame interpolated mite motion image set; 305. If the number of broken frames is greater than a preset broken frame threshold, the mite motion image set is exported to a preset recycling pool.

[0032] In this embodiment, by analyzing and processing the continuity of each frame of the mite motion image, the integrity and coherence of the mite motion trajectory can be ensured, thereby greatly improving the accuracy of subsequent data processing and analysis. This process not only improves the readability of the mite motion trajectory, but also optimizes the recognition and analysis of the mite motion behavior. For example, when the continuity detection shows that the mite motion image set is in a discontinuous state, the system can promptly discover and take measures to perform frame interpolation processing on the mite motion image set, thereby obtaining a more complete and accurate mite motion image set. This frame interpolation processing not only avoids data analysis errors caused by inter-frame missing, but also ensures the quality and effect of data processing. In addition, when the number of broken frames exceeds the preset broken frame threshold, the system will export the mite motion image set to a preset recycling pool, discard the part of the image that cannot be repaired, and ensure the integrity and reliability of the data.

[0033] To facilitate understanding, the following example is used to assist in explanation: for each continuous motion image of the mite, the mites are first divided into two categories according to their tracking results: the first category, the tracking results are continuous and uninterrupted, for example, in the ten frames of images currently tracked, the mite has an image coordinate bbox in each frame, and its image coordinates are continuous. This category is treated as normal, that is, it flows directly to the next step; the second category, the tracking results are discontinuous and have stages. For example, in the ten frames of images currently tracked, the mite has no position coordinate bbox in the 4th frame. For this case, the interpolation method is used, and the results of the 3rd and 5th frames are averaged. Let the position coordinate bbox of the 3rd frame be 3 The position coordinate bbox of the 5th frame is (x3, y3, w3, h3). 5 is (x5, y5, w5, h5), then the position coordinate bbox of the 4th frame 4 (x4, y4, w4, h4) is replaced by the average value of two adjacent frames, where the x value and y value represent the coordinate values; in addition, if the mite has no tracking position coordinates for more than 3 consecutive frames, the calculation of the similarity of the current mite is abandoned.

[0034] See also Figure 4 , a fourth embodiment of the method for detecting live mites in the embodiments of the present invention includes: 401. Obtain a mite sample set vector matrix according to the detection image sample set, and calculate a mite covariance matrix according to the mite sample set vector matrix; 402. Perform eigenvalue decomposition on the mite covariance matrix, and sort the eigenvalues ​​in descending order of values ​​to obtain N eigenvalues ​​with the highest order; 403. Obtain corresponding eigenvectors according to the N eigenvalues, and perform standardization processing on each eigenvector to form an eigenvector matrix; 404. Obtain PCA feature information according to the feature vector matrix and the mite sample set vector matrix; 405. Perform grayscale and gamma correction processing on the detection image sample set, and calculate the pixel gradient of the processed detection image sample set according to a preset operator; 406. Counting pixel gradients in a preset pixel area to form a gradient histogram; 407. Obtain a feature vector of the gradient histogram to form HOG feature information.

[0035] In this embodiment, the mite sample set vector matrix is ​​extracted from the detection image sample set, which can capture the key information of the mite sample. Subsequently, the calculated mite covariance matrix further reveals the correlation between samples; the mite covariance matrix is ​​decomposed by eigenvalues ​​and sorted according to the numerical value. This process ensures that the system can identify the eigenvalues ​​that are most critical to distinguishing different mite samples. By screening out the top N eigenvalues, the system can focus on the most representative features, thereby improving the efficiency and effect of feature extraction; the eigenvectors corresponding to these key eigenvalues ​​are obtained, and they are standardized to form an eigenvector matrix. This process not only makes the eigenvectors more standardized, but also provides a clear and consistent data format for subsequent PCA feature extraction. Using the eigenvector matrix and the mite sample set vector matrix, PCA feature information can be obtained. PCA, as a powerful dimensionality reduction technology, can effectively reduce the dimension of the data while retaining the most important information, which not only simplifies the calculation process, but also improves the processing speed and efficiency.

[0036] Grayscale and gamma correction are performed on the detection image sample set to eliminate the impact of illumination changes on the image and ensure the accuracy of subsequent processing; the pixel gradient of the processed detection image sample set is calculated to help capture the detailed information in the image; the pixel gradient in the preset pixel area is counted to form a gradient histogram. This process helps the system better understand the texture features of different areas in the image, and the gradient histogram, as an effective image description method, can provide rich data support for HOG feature extraction; the feature vector of the gradient histogram is obtained to form HOG feature information. This process enables us to extract directional local features from the image. HOG feature information is very effective in describing the shape and texture in the image, which helps to improve the accuracy of mite detection and classification.

[0037] See also Figure 5 The fifth embodiment of the method for detecting live mites in the embodiment of the present invention includes: 501. Add a sliding detection window to each image in the mite detection image sample set, and obtain brightness data, contrast data, and structure data of each image through the sliding detection window; 502. Calculate the structural similarity between adjacent images according to the brightness data, the contrast data and the structural data, and perform mean calculation according to each structural similarity to obtain a first structural similarity result; 503. Calculate the cosine similarity of the PCA feature information using a cosine similarity algorithm, and form a second structural similarity result according to each cosine similarity; 504. Calculate the cosine similarity of the HOG feature information using a cosine similarity algorithm, and form a third structural similarity result according to each cosine similarity.

[0038] In this embodiment, when calculating the similarity of mite images, the system first combines brightness data, contrast data and structural data to enable the system to analyze the images from multiple dimensions, which helps to capture subtle differences in the images, calculate the structural similarities between adjacent images, and calculate the mean of these similarity values, effectively smoothing out deviations caused by some accidental factors, thereby obtaining a more stable and reliable structural similarity result.

[0039] Secondly, the cosine similarity algorithm is used to calculate the similarity of PCA (principal component analysis) feature information. This method can effectively measure the angle between feature vectors, thereby judging their similarity. Since PCA can extract the main features of an image, the second structural similarity result obtained by the cosine similarity algorithm can reflect the similarity of the images in terms of their main features.

[0040] Similarly, the cosine similarity calculation of HOG (Histogram of Oriented Gradients) feature information focuses on the local texture information of the image. HOG features can capture the shape and edge information of objects in the image. Therefore, the third structural similarity result obtained by the cosine similarity algorithm can further supplement and improve the evaluation of image similarity.

[0041] Combining the above three different structural similarity results, the system can make a more comprehensive and accurate assessment of the similarity of mite images as a whole. This multi-angle and multi-level analysis method not only improves the accuracy of similarity calculation, but also better adapts to the image processing needs under different environments and conditions.

[0042] It should be noted that the cosine similarity calculation formula is as follows:

[0043] Where F represents the image feature Feature, i represents the i-th frame, N represents the number of images in the calculation, and F i and F i+1 Represents the image features of the i-th frame and the i+1-th frame.

[0044] See also Figure 6 The sixth embodiment of the method for detecting live mites in the embodiments of the present invention includes: 601. Determine whether the first structural similarity result is greater than a first preset similarity threshold to obtain a first comparison result; 602. Determine whether each cosine similarity in the second structural similarity result is greater than a second preset similarity threshold, and count the number of cosine similarities greater than the second preset similarity threshold to obtain a second comparison result; 603. Determine whether each cosine similarity in the third structural similarity result is greater than a third preset similarity threshold, and count the number of cosine similarities greater than the third preset similarity threshold to obtain a third comparison result; 604. Obtaining a live mite score value according to the first comparison result, the second comparison result, and the third comparison result; 605. Compare the live mite score value with a preset live reference value to obtain a live mite detection result.

[0045] In this embodiment, in the process of identifying live mites, a first structural similarity comparison is first performed. By calculating and comparing the similarity between the target sample and the known live mite structure, it is determined whether it exceeds the first preset similarity threshold. If the first comparison result shows that the similarity is insufficient, then the sample is likely to be a non-living or non-target species of mite, thereby avoiding further erroneous analysis.

[0046] The system then enters the second stage to analyze the second structural similarity results. In this stage, the system calculates multiple cosine similarity values ​​and determines whether they exceed the second preset similarity threshold one by one. By counting the number of cosine similarity values ​​that meet the conditions, the system can obtain a more accurate second comparison result. This process helps to identify subtle differences that may be overlooked in the first stage, further improving the accuracy of recognition; by calculating and comparing cosine similarity values ​​and counting the number that exceeds the third preset similarity threshold, the third comparison result is obtained. The analysis at this stage helps to capture features that may not be fully recognized in the first two stages, thereby further improving the overall recognition accuracy.

[0047] Combining the results of the first, second, and third comparisons, the system will calculate a comprehensive score for live mites. This score is based on the analysis results of multiple dimensions and can more comprehensively reflect the live characteristics of the sample. Finally, the system will compare this score with the preset live benchmark value. If the score meets the conditions of the live benchmark value, the sample is judged to be a live mite; otherwise, it is judged to be non-live. This comprehensive scoring and comparison method not only improves the accuracy of identification, but also reduces the possibility of misjudgment and missed judgment, ensuring the reliability of the test results.

[0048] like Fig.11 As shown, for ease of understanding, the following example is used to assist in the explanation: Here, only an optional case is given to obtain the mite similarity information of the past 10 frames of pictures, Threshold SSIM (The first preset similarity threshold) is set to 0.8. If (First structural similarity result) is greater than 0.8, then the similarity score P SIM Add 1; Threshold PCA (Second preset similarity threshold) is set to 0.95. If PCA with a score greater than 0.95 in the second structural similarity result i If the number is greater than 3, the similarity score P SIM Add 1; set Threshold HOG (The third preset similarity threshold) is 0.85. If (The third structural similarity result) is greater than 0.85, then the similarity score P SIM Add 1. Finally, the similarity scores of all mites are counted, and the preset living body baseline value is 2. If P SIM If the number of mites is greater than or equal to 2, they will be judged as dead mites, that is, the similarity between the images is too high. SIM Mites with a count less than 2 will be considered live mites.

[0049] The above describes the live mite detection method in the embodiment of the present invention. The following describes the live mite detection device in the embodiment of the present invention. Figure 7 , an embodiment of the live mite detection device in the embodiment of the present invention includes: An imaging module 701 is used to obtain a continuous sequence of mite images based on a large field of view imaging technique; A tracking module 702 is used to obtain bbox detection frame information of each frame image of the mite image sequence using a multi-target tracking model; A continuous module 703 is used to obtain a mite motion image set according to the bbox detection frame information and the mite image sequence, and determine whether the mite motion image set is in a continuous state; A normalization module 704 is used for normalizing each frame of the mite image in the mite motion image set to obtain a mite detection image sample set if the mite motion image set is in a continuous state; A feature module 705 is used to extract features from a sample set of mite detection images to obtain PCA feature information and HOG feature information; A calculation module 706 is used to perform inter-frame similarity calculation based on the mite detection image sample set, PCA feature information and HOG feature information to obtain a first structural similarity result, a second structural similarity result and a third structural similarity result; The scoring module 707 is used to score the live mites according to the first structure similarity result, the second structure similarity result and the third structure similarity result to obtain the live mites detection result.

[0050] In this embodiment, the imaging module 701 and the tracking module 702 capture static images of mites at different time points of the mite sample through a high-resolution large-field camera, and record their dynamic behaviors to obtain a continuous sequence of mite images. The movement of each mite and its legs can be distinguished through the mite image sequence; the continuous module 703 and the normalization module 704 can accurately identify the specific position of the mite in each frame of the image by using a multi-target tracking model. By analyzing the continuous frame images, the moving path, speed and behavior pattern of the mite can be observed, thereby improving the recognition accuracy of the activity of the mite; the feature module 705 describes the image texture and shape features by calculating the gradient direction histogram of the local area of ​​the image, and by extracting the main components and edge information in the image, the system can have a deeper understanding of the intrinsic characteristics of the mite image, which is helpful to evaluate the similarity of the mite images; the calculation module 706 and the scoring module 707 can determine whether the mites are the same individual or whether there are significant morphological changes by comparing the similarities between different frames, and determine whether the mites are alive based on the similarity results.

[0051] above Figure 7 The live mite detection device in the embodiment of the present invention is described in detail from the perspective of modular functional entities. The live mite detection device in the embodiment of the present invention is described in detail from the perspective of hardware processing.

[0052] Figure 88 is a schematic diagram of the structure of a live mite detection device provided by an embodiment of the present invention. The live mite detection device 800 may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) 810 (for example, one or more processors) and a memory 820, and one or more storage media 830 (for example, one or more mass storage devices) storing application programs 833 or data 832. Among them, the memory 820 and the storage medium 830 may be temporary storage or permanent storage. The program stored in the storage medium 830 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the live mite detection device 800. Furthermore, the processor 810 may be configured to communicate with the storage medium 830, and execute a series of instruction operations in the storage medium 830 on the live mite detection device 800 to implement the steps of the live mite detection method provided in the above-mentioned method embodiments.

[0053] The live mite detection device 800 may also include one or more power supplies 840, one or more wired or wireless network interfaces 850, one or more input and output interfaces 860, and / or one or more operating systems 831, such as Windows Serve, Mac OS X, Unix, Linux, FreeBSD, etc. It will be appreciated by those skilled in the art that Figure 8 The structure of the live mite detection device shown does not constitute a limitation on the live mite detection device, and may include more or fewer components than shown in the figure, or a combination of certain components, or a different arrangement of components.

[0054] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions are executed on a computer, the computer executes the steps of the live mite detection method.

[0055] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described system, device, or unit can refer to the corresponding process in the aforementioned method embodiment and will not be repeated here.

[0056] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the whole or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0057] Finally, it should be noted that the above description is only a preferred example of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for detecting live mites, characterized in that: include: Acquire continuous mite image sequences based on large field of view imaging technology; The multi-target tracking model is used to obtain the bbox detection frame information of each frame image in the mite image sequence; Obtain a set of mite motion images according to the bbox detection frame information and the mite image sequence, and determine whether the mite motion image set is in a continuous state; If the mite motion image set is in a continuous state, normalization processing is performed on each frame of the mite image in the mite motion image set to obtain a mite detection image sample set; Perform feature extraction on the sample set of mite detection images to obtain PCA feature information and HOG feature information; Calculating inter-frame similarity based on the mite detection image sample set, PCA feature information, and HOG feature information to obtain a first structural similarity result, a second structural similarity result, and a third structural similarity result; A live mite score is performed according to the first structure similarity result, the second structure similarity result and the third structure similarity result to obtain a live mite detection result.

2. The method for detecting live mites according to claim 1, characterized in that: The method of using the multi-target tracking model to obtain the detection frame information of each frame image in the mite image sequence includes: Perform forward reasoning on each frame of the mite image sequence through the YOLOV5 model to obtain the first detection frame information; Construct a multi-target tracking model based on the SORT algorithm; Calling the Kalman filter algorithm from the multi-target tracking model, and performing motion modeling on each frame image in the mite image sequence based on the Kalman filter algorithm to obtain a target mite motion model; Predicting the coordinates of the target mite in the next frame according to the target mite motion model to obtain the second detection frame information; The Hungarian algorithm is used to calculate the intersection-over-union ratio coefficient between the first detection frame information and the second detection frame information, and the intersection-over-union ratio coefficient is compared with a preset matching threshold; If the intersection-over-union ratio coefficient is greater than the preset matching threshold, the first detection box information and the second detection box information are combined to obtain the bbox detection box information.

3. The method for detecting live mites according to claim 1, characterized in that: The step of obtaining a mite motion image set according to the bbox detection frame information and the mite image sequence, and determining whether the mite motion image set is in a continuous state, includes: According to the preset continuous detection frame number and bbox detection frame information, the mite image of each mite is obtained from each frame image of the mite image sequence, and the mite head portraits are associated and matched to form a mite motion image set; The image coordinates of each frame of the mite image in the mite motion image set are obtained, and whether the mite motion image set is in a continuous state is determined according to the continuity of the image coordinates of each frame.

4. The method for detecting live mites according to claim 3, characterized in that: After obtaining the image coordinates of each frame of the mite image in the mite motion image set and judging whether the mite motion image set is in a continuous state according to the continuity of the image coordinates of each frame, the method further includes: If the mite motion image set is in a discontinuous state, the number of broken frames of each frame image coordinate is obtained, and the number of broken frames is compared with a preset broken frame threshold; If the number of broken frames is less than or equal to the preset broken frame threshold, the interpolation method is used to perform frame interpolation processing on the mite motion image set to obtain a frame interpolated mite motion image set; If the number of broken frames is greater than a preset broken frame threshold, the mite motion image set is exported to a preset recycling pool.

5. The method for detecting live mites according to claim 4, characterized in that: The feature extraction of the mite detection image sample set to obtain PCA feature information and HOG feature information includes: Obtain a mite sample set vector matrix according to the detection image sample set, and calculate a mite covariance matrix according to the mite sample set vector matrix; Perform eigenvalue decomposition on the mite covariance matrix, and sort the eigenvalues ​​in descending order to obtain the top N eigenvalues; Obtain the corresponding eigenvectors according to the N eigenvalues, and perform standardization on each eigenvector to form an eigenvector matrix; The PCA feature information is obtained according to the feature vector matrix and the mite sample set vector matrix; Grayscale and gamma correction are performed on the detection image sample set, and pixel gradients of the processed detection image sample set are calculated according to a preset operator; Counting pixel gradients within a preset pixel area to form a gradient histogram; Get the feature vector of the gradient histogram to form HOG feature information.

6. The method for detecting live mites according to claim 1, characterized in that: The inter-frame similarity calculation is performed according to the mite detection image sample set, the PCA feature information and the HOG feature information to obtain a first structural similarity result, a second structural similarity result and a third structural similarity result, including: Add a sliding detection window to each image in the mite detection image sample set, and obtain brightness data, contrast data and structure data of each image through the sliding detection window; Calculating the structural similarity between adjacent images according to the brightness data, the contrast data and the structural data, and performing mean calculation according to each structural similarity to obtain a first structural similarity result; The cosine similarity algorithm is used to calculate the cosine similarity of the PCA feature information, and a second structural similarity result is formed according to each cosine similarity; The cosine similarity algorithm is used to calculate the cosine similarity of the HOG feature information, and a third structural similarity result is formed according to each cosine similarity.

7. The method for detecting live mites according to claim 6, characterized in that: The performing of a mite liveness scoring according to the first structure similarity result, the second structure similarity result and the third structure similarity result to obtain a live mite detection result includes: Determine whether the first structural similarity result is greater than a first preset similarity threshold to obtain a first comparison result; Determine whether each cosine similarity in the second structural similarity result is greater than a second preset similarity threshold, and count the number of cosine similarities greater than the second preset similarity threshold to obtain a second comparison result; Determine whether each cosine similarity in the third structural similarity result is greater than a third preset similarity threshold, and count the number of cosine similarities greater than the third preset similarity threshold to obtain a third comparison result; Obtaining a live mite score value according to the first comparison result, the second comparison result, and the third comparison result; The live mite score value is compared with the preset live benchmark value to obtain the live mite detection result.

8. A live mite detection device, characterized in that: include: An imaging module, used for acquiring a continuous sequence of mite images based on a large field of view imaging technique; A tracking module is used to obtain the bbox detection frame information of each frame image of the mite image sequence using a multi-target tracking model; A continuous module is used to obtain a mite motion image set according to the bbox detection frame information and the mite image sequence, and to determine whether the mite motion image set is in a continuous state; A normalization module is used for normalizing each frame of the mite image in the mite motion image set if the mite motion image set is in a continuous state, so as to obtain a mite detection image sample set; The feature module is used to extract features from the sample set of mite detection images to obtain PCA feature information and HOG feature information; A calculation module, used for performing inter-frame similarity calculation based on the mite detection image sample set, PCA feature information and HOG feature information to obtain a first structural similarity result, a second structural similarity result and a third structural similarity result; The scoring module is used to score the liveness of mites according to the first structure similarity result, the second structure similarity result and the third structure similarity result to obtain the liveness mite detection result.

9. A live mite detection device, characterized in that: The live mite detection device comprises: a memory and at least one processor, wherein instructions are stored in the memory; At least one of the processors calls the instructions in the memory to enable the live mite detection device to perform each step of the live mite detection method as described in any one of claims 1 to 7.

10. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instructions are executed by the processor, the steps of the method for detecting live mites as described in any one of claims 1 to 7 are implemented.

Images