Live mite detection method, device, apparatus and storage medium

Abstract

The present application relates to the technical field of image recognition, and particularly relates to a living mite detection method, device and equipment and a storage medium, through a large field of view technology, static images of mites at different time points are captured for mite samples to obtain a continuous mite image sequence; a multi-target tracking model is used to accurately identify the specific position of each frame of mite in the image, the moving path, speed and behavior pattern of the mite can be observed, and the identification precision of the activity of the mite is improved; the gradient direction histogram of a local area of the image is calculated to describe the texture and shape features of the image, the main components and edge information in the image are extracted, the system can more deeply understand the internal features of the mite image, and the similarity of the mite image can be evaluated; by comparing the similarity between different frames, the system can determine whether the mites are the same individual or whether there is a significant morphological change, and whether the mites are living bodies is determined according to the similarity result.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and in particular to a method, apparatus, device, and storage medium for detecting live mites. Background Technology

[0002] The number of live mites is a crucial indicator of the mite-proof performance of daily necessities. Traditional methods for detecting live mites typically involve manual counting. This involves creating a localized image of the mites in a sample using a microscope, and then visually identifying whether the mites are moving to determine their liveness. This method relies heavily on manual operation and carries significant human risks, including the movement of the microscopic imaging area, adjustment of the focus, and visual counting. Furthermore, when a person is fatigued after prolonged work, the probability of miscounting increases significantly, affecting the accuracy of the test results.

[0003] It is evident that existing technologies still need improvement and enhancement. Summary of the Invention

[0004] In order to overcome the shortcomings of the prior art, the present invention aims to provide a method, device, equipment and storage medium for detecting live mites, which is based on large field-of-view imaging to acquire images that continuously cover the entire mite sample, and judges the liveness of mites based on the image features of each image, so as to improve the accuracy of live mite identification.

[0005] The first aspect of this invention provides a method for detecting live mites, comprising: acquiring a continuous mite image sequence based on a large field-of-view imaging technique; acquiring bounding box (bbox) information of each frame of the mite image sequence using a multi-target tracking model; acquiring a set of mite motion images based on the bbox information and the mite image sequence, and determining whether the set of mite motion images is continuous; if the set of mite motion images is continuous, normalizing each frame of the mite motion image set to obtain a mite detection image sample set; extracting features from the mite detection image sample set to obtain PCA feature information and HOG feature information; calculating inter-frame similarity based on the mite detection image sample set, PCA feature information, and HOG feature information to obtain a first structural similarity result, a second structural similarity result, and a third structural similarity result; and scoring the liveness of mites based on the first structural similarity result, the second structural similarity result, and the third structural similarity result to obtain a live mite detection result.

[0006] Optionally, in a first implementation of the first aspect of the present invention, the step of obtaining the detection box information of each frame of the mite image sequence using a multi-target tracking model includes: performing forward inference on each frame of the mite image sequence using a YOLOv5 model to obtain first detection box information; constructing a multi-target tracking model based on the SORT algorithm; calling the Kalman filter algorithm from the multi-target tracking model and performing motion modeling on each frame of the mite image sequence based on the Kalman filter algorithm to obtain a target mite motion model; predicting the coordinates of the target mite in the next frame based on the target mite motion model to obtain second detection box information; calculating the cross-union ratio (CUP) between the first and second detection box information using the Hungarian algorithm and comparing the CUP with a preset matching threshold; if the CUP is greater than the preset matching threshold, combining the first and second detection box information to obtain bbox detection box information.

[0007] Optionally, in a second implementation of the first aspect of the present invention, the step of obtaining a set of mite motion images based on the bbox detection box information and the mite image sequence, and determining whether the set of mite motion images is in a continuous state, includes: obtaining the mite image of each mite from each frame of the mite image sequence based on a preset number of continuous detection frames and the bbox detection box information, and associating and matching each mite head image to form a set of mite motion images; obtaining the image coordinates of each frame of mite images in the set of mite motion images, and determining whether the set of mite motion images is in a continuous state based on the continuity of the image coordinates of each frame.

[0008] Optionally, in the third implementation of the first aspect of the present invention, after obtaining the image coordinates of each frame of mite image in the mite motion image set and determining whether the mite motion image set is in a continuous state based on the continuity of the image coordinates of each frame, the method further includes: if the mite motion image set is in a discontinuous state, obtaining the number of broken frames of each frame image coordinate and comparing the number of broken frames with a preset broken frame threshold; if the number of broken frames is less than or equal to the preset broken frame threshold, performing frame interpolation on the mite motion image set to obtain a frame-interpolated mite motion image set; if the number of broken frames is greater than the preset broken frame threshold, exporting the mite motion image set to a preset recycling pool.

[0009] Optionally, in the fourth implementation of the first aspect of the present invention, the step of extracting features from the mite detection image sample set to obtain PCA feature information and HOG feature information includes: obtaining a mite sample set vector matrix based on the detection image sample set, and calculating a mite covariance matrix based on the mite sample set vector matrix; performing eigenvalue decomposition on the mite covariance matrix, and sorting the feature values ​​in descending order of value to obtain the top N feature values; obtaining the corresponding feature vectors based on the N feature values, and standardizing each feature vector to form a feature vector matrix; obtaining PCA feature information based on the feature vector matrix and the mite sample set vector matrix; performing grayscale and gamma correction processing on the detection image sample set, and calculating the pixel gradient of the processed detection image sample set according to a preset operator; statistically analyzing the pixel gradients within a preset pixel region to form a gradient histogram; and obtaining the feature vectors of the gradient histogram to form HOG feature information.

[0010] Optionally, in a fifth implementation of the first aspect of the present invention, the step of calculating inter-frame similarity based on the mite detection image sample set, PCA feature information, and HOG feature information to obtain a first structural similarity result, a second structural similarity result, and a third structural similarity result includes: adding a sliding detection window to each image in the mite detection image sample set, and obtaining brightness data, contrast data, and structural data of each image through the sliding detection window; calculating the structural similarity between adjacent images based on the brightness data, contrast data, and structural data, and calculating the mean of each structural similarity to obtain a first structural similarity result; calculating the cosine similarity of the PCA feature information using a cosine similarity algorithm, and forming a second structural similarity result based on each cosine similarity; calculating the cosine similarity of the HOG feature information using a cosine similarity algorithm, and forming a third structural similarity result based on each cosine similarity.

[0011] Optionally, in a sixth implementation of the first aspect of the present invention, the step of performing a live mite score based on the first structural similarity result, the second structural similarity result, and the third structural similarity result to obtain a live mite detection result includes: determining whether the first structural similarity result is greater than a first preset similarity threshold to obtain a first comparison result; determining whether each cosine similarity in the second structural similarity result is greater than the second preset similarity threshold, and counting the number of cosine similarities greater than the second preset similarity threshold to obtain a second comparison result; determining whether each cosine similarity in the third structural similarity result is greater than the third preset similarity threshold, and counting the number of cosine similarities greater than the third preset similarity threshold to obtain a third comparison result; obtaining a live mite score value based on the first comparison result, the second comparison result, and the third comparison result; and comparing the live mite score value with a preset live baseline value to obtain a live mite detection result.

[0012] A second aspect of the present invention provides a live mite detection device, comprising: an imaging module for acquiring a continuous mite image sequence based on a large field-of-view imaging technique; a tracking module for acquiring bounding box (bbox) information of each frame of the mite image sequence using a multi-target tracking model; a continuity module for acquiring a set of mite motion images based on the bbox information and the mite image sequence, and determining whether the set of mite motion images is continuous; and a normalization module for normalizing each frame of the mite motion image set if the set of mite motion images is continuous. The system is configured to: obtain a mite detection image sample set; perform feature extraction on the mite detection image sample set to obtain PCA feature information and HOG feature information; perform inter-frame similarity calculation based on the mite detection image sample set, PCA feature information, and HOG feature information to obtain a first structural similarity result, a second structural similarity result, and a third structural similarity result; and perform live mite scoring based on the first structural similarity result, the second structural similarity result, and the third structural similarity result to obtain live mite detection results.

[0013] A third aspect of the present invention provides a live mite detection device, the live mite detection device comprising: a memory and at least one processor, the memory storing instructions; the at least one processor calling the instructions in the memory to cause the live mite detection device to perform the various steps of the live mite detection method described in any of the preceding claims.

[0014] A fourth aspect of the present invention provides a computer-readable storage medium storing instructions that, when executed by a processor, implement the steps of the live mite detection method described in any of the preceding claims.

[0015] In the technical solution of this invention, a high-resolution, wide-field-of-view camera captures static images of mites at different time points, recording their dynamic behavior to obtain a continuous mite image sequence. This image sequence allows for the identification of each mite and the movement of its legs. A multi-target tracking model accurately identifies the specific location of mites in each frame of the image. Analysis of consecutive frames reveals the mite's movement path, speed, and behavioral patterns, improving the accuracy of mite activity identification. The system describes image texture and shape features by calculating the gradient orientation histogram of local image regions. Extracting the main components and edge information from the image allows for a deeper understanding of the intrinsic features of the mite images, aiding in the assessment of image similarity. By comparing the similarity between different frames, the system can determine whether the mites are the same individual or whether there are significant morphological changes, and based on the similarity results, determine whether the mites are alive. Attached Figure Description

[0016] Figure 1 This is a first flowchart of a live mite detection method provided in an embodiment of the present invention;

[0017] Figure 2 This is a second flowchart of the live mite detection method provided in an embodiment of the present invention;

[0018] Figure 3 This is a third flowchart of the live mite detection method provided in the embodiments of the present invention;

[0019] Figure 4 This is a fourth flowchart of the live mite detection method provided in the embodiments of the present invention;

[0020] Figure 5 This is a fifth flowchart of the live mite detection method provided in the embodiments of the present invention;

[0021] Figure 6 This is the sixth flowchart of the live mite detection method provided in the embodiments of the present invention;

[0022] Figure 7 This is a schematic diagram of the structure of the live mite detection device provided in an embodiment of the present invention;

[0023] Figure 8 This is a schematic diagram of the structure of the live mite detection device provided in an embodiment of the present invention;

[0024] Figure 9 Example image of optical imaging of a mite sample;

[0025] Figure 10 A flowchart illustrating the normalization process for mite images in the live mite detection method provided in this embodiment of the invention.

[0026] Figure 11 A flowchart illustrating the liveness scoring of mite images in the live mite detection method provided in this embodiment of the invention. Detailed Implementation

[0027] This invention provides a method, apparatus, device, and storage medium for detecting live mites. The invention captures static images of mites at different time points using a high-resolution, wide-field-of-view camera, recording their dynamic behavior to obtain a continuous mite image sequence. This image sequence allows for the identification of each mite and the movement of its legs. A multi-target tracking model accurately identifies the specific location of mites in each frame of the image. Analysis of consecutive frames reveals the mite's movement path, speed, and behavioral patterns, improving the accuracy of mite activity identification. The system describes image texture and shape features by calculating the gradient orientation histogram of local image regions. Extracting key components and edge information from the image allows for a deeper understanding of the intrinsic features of mite images, aiding in the assessment of image similarity. By comparing the similarity between different frames, the system can determine whether the mites belong to the same individual or exhibit significant morphological changes, and ultimately, determine whether the mites are alive based on the similarity results.

[0028] The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" or "having" and any variations thereof are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0029] For ease of understanding, the specific process of the embodiments of the present invention is described below. Please refer to [link / reference]. Figure 1 One embodiment of the live mite detection method in this invention includes:

[0030] 101. Acquiring continuous mite image sequences based on large field-of-view imaging technology;

[0031] 102. Use a multi-target tracking model to obtain the bounding box information of each frame of a mite image sequence;

[0032] 103. Obtain a set of mite motion images based on the bounding box information and the mite image sequence, and determine whether the set of mite motion images is continuous.

[0033] In this embodiment, a high-resolution, wide-field-of-view camera is used to image the mite sample, acquiring a continuous sequence of mite images. This image sequence ensures the integrity and continuity of the sample. This process not only captures static images of mites at different time points but also records their dynamic behavior, enabling the identification of the movement of each mite and its legs. A multi-target tracking model is used to acquire the bounding box information of each frame, achieving precise localization of multiple mites within the same image. The multi-target tracking model accurately identifies the specific location of each mite in each frame, providing a data foundation for subsequent motion analysis and behavioral pattern recognition. The acquisition of the mite motion image set and the determination of its continuity help analyze the mite's movement trajectory and behavioral characteristics. Through the analysis of continuous frame images, the system can observe the mite's movement path, speed, and behavioral patterns, which is of great significance for understanding mite activity.

[0034] like Figure 9 As shown, the image in the figure is a mite image obtained by optical imaging of a mite sample using a high-resolution wide field-of-view camera.

[0035] 104. If the mite motion image set is continuous, normalize each frame of the mite motion image set to obtain a mite detection image sample set.

[0036] In this embodiment, the continuous mite motion image set is normalized to improve the accuracy and efficiency of subsequent processing. Since the target size of the images obtained from target tracking is inconsistent, that is, the size of the detection box of the same target in each frame will change, and the similarity calculation of image information needs to ensure that the two vector dimensions are the same, a set of continuous mite images needs to be processed to have a uniform size.

[0037] like Figure 10 As shown, the specific steps are as follows: Traverse the information of the mite motion image set, count the length and width of each frame image, record the maximum value of the length and width respectively, use this maximum value as a template to fill each image, and finally obtain the mite detection image sample set.

[0038] 105. Extract features from the mite detection image sample set to obtain PCA feature information and HOG feature information;

[0039] 106. Calculate inter-frame similarity based on the mite detection image sample set, PCA feature information, and HOG feature information to obtain the first structural similarity result, the second structural similarity result, and the third structural similarity result;

[0040] 107. Based on the first structural similarity result, the second structural similarity result, and the third structural similarity result, a live mite score is calculated to obtain the live mite detection result.

[0041] In this embodiment, PCA and HOG feature information are extracted from the mite detection image sample set. PCA features can reflect the main contour information of the image and retain the main structural features of the image through dimensionality reduction. HOG features can reflect the edge direction information of the image. The image texture and shape features are described by calculating the gradient direction histogram of the local region of the image. By extracting the main components and edge information in the image, the system can gain a deeper understanding of the intrinsic features of the mite image.

[0042] Frame-to-frame similarity calculations yield structural similarity results, which help assess the similarity of mite images. By comparing the similarity between different frames, the system can determine whether the mites belong to the same individual or whether there are significant morphological changes, which is crucial for liveness detection. Based on the similarity results, the system determines whether the mites are alive and ultimately outputs the live mite detection result. Combining the data and analysis results obtained from the above steps, the system can accurately score whether mites are alive, thus providing reliable detection results for the dynamic statistics of live mites.

[0043] In this embodiment of the invention, a high-resolution, wide-field-of-view camera captures static images of mites at different time points, recording their dynamic behavior to obtain a continuous mite image sequence. This image sequence allows for the identification of each mite and the movement of its legs. A multi-target tracking model accurately identifies the specific location of mites in each frame of the image. Analysis of consecutive frames reveals the mite's movement path, speed, and behavioral patterns, improving the accuracy of mite activity identification. The system describes image texture and shape features by calculating the gradient orientation histogram of local image regions. Extracting key components and edge information from the image allows for a deeper understanding of the intrinsic characteristics of mite images, aiding in the assessment of image similarity. By comparing the similarity between different frames, the system can determine whether the mites are the same individual or whether there are significant morphological changes, and based on the similarity results, determine whether the mites are alive.

[0044] Please see Figure 2 The second embodiment of the live mite detection method in this invention includes:

[0045] 201. Using the YOLOv5 model, perform forward inference on each frame of the mite image sequence to obtain the first detection box information;

[0046] 202. Construct a multi-target tracking model based on the SORT algorithm;

[0047] 203. Call the Kalman filter algorithm from the multi-target tracking model, and perform motion modeling on each frame of the mite image sequence based on the Kalman filter algorithm to obtain the target mite motion model;

[0048] 204. Predict the coordinates of the target mite in the next frame based on the target mite motion model to obtain the second detection box information;

[0049] 205. Calculate the cross-union ratio (CUP) between the first and second detection box information using the Hungarian algorithm, and compare the CUP with a preset matching threshold.

[0050] 206. If the crossover ratio is greater than the preset matching threshold, the first detection box information and the second detection box information are combined to obtain the bbox detection box information.

[0051] In this embodiment, the YOLOv5 model is used to perform forward inference on the mite image sequence to obtain preliminary detection box information for mites in each frame. This process improves detection accuracy, ensures processing speed, and uses this first detection box information as a baseline reference. A multi-target tracking model based on the SORT algorithm is then constructed. This model uses the Kalman filter algorithm to build a motion model of the target mite and predicts the coordinates of the target mite in the next frame based on this model, thereby obtaining the second detection box information. This process not only enhances the accuracy of predictions but also provides reliable data support for subsequent detection box information fusion. After obtaining the first and second detection box information, the Hungarian algorithm is used to calculate the intersection-union ratio (IU / R) between them and compares it with a preset matching threshold. This method ensures the matching degree of the detection box information, effectively filtering out high-quality detection results. Finally, when the IU / R is greater than the preset matching threshold, the system combines the first and second detection box information to obtain the final bounding box (bbox) information. This synthesis process improves the accuracy and stability of the detection box information and enhances the robustness of the data by combining predicted detection box information. The entire process remains stable under different environmental conditions, improving the model's recognition performance in complex scenes. Whether it's changes in lighting, background interference, or the rapid movement of mites, the model can adapt and accurately identify target mites. This robustness allows researchers to conduct effective monitoring under various experimental conditions, providing a solid foundation for subsequent research and applications.

[0052] To aid understanding, the following explanation is provided: First, the YOLOv5 model is used as the detector. Each image in the mite image sequence is input into the YOLOv5 model, and then processed into images of the same size before forward inference. At the same time, the target category information and detection box information, i.e., the first detection box information, are output. Then, non-maximum suppression is performed on the first detection box information to remove redundant detection boxes superimposed on the same target.

[0053] After obtaining the detection results from the multi-target tracking model, the model uses Kalman filtering to predict the mathematical model of target motion, obtaining predicted values. Then, the Hungarian algorithm correlates the predicted values ​​obtained from the Kalman filtering with the observation values ​​obtained from the detector to find the optimal estimate. The specific process is as follows: Now, it is necessary to obtain the detection box information of a certain target at frame t, denoted as bbox. t optimal (This includes the width w and height h of the target rectangle, as well as the x and y coordinates of the rectangle's center point within the entire image.) Currently known information includes the bounding box information (bbox) for frames 1 to t-1. 1 ~t-1 Next, the detection box information of frame t is obtained through two methods. The first method is through Kalman filtering mathematical modeling, based on the bounding boxes of frames 1 to t-1. 1~t-1 Predict the detection bounding box information for frame t, denoted as bbox. t prediction The second method is to directly detect the target bounding box information in frame t using a detector, denoted as bbox. t detection Finally, the Hungarian algorithm is used to analyze the bounding box of frame t-1. 1~t-1 and bbox t detection The process of performing correlation matching actually involves calculating the intersection-union (IU) ratio of the bounding boxes (bboxes) in two frames. If the IU ratio is greater than a preset threshold, it indicates that the bboxes have been successfully matched. Finally, the bboxes are combined... t prediction and bbox t detection Get more accurate bbox t optimal The bbox t optimal This refers to the bbox detection frame information.

[0054] Please see Figure 3 The third embodiment of the live mite detection method in this invention includes:

[0055] 301. Based on the preset number of consecutive detection frames and bbox detection information, obtain the mite image of each mite from each frame of the mite image sequence, and perform association matching on each mite head image to form a set of mite motion images;

[0056] 302. Obtain the image coordinates of each frame of mite images in the mite motion image set, and determine whether the mite motion image set is in a continuous state based on the continuity of the image coordinates of each frame.

[0057] 303. If the mite motion image set is in a discontinuous state, obtain the number of broken frames for each frame image coordinate and compare the number of broken frames with the preset broken frame threshold.

[0058] 304. If the number of broken frames is less than or equal to the preset broken frame threshold, the frame interpolation method is used to perform frame interpolation on the mite motion image set to obtain the frame interpolated mite motion image set.

[0059] 305. If the number of broken frames is greater than the preset broken frame threshold, the mite motion image set will be exported to the preset recycling pool.

[0060] In this embodiment, by analyzing and processing the continuity of each frame of mite movement images, the integrity and coherence of the mite movement trajectory can be ensured, thereby greatly improving the accuracy of subsequent data processing and analysis. This process not only improves the readability of the mite movement trajectory but also optimizes the identification and analysis of mite movement behavior. For example, when continuity detection shows that the mite movement image set is discontinuous, the system can promptly detect and take measures to perform frame interpolation on the mite movement image set, thereby obtaining a more complete and accurate mite movement image set. This frame interpolation process not only avoids data analysis errors caused by missing frames but also ensures the quality and effectiveness of data processing. Furthermore, when the number of broken frames exceeds a preset broken frame threshold, the system will export the mite movement image set to a preset recycling pool, discarding the unrepairable images to ensure the integrity and reliability of the data.

[0061] To aid understanding, the following example illustrates the process: For continuous motion images of each mite, mites are first categorized into two groups based on their tracking results: The first group has continuous, uninterrupted tracking. For example, in the current ten-frame tracking images, the mite has image coordinates (bbox) in every frame, and these coordinates are continuous. This group is processed normally, proceeding directly to the next step. The second group has discontinuous, phased tracking results. For example, in the current ten-frame tracking images, the mite has no position coordinates (bbox) in the fourth frame. In this case, frame interpolation is used, averaging the results from the third and fifth frames, and setting the position coordinates (bbox) of the third frame as... 3 The position coordinates (bbox) of frame 5 are (x3, y3, w3, h3). 5Given (x5, y5, w5, h5), the position coordinates of the bbox in frame 4 are... 4 (x4, y4, w4, h4) is replaced by the average of two adjacent frames, where x and y values ​​represent coordinate values; in addition, if the mite has no tracked position coordinates for more than 3 consecutive frames, the similarity calculation of the current mite is abandoned.

[0062] Please see Figure 4 The fourth embodiment of the live mite detection method in this invention includes:

[0063] 401. Obtain the mite sample set vector matrix based on the detected image sample set, and calculate the mite covariance matrix based on the mite sample set vector matrix.

[0064] 402. Perform eigenvalue decomposition on the mite covariance matrix and sort the eigenvalues ​​in descending order of value to obtain the top N eigenvalues.

[0065] 403. Obtain the corresponding eigenvectors based on N eigenvalues, and standardize each eigenvector to form an eigenvector matrix;

[0066] 404. Obtain PCA feature information based on the feature vector matrix and the mite sample set vector matrix;

[0067] 405. Perform grayscale and gamma correction processing on the detection image sample set, and calculate the pixel gradient of the processed detection image sample set according to the preset operator;

[0068] 406. Calculate the pixel gradient within the preset pixel region to form a gradient histogram;

[0069] 407. Obtain the feature vectors of the gradient histogram to form HOG feature information.

[0070] In this embodiment, extracting the mite sample set vector matrix from the detected image sample set captures key information about the mite samples. Subsequently, the calculated mite covariance matrix further reveals the correlation between samples. Eigenvalue decomposition is performed on the mite covariance matrix, and the values ​​are sorted according to their magnitude. This process ensures that the system can identify the most critical feature values ​​for distinguishing different mite samples. By selecting the top N feature values, the system can focus on the most representative features, thereby improving the efficiency and effectiveness of feature extraction. Feature vectors corresponding to these key feature values ​​are obtained and standardized to form a feature vector matrix. This process not only makes the feature vectors more standardized but also provides a clear and consistent data format for subsequent PCA feature extraction. Using the feature vector matrix and the mite sample set vector matrix, PCA feature information can be obtained. PCA, as a powerful dimensionality reduction technique, can effectively reduce the dimensionality of data while retaining the most important information. This not only simplifies the calculation process but also improves processing speed and efficiency.

[0071] Grayscale conversion and gamma correction of the detection image sample set are performed to eliminate the influence of illumination changes on the image and ensure the accuracy of subsequent processing. Calculating the pixel gradient of the processed detection image sample set helps to capture detailed information in the image. Statistical analysis of pixel gradients within a preset pixel region to form a gradient histogram helps the system better understand the texture features of different regions in the image. As an effective image description method, the gradient histogram can provide rich data support for HOG feature extraction. Obtaining the feature vector of the gradient histogram to form HOG feature information allows us to extract directional local features from the image. HOG feature information is very effective in describing the shape and texture in the image, which helps to improve the accuracy of mite detection and classification.

[0072] Please see Figure 5 The fifth embodiment of the live mite detection method in this invention includes:

[0073] 501. Add a sliding detection window to each image in the mite detection image sample set, and obtain the brightness data, contrast data and structure data of each image through the sliding detection window;

[0074] 502. Calculate the structural similarity between adjacent images based on brightness data, contrast data, and structural data, and calculate the mean of each structural similarity to obtain the first structural similarity result.

[0075] 503. The cosine similarity algorithm is used to calculate the cosine similarity of PCA feature information, and the second structural similarity result is formed based on each cosine similarity.

[0076] 504. The cosine similarity algorithm is used to calculate the cosine similarity of HOG feature information, and the third structural similarity result is formed based on each cosine similarity.

[0077] In this embodiment, when calculating the similarity of mite images, the system first combines brightness data, contrast data, and structural data to analyze the images from multiple dimensions. This helps to capture subtle differences in the images, calculate the structural similarity between adjacent images, and calculate the mean of these similarity values. This effectively smooths out some deviations caused by accidental factors, thereby obtaining a more stable and reliable structural similarity result.

[0078] Secondly, the cosine similarity algorithm is used to calculate the similarity of PCA (Principal Component Analysis) feature information. This method can effectively measure the angle between feature vectors, thereby determining their degree of similarity. Since PCA can extract the main features of an image, the second structural similarity result obtained by the cosine similarity algorithm can reflect the similarity of images in their main features.

[0079] Similarly, the cosine similarity calculation of HOG (Histogram of Oriented Gradients) feature information focuses on the local texture information of the image. HOG features can capture the shape and edge information of objects in the image. Therefore, the third structural similarity result obtained by the cosine similarity algorithm can further supplement and improve the evaluation of image similarity.

[0080] By combining the results of these three different structural similarities, the system can provide a more comprehensive and accurate assessment of the similarity of mite images. This multi-angle, multi-level analysis method not only improves the accuracy of similarity calculation but also better adapts to image processing needs under different environments and conditions.

[0081] It should be noted that the formula for calculating cosine similarity is as follows:

[0082]

[0083] In the formula, F represents the image feature, i represents the i-th frame, and N represents the number of images in this set of images. i and F i+1 This represents the image features of the i-th frame and the (i+1)-th frame.

[0084] Please see Figure 6 The sixth embodiment of the live mite detection method in this invention includes:

[0085] 601. Determine whether the first structural similarity result is greater than the first preset similarity threshold to obtain the first comparison result;

[0086] 602. Determine whether each cosine similarity in the second structural similarity result is greater than the second preset similarity threshold, and count the number of cosine similarities greater than the second preset similarity threshold to obtain the second comparison result;

[0087] 603. Determine whether each cosine similarity in the third structural similarity result is greater than the third preset similarity threshold, and count the number of cosine similarities greater than the third preset similarity threshold to obtain the third comparison result;

[0088] 604. Based on the first comparison result, the second comparison result, and the third comparison result, obtain the live mite score;

[0089] 605. Compare the live mite score with the preset live baseline value to obtain the live mite detection result.

[0090] In this embodiment, during the process of identifying live mites, a first structural similarity comparison is first performed. By calculating and comparing the similarity between the target sample and the known live mite structure, it is determined whether it exceeds a first preset similarity threshold. If the first comparison result shows insufficient similarity, then the sample is likely to be a non-live or non-target species of mite, thereby avoiding further erroneous analysis.

[0091] The system then enters the second stage, analyzing the second structural similarity results. In this stage, the system calculates multiple cosine similarity values ​​and checks whether each exceeds a second preset similarity threshold. By counting the number of cosine similarity values ​​that meet the criteria, the system obtains a more accurate second comparison result. This process helps identify subtle differences that might have been overlooked in the first stage, further improving recognition accuracy. The third comparison result is obtained by calculating and comparing cosine similarity values ​​and counting the number exceeding a third preset similarity threshold. This stage of analysis helps capture features that might not have been fully recognized in the first two stages, thereby further improving the overall recognition accuracy.

[0092] By combining the results of the first, second, and third comparisons, the system calculates a comprehensive live mite score. This score is based on multi-dimensional analysis and provides a more comprehensive reflection of the sample's liveness characteristics. Finally, the system compares this score with a preset liveness benchmark. If the score meets the benchmark criteria, the sample is identified as containing live mites; otherwise, it is identified as non-live. This comprehensive scoring and comparison method not only improves the accuracy of identification but also reduces the possibility of false positives and false negatives, ensuring the reliability of the detection results.

[0093] like Figure 11As shown, to facilitate understanding, the following example is provided: Only one optional case is given here, taking the mite similarity information from the past 10 frames of images, Threshold. SSIM (The first preset similarity threshold) is set to 0.8. If If the first structural similarity result is greater than 0.8, then the similarity score P... SIM Add 1; Threshold PCA (The second preset similarity threshold) is set to 0.95. PCA values ​​greater than 0.95 in the second structural similarity results i If the number is greater than 3, then the similarity score P SIM Add 1; set Threshold HOG (The third preset similarity threshold) is 0.85. If If the (third structural similarity result) is greater than 0.85, then the similarity score P SIM Add 1. Finally, calculate the similarity score for all mites, with a preset live baseline value of 2. If P SIM A number of mites greater than or equal to 2 will be considered dead mites, meaning the similarity between images is too high. If P SIM Mites with a count less than 2 will be considered live mites.

[0094] The above describes the method for detecting live mites in embodiments of the present invention. The following describes the device for detecting live mites in embodiments of the present invention. Please refer to [link / reference]. Figure 7 One embodiment of the live mite detection device in this invention includes:

[0095] Imaging module 701 is used to acquire a continuous sequence of mite images based on large field-of-view imaging technology;

[0096] The tracking module 702 is used to obtain the bounding box information of each frame of the mite image sequence using a multi-target tracking model;

[0097] The continuous module 703 is used to obtain a set of mite motion images based on the bbox detection box information and the mite image sequence, and to determine whether the set of mite motion images is in a continuous state.

[0098] The normalization module 704 is used to normalize each frame of mite images in the mite motion image set if the mite motion image set is in a continuous state, so as to obtain a mite detection image sample set.

[0099] Feature module 705 is used to extract features from the mite detection image sample set to obtain PCA feature information and HOG feature information;

[0100] The calculation module 706 is used to perform inter-frame similarity calculation based on the mite detection image sample set, PCA feature information and HOG feature information to obtain the first structural similarity result, the second structural similarity result and the third structural similarity result;

[0101] The scoring module 707 is used to score the live mites based on the first structural similarity result, the second structural similarity result, and the third structural similarity result, so as to obtain the live mite detection result.

[0102] In this embodiment, the imaging module 701 and the tracking module 702 capture static images of mites at different time points using a high-resolution, wide-field-of-view camera, recording their dynamic behavior to obtain a continuous mite image sequence. This image sequence allows for the identification of each mite and the movement of its legs. The continuity module 703 and the normalization module 704 utilize a multi-target tracking model to accurately identify the specific location of mites in each frame of the image. Analysis of continuous frame images allows for the observation of the mite's movement path, speed, and behavioral patterns, improving the accuracy of mite activity identification. The feature module 705 describes image texture and shape features by calculating the gradient orientation histogram of local image regions. By extracting the main components and edge information in the image, the system can gain a deeper understanding of the intrinsic features of the mite images, aiding in the assessment of mite image similarity. The calculation module 706 and the scoring module 707 compare the similarity between different frames to determine whether the mites are the same individual or whether there are significant morphological changes. Based on the similarity results, the system determines whether the mites are alive.

[0103] above Figure 7 The live mite detection device in this embodiment of the invention will be described in detail from the perspective of modular functional entities. The live mite detection device in this embodiment of the invention will be described in detail from the perspective of hardware processing.

[0104] Figure 8This is a schematic diagram of the structure of a live mite detection device 800 provided in an embodiment of the present invention. The live mite detection device 800 can vary significantly due to different configurations or performance. It may include one or more central processing units (CPUs) 810 (e.g., one or more processors) and a memory 820, and one or more storage media 830 (e.g., one or more mass storage devices) storing application programs 833 or data 832. The memory 820 and storage media 830 can be temporary or persistent storage. The program stored in the storage media 830 may include one or more modules (not shown in the diagram), each module including a series of instruction operations on the live mite detection device 800. Furthermore, the processor 810 may be configured to communicate with the storage media 830 and execute the series of instruction operations in the storage media 830 on the live mite detection device 800 to implement the steps of the live mite detection method provided in the above-described method embodiments.

[0105] The live mite detection device 800 may also include one or more power supplies 840, one or more wired or wireless network interfaces 850, one or more input / output interfaces 860, and / or one or more operating systems 831, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. Those skilled in the art will understand that... Figure 8 The illustrated structure of the live mite detection device does not constitute a limitation on the live mite detection device. It may include more or fewer components than illustrated, or combine certain components, or have different component arrangements.

[0106] The present invention also provides a computer-readable storage medium, which can be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium, wherein the computer-readable storage medium stores instructions that, when executed on a computer, cause the computer to perform the steps of the live mite detection method.

[0107] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the system, device, or unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0108] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0109] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for detecting live mites, characterized in that, include: A continuous sequence of mite images was acquired using large field-of-view imaging technology. A multi-target tracking model is used to obtain the bounding box information of each frame of a mite image sequence; Based on the bounding box information and the mite image sequence, obtain a set of mite motion images and determine whether the set of mite motion images is continuous. If the mite motion image set is continuous, then each frame of the mite motion image set is normalized to obtain a mite detection image sample set. Feature extraction was performed on the mite detection image sample set to obtain PCA feature information and HOG feature information; Inter-frame similarity is calculated based on the mite detection image sample set, PCA feature information, and HOG feature information to obtain the first structural similarity result, the second structural similarity result, and the third structural similarity result. Based on the first structural similarity result, the second structural similarity result, and the third structural similarity result, a live mite score is calculated to obtain the live mite detection result; The method of obtaining bounding box (bbox) information for each frame of a mite image sequence using a multi-object tracking model includes: The YOLOv5 model is used to perform forward inference on each frame of the mite image sequence to obtain the first detection box information; A multi-target tracking model is constructed based on the SORT algorithm; The Kalman filter algorithm is called from the multi-target tracking model, and motion modeling is performed on each frame of the mite image sequence based on the Kalman filter algorithm to obtain the target mite motion model. The coordinates of the target mite in the next frame are predicted based on the target mite motion model to obtain the second detection box information; The cross-union ratio (CUR) between the first and second detection box information is calculated using the Hungarian algorithm, and then compared with a preset matching threshold. If the crossover ratio is greater than the preset matching threshold, the first detection box information and the second detection box information are combined to obtain the bbox detection box information.

2. The method for detecting live mites according to claim 1, characterized in that, The step of obtaining a set of mite motion images based on the bounding box information and the mite image sequence, and determining whether the set of mite motion images is continuous, includes: Based on the preset number of consecutive detection frames and bbox detection information, the mite image of each mite is obtained from each frame of the mite image sequence, and the mite head images are associated and matched to form a set of mite motion images. Obtain the image coordinates of each mite image in the mite motion image set, and determine whether the mite motion image set is continuous based on the continuity of the image coordinates of each frame.

3. The method for detecting live mites according to claim 2, characterized in that, After determining whether the mite motion image set is continuous based on the continuity of the image coordinates of each frame in the acquired mite motion image set, the following steps are also included: If the mite motion image set is in a discontinuous state, the number of broken frames of each frame image coordinate is obtained and compared with the preset broken frame threshold. If the number of broken frames is less than or equal to the preset broken frame threshold, the frame interpolation method is used to perform frame interpolation on the mite motion image set to obtain the frame interpolated mite motion image set. If the number of broken frames exceeds the preset broken frame threshold, the mite motion image set will be exported to the preset recycling pool.

4. The method for detecting live mites according to claim 3, characterized in that, The feature extraction of the mite detection image sample set to obtain PCA feature information and HOG feature information includes: The mite sample set vector matrix is ​​obtained from the detected image sample set, and the mite covariance matrix is ​​calculated based on the mite sample set vector matrix. The covariance matrix of mites is decomposed into eigenvalues, and the eigenvalues ​​are sorted in descending order of their values ​​to obtain the top N eigenvalues. The corresponding eigenvectors are obtained based on N eigenvalues, and each eigenvector is standardized to form an eigenvector matrix; PCA feature information is obtained based on the feature vector matrix and the mite sample set vector matrix; The detection image sample set is subjected to grayscale conversion and gamma correction, and the pixel gradient of the processed detection image sample set is calculated according to the preset operator. Calculate the pixel gradient within a preset pixel region to form a gradient histogram; Obtain the feature vectors of the gradient histogram to form HOG feature information.

5. The method for detecting live mites according to claim 1, characterized in that, The step of calculating inter-frame similarity based on the mite detection image sample set, PCA feature information, and HOG feature information to obtain a first structural similarity result, a second structural similarity result, and a third structural similarity result includes: A sliding detection window is added to each image in the mite detection image sample set, and the brightness data, contrast data and structural data of each image are obtained through the sliding detection window; The structural similarity between adjacent images is calculated based on brightness data, contrast data, and structural data, and the mean of each structural similarity is calculated to obtain the first structural similarity result. The cosine similarity algorithm is used to calculate the cosine similarity of PCA feature information, and a second structural similarity result is formed based on each cosine similarity. The cosine similarity algorithm is used to calculate the cosine similarity of HOG feature information, and a third structural similarity result is formed based on each cosine similarity.

6. The method for detecting live mites according to claim 5, characterized in that, The process of scoring live mites based on the first structural similarity result, the second structural similarity result, and the third structural similarity result to obtain live mite detection results includes: Determine whether the first structural similarity result is greater than the first preset similarity threshold to obtain the first comparison result; Determine whether each cosine similarity in the second structural similarity result is greater than the second preset similarity threshold, and count the number of cosine similarities greater than the second preset similarity threshold to obtain the second comparison result; Determine whether each cosine similarity in the third structural similarity result is greater than the third preset similarity threshold, and count the number of cosine similarities greater than the third preset similarity threshold to obtain the third comparison result; The live mite score was obtained based on the first comparison result, the second comparison result, and the third comparison result; The live mite score is compared with the preset live baseline value to obtain the live mite detection result.

7. A live mite detection device, characterized in that, include: The imaging module is used to acquire a continuous sequence of mite images based on a large field-of-view imaging technology. The tracking module is used to obtain the bounding box (bbox) information of each frame of the mite image sequence using a multi-target tracking model; it performs forward inference on each frame of the mite image sequence using a YOLOv5 model to obtain the first bounding box information; it constructs a multi-target tracking model based on the SORT algorithm; it calls the Kalman filter algorithm from the multi-target tracking model and performs motion modeling on each frame of the mite image sequence based on the Kalman filter algorithm to obtain the target mite motion model; and it predicts the coordinates of the target mite in the next frame based on the target mite motion model to obtain the second bounding box information. The cross-union ratio (CUR) between the first and second detection box information is calculated using the Hungarian algorithm, and then compared with a preset matching threshold. If the crossover ratio is greater than the preset matching threshold, the first detection box information and the second detection box information are combined to obtain the bbox detection box information. The continuous module is used to obtain a set of mite motion images based on the bounding box information and the mite image sequence, and to determine whether the set of mite motion images is continuous. The normalization module is used to normalize each frame of mite images in the mite motion image set if the mite motion image set is continuous, so as to obtain a mite detection image sample set. The feature module is used to extract features from the mite detection image sample set to obtain PCA feature information and HOG feature information; The calculation module is used to perform inter-frame similarity calculation based on the mite detection image sample set, PCA feature information and HOG feature information to obtain the first structural similarity result, the second structural similarity result and the third structural similarity result; The scoring module is used to score the live mites based on the first structural similarity result, the second structural similarity result, and the third structural similarity result, so as to obtain the live mite detection result.

8. A live mite detection device, characterized in that, The live mite detection device includes: a memory and at least one processor, wherein the memory stores instructions; At least one of the processors invokes the instructions in the memory to cause the live mite detection device to perform the steps of the live mite detection method as described in any one of claims 1-6.

9. A computer-readable storage medium storing instructions thereon, characterized in that, When the instructions are executed by the processor, they implement the steps of the live mite detection method as described in any one of claims 1-6.

Images