A method, device and system for judging a wet digestion state and a storage medium

Abstract

The application discloses a wet digestion state judgment method, device and system and a storage medium, which are used for judging the wet digestion state. The method detects the current image to be detected through multiple random forward aggregations to obtain multiple groups of detection results; candidate detection boxes in the detection results are aligned; the mean value and variance of the confidence score of the aligned candidate detection boxes are calculated; the time sequence stability index of the current image to be detected is determined according to the mean value and variance of the confidence score of the aligned candidate detection boxes; the number of stable frames in the continuous N+1 frames of images to be detected is determined according to the time sequence stability index and the variance of the confidence score of the N frames of images to be detected before the current frame of image to be detected and the current image to be detected; and when the number of stable frames is greater than a first preset threshold, it is determined that the current detection state of the content of the test tube is undigested. The continuous verification in the time dimension improves the accuracy of the wet digestion state judgment.

Description

Technical Field

[0001] This application relates to the field of wet digestion technology, and in particular to a method, apparatus, system and storage medium for determining the state of wet digestion. Background Technology

[0002] Wet digestion is an important sample pretreatment technique in chemical analysis. It uses strong acids to decompose the sample under heating conditions, converting the analyte into a form suitable for analysis. Traditionally, the degree of digestion in wet digestion is judged primarily by visual inspection, i.e., by observing whether digestion is complete. This method has significant limitations: first, it is highly subjective, with inconsistent judgment standards among different operators; second, it is inefficient, requiring continuous manual observation; and third, it is easily affected by the operating environment, such as changes in lighting conditions.

[0003] While existing technologies include traditional computer vision methods, they perform poorly under complex backgrounds and lighting conditions, making it difficult to meet the needs of high-precision fields such as pharmaceutical manufacturing.

[0004] Therefore, how to provide a method for judging the state of wet digestion in order to improve the accuracy of wet digestion state judgment has become an urgent technical problem to be solved. Summary of the Invention

[0005] This application provides a method, system, and storage medium for determining the state of wet digestion, in order to improve the accuracy of determining the state of wet digestion.

[0006] This application provides a method for determining the state of wet digestion, including: Obtain the current image to be detected; The current image to be detected is detected by multiple random forward aggregations to obtain multiple sets of detection results, wherein the detection results include at least candidate detection boxes of test tube contents and confidence scores corresponding to the candidate detection boxes; Align the candidate detection boxes in the detection results; Calculate the mean and variance of the confidence scores of the aligned candidate detection boxes; The variance adjustment score is obtained based on the mean and variance of the confidence scores of the aligned candidate detection boxes, and the temporal stability index of the current image to be detected is determined based on the variance adjustment score. Based on the temporal stability index and confidence score variance of the current frame to be detected and the previous N frames to be detected, the number of stable frames in the N+1 consecutive frames to be detected is determined, where the number of stable frames is negatively correlated with the degree of resolution. When the number of stable frames is greater than the first preset threshold, the current detection status of the test tube contents is determined to be undigested.

[0007] The beneficial effects of this application are as follows: This application obtains candidate box confidence scores by performing multiple random forward aggregations on the current image to be detected. After alignment, the mean and variance of the confidence scores of the candidate detection boxes are calculated to obtain a temporal stability index. The temporal stability index is then used to analyze the number of stable frames in N+1 consecutive frames, and finally, the resolution state is determined based on a threshold. This application improves the robustness of state determination through continuous verification in the time dimension. A state is only confirmed when the count continuously accumulates to a threshold, ensuring that state transitions only occur in scenarios where the target behavior is continuous and consistent, thereby improving the accuracy of wet resolution state determination.

[0008] In one embodiment, obtaining the current image to be detected includes: Get the original image of the current test tube; Calculate the vertical gradient energy column-wise for the original image; The vertical gradient energy is smoothed. Determine the position of the test tube spindle corresponding to the peak value of the vertical gradient energy after smoothing; The peak value of the vertical gradient energy is extended to the left and right to determine the lateral range of the test tube; The current image to be detected is obtained from the image of the horizontal range of the test tube.

[0009] In one embodiment, obtaining the current image to be detected from the image of the lateral range of the test tube includes: A region of a predetermined proportion is selected from the bottom of the image across the horizontal range of the test tube as the ROI image of the original image; The ROI image is determined as the current image to be detected.

[0010] In one embodiment, aligning the candidate detection boxes in the detection results includes: Obtain the candidate bounding box with the highest confidence score from multiple sets of detection results; Calculate the IoU value between the remaining candidate detection boxes and the candidate detection box with the highest confidence score; When there is a target candidate detection box with an IoU value greater than the second preset threshold in the remaining candidate detection boxes, the target candidate detection box is aligned with the candidate detection box with the highest confidence score.

[0011] In one embodiment, aligning the candidate detection boxes in the detection results includes: The IoU value is calculated by comparing each pair of the multiple detection results; When there are two candidate detection boxes with an IoU value greater than the second preset threshold, the two candidate detection boxes are aligned.

[0012] In one embodiment, the second preset threshold is determined as follows: The IoU values ​​are calculated by comparing each pair of the multiple detection results to obtain the IoU matrix; Calculate the proportion of off-diagonal elements in the IoU matrix that are below the overlap threshold to obtain the global crowding of the current image to be detected; Substituting the global congestion level into the following formula, a second preset threshold is obtained:

[0013] in, t The second preset threshold is adaptive; base The base threshold; k is the adjustment coefficient; d is the global crowding level; t min The lower limit of the threshold; t max This represents the upper limit of the threshold.

[0014] In one embodiment, determining the temporal stability index of the current image to be detected based on the variance adjustment score includes: Applying the variance adjustment fraction to the following formula yields the time series stability index:

[0015] in, Let be the time series stability index at the current time t; The time series stability index is the index of the previous time step t-1; For smoothing coefficients; The target detection bounding box for the current frame (time step t); It is a geometric function that reflects the geometric relationship between the current frame detection box and historical frames.

[0016] In one embodiment, determining the number of stable frames in a series of N+1 consecutive frames of images to be detected based on the temporal stability index and confidence score variance of the current frame image to be detected and the previous N frames image to be detected includes: When the time series stability index is greater than the first preset value and the confidence score variance is less than the second preset value, the stability counter is incremented by 1. Otherwise, the stable counter is reset to zero.

[0017] This application also provides a device for determining the state of wet digestion, comprising: The acquisition module is used to acquire the current image to be detected; The detection module is used to detect the current image to be detected by multiple random forward aggregations to obtain multiple sets of detection results, wherein the detection results include at least candidate detection boxes of test tube contents and confidence scores corresponding to the candidate detection boxes; An alignment module is used to align candidate detection boxes in the detection results; The calculation module is used to calculate the mean and variance of the confidence scores of the aligned candidate detection boxes. The first determining module is used to obtain a variance adjustment score based on the mean confidence score and the variance of the confidence score of the aligned candidate detection boxes, and to determine the temporal stability index of the current image to be detected based on the variance adjustment score. The second determining module is used to determine the number of stable frames in a continuous N+1 frames of images to be detected based on the temporal stability index and confidence score variance of the current frame image to be detected and the previous N frames image to be detected. The number of stable frames is negatively correlated with the degree of resolution. The third determining module is used to determine that the current detection status of the test tube contents is undigested when the number of stable frames is greater than the first preset threshold.

[0018] In one embodiment, the first acquisition module includes: The first acquisition submodule is used to acquire the original image of the current test tube; The first calculation submodule is used to calculate the vertical gradient energy of the original image column by column; A smoothing submodule is used to smooth the vertical gradient energy; The determination submodule is used to determine the position of the test tube spindle corresponding to the peak value of the vertical gradient energy after smoothing. An extension submodule is used to extend the peak value of the vertical gradient energy to the left and right to determine the lateral range of the test tube; The second acquisition submodule is used to acquire the current image to be detected from the image of the horizontal range of the test tube.

[0019] In one embodiment, the second acquisition submodule is further configured to: A region of a predetermined proportion is selected from the bottom of the image across the horizontal range of the test tube as the ROI image of the original image; The ROI image is determined as the current image to be detected.

[0020] In one embodiment, the alignment module includes: The third acquisition submodule is used to obtain the candidate detection box with the highest confidence score from multiple sets of detection results; The second calculation submodule is used to calculate the IoU value between the remaining candidate detection boxes and the candidate detection box with the highest confidence score; The first alignment submodule is used to align the target candidate detection box with the candidate detection box with the highest confidence score when there is a target candidate detection box with an IoU value greater than a second preset threshold in the remaining candidate detection boxes.

[0021] In one embodiment, the alignment module includes: The third calculation submodule is used to compare the multiple detection results pairwise to calculate the IoU value; The second alignment submodule is used to align the two candidate detection boxes when there are two candidate detection boxes with an IoU value greater than a second preset threshold.

[0022] In one embodiment, the second preset threshold is determined as follows: The IoU values ​​are calculated by comparing each pair of the multiple detection results to obtain the IoU matrix; Calculate the proportion of off-diagonal elements in the IoU matrix that are below the overlap threshold to obtain the global crowding of the current image to be detected; Substituting the global congestion level into the following formula, a second preset threshold is obtained:

[0023] in, t The second preset threshold is adaptive; base The base threshold; k is the adjustment coefficient; d is the global crowding level; t min The lower limit of the threshold; t max This represents the upper limit of the threshold.

[0024] In one embodiment, the first determining module includes: Substitute the values ​​into the submodule, which applies the variance adjustment fraction to the following formula to obtain the time series stability index:

[0025] in, Let be the time series stability index at the current time t; The time series stability index is the index of the previous time step t-1; For smoothing coefficients; The target detection bounding box for the current frame (time step t); It is a geometric function that reflects the geometric relationship between the current frame detection box and historical frames.

[0026] In one embodiment, the second determining module is further configured to: When the time series stability index is greater than the first preset value and the confidence score variance is less than the second preset value, the stability counter is incremented by 1. Otherwise, the stable counter is reset to zero.

[0027] This application also provides a system for determining the state of wet digestion, including: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to implement the wet digestion state determination method described in any of the above embodiments.

[0028] This application also provides a computer-readable storage medium, which, when the instructions in the storage medium are executed by the processor corresponding to the wet digestion state determination system, enables the wet digestion state determination system to implement the wet digestion state determination method described in any of the above embodiments.

[0029] Other features and advantages of this application will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings.

[0030] The technical solution of this application will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0031] The accompanying drawings are provided to further illustrate the present application and form part of the specification. They are used together with the embodiments of the present application to explain the application and do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart of a method for determining the state of wet digestion in one embodiment of this application; Figure 2 This is a schematic diagram of a wet digestion state determination device according to an embodiment of this application; Figure 3 This is a schematic diagram of the hardware structure of a wet digestion state determination system according to an embodiment of this application. Detailed Implementation

[0032] The preferred embodiments of this application are described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit this application.

[0033] Figure 1 This is a flowchart of a method for determining the state of wet digestion in one embodiment of this application, as shown below. Figure 1 As shown, the method can be implemented as follows: S101-S107: In step S101, the current image to be detected is acquired; In step S102, the current image to be detected is detected by multiple random forward aggregations to obtain multiple sets of detection results, wherein the detection results include at least candidate detection boxes of test tube contents and confidence scores corresponding to the candidate detection boxes; In step S103, the candidate detection boxes in the detection results are aligned; In step S104, the mean confidence score and variance of the confidence score of the aligned candidate detection boxes are calculated. In step S105, the variance adjustment score is obtained based on the mean confidence score and variance of the aligned candidate detection boxes, and the temporal stability index of the current image to be detected is determined based on the variance adjustment score. In step S106, the number of stable frames in the N+1 consecutive frames of images to be detected is determined based on the temporal stability index and confidence score variance of the current frame image to be detected and the N frames of images to be detected before the current frame image to be detected. The number of stable frames is negatively correlated with the degree of resolution. In step S107, when the number of stable frames is greater than the first preset threshold, the current detection status of the test tube contents is determined to be undigested.

[0034] In this application, the current image to be inspected is acquired. The image can be acquired using a high-definition camera, such as a 1080p resolution industrial camera, a 30fps frame rate, and a high-definition camera supporting night vision mode. During acquisition, the light source needs to be controlled, for example, by using multiple LED light sources to ensure the contents of the test tube are clearly visible. Preprocessing operations such as noise filtering, illumination equalization, and geometric correction can be performed on the acquired image. Furthermore, during image acquisition in this application, a robotic arm can rotate the test tube to acquire images from multiple angles.

[0035] During wet digestion, the position of the test tube, lighting, or camera angle may change slightly over time, such as when the experimenter bumps the table or the lighting shifts. These changes can lead to misjudgments by the detection system—for example, misdetecting the edge of the precipitate, thus affecting the experimental results. Therefore, in one embodiment of this application, when acquiring the current image to be detected, automatic drift correction is used to stabilize the position of the test tube's main axis and the bottom ROI, mitigating composition / camera / lighting drift and ensuring that the system can always accurately lock onto the key areas of the test tube (such as the bottom precipitate). Specifically, the original image containing the test tube is acquired, for example, the BGR image of the current frame, which contains a complete image of the test tube. Then, the vertical gradient energy is calculated column-wise for the original image, and the vertical gradient energy is smoothed. Here, gradient is a measure of the change in pixel grayscale values ​​in an image, typically used for edge or texture detection. Vertical gradient energy is a quantified form of gradient magnitude, usually defined as the square or absolute value of the gradient, used to measure the edge strength or energy change in the vertical direction. In image processing, gradients are calculated by comparing the grayscale differences between adjacent pixels (e.g., using the Sobel operator) or by calculating partial derivatives. Since gradient calculations are typically sensitive to brightness changes, the original image is often processed into grayscale, converting a color image (e.g., BGR format) to grayscale. This means each pixel retains only its brightness information (grayscale value from 0 to 255) to simplify subsequent processing. Then, the vertical gradient energy is calculated column-wise on the grayscale-processed image, specifically along the y-axis, to accurately capture local features (e.g., the horizontal edges of a test tube image). Furthermore, the vertical gradient energy can be smoothed, for example, by using filtering (e.g., mean filtering) to suppress noise or minor gradient fluctuations, preserving significant edges and avoiding false detections. Furthermore, the peak value of the smoothed vertical gradient energy corresponds to the position of the test tube's main axis. Based on this, the peak value of the vertical gradient energy is expanded to the left and right to determine the test tube's lateral range. For example, the boundaries are expanded to the left and right until the energy value is lower than a first preset proportion (e.g., 30%) of the peak energy. At this point, the region width reflects the actual span of the main structure. In addition, if the expanded region width is less than a second preset proportion (e.g., 10%) of the total image width, it is symmetrically padded to a predetermined minimum width to ensure the stability of subsequent processing. Then, the current image to be detected is obtained from the image of the test tube's lateral range. For example, a region of a preset proportion is selected from the bottom of the image of the test tube's lateral range as the ROI image of the original image, and the ROI image is determined as the current image to be detected.

[0036] Furthermore, to eliminate the impact of long-term equipment or environmental drift, this application also calculates a drift score based on the ROI image and performs self-calibration through a "daily baseline drift score," forming a closed loop of "baseline-alarm-update" for long-term chemical scenarios. Specifically, stable ROI feature values ​​are obtained by exponential smoothing of the ROI image, where the ROI feature values ​​can be the average gradient energy, average pixel value, etc. of the ROI image. In one embodiment, smoothing is performed using the following formula:

[0037] in, These are the smoothed ROI eigenvalues ​​at time t. The original ROI feature value at the current moment; γ is the decay rate, a parameter used to control the smoothness.

[0038] Then, a drift score for the current image under test is calculated based on the stable ROI feature values. Specifically, a drift score is formed by comparing the brightness / texture fluctuations with the overall image based on the stable ROI feature values, which is used to measure the deviation between the current state and the normal state. In one embodiment of this application, the drift score is calculated according to the following formula:

[0039] in, The drift score for the current image under test is used to quantify the brightness fluctuation of the ROI region, and the value range is limited to the [0,1] interval; for The variance of grayscale values ​​in a region reflects the degree of dispersion of pixel brightness within that region; ε is the variance of the grayscale of the entire image, which serves as a normalization benchmark to eliminate the impact of overall image brightness variations on the score; ε is a preset constant, such as 1e-6, to prevent calculation errors caused by a zero denominator.

[0040] Then, "normal frames" are counted based on whether the drift score continuously falls within the normal range; if there are continuous abnormalities, a drift alarm is recorded. Specifically, the baseline of the drift score in historical data is obtained, including the baseline mean. μ ref (i.e., the average of historical drift scores) and baseline standard deviation σ ref(i.e., the standard deviation of historical drift scores); when the drift score of the current frame exceeds the baseline standard deviation, which is a preset multiple of the baseline mean, it is considered an abnormal fluctuation, triggering an alarm; when the number of alarms reaches a preset number, a baseline update or recalibration prompt is triggered, for example, reconstructing the baseline with an empty frame (an interference-free reference frame) to reset the historical baseline and eliminate the impact of long-term device or environmental drift. In one embodiment, when the drift score of the current frame exceeds the baseline standard deviation, a frame discarding mechanism is triggered to prevent erroneous data from contaminating subsequent processing. Otherwise, the drift score of the current frame is considered to be slightly out of bounds, and logs are recorded or adaptive baseline adjustment is triggered, without discarding the current frame to maintain data continuity. Of course, decisions can also be made dynamically based on the historical trend of the drift score, for example: if there is no improvement after 3 consecutive alarms, frame discarding is enabled; otherwise, retransmission or error correction is attempted.

[0041] Upon obtaining the current image to be detected, multiple random forward aggregations are performed on the image to obtain multiple sets of detection results. These results include at least candidate bounding boxes for the test tube contents and their corresponding confidence scores. Multiple random forward aggregations (MCDropout) are a technique for evaluating model uncertainty during the inference phase. In object detection or classification tasks, multiple random inferences reduce transient noise and quantify the confidence of model predictions. During each inference, the model randomly discards a portion of neurons (Dropout activation). For example, running the detection model 3-7 times (default 3 times) on the same frame image can yield different candidate bounding boxes B. ti and confidence score S ti Repeating this process R times yields multiple detection results. The detection model can be a deep learning model for object detection, such as an improved YOLOv11 object detection model, to analyze and process the acquired images. For example, multi-level features are extracted using an improved YOLOv11 backbone network, these multi-level features are fused using the SPFF module, and then the detection results, including bounding boxes and corresponding confidence scores, are generated using the C2PSA module. This application does not specifically limit the selection of the deep learning model.

[0042] Then, the candidate detection boxes in the detection results are aligned. Multiple detection results are aligned. Since IoU (Intersection over Union) is a core evaluation metric in computer vision used for object detection or image segmentation, used to quantify the degree of overlap of candidate boxes, alignment can be performed according to IoU. When there are two candidate detection boxes with an IoU greater than a preset value, it is determined that the two candidate detection boxes correspond to the same test tube contents, and the two candidate detection boxes are aligned. For example, boxes with an IoU > 50% are considered to be the same test tube contents. In one embodiment, the candidate detection box with the highest confidence score is obtained from multiple sets of detection results; the IoU value between the remaining candidate detection boxes and the candidate detection box with the highest confidence score is calculated; when there is a target candidate detection box with an IoU value greater than a second preset threshold among the remaining candidate detection boxes, the target candidate detection box is aligned with the candidate detection box with the highest confidence score. In another embodiment, the multiple detection results are compared pairwise to calculate the IoU value; when there are two candidate detection boxes with an IoU value greater than a second preset threshold, the two candidate detection boxes are aligned. At this time, a greedy algorithm can be used to ensure that each box is matched only once (i.e., asymmetric matching).

[0043] The aforementioned second preset threshold can be a pre-defined fixed value. Based on this fixed IoU threshold, basic target box filtering is performed, and preliminary candidate detection boxes and confidence scores are output. The IoU threshold can also be dynamically adjusted using density awareness to further filter redundant boxes or retain dense targets. Specifically, in dense small target / foam scenarios, the IoU threshold is dynamically adjusted: it is widened when dense and tightened when sparse, reducing the trade-off between missed detections and false detections.

[0044] When using a dynamic threshold, it can be determined as follows: Input N candidate detection boxes and their corresponding confidence scores for the current image to be tested, as well as the basic IoU threshold and adjustment parameters. Then, perform density estimation: if the number of candidate detection boxes N≤1, directly return all boxes. When N>1, perform pairwise comparisons on the multiple detection results to calculate the IoU value and obtain the IoU matrix; calculate the proportion of off-diagonal elements in the IoU matrix that are below the overlap threshold to obtain the global crowding degree of the current image to be tested. Specifically, the global crowding degree is calculated according to the following formula:

[0045] in, d The congestion level, with a value ranging from [0,1], reflects the density of overlapping targets in the current detection bounding box; M ij The IoU value between the i-th and j-th detection boxes, ranging from [0,1], measures the degree of overlap between the two boxes; τ dThis is the crowding threshold; if the IoU between two boxes exceeds this value, they are considered to overlap.

[0046] d A value close to 1 indicates that the target is highly dense, and the threshold needs to be relaxed to avoid missed detections. d A value approaching 0 indicates a sparse distribution of the target, making it suitable to tighten the threshold. Therefore, the basic IoU threshold can be linearly adjusted based on crowding and truncated within an acceptable range: the more crowded the area, the higher the threshold; the sparser the area, the lower the threshold. Specifically, the threshold can be adjusted using the following formula:

[0047] in, t This is the adaptive IoU threshold, i.e., the second preset threshold; base The base threshold is usually a preset fixed value (e.g., 0.5) used as a benchmark for adjustment; k is an adjustment coefficient that controls the strength of the influence of density on the threshold, with a default value of 0.3 and a range of [0.2, 0.5]; d is the crowding degree; t min The lower limit of the threshold is fixed at 0.3 to ensure the lowest possible judgment standard; t max The upper limit of the threshold is fixed at 0.8 to avoid excessive relaxation leading to false detections.

[0048] The adjusted threshold can then be used to filter candidate detection boxes, obtaining a set of retained boxes and their corresponding confidence scores. Simultaneously, the crowding level of the current image to be detected and the actual threshold used can be recorded for subsequent analysis or visualization. Adaptive thresholding can better handle complex scenarios, improving detection accuracy and recall.

[0049] Furthermore, the mean and variance of the confidence scores of the aligned candidate detection boxes can be calculated. For example, to calculate the mean and variance of the R confidence scores for the same test tube contents to reflect the model's prediction consistency for the target in the current frame, the average confidence scores for the same test tube contents can be calculated using the following formula:

[0050] in, The average confidence score for the contents of the same test tube reflects the average confidence level; a higher value indicates a greater overall confidence level of the model. R represents the number of random inferences. S ti This represents the confidence score obtained for each inference.

[0051] Meanwhile, the variance of the confidence scores for the contents of the same test tube is calculated using the following formula:

[0052] in, The variance of confidence level for the contents of the same test tube is defined as follows: low variance indicates consistent results from multiple tests, while high variance indicates uncertainty or noise interference in the model. R represents the average confidence score for the contents of the same test tube; R is the number of random inferences; S ti This represents the confidence score obtained for each inference.

[0053] The variance adjustment score is obtained based on the mean and variance of the confidence scores of the aligned candidate detection boxes, and the temporal stability index of the current image to be detected is determined based on the variance adjustment score. In this application, the confidence score can be adjusted by soft adjustment or hard gating to obtain the variance adjustment score.

[0054] Soft adjustment dynamically adjusts the model output using probabilities or thresholds, retaining some results with higher uncertainty and avoiding direct discarding. For example, confidence scores can be adjusted using the following formula:

[0055] in, This refers to the variance adjustment score after adjusting for the confidence score. The average confidence score; α represents the confidence level variance; α represents the adjustment strength. This is the variance threshold. Typically, α ∈ [0.5, 2]. ∈[0.005,0.05] (default 0.015).

[0056] Hard gating, on the other hand, filters results directly based on an absolute threshold, retaining only predictions with high certainty and discarding the rest. For example, confidence scores can be adjusted using the following formula:

[0057] in, This refers to the variance adjustment score after adjusting for the confidence score. The average confidence score; As an indicator function, when variance Less than the variance threshold Output 1 if the condition is met, otherwise output 0. In practical applications, adjustments can be made according to the actual situation. For example, soft adjustment is preferred when there is high data noise or a high recall rate is required, while hard gating is preferred when the system has high real-time requirements or subsequent modules are sensitive to false detections.

[0058] In addition, this application quantifies the stability of temporal data (such as object detection box sequences) using the Temporal Stability Index (EMA) to address the boundary jitter problem in single-frame detection.

[0059]

[0060] in, Let be the time series stability index at the current time t; The time series stability index is the index of the previous time step t-1; This is a smoothing coefficient, and its value is a number between 0 and 1. It is a geometric function that reflects the geometric relationship between the current frame detection box and historical frames.

[0061] Geometric functions The appropriate function can be selected based on actual needs; for example, in one embodiment, a geometric function... Choose the following normalized offset formula:

[0062] in, This represents the normalized offset, used to quantify the degree of geometric change of the current frame's bounding box relative to historical frames; The target detection bounding box for the current frame (time step t) is typically represented as a coordinate vector (x, y, w, h) (i.e., center point coordinates, width, and height); B t-1 This is the target detection bounding box of the previous frame (time step t-1).

[0063] In another embodiment, geometric functions Choose the following formula for area change:

[0064] in, The stability of area change is used to constrain abrupt changes in the target size; the closer the area ratio is to 1, the higher the stability. B is the target detection bounding box for the current frame (time step t); t-1 The target detection bounding box of the previous frame (time step t-1); This represents the area of ​​the target detection bounding box.

[0065] Furthermore, to reduce state jitter and false negatives and ensure the stability of state determination, this application obtains the temporal stability index and confidence score variance of the current image to be detected and the N frames of images to be detected preceding the current image to be detected. Based on the temporal stability index and confidence score variance of the current image to be detected and the N frames of images to be detected preceding the current image to be detected, the number of stable frames in the N+1 consecutive frames of images to be detected is determined, wherein the number of stable frames is negatively correlated with the degree of resolution; when the number of stable frames is greater than a first preset threshold, the current detection state of the test tube contents is determined to be unresolved. Otherwise, when the number of stable frames is less than a second preset threshold, the current detection state is determined to be resolved; when the number of stable frames is between the first preset threshold and the second preset threshold, the current detection state is determined to be a transitional state. Thus, the robustness of state determination is improved through continuous verification in the time dimension. Only when the count continuously accumulates to the threshold is the state confirmed, ensuring that state transitions only occur in scenarios where the target behavior is continuous and consistent.

[0066] The beneficial effects of this application are as follows: This application obtains candidate box confidence scores by performing multiple random forward aggregations on the current image to be detected. After alignment, the mean and variance of the confidence scores of the candidate detection boxes are calculated to obtain a temporal stability index. The temporal stability index is then used to analyze the number of stable frames in N+1 consecutive frames, and finally, the resolution state is determined based on a threshold. This application improves the robustness of state determination through continuous verification in the time dimension. A state is only confirmed when the count continuously accumulates to a threshold, ensuring that state transitions only occur in scenarios where the target behavior is continuous and consistent, thereby improving the accuracy of wet resolution state determination.

[0067] In one embodiment, step S101 above can be implemented as steps A1-A6 as follows: In step A1, the original image containing the test tube is obtained; In step A2, the vertical gradient energy is calculated column-wise for the original image; In step A3, the vertical gradient energy is smoothed. In step A4, the position of the test tube spindle corresponding to the peak value of the vertical gradient energy after smoothing is determined; In step A5, the peak value of the vertical gradient energy is extended to the left and right to determine the lateral range of the test tube; In step A6, the current image to be detected is obtained from the image of the horizontal range of the test tube.

[0068] In one embodiment, step A6 above can be implemented as steps A61-A62: In step A61, a region of a preset proportion is selected from the bottom of the image of the test tube's horizontal range as the ROI image of the original image; In step A62, the ROI image is determined to be the current image to be detected.

[0069] In one embodiment, step S103 above can be implemented as steps B1-B3 as follows: In step B1, the candidate detection box with the highest confidence score is obtained from multiple sets of detection results; In step B2, the IoU value between the remaining candidate detection boxes and the candidate detection box with the highest confidence score is calculated; In step B3, when there is a target candidate detection box with an IoU value greater than the second preset threshold in the remaining candidate detection boxes, the target candidate detection box is aligned with the candidate detection box with the highest confidence score.

[0070] In one embodiment, step S103 above can be implemented as steps C1-C2 as follows: In step C1, the multiple detection results are compared pairwise to calculate the IoU value; In step C2, when there are two candidate detection boxes with an IoU value greater than the second preset threshold, the two candidate detection boxes are aligned.

[0071] In one embodiment, the second preset threshold in step C2 above is determined by the following steps C21-C23: In step C21, the IoU values ​​are calculated by comparing each pair of the multiple detection results to obtain the IoU matrix; In step C22, the proportion of off-diagonal elements in the IoU matrix that are below the low threshold of overlap is calculated to obtain the global crowding degree of the current image to be detected; In step C23, the global congestion level is substituted into the following formula to obtain the second preset threshold:

[0072] in, t To adapt to the second preset threshold; base The base threshold; k is the adjustment coefficient; d is the global crowding level; t min t is the lower limit of the threshold; max This represents the upper limit of the threshold.

[0073] In one embodiment, the step S105 above, which involves determining the temporal stability index of the current image to be detected based on the variance adjustment score, can be implemented as follows: Applying the variance adjustment fraction to the following formula yields the time series stability index:

[0074] in, Let be the time series stability index at the current time t; The time series stability index is the index of the previous time step t-1; For smoothing coefficients; The target detection bounding box for the current frame (time step t); It is a geometric function that reflects the geometric relationship between the current frame detection box and historical frames.

[0075] In one embodiment, step S107 can be implemented as follows: When the time series stability index is greater than a first preset value and the confidence score variance is less than a second preset value, the stability counter is incremented by 1; otherwise, the stability counter is cleared to zero.

[0076] Figure 2 This is a schematic diagram of a wet digestion state determination device according to an embodiment of this application, as shown below. Figure 2 As shown, the device includes: The acquisition module 201 is used to acquire the current image to be detected; The detection module 202 is used to detect the current image to be detected by multiple random forward aggregations to obtain multiple sets of detection results, wherein the detection results include at least candidate detection boxes of test tube contents and confidence scores corresponding to the candidate detection boxes; Alignment module 203 is used to align candidate detection boxes in the detection results; Calculation module 204 is used to calculate the mean confidence score and variance of the confidence score of the aligned candidate detection boxes; The first determining module 205 is used to obtain a variance adjustment score based on the mean confidence score and the variance of the confidence score of the aligned candidate detection boxes, and to determine the temporal stability index of the current image to be detected based on the variance adjustment score. The second determining module 206 is used to determine the number of stable frames in a continuous N+1 frames of images to be detected based on the temporal stability index and confidence score variance of the current frame image to be detected and the previous N frames image to be detected, wherein the number of stable frames is negatively correlated with the degree of resolution. The third determining module 207 is used to determine that the current detection status of the test tube contents is undigested when the number of stable frames is greater than the first preset threshold.

[0077] In one embodiment, the first acquisition module includes: The first acquisition submodule is used to acquire the original image of the current test tube; The first calculation submodule is used to calculate the vertical gradient energy of the original image column by column; A smoothing submodule is used to smooth the vertical gradient energy; The determination submodule is used to determine the position of the test tube spindle corresponding to the peak value of the vertical gradient energy after smoothing. An extension submodule is used to extend the peak value of the vertical gradient energy to the left and right to determine the lateral range of the test tube; The second acquisition submodule is used to acquire the current image to be detected from the image of the horizontal range of the test tube.

[0078] In one embodiment, the second acquisition submodule is further configured to: A region of a predetermined proportion is selected from the bottom of the image across the horizontal range of the test tube as the ROI image of the original image; The ROI image is determined as the current image to be detected.

[0079] In one embodiment, the alignment module includes: The third acquisition submodule is used to obtain the candidate detection box with the highest confidence score from multiple sets of detection results; The second calculation submodule is used to calculate the IoU value between the remaining candidate detection boxes and the candidate detection box with the highest confidence score; The first alignment submodule is used to align the target candidate detection box with the candidate detection box with the highest confidence score when there is a target candidate detection box with an IoU value greater than a second preset threshold in the remaining candidate detection boxes.

[0080] In one embodiment, the alignment module includes: The third calculation submodule is used to compare the multiple detection results pairwise to calculate the IoU value; The second alignment submodule is used to align the two candidate detection boxes when there are two candidate detection boxes with an IoU value greater than a second preset threshold.

[0081] In one embodiment, the second preset threshold is determined as follows: The IoU values ​​are calculated by comparing each pair of the multiple detection results to obtain the IoU matrix; Calculate the proportion of off-diagonal elements in the IoU matrix that are below the overlap threshold to obtain the global crowding of the current image to be detected; Substituting the global congestion level into the following formula, a second preset threshold is obtained:

[0082] in, t The second preset threshold is adaptive; base The base threshold; k is the adjustment coefficient; d is the global crowding level; t min t is the lower limit of the threshold; max This represents the upper limit of the threshold.

[0083] In one embodiment, the first determining module includes: Substitute the values ​​into the submodule, which applies the variance adjustment fraction to the following formula to obtain the time series stability index:

[0084] in, Let be the time series stability index at the current time t; The time series stability index is the index of the previous time step t-1; For smoothing coefficients; The target detection bounding box for the current frame (time step t); It is a geometric function that reflects the geometric relationship between the current frame detection box and historical frames.

[0085] In one embodiment, the second determining module is further configured to: When the time series stability index is greater than the first preset value and the confidence score variance is less than the second preset value, the stability counter is incremented by 1. Otherwise, the stable counter is reset to zero.

[0086] Figure 3 This is a schematic diagram of the hardware structure of a wet digestion state determination system according to an embodiment of this application, as shown below. Figure 3 As shown, the wet digestion state determination system includes: At least one processor 320; and, Memory 304 communicatively connected to the at least one processor 320; wherein, The memory 304 stores instructions that can be executed by the at least one processor 320 to implement the wet digestion state determination method described in any of the above embodiments.

[0087] Reference Figure 3 The wet digestion status determination system 300 may include one or more of the following components: processing component 302, memory 304, power supply component 306, multimedia component 308, audio component 310, input / output (I / O) interface 312, sensor component 314, and communication component 316.

[0088] Processing component 302 typically controls the overall operation of the wet digestion status determination system 300. Processing component 302 may include one or more processors 320 to execute instructions to complete all or part of the steps of the method described above. Furthermore, processing component 302 may include one or more modules to facilitate interaction between processing component 302 and other components. For example, processing component 302 may include a multimedia module to facilitate interaction between multimedia component 308 and processing component 302.

[0089] Memory 304 is configured to store various types of data to support the operation of the wet digestion status determination system 300. Examples of this data include instructions for any application or method operating on the wet digestion status determination system 300, such as text, images, videos, etc. Memory 304 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0090] Power supply component 306 provides power to various components of the wet digestion status determination system 300. Power supply component 306 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the wet digestion status determination system 300.

[0091] The multimedia component 308 includes a screen that provides an output interface between the wet digestion state determination system 300 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of touch or swipe actions but also the duration and pressure associated with the touch or swipe operation. In some embodiments, the multimedia component 308 may also include a front-facing camera and / or a rear-facing camera. When the wet digestion state determination system 300 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0092] Audio component 310 is configured to output and / or input audio signals. For example, audio component 310 includes a microphone (MIC) configured to receive external audio signals when the wet digestion status determination system 300 is in an operating mode, such as alarm mode, recording mode, voice recognition mode, and voice output mode. The received audio signals may be further stored in memory 304 or transmitted via communication component 316. In some embodiments, audio component 310 also includes a speaker for outputting audio signals.

[0093] I / O interface 312 provides an interface between processing component 302 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0094] Sensor assembly 314 includes one or more sensors for providing various aspects of the wet digestion state determination system 300's state assessment. For example, sensor assembly 314 may include a sound sensor. Additionally, sensor assembly 314 can detect the on / off state of the wet digestion state determination system 300, the relative positioning of components (e.g., the display and keypad of the wet digestion state determination system 300), and the operating state of the wet digestion state determination system 300 or a component thereof, such as the operating state of the air distribution plate, structural state, and discharge scraper, as well as the orientation, acceleration / deceleration, and temperature changes of the wet digestion state determination system 300. Sensor assembly 314 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 314 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 314 may further include an accelerometer, a gyroscope, a magnetic sensor, a pressure sensor, a material buildup thickness sensor, or a temperature sensor.

[0095] Communication component 316 is configured to enable the wet digestion status determination system 300 to communicate with other devices and cloud platforms via wired or wireless means. The wet digestion status determination system 300 can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 316 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 316 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0096] In an exemplary embodiment, the wet digestion state determination system 300 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the wet digestion state determination method described in any of the above embodiments.

[0097] This application also provides a computer-readable storage medium, which, when the instructions in the storage medium are executed by the processor corresponding to the wet digestion state determination system, enables the wet digestion state determination system to implement the wet digestion state determination method described in any of the above embodiments.

[0098] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0099] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0100] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0101] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0102] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for determining the state of wet digestion, characterized in that, include: Obtain the current image to be detected; The current image to be detected is detected by multiple random forward aggregations to obtain multiple sets of detection results, wherein the detection results include at least candidate detection boxes of test tube contents and confidence scores corresponding to the candidate detection boxes; Align the candidate detection boxes in the detection results; Calculate the mean and variance of the confidence scores of the aligned candidate detection boxes; The variance adjustment score is obtained based on the mean and variance of the confidence scores of the aligned candidate detection boxes, and the temporal stability index of the current image to be detected is determined based on the variance adjustment score. Based on the temporal stability index and confidence score variance of the current frame to be detected and the previous N frames to be detected, the number of stable frames in the N+1 consecutive frames to be detected is determined, where the number of stable frames is negatively correlated with the degree of resolution. When the number of stable frames is greater than the first preset threshold, the current detection status of the test tube contents is determined to be undigested. The step of obtaining the variance-adjusted score based on the mean and variance of the confidence scores of the aligned candidate detection boxes includes: The confidence score is adjusted by soft adjustment or hard gating to obtain the variance adjustment score. Soft adjustment dynamically adjusts the model output by probability or threshold, retaining some results with high uncertainty; hard gating directly filters the results based on an absolute threshold, retaining only predictions with high certainty and discarding the rest. The step of determining the temporal stability index of the current image to be detected based on the variance adjustment score includes: Applying the variance adjustment fraction to the following formula yields the time series stability index: in, Let be the time series stability index at the current time t; The time series stability index is the index of the previous time step t-1; This is a smoothing coefficient, and its value is a number between 0 and 1. The target detection bounding box for the current frame; It is a geometric function that reflects the geometric relationship between the current frame detection box and historical frames.

2. The method as described in claim 1, characterized in that, The step of obtaining the current image to be detected includes: Get the original image of the current test tube; Calculate the vertical gradient energy column-wise for the original image; The vertical gradient energy is smoothed. Determine the position of the test tube spindle corresponding to the peak value of the vertical gradient energy after smoothing; The peak value of the vertical gradient energy is extended to the left and right to determine the lateral range of the test tube; The current image to be detected is obtained from the image of the horizontal range of the test tube.

3. The method as described in claim 1, characterized in that, The step of aligning the candidate detection boxes in the detection results includes: Obtain the candidate bounding box with the highest confidence score from multiple sets of detection results; Calculate the IoU value between the remaining candidate detection boxes and the candidate detection box with the highest confidence score; When there is a target candidate detection box with an IoU value greater than the second preset threshold in the remaining candidate detection boxes, the target candidate detection box is aligned with the candidate detection box with the highest confidence score.

4. The method as described in claim 1, characterized in that, The step of aligning the candidate detection boxes in the detection results includes: The IoU value is calculated by comparing each pair of the multiple sets of detection results; When there are two candidate detection boxes with an IoU value greater than the second preset threshold, the two candidate detection boxes are aligned.

5. The method as described in claim 3 or 4, characterized in that, The second preset threshold is determined in the following way: The IoU values ​​are calculated by comparing each pair of the multiple sets of detection results to obtain the IoU matrix; Calculate the global congestion level based on the IoU matrix and the following formula: in, d The congestion level, with a value ranging from [0,1], reflects the density of overlapping targets in the current detection bounding box; M ij The IoU value between the i-th and j-th detection boxes is in the range [0,1], which measures the degree of overlap between the two boxes. τ d This is the crowding threshold; if the IoU between two boxes exceeds this value, they are considered to overlap. A second preset threshold is obtained based on the global congestion level.

6. The method as described in claim 1, characterized in that, The step of determining the number of stable frames in a series of N+1 consecutive frames of images to be detected based on the temporal stability index and confidence score variance of the current frame image to be detected and the previous N frames image to be detected includes: When the time series stability index is greater than the first preset value and the confidence score variance is less than the second preset value, the stability counter is incremented by 1. Otherwise, the stable counter is reset to zero.

7. A device for determining the state of wet digestion, used in the method for determining the state of wet digestion as described in any one of claims 1-6, characterized in that, include: The acquisition module is used to acquire the current image to be detected; The detection module is used to detect the current image to be detected by multiple random forward aggregations to obtain multiple sets of detection results, wherein the detection results include at least candidate detection boxes of test tube contents and confidence scores corresponding to the candidate detection boxes; An alignment module is used to align candidate detection boxes in the detection results; The calculation module is used to calculate the mean and variance of the confidence scores of the aligned candidate detection boxes. The first determining module is used to obtain a variance adjustment score based on the mean confidence score and the variance of the confidence score of the aligned candidate detection boxes, and to determine the temporal stability index of the current image to be detected based on the variance adjustment score. The second determining module is used to determine the number of stable frames in a continuous N+1 frames of images to be detected based on the temporal stability index and confidence score variance of the current frame image to be detected and the previous N frames image to be detected. The number of stable frames is negatively correlated with the degree of resolution. The third determining module is used to determine that the current detection status of the test tube contents is undigested when the number of stable frames is greater than the first preset threshold.

8. A system for determining the state of wet digestion, characterized in that, include: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to implement the wet digestion state determination method as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, When the instructions in the storage medium are executed by the processor corresponding to the wet digestion status determination system, the wet digestion status determination system is able to implement the wet digestion status determination method as described in any one of claims 1-6.

Images