Method for judging liquid dripping state of chemical device

Abstract

The invention discloses a method for judging a liquid dripping state of a chemical device, which relates to the technical field of liquid dripping detection and comprises the following steps of: acquiring video frame data; performing liquid water drop detection by adopting the liquid water drop detection model; a liquid level detection model is adopted for liquid level detection; extracting a region of interest from the liquid level result; performing feature extraction on the region of interest to obtain video frame features; performing similarity calculation on the video frame features of every two adjacent video frames; an adaptive threshold algorithm is adopted to generate an adaptive threshold according to the similarity features; and judging whether liquid drops exist or not. The liquid drop detection and the liquid level detection are performed through the target detection algorithm, the similarity calculation and the adaptive threshold algorithm are combined, the automatic real-time detection of the liquid dripping state is realized in a combined manner, and the flexibility and the adaptability of the liquid dripping state detection are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of liquid drop detection, and in particular to a method for judging liquid drop state of a chemical device. BACKGROUND

[0002] In the automatic experimental device of the experimental platform, the demand for monitoring the liquid drop state is particularly prominent, which is of great significance for the current chemical filtration, column (adsorption filtration) reaction process judgment. When there is a liquid drop state, it means that the previous chemical filtration operation is still continuing, and if the liquid drop state disappears, it means that the previous chemical filtration operation is terminated. Therefore, in a closed experimental device, how to accurately monitor the liquid drop state of the liquid in the reaction bottle and balance between real-time and accuracy has become a very challenging task.

[0003] In the prior art, Chinese patent CN108761551A proposes a liquid drop detection method based on a ray type density sensor, which belongs to the technical field of liquid drop detection. The method mainly realizes it by deploying a ray emitter, a ray receiving sensor, a single-chip microcomputer and a buzzer on the dropper. The ray emitter will emit cesium-137 or cobalt-60 isotope beam gamma rays. By using the different intensities of gamma rays attenuated when penetrating different substances, it is detected whether there is a liquid drop phenomenon in the dropper. Although the method has stable performance and long service life, the method is only suitable for the dropper device proposed in the patent, and if other experimental equipment of different shapes is used, the ray emitter and the ray receiving device need to be re-adapted, resulting in poor applicability and high cost. Chinese patent CN117073911A proposes a space liquid drop detection system, method and computer equipment. The method acquires a detection image by collecting the area irradiated by a laser through an image acquisition device, and then realizes water leakage detection through an image matching method. Although the method is not limited by experimental equipment, the image matching method needs to set a template image. If the test environment is changed, a new template needs to be replaced, which also limits its use environment.

[0004] In summary, the above-mentioned prior art is limited by experimental equipment and test environment, and the flexibility and adaptability of liquid drop state detection are poor. SUMMARY

[0005] The present application provides a method for judging the liquid drop state of a chemical device to solve the problem that the existing liquid drop state detection technology is limited by experimental equipment and test environment, and the flexibility and adaptability of liquid drop state detection are poor.

[0006] In one aspect, the present application provides a method for judging the liquid drop state of a chemical device, comprising the following steps:

[0007] Step one, collect video frame data in a specified time window.

[0008] Step two, use a liquid water droplet detection model based on a target detection algorithm to detect liquid water droplets in the video frame data. If liquid water droplets are detected, return to step one. If not, proceed to the next step.

[0009] Step three, use a liquid level detection model based on a target detection algorithm to detect liquid level in the video frame data. If no liquid level is detected, return to step one. If detected, proceed to the next step.

[0010] Step four, extract the region of interest from the liquid level detection model detected liquid level result.

[0011] Step five, extract features from the region of interest to obtain video frame features.

[0012] Step six, calculate the similarity of the video frame features of each pair of adjacent video frames in the specified time window to obtain several similarity features.

[0013] Step seven, use an adaptive threshold algorithm to generate an adaptive threshold based on the similarity features.

[0014] Step eight, determine whether there is liquid dripping in the specified time window based on the adaptive threshold and the similarity features. If there is, return to step one. If not, it means the previous chemical operation has been completed.

[0015] In one possible implementation, in step one, after obtaining the video frame data, the video frame data is cropped to extract a chemical device region image.

[0016] In one possible implementation, in step two, before liquid water droplet detection, the chemical device region image is cropped to extract a liquid dripping region image.

[0017] The liquid water droplet detection model is used to detect liquid water droplets in the liquid dripping region image.

[0018] The liquid water droplet detection model is obtained by training a Yolov8 network using a liquid dripping region image training dataset.

[0019] In one possible implementation, in step three, the liquid level detection model is used to detect liquid level in the chemical device region image.

[0020] The liquid level detection model is obtained by training a Yolov8 network using a chemical device region image training dataset.

[0021] In one possible implementation, in step four, the center point of the detection box of the liquid level result is first determined, and then the region of interest is extracted based on a pre-set threshold of interest.

[0022] In one possible implementation, in step five, the LBP histogram feature extraction algorithm is used to extract features from the region of interest.

[0023] In one possible implementation, in step six, the similarity calculation employs the Manhattan similarity algorithm.

[0024] In one possible implementation, in step seven, the adaptive thresholding algorithm generates an adaptive threshold based on the standard deviation and mean of the similarity features; the adaptability of the adaptive threshold is achieved as the time window moves.

[0025] In one possible implementation, step eight includes:

[0026] Each similarity feature is compared with the adaptive threshold. If at least one similarity feature is greater than the adaptive threshold within the specified time window, it indicates that liquid droplets are falling within the specified time window, and the process returns to step one. Otherwise, it indicates that no liquid droplets are falling, and the previous chemical operation has been completed.

[0027] The method for determining the dripping state of liquid in a chemical apparatus according to this application has the following advantages:

[0028] This application employs a target detection algorithm for liquid droplet and liquid level detection, combined with similarity calculation and an adaptive threshold algorithm, to achieve automated real-time detection of liquid droplet states, thus improving the flexibility and adaptability of liquid droplet state detection. The method utilizes a dual-judgment approach: first, a judgment based on liquid droplet detection; and second, a judgment based on liquid level changes determined by liquid level detection, similarity calculation, and adaptive thresholding. This dual-judgment approach enhances the reliability of liquid droplet state detection.

[0029] The method presented in this application is a purely vision-based solution, relying entirely on image data captured by a camera to achieve real-time detection of liquid droplets, completely eliminating reliance on traditional sensors and other hardware. This not only enables remote, non-contact, and unmanned automated liquid droplet detection but also significantly reduces the potential harm of chemicals to researchers. Furthermore, as a purely vision-based solution, it is not limited by the environment and can be adapted to various business scenarios through algorithm adjustments, greatly reducing application costs.

[0030] The method of this application uses an adaptive threshold algorithm to generate an adaptive threshold based on similarity features. It determines whether liquid droplets exist within a specified time window based on the adaptive threshold and similarity features. It does not require pre-setting parameters or environmental template features, can adapt to various working environments, has good stability and generalization, and can maintain high-precision detection results under different conditions, further improving the reliability and applicability of liquid droplet state detection. Attached Figure Description

[0031] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0032] Figure 1 A flowchart illustrating a method for determining the liquid dripping state in a chemical apparatus, provided in an embodiment of this application;

[0033] Figure 2 Examples of video frame data provided in embodiments of this application;

[0034] Figure 3 An example of a chemical device region image obtained by cropping video frame data, provided in an embodiment of this application;

[0035] Figure 4 This is an example of a liquid droplet area image obtained by cropping an image of a chemical device area according to an embodiment of this application.

[0036] Figure 5 This is an example of a result diagram of liquid droplet detection provided in an embodiment of this application;

[0037] Figure 6 This is an example of a result graph of liquid level detection provided in an embodiment of this application;

[0038] Figure 7 This is an example of a result image of the region of interest extracted according to an embodiment of this application. Detailed Implementation

[0039] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0040] like Figure 1As shown in the figure, this application provides a method for determining the liquid dripping state of a chemical device, including the following steps:

[0041] Step 1: Collect video frame data within the specified time window.

[0042] Step 2: Use a liquid droplet detection model based on target detection algorithm to detect liquid droplets in the video frame data. If liquid droplets are detected, return to step 1; otherwise, proceed to the next step.

[0043] Step 3: Use a liquid level detection model based on target detection algorithm to detect the liquid level in the video frame data. If no liquid level is detected, return to step 1; if a liquid level is detected, proceed to the next step.

[0044] Step 4: Extract the region of interest from the liquid level detection results detected by the liquid level detection model.

[0045] Step 5: Extract features from the region of interest to obtain video frame features.

[0046] Step 6: Calculate the similarity of video frame features for each pair of adjacent video frames within the specified time window to obtain several similarity features.

[0047] Step 7: Use an adaptive threshold algorithm to generate an adaptive threshold based on the similarity features.

[0048] Step 8: Determine whether liquid droplets exist within the specified time window based on the adaptive threshold and the similarity feature. If they do, return to step 1. If they do not, it means that the previous chemical operation has been completed.

[0049] For example, in step one, after obtaining the video frame data, the video frame data is cropped to extract the image of the chemical device area.

[0050] Specifically, in this embodiment, RTSP (Real-Time Streaming Protocol) streaming technology is used to acquire video frame data of the chemical apparatus (a reaction vessel in this embodiment) in real time from a camera. The video frame data mainly describes the liquid dripping process in the reaction vessel, capturing key phenomena such as liquid droplet and liquid level changes, providing high-quality input data for subsequent target detection, texture feature extraction, and similarity calculation. Figure 2 The image shows an example of video frame data.

[0051] After obtaining the video frame data, a masking technique is used to crop the video frame data and extract the image of the chemical device area to eliminate background interference. For example... Figure 3 The image shown is an example of a region image of a chemical apparatus.

[0052] For example, in step two, before performing liquid droplet detection, the image of the chemical device area is cropped to extract the image of the liquid droplet area.

[0053] The liquid droplet detection model is used to detect liquid droplets in the image of the liquid droplet area.

[0054] The liquid droplet detection model is obtained by training a Yolov8 network using a training dataset of liquid droplet region images.

[0055] Specifically, in this embodiment, the image of the chemical device area is cropped using masking technology to extract the image of the liquid dripping area. For example... Figure 4 The image shown is an example of an image of the liquid droplet area.

[0056] In this embodiment, the liquid droplet area image training dataset includes a collection of pre-annotated historical liquid droplet area images. Using this dataset, a deep learning model is trained on an existing Yolov8 network. The resulting liquid droplet detection model can accurately identify and locate liquid droplets, thereby monitoring the progress of the chemical reaction in real time. When the liquid droplet detection model detects a liquid droplet, it indicates that the previous chemical operation (such as filtering or column chromatography) is still in progress. The process returns to step one, re-acquiring and updating the video frame data within the time window. When the model does not detect a liquid droplet, the next step, liquid level detection, is executed. Figure 5 The image shown is an example of the results of liquid droplet detection.

[0057] For example, in step three, the liquid level detection model is used to detect the liquid level in the image of the chemical device area.

[0058] The liquid level detection model was obtained by training a Yolov8 network using a training dataset of images of the chemical device area.

[0059] Specifically, in this embodiment, the chemical device region image training dataset includes a collection of pre-annotated historical chemical device region images. The existing Yolov8 network is trained using this chemical device region image training dataset, resulting in a liquid level detection model capable of accurately identifying and locating liquid levels. When the liquid level detection model fails to detect a liquid level, it indicates that the previous chemical operation has not yet begun, and the process returns to step one; when the liquid level detection model detects a liquid level, the next step is executed to extract the region of interest. For example... Figure 6 The image shown is an example of the results of liquid level detection.

[0060] For example, in step four, the center point of the detection frame of the liquid level result is first determined, and then the region of interest is extracted based on a pre-set threshold of interest.

[0061] Specifically, the formula for extracting the region of interest is as follows:

[0062] I roi =(x min ,y mon ,x max ,y max )=(x c -T roi ,y c -T roi ,x c +T roi ,y c +T roi ).

[0063] Among them, (x c ,y c T is the center point of the detection frame for the liquid level result. roi It is the interest threshold, I roi This is the region of interest for the liquid level results. For example... Figure 7 The image shown is an example of the result of extracting the region of interest.

[0064] For example, in step five, the LBP histogram feature extraction algorithm is used to extract features from the region of interest.

[0065] Specifically, the formula for feature extraction using the LBP histogram feature extraction algorithm is as follows:

[0066] f t =concatenate(M LBP (I roi )).

[0067] Among them, M LBP () represents the LBP histogram feature extraction algorithm, I roi This represents the region of interest in the liquid level results; `concatenate()` represents feature concatenation; `f` t Here, t represents the LBP histogram features (i.e., video frame features), and t represents the current time.

[0068] For example, in step six, the similarity calculation uses the Manhattan similarity algorithm.

[0069] Specifically, in this embodiment, the length of the time window is set to 10 frames, resulting in 10 LBP histogram features (i.e., video frame features). in, This indicates the video frame characteristics of the 10th frame.

[0070] The Manhattan similarity algorithm is used to calculate the similarity of video frame features between pairs of adjacent video frames, resulting in nine similarity features:

[0071]

[0072] in, It is the tth i LBP histogram features at each time step (i.e., video frame features), It is the tth i+1 LBP histogram features at time S. man () indicates the Manhattan similarity algorithm. Indicates the t-th i Video frame features at time t and the t-th i+1 The Manhattan similarity value (i.e., similarity feature) of the video frame features at a given time.

[0073] For example, in step seven, the adaptive threshold algorithm generates an adaptive threshold based on the standard deviation and mean of the similarity features; the adaptability of the adaptive threshold is achieved as the time window moves.

[0074] Specifically, in this embodiment, the adaptive thresholding algorithm calculates the standard deviation and mean of all similarity features within a specified time window, and sets the sum of k times the standard deviation and the mean as the adaptive threshold for the current time window. As the time window moves, new video frame data is continuously added, while the oldest video frame data is discarded. Whenever the time window moves, the standard deviation and mean of all similarity features within the current time window are recalculated, and the sum of k times the new standard deviation and the mean is updated as the latest adaptive threshold. The adaptive threshold can adaptively adjust according to changes in video frame data, ensuring the accuracy and robustness of the algorithm. This dynamic adjustment mechanism allows the adaptive threshold to better reflect real-time changes in video frame data, thereby improving overall performance and reliability.

[0075] In this embodiment, the adaptive threshold is calculated using the following formula:

[0076]

[0077] Among them, T dynamic For adaptive thresholding, Std() is the standard deviation, Mean() is the mean, and k represents the multiple of the standard deviation, which can be manually adjusted.

[0078] For example, step eight includes:

[0079] Each similarity feature is compared with the adaptive threshold. If at least one similarity feature is greater than the adaptive threshold within the specified time window, it indicates that liquid droplets are falling within the specified time window, and the process returns to step one. Otherwise, it indicates that no liquid droplets are falling, and the previous chemical operation has been completed.

[0080] Specifically, the formula for comparing each similarity feature with the adaptive threshold is as follows:

[0081]

[0082] When at least one similarity feature Greater than the adaptive threshold T dynamic When the status is 0, it indicates that liquid is dripping within the specified time window, and the process returns to step one; when all... All are less than or equal to the adaptive threshold T dynamic When the state is 1, it indicates that no liquid is dripping and the previous chemical operation has been completed.

[0083] This application's embodiments utilize a target detection algorithm for liquid droplet detection and liquid level detection, combined with similarity calculation and an adaptive threshold algorithm, to achieve automated real-time detection of liquid droplet states, improving the flexibility and adaptability of liquid droplet state detection. The method employs a dual-judgment approach: first, judgment based on liquid droplet detection; second, judgment based on liquid level detection combined with similarity calculation and adaptive thresholding to assess liquid level changes. This dual-judgment approach enhances the reliability of liquid droplet state detection.

[0084] The method presented in this application is a purely vision-based solution, relying entirely on image data captured by a camera to achieve real-time detection of liquid droplets, completely eliminating reliance on traditional sensors and other hardware. This not only enables remote, non-contact, and unmanned automated liquid droplet detection but also significantly reduces the potential harm of chemicals to researchers. Furthermore, as a purely vision-based solution, it is not limited by the environment and can be adapted to various business scenarios through algorithm adjustments, greatly reducing application costs.

[0085] The method of this application uses an adaptive threshold algorithm to generate an adaptive threshold based on similarity features. It determines whether liquid droplets exist within a specified time window based on the adaptive threshold and similarity features. It does not require pre-setting parameters or environmental template features, can adapt to various working environments, has good stability and generalization, and can maintain high-precision detection results under different conditions, further improving the reliability and applicability of liquid droplet state detection.

[0086] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0087] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for determining the liquid dripping state in a chemical apparatus, characterized in that, Includes the following steps: Step 1: Collect video frame data within the specified time window; Step 2: Use a liquid droplet detection model based on target detection algorithm to detect liquid droplets in the video frame data. If liquid droplets are detected, return to step 1; otherwise, proceed to the next step. Step 3: Use a liquid level detection model based on target detection algorithm to detect the liquid level in the video frame data. If no liquid level is detected, return to step 1; if a liquid level is detected, proceed to the next step. Step 4: Extract the region of interest from the liquid level detection results detected by the liquid level detection model; Step 5: Extract features from the region of interest to obtain video frame features; Step 6: Calculate the similarity of video frame features for each pair of adjacent video frames within the specified time window to obtain several similarity features; Step 7: Use an adaptive threshold algorithm to generate an adaptive threshold based on the similarity features; Step 8: Determine whether liquid droplets exist within the specified time window based on the adaptive threshold and the similarity feature. If they do, return to step 1. If they do not, it means that the previous chemical operation has been completed.

2. The method for determining the liquid dripping state of a chemical apparatus according to claim 1, characterized in that, In step one, after obtaining the video frame data, the video frame data is cropped to extract the image of the chemical device area.

3. The method for determining the liquid dripping state of a chemical apparatus according to claim 2, characterized in that, In step two, before detecting liquid droplets, the image of the chemical device area is cropped to extract the image of the liquid droplet area; The liquid droplet detection model is used to detect liquid droplets in the image of the liquid droplet area. The liquid droplet detection model is obtained by training a Yolov8 network using a training dataset of liquid droplet region images.

4. The method for determining the liquid dripping state in a chemical apparatus according to claim 2, characterized in that, In step three, the liquid level detection model is used to detect the liquid level in the image of the chemical device area; The liquid level detection model was obtained by training a Yolov8 network using a training dataset of images of the chemical device area.

5. The method for determining the liquid dripping state in a chemical apparatus according to claim 1, characterized in that, In step four, the center point of the detection frame of the liquid level result is first determined, and then the region of interest is extracted based on the pre-set threshold of interest.

6. The method for determining the liquid dripping state of a chemical apparatus according to claim 1, characterized in that, In step five, the LBP histogram feature extraction algorithm is used to extract features from the region of interest.

7. The method for determining the liquid dripping state in a chemical apparatus according to claim 1, characterized in that, In step six, the similarity calculation uses the Manhattan similarity algorithm.

8. The method for determining the liquid dripping state of a chemical apparatus according to claim 1, characterized in that, In step seven, the adaptive threshold algorithm generates an adaptive threshold based on the standard deviation and mean of the similarity features; The adaptability of the adaptive threshold is achieved as the time window moves.

9. The method for determining the liquid dripping state of a chemical apparatus according to claim 1, characterized in that, Step eight includes: Each similarity feature is compared with the adaptive threshold. If at least one similarity feature is greater than the adaptive threshold within the specified time window, it indicates that liquid droplets are falling within the specified time window, and the process returns to step one. Otherwise, it indicates that no liquid droplets are falling, and the previous chemical operation has been completed.

Images