An adaptive liquid recognition method based on unsupervised learning

Abstract

The application discloses a kind of self-adapting liquid distribution recognition methods based on unsupervised learning, it is related to liquid distribution recognition technical field, including: including the following steps: collecting continuous video frame data in liquid distribution process;Extract region of interest;Extract multiple features in region of interest;According to the feature extracted, calculate the feature difference value corresponding to current frame;Dynamic weighted fusion calculates the fusion difference value corresponding to current frame;The fusion difference value corresponding to current frame is adaptively quartile range anomaly detection and structure similarity anomaly detection, realizes liquid distribution recognition.The application is based on adaptive quartile range anomaly detection and structure similarity anomaly detection, respectively from pixel level level and global level level, carry out double-scale anomaly detection, improve the accuracy and stability of liquid distribution recognition.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of liquid separation recognition, and in particular to an adaptive liquid separation recognition method based on unsupervised learning. BACKGROUND

[0002] In chemical experiments, researchers often need to perform liquid separation operation on two immiscible liquids. The traditional method requires the experimenter to constantly observe the liquid interface and manually open and close the valve when the interface is observed to achieve liquid separation. This method not only consumes time and effort, but also requires frequent manual intervention, greatly increasing the workload of the experimenter. Therefore, it is necessary to study the liquid separation recognition method.

[0003] In the prior art, Chinese patent CN118609053A discloses a liquid separation recognition method, system, device and storage medium, which obtains real-time image information of the liquid surface of the reaction kettle during the extraction process, extracts features from the image information, and performs target detection to obtain the reaction kettle liquid surface image features and target detection results at each time; and judges the completion of the extraction reaction by comparing the image features and the target detection results, and separates the liquid according to the completion of the extraction reaction; when separating the liquid, the liquid surface real-time image information of the liquid separation pipe is extracted, and whether the liquid separation is completed is judged according to the comparison results of the image features at the front and rear moments and the position of the reaction kettle layer interface, so as to realize liquid separation recognition.

[0004] However, the accuracy and stability of the liquid separation recognition of the above-mentioned prior art need to be improved. SUMMARY

[0005] The present application provides an adaptive liquid separation recognition method based on unsupervised learning to solve the problem that the accuracy and stability of the liquid separation recognition of the prior art need to be improved.

[0006] In one aspect, the present application provides an adaptive liquid separation recognition method based on unsupervised learning, comprising the following steps:

[0007] Step one, collecting continuous video frame data in the liquid separation process.

[0008] Step two, extracting the region of interest in the continuous video frame data.

[0009] Step three, extracting the HSV histogram feature, edge density feature and LBP texture feature in the region of interest.

[0010] Step four, according to the extracted features, calculating the feature difference value between the current frame and the previous frame with a fixed time interval to obtain the feature difference value corresponding to the current frame.

[0011] Step five, the feature difference value corresponding to the current frame is dynamically weighted and fused to obtain a fusion difference value corresponding to the current frame.

[0012] Step six, the fusion difference value corresponding to the current frame is adaptively quartile range anomaly detected, if judged as normal, return to step one, if judged as abnormal, then proceed to the next step.

[0013] Step seven, the structure similarity between the current frame and the previous frame with a fixed time interval is calculated to obtain a structure similarity index corresponding to the current frame, and the structure similarity index corresponding to the current frame is subjected to structure similarity anomaly detection, if judged as normal, return to step one, if judged as abnormal, then the separation is completed.

[0014] In a possible implementation, in step two, the region of interest refers to a separation change sensitive region, which is determined through geometric calibration based on the fixed spatial position relationship between the acquisition device and the separation device.

[0015] In a possible implementation, in step three, the HSV histogram feature, the edge density feature and the LBP texture feature are stored in a fixed length feature sliding window, and a first-in first-out rule is followed.

[0016] In a possible implementation, in step four, the feature difference value corresponding to the current frame includes: an HSV histogram feature difference value corresponding to the current frame, an edge density feature difference value corresponding to the current frame, and an LBP texture feature difference value corresponding to the current frame.

[0017] The HSV histogram feature difference value is calculated by using a chi-square distance.

[0018] The edge density feature difference value is calculated by using a Hausdorff distance.

[0019] The LBP texture feature difference value is calculated by using a Euclidean distance.

[0020] In a possible implementation, in step five, based on the features of all historical frames in the feature sliding window, the coefficients of variation of the HSV histogram feature, the edge density feature and the LBP texture feature are calculated respectively, the three coefficients of variation are normalized to obtain three dynamic weight coefficients, and the feature difference value corresponding to the current frame is dynamically weighted and fused based on the dynamic weight coefficients to obtain a fusion difference value corresponding to the current frame.

[0021] In a possible implementation, in step six, the fusion difference values corresponding to all the historical frames in the feature sliding window are stored in the difference value sliding window, the quartile and interquartile range of all the fusion difference values in the difference value sliding window are calculated, the adaptive interquartile range anomaly threshold is obtained, and adaptive interquartile range anomaly detection is performed on the fusion difference value corresponding to the current frame according to the interquartile range anomaly threshold.

[0022] In a possible implementation, in step six, if the fusion difference value corresponding to the current frame is determined to be normal, the fusion difference value corresponding to the current frame is added to the difference value sliding window, and the first fusion difference value in the difference value sliding window is discarded, and the method returns to step one.

[0023] If the fusion difference value corresponding to the current frame is determined to be abnormal, the fusion difference value corresponding to the current frame is not added to the difference value sliding window, and the next step is performed.

[0024] In a possible implementation, in step seven, the structural similarity indexes corresponding to all the historical frames in the feature sliding window are stored in the structural similarity sliding window, the mean and variance of all the structural similarity indexes in the structural similarity sliding window are calculated, the structural similarity anomaly threshold is obtained, and structural similarity anomaly detection is performed on the structural similarity index corresponding to the current frame according to the structural similarity anomaly threshold.

[0025] In a possible implementation, in step seven, if the structural similarity index corresponding to the current frame is determined to be normal, the structural similarity index corresponding to the current frame is added to the structural similarity sliding window, and the first structural similarity index in the structural similarity sliding window is discarded, and the method returns to step one.

[0026] If the structural similarity index corresponding to the current frame is determined to be abnormal, the structural similarity index corresponding to the current frame is not added to the structural similarity sliding window, and the method is completed.

[0027] The adaptive separation recognition method based on unsupervised learning has the following advantages:

[0028] Through adaptive interquartile range anomaly detection and structural similarity anomaly detection, double-scale anomaly detection is performed from the pixel level and the global level, respectively, so that the accuracy and stability of separation recognition are improved. The adaptive interquartile range anomaly detection is at the pixel level and is used for anomaly detection of the fusion difference value, and the structural similarity anomaly detection is at the global level and is used for anomaly detection of the structural similarity index.

[0029] The variation coefficients of the HSV histogram features, the edge density features and the LBP texture features are calculated, the three variation coefficients are normalized, three dynamic weight coefficients are obtained, the feature difference values corresponding to the current frame are dynamically weighted and fused based on the dynamic weight coefficients, the fusion difference values corresponding to the current frame are obtained, the cross-scene adaptive multi-modal feature fusion is realized, and the scene adaptability is improved.

[0030] The dynamic weighted fusion and the double-scale anomaly detection in the application do not need to preset fixed parameters, but dynamically adjust the parameters according to the features in the actual application scene, and the unsupervised learning method not only enhances the generalization ability, but also ensures the effectiveness in various complex environments. BRIEF DESCRIPTION OF DRAWINGS

[0031] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0032] Figure 1 A flowchart of an adaptive liquid recognition method based on unsupervised learning provided by an embodiment of the application. DETAILED DESCRIPTION

[0033] The technical solutions in the embodiments of the application will be described clearly and completely in the embodiments of the application combined with the drawings. Obviously, the described embodiments are only some of the embodiments of the application, not all. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor are within the protection scope of the application.

[0034] As shown in the drawings, Figure 1 The embodiment of the application provides an adaptive liquid recognition method based on unsupervised learning, which comprises the following steps:

[0035] Step one, collecting continuous video frame data in the liquid separation process.

[0036] Step two, extracting a region of interest in the continuous video frame data.

[0037] Step three, extracting HSV histogram features, edge density features and LBP texture features in the region of interest.

[0038] Step four, according to the extracted features, calculating the feature difference value between the current frame and the previous frame with a fixed time interval, to obtain the feature difference value corresponding to the current frame.

[0039] Step five, the feature difference value corresponding to the current frame is dynamically weighted and fused to obtain a fusion difference value corresponding to the current frame.

[0040] Step six, the fusion difference value corresponding to the current frame is adaptively detected by quartile distance anomaly detection, if it is judged to be normal, return to step one, if it is judged to be abnormal, then the next step is performed.

[0041] Step seven, the structure similarity between the current frame and the previous frame with a fixed time interval is calculated to obtain a structure similarity index corresponding to the current frame, and the structure similarity index corresponding to the current frame is detected by structure similarity anomaly detection, if it is judged to be normal, return to step one, if it is judged to be abnormal, then the separation is completed.

[0042] Specifically, in the embodiment, in step one, the continuous video frame data in the separation process is collected by a fixed industrial camera.

[0043] Exemplarily, in step two, the region of interest refers to a separation change sensitive region, which is determined by geometric calibration based on the fixed spatial position relationship between the collection device and the separation device.

[0044] Specifically, in the embodiment, the separation change sensitive region is determined by geometric calibration based on the fixed spatial position relationship between the fixed industrial camera and the separation device, so that the region of interest in the continuous video frame data can be extracted, and the interference of background irrelevant information can be eliminated.

[0045] Exemplarily, in step three, the HSV histogram feature, the edge density feature and the LBP texture feature are stored in a fixed length feature sliding window, and the first-in first-out rule is followed.

[0046] Specifically, the HSV histogram feature, the edge density feature and the LBP texture feature in the region of interest are stored in the feature sliding window, so that the dynamic update of time series data can be realized.

[0047] Exemplarily, in step four, the feature difference value corresponding to the current frame includes: the HSV histogram feature difference value corresponding to the current frame, the edge density feature difference value corresponding to the current frame, and the LBP texture feature difference value corresponding to the current frame.

[0048] The HSV histogram feature difference value is calculated by using chi-square distance.

[0049] The edge density feature difference value is calculated by using Hausdorff distance.

[0050] The LBP texture feature difference value is calculated by using Euclidean distance.

[0051] Specifically, the feature difference value of the current frame and the previous frame in the fixed time interval is calculated by using the chi-square distance, the Hausdorff distance and the Euclidean distance, and the distance descriptors reflect the degree of visual change between the current frame and the previous frame in the fixed time interval.

[0052] For example, in step five, the coefficients of variation of the HSV histogram feature, the edge density feature and the LBP texture feature are respectively calculated based on the features of all the historical frames in the feature sliding window, the three coefficients of variation are normalized to obtain three dynamic weight coefficients, and the feature difference value corresponding to the current frame is dynamically weighted and fused based on the dynamic weight coefficients to obtain a fusion difference value corresponding to the current frame.

[0053] Specifically, the coefficient of variation is used to quantify the stability of the feature by using the ratio of the standard deviation to the mean. In this embodiment, the coefficient of variation of the HSV histogram feature is the ratio of the standard deviation to the mean of the HSV histogram features of all the historical frames in the feature sliding window, the coefficient of variation of the edge density feature is the ratio of the standard deviation to the mean of the edge density features of all the historical frames in the feature sliding window, and the coefficient of variation of the LBP texture feature is the ratio of the standard deviation to the mean of the LBP texture features of all the historical frames in the feature sliding window.

[0054] In this embodiment, the three coefficients of variation are normalized to obtain three dynamic weight coefficients, and the feature difference value corresponding to the current frame is dynamically weighted and fused, so that the features with high stability occupy a higher weight in the decision.

[0055] For example, in step six, the fusion difference values corresponding to all the historical frames in the feature sliding window are stored in the difference value sliding window, the quartiles and the interquartile range of all the fusion difference values in the difference value sliding window are calculated to obtain an adaptive interquartile range anomaly threshold, and the fusion difference value corresponding to the current frame is adaptively detected by using the interquartile range anomaly threshold.

[0056] Specifically, in this embodiment, the quartiles of all the fusion difference values in the difference value sliding window are calculated, including the lower quartile Q1 (25% quartile), the median Q2 (50% quartile) and the upper quartile Q3 (75% quartile), and the interquartile range is equal to the difference between the upper quartile Q3 and the lower quartile Q1.

[0057] In this embodiment, the interquartile range anomaly threshold is obtained by the following formula:

[0058] T IQR = Q3 + k * IQR.

[0059] Wherein, T IQR represents the interquartile range anomaly threshold, k represents the interquartile range weight coefficient, and IQR represents the interquartile range.

[0060] In step six, if the fusion difference value corresponding to the current frame is judged to be normal, the fusion difference value corresponding to the current frame is added to the difference value sliding window, and the first fusion difference value in the difference value sliding window is discarded, and the process returns to step one.

[0061] If the fusion difference value corresponding to the current frame is judged to be abnormal, the fusion difference value corresponding to the current frame is not added to the difference value sliding window, and the next step is performed.

[0062] Specifically, in the embodiment, if the fusion difference value corresponding to the current frame is less than the quartile range abnormal threshold, it is judged to be normal; if the fusion difference value corresponding to the current frame is greater than or equal to the quartile range abnormal threshold, it is judged to be abnormal.

[0063] In the embodiment, the fusion difference value corresponding to the current frame when judged to be abnormal is not added to the difference value sliding window, so as to avoid the influence of high abnormal values on the mean baseline of the difference value sliding window.

[0064] In step seven, all the structural similarity indexes corresponding to the historical frames in the feature sliding window are stored in the structural similarity sliding window, the mean and variance of all the structural similarity indexes in the structural similarity sliding window are calculated to obtain a structural similarity abnormal threshold, and the structural similarity index corresponding to the current frame is subjected to structural similarity abnormality detection according to the structural similarity abnormal threshold.

[0065] Specifically, in the embodiment, the structural similarity abnormal threshold is obtained by the following formula:

[0066]

[0067] wherein, T SSIM represents the structural similarity abnormal threshold, μ represents the mean of the structural similarity index, j represents the structural similarity variance weight coefficient, represents the variance of the structural similarity index.

[0068] In step seven, if the structural similarity index corresponding to the current frame is judged to be normal, the structural similarity index corresponding to the current frame is added to the structural similarity sliding window, and the first structural similarity index in the structural similarity sliding window is discarded, and the process returns to step one.

[0069] If the structural similarity index corresponding to the current frame is judged to be abnormal, the structural similarity index corresponding to the current frame is not added to the structural similarity sliding window, and the process is completed.

[0070] Specifically, in the embodiment, if the structural similarity index corresponding to the current frame is less than the structural similarity anomaly threshold, it is judged as normal; if the structural similarity index corresponding to the current frame is greater than or equal to the structural similarity anomaly threshold, it is judged as abnormal.

[0071] In the embodiment, the structural similarity index corresponding to the current frame when judged as abnormal is not added to the structural similarity sliding window, so as to avoid affecting the mean baseline of the structural similarity sliding window when the method of the application is still running subsequently.

[0072] In the embodiment, when the fusion difference value corresponding to the current frame in step six is judged as abnormal, step seven is entered, and when the structural similarity index corresponding to the current frame in step seven is also judged as abnormal, it is confirmed that the separation is completed. That is, the adaptive quartile distance anomaly detection and the structural similarity anomaly detection are both judged as abnormal, and then it is confirmed that the separation is completed, and the valve closing signal is sent.

[0073] The embodiment of the application improves the accuracy and stability of separation recognition by performing double-scale anomaly detection from the pixel-level and global-level based on adaptive quartile distance anomaly detection and structural similarity anomaly detection. The adaptive quartile distance anomaly detection is at the pixel-level and is used for anomaly detection of the fusion difference value; the structural similarity anomaly detection is at the global-level and is used for anomaly detection of the structural similarity index.

[0074] The three dynamic weight coefficients are obtained by calculating the coefficients of variation of the HSV histogram features, the edge density features and the LBP texture features, and normalizing the three coefficients of variation. The dynamic weight coefficients are used for dynamic weighted fusion of the feature difference value corresponding to the current frame, so as to obtain the fusion difference value corresponding to the current frame, realize cross-scene adaptive multi-modal feature fusion, and improve the scene adaptability.

[0075] The dynamic weighted fusion and the double-scale anomaly detection in the application do not need to preset fixed parameters, but dynamically adjust the parameters according to the actual application scene. This unsupervised learning method not only enhances the generalization ability, but also ensures its effectiveness in various complex environments.

[0076] Although the preferred embodiments of the application have been described, those skilled in the art can make other changes and modifications to the embodiments once they know the basic inventive concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the application.

[0077] Obviously, many modifications and variations of the present application are possible in light of the above teachings. It is, therefore, to be understood that within the scope of the appended claims and their equivalents, the application can be practiced otherwise than as specifically described.

Claims

1. An adaptive liquid separation and recognition method based on unsupervised learning, characterized in that: The following steps are involved: Step 1: Collect continuous video frame data during the liquid dispensing process; Step 2: extracting a region of interest from the continuous video frame data; Step 3: extracting HSV histogram features, edge density features and LBP texture features in the region of interest; Step 4: Based on the extracted features, calculate the feature difference value between the current frame and the previous frame at a fixed time interval to obtain the feature difference value corresponding to the current frame; Step 5: Dynamically weight the feature difference values ​​corresponding to the current frame to obtain the fused difference value corresponding to the current frame; Step 6: Perform adaptive interquartile range anomaly detection on the fusion difference value corresponding to the current frame. If it is judged to be normal, return to step 1; if it is judged to be abnormal, proceed to the next step; In step seven, the structural similarity between the current frame and the previous frame at a fixed time interval is calculated to obtain the structural similarity index corresponding to the current frame. The structural similarity anomaly detection is performed on the structural similarity index corresponding to the current frame. If it is judged to be normal, the process returns to step one. If it is judged to be abnormal, the liquid separation is completed.

2. The method for adaptive liquid separation and recognition based on unsupervised learning according to claim 1, characterized in that: In step 2, the region of interest refers to a region sensitive to liquid separation changes, which is determined by geometric calibration based on a fixed spatial position relationship between the acquisition device and the liquid separation device.

3. The method for adaptive liquid separation and recognition based on unsupervised learning according to claim 1, characterized in that: In step three, the HSV histogram features, the edge density features, and the LBP texture features are stored in a fixed-length feature sliding window, following a first-in-first-out rule.

4. The method for adaptive liquid separation and recognition based on unsupervised learning according to claim 1, characterized in that: In step 4, the feature difference value corresponding to the current frame includes: the HSV histogram feature difference value corresponding to the current frame, the edge density feature difference value corresponding to the current frame, and the LBP texture feature difference value corresponding to the current frame; The HSV histogram feature difference value is calculated using chi-square distance; The edge density feature difference value is calculated using Hausdorff distance; The LBP texture feature difference value is calculated using Euclidean distance.

5. The method for adaptive liquid separation and recognition based on unsupervised learning according to claim 1, characterized in that: In step five, based on the features of all historical frames in the feature sliding window, the coefficients of variation of the HSV histogram feature, the edge density feature, and the LBP texture feature are calculated respectively, and the three coefficients of variation are normalized to obtain three dynamic weight coefficients. Based on the dynamic weight coefficients, the feature difference values ​​corresponding to the current frame are dynamically weighted and fused to obtain the fused difference value corresponding to the current frame.

6. The method for adaptive liquid separation and recognition based on unsupervised learning according to claim 1, characterized in that: In step six, the fused difference values ​​corresponding to all historical frames in the feature sliding window are stored in the difference value sliding window, and the quartiles and interquartile ranges of all fused difference values ​​in the difference value sliding window are calculated to obtain an adaptive interquartile range anomaly threshold. According to the interquartile range anomaly threshold, adaptive interquartile range anomaly detection is performed on the fused difference value corresponding to the current frame.

7. The method for adaptive liquid separation and recognition based on unsupervised learning according to claim 6, characterized in that: In step 6, if the fused difference value corresponding to the current frame is judged to be normal, the fused difference value corresponding to the current frame is added to the difference value sliding window, and the first fused difference value in the difference value sliding window is discarded, and the process returns to step 1; If the fused difference value corresponding to the current frame is judged to be abnormal, the fused difference value corresponding to the current frame is not added to the difference value sliding window, and the next step is performed.

8. The method for adaptive liquid separation and recognition based on unsupervised learning according to claim 1, characterized in that: In step seven, the structural similarity indexes corresponding to all historical frames in the feature sliding window are stored in the structural similarity sliding window, the mean and variance of all structural similarity indices in the structural similarity sliding window are calculated, and the structural similarity anomaly threshold is obtained. The structural similarity anomaly detection is performed on the structural similarity index corresponding to the current frame according to the structural similarity anomaly threshold.

9. The method for adaptive liquid separation and recognition based on unsupervised learning according to claim 8, characterized in that: In step 7, if the structural similarity index corresponding to the current frame is judged to be normal, the structural similarity index corresponding to the current frame is added to the structural similarity sliding window, and the first structural similarity index in the structural similarity sliding window is discarded, and the process returns to step 1; If the structural similarity index corresponding to the current frame is judged to be abnormal, the structural similarity index corresponding to the current frame is not added to the structural similarity sliding window, and the liquid separation is completed.

Images