Liquid volume prediction method and device
Through the combination of deep learning model and Kalman filtering algorithm, the time-consuming and error problems of centrifugal liquid volume measurement in traditional methods are solved, and high-precision liquid volume recognition and prediction are achieved, supporting automation and intelligent control of the centrifugal process.
Patent Information
- Application Number
- CN202411090826.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-09
- Publication Date
- 2024-11-22
AI Technical Summary
The traditional method of manual observation and measuring centrifugal liquid stratification is time-consuming and labor-intensive, and it is easy to introduce artificial errors, making it difficult to achieve accurate liquid volume calculations, limiting the control of centrifugal conditions and the optimization of separation process.
The deep learning model is used to process centrifugal liquid image, and the target area is automatically marked through the position recognition model, and the liquid volume prediction is performed in combination with the Kalman filtering algorithm to improve the stability and accuracy of the recognition results.
It realizes high-precision identification and prediction of centrifugal liquid volume, reduces sensitivity to environmental changes, improves the reliability and data stability of identification results, and supports automated and intelligent centrifugal process control.
Smart Images

Figure CN119006788A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of image processing, and in particular to a liquid volume prediction method and device. Background Art
[0002] Centrifugation is a commonly used separation and purification method in many fields such as biomedicine, chemical analysis, and pharmaceutical engineering. During the centrifugation process, substances of different densities will be separated into layers under the action of centrifugal force, forming a clear interface, which is crucial for subsequent experimental analysis. However, the traditional method of manually observing and measuring the stratification of liquids in centrifuge cups is not only time-consuming and labor-intensive, but also prone to human errors.
[0003] Therefore, developing a technical solution that can accurately calculate the liquid volume of centrifuged liquid is of great significance for accurately controlling centrifugal conditions, optimizing separation processes, and promoting the automation and intelligent development of related fields. Summary of the invention
[0004] The main purpose of the present invention is to provide a liquid volume prediction method and device, aiming to solve the technical problems in the prior art.
[0005] To achieve the above objectives, the present disclosure proposes a liquid volume prediction method, comprising:
[0006] Acquire an image to be identified, where the image to be identified is an image of centrifugal liquid in a centrifugal cup;
[0007] Inputting the image to be identified into a pre-trained position recognition model to obtain a position recognition result corresponding to the image to be identified, wherein the position recognition result includes bounding box information for marking a target area in the image to be identified;
[0008] Calculating the visual measurement volume of the target liquid corresponding to the target area according to the position recognition result;
[0009] The liquid volume corresponding to the target area is predicted according to the visually measured volume.
[0010] Optionally, predicting the liquid volume corresponding to the target area according to the visually measured volume includes:
[0011] determining an initial volume of the target liquid based on the visually measured volume;
[0012] Acquiring pump speed information of a peristaltic pump used to transport the target liquid;
[0013] Based on Kalman filtering, prediction is performed according to the initial volume and the pump speed information to obtain a predicted volume of the target liquid at the next time step;
[0014] determining a target predicted volume based on the predicted volume and a first visual measurement of the target liquid at a next time step;
[0015] The liquid volume corresponding to the target area is determined according to the target predicted volume.
[0016] Optionally, before determining the liquid volume corresponding to the target area according to the target predicted volume, the method further includes:
[0017] Determining whether the target prediction volume satisfies a preset iteration stop condition;
[0018] When the target predicted volume does not satisfy the iteration stop condition, updating the initial volume according to the target predicted volume;
[0019] Performing prediction according to the updated initial volume to update the predicted volume of the target liquid;
[0020] The target predicted volume is updated based on the updated predicted volume and a second visual measurement result of the target liquid at a next time step.
[0021] Optionally, determining a target predicted volume based on the predicted volume and a first visual measurement result of the target liquid at a next time step comprises:
[0022] Determining whether the first visual measurement result meets a preset usability condition;
[0023] In a case where the first visual measurement result does not satisfy the availability condition, determining the predicted volume as the target predicted volume;
[0024] The available condition is that the first visual measurement result contains a first visual measurement volume of the target liquid at the next time step, and the first visual measurement volume is within a preset liquid volume range.
[0025] Optionally, it also includes:
[0026] When the first visual measurement result satisfies the availability condition, calculating a difference value between the predicted volume and the first visual measurement volume;
[0027] Determining whether the predicted volume meets a preset error condition according to the difference value;
[0028] In a case where the predicted volume does not satisfy the error condition, determining the predicted volume as the target predicted volume;
[0029] When the predicted volume satisfies the error condition, data fusion is performed according to the first visual measurement result and the predicted volume to obtain the target predicted volume.
[0030] Optionally, calculating the visual measurement volume of the target liquid corresponding to the target area according to the position recognition result includes:
[0031] Determining a target area position of the target area according to the position recognition result;
[0032] Calculating a target pixel distance from the target area to the inner wall of the centrifuge cup according to the target area position and the area position of the centrifuge cup;
[0033] Obtaining a quadratic fitting function corresponding to the centrifugal cup, wherein the quadratic fitting function is used to characterize a functional mapping relationship between a liquid volume and a pixel distance of the liquid in the centrifugal cup;
[0034] The visual measurement volume of the target liquid is calculated according to the target pixel distance and the quadratic fitting function.
[0035] Optionally, obtaining the quadratic fitting function corresponding to the centrifuge cup includes:
[0036] Acquire a calibration experimental data set, wherein the calibration experimental data set includes data points between multiple groups of liquid volumes and pixel distances corresponding to the centrifugal cup;
[0037] The calibration experimental data set is fitted by the least square method to obtain the quadratic fitting function.
[0038] Optionally, the training method of the location recognition model includes:
[0039] Collecting a centrifugal scene image of the centrifugal liquid, wherein the centrifugal scene image includes a centrifugal liquid image and a background image;
[0040] Performing image preprocessing on the centrifugal scene image to obtain centrifugal scene image data;
[0041] Performing data annotation on the centrifugal scene image data to mark the area position of the target area with a bounding box on the centrifugal liquid image;
[0042] Based on the centrifugal scene image data after data annotation, the initial model is trained for multiple rounds of iterations to obtain the position recognition model.
[0043] Optionally, the location recognition model is a YOLO network model.
[0044] In addition, to achieve the above-mentioned purpose, the present disclosure also provides a liquid volume prediction device, the liquid volume prediction device comprising:
[0045] An acquisition module, used for acquiring an image to be identified, wherein the image to be identified is an image of centrifugal liquid in a centrifugal cup;
[0046] A position recognition module, used for inputting the image to be recognized into a pre-trained position recognition model to obtain a position recognition result corresponding to the image to be recognized, wherein the position recognition result includes bounding box information for marking a target area in the image to be recognized;
[0047] A calculation module, used for calculating the visual measurement volume of the target liquid corresponding to the target area according to the position recognition result;
[0048] A prediction module is used to predict the liquid volume corresponding to the target area according to the visual measurement volume.
[0049] In addition, to achieve the above objectives, the present disclosure also provides a computer-readable storage medium, on which a computer program is stored, and a processor executes the computer program to implement the above method.
[0050] In addition, to achieve the above-mentioned purpose, the present disclosure also provides an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the above-mentioned method.
[0051] In addition, to achieve the above objectives, the present disclosure also provides a computer program product, which implements the above method when executed by a processor.
[0052] Through the above technical solution, the image to be identified is first obtained, and the image to be identified is input into the pre-trained position recognition model to obtain the position recognition result corresponding to the centrifugal image, wherein the position recognition result includes the bounding box information for marking the target area in the image to be identified. Then, according to the position recognition result, the visual measurement volume of the target liquid corresponding to the target area in the centrifugal image is calculated, and then the liquid volume corresponding to the target area is predicted according to the visual measurement volume.
[0053] In this way, by introducing a deep learning model, the key features of the target area can be automatically extracted from complex centrifugal images, achieving high-precision recognition of these features. Compared with traditional image processing algorithms, deep learning models are less sensitive to environmental changes such as light, shadows, and sidewall artifacts, and the recognition results are more stable and reliable. This high-precision recognition capability provides a solid foundation for subsequent applications such as cell separation and volume measurement. At the same time, in order to further improve the stability and reliability of recognition, this technical solution also innovatively combines the visual measurement volume output by the deep learning model to predict the liquid volume, so as to improve the stability and accuracy of the data. This improvement in data processing and fusion capabilities makes this technical solution more reliable in complex and changeable practical application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS
[0054] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying any creative work.
[0055] Figure 1 A schematic diagram of the device structure of the hardware operating environment involved in the embodiment of the present disclosure;
[0056] Figure 2 A schematic flow chart of a liquid volume prediction method according to an embodiment of the present disclosure;
[0057] Figure 3 A schematic diagram of a position recognition result involved in an embodiment of the present disclosure;
[0058] Figure 4 It is a structural block diagram of a liquid volume prediction device involved in the embodiment of the present disclosure.
[0059] The realization of the objectives, functional features and advantages of the present disclosure will be further described in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION
[0060] The following will be combined with the drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.
[0061] Reference Figure 1 , Figure 1The figure is a schematic diagram of the device structure of the hardware operating environment involved in the embodiment of the present disclosure.
[0062] Typically, the device comprises: at least one processor 301, a memory 302, and a liquid volume prediction program stored in the memory 302 and executable on the processor 301, wherein the liquid volume prediction program is configured to implement the steps of the liquid volume prediction method as described above.
[0063] The processor 301 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 301 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 301 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 301 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. The processor 301 may also include an AI (Artificial Intelligence) processor, which is used to process the operation of the liquid volume prediction method, so that the liquid volume prediction method model can be trained and learned autonomously to improve efficiency and accuracy.
[0064] The memory 302 may include one or more storage media, which may be non-transitory. The memory 302 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory storage medium in the memory 302 is used to store at least one instruction, which is used to be executed by the processor 301 to implement the liquid volume prediction method provided in the method embodiment of the present disclosure.
[0065] In some embodiments, the terminal may further optionally include: a communication interface 303 and at least one peripheral device. The processor 301, the memory 302 and the communication interface 303 may be connected via a bus or a signal line. Each peripheral device may be connected to the communication interface 303 via a bus, a signal line or a circuit board. Specifically, the peripheral device includes: at least one of a radio frequency circuit 304, a display screen 305 and a power supply 306.
[0066] The communication interface 303 may be used to connect at least one peripheral device related to I / O (Input / Output) to the processor 301 and the memory 302. In some embodiments, the processor 301, the memory 302, and the communication interface 303 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 301, the memory 302, and the communication interface 303 may be implemented on a separate chip or circuit board, which is not limited in this embodiment.
[0067] The radio frequency circuit 304 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 304 communicates with the communication network and other communication devices through electromagnetic signals. The radio frequency circuit 304 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals into electrical signals. Optionally, the radio frequency circuit 304 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, and the like. The radio frequency circuit 304 can communicate with other terminals through at least one wireless communication protocol. The wireless communication protocol includes, but is not limited to: a metropolitan area network, various generations of mobile communication networks (2G, 3G, 4G and 5G), a wireless local area network and / or a WiFi (Wireless Fidelity) network. In some embodiments, the radio frequency circuit 304 may also include circuits related to NFC (Near Field Communication), which is not limited in the present disclosure.
[0068] The display screen 305 is used to display the UI (User Interface). The UI may include graphics, text, icons, videos, and any combination thereof. When the display screen 305 is a touch display screen, the display screen 305 also has the ability to collect touch signals on the surface or above the surface of the display screen 305. The touch signal can be input to the processor 301 as a control signal for processing. At this time, the display screen 305 can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, the display screen 305 can be one, the front panel of the electronic device; in other embodiments, the display screen 305 can be at least two, respectively arranged on different surfaces of the electronic device or in a folding design; in some embodiments, the display screen 305 can be a flexible display screen, arranged on a curved surface or a folding surface of the electronic device. Even, the display screen 305 can also be set to a non-rectangular irregular shape, that is, a special-shaped screen. The display screen 305 can be made of materials such as LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode, organic light-emitting diode).
[0069] The power supply 306 is used to power various components in the electronic device. The power supply 306 can be AC power, DC power, a disposable battery, or a rechargeable battery. When the power supply 306 includes a rechargeable battery, the rechargeable battery can support wired charging or wireless charging. The rechargeable battery can also be used to support fast charging technology. Those skilled in the art will understand that Figure 1 The structure shown in the figure does not constitute a limitation of the device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.
[0070] In addition, an embodiment of the present disclosure further proposes a storage medium, on which a liquid volume prediction program is stored, and when the liquid volume prediction program is executed by a processor, the steps of the liquid volume prediction method described above are implemented. Therefore, it will not be repeated here. In addition, the description of the beneficial effects of adopting the same method will not be repeated. For technical details not disclosed in the storage medium embodiment involved in the present disclosure, please refer to the description of the method embodiment of the present disclosure. As an example, the program instructions can be deployed to be executed on one device, or on multiple devices located at one location, or on multiple devices distributed at multiple locations and interconnected by a communication network.
[0071] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the above-mentioned program can be stored in a storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the above-mentioned storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.
[0072] In the field of cell centrifugation, especially when using centrifuge cups to separate PBMCs (Peripheral Blood Mononuclear Cells), although visual methods can be used to identify red blood cell stratification and gas-liquid interfaces to achieve a certain degree of precise control, existing technologies mainly rely on traditional image processing algorithms, such as edge detection, etc. These algorithms face significant limitations in practical applications.
[0073] First, they are highly sensitive to the environment. Environmental factors such as changes in light, reflections from the side walls of the centrifuge cup, or shadows may interfere with the recognition results, leading to unstable and inaccurate recognition results. Second, traditional algorithms are often unable to handle complex image features and find it difficult to accurately capture subtle changes in red blood cell stratification and gas-liquid interfaces, thus limiting their application in high-precision separation control. In addition, these algorithms usually require a lot of manual parameter adjustment and optimization, which increases the complexity and cost of use.
[0074] In view of this, the present disclosure proposes a liquid volume prediction method and device. By using deep learning technology and training a complex and powerful neural network model, high-precision recognition of red blood cell stratification and gas-liquid interface is achieved. The Kalman filter algorithm is also introduced to filter the visual recognition results to solve the above technical problems.
[0075] Reference Figure 2 , Figure 2 The figure is a flow chart of a method for predicting liquid volume according to an embodiment of the present disclosure, comprising the following steps:
[0076] Step S11: obtaining an image to be identified, where the image to be identified is an image of centrifugal liquid in a centrifugal cup.
[0077] Step S12: inputting the image to be identified into a pre-trained position recognition model to obtain a position recognition result corresponding to the image to be identified, wherein the position recognition result includes bounding box information for marking a target area in the image to be identified.
[0078] Step S13: Calculate the visual measurement volume of the target liquid corresponding to the target area according to the position recognition result.
[0079] Step S14: predicting the liquid volume corresponding to the target area according to the visually measured volume.
[0080] It should be understood that the target area can be a certain layered area in the centrifuged liquid, such as the red blood cell layered area. It can also be multiple layered areas in the centrifuged liquid, such as the entire centrifuged liquid in the centrifuge cup. The disclosed embodiments do not specifically limit this.
[0081] Through the above technical solution, the image to be identified is first obtained, and the image to be identified is input into the pre-trained position recognition model to obtain the position recognition result corresponding to the centrifugal image, wherein the position recognition result includes the bounding box information for marking the target area in the image to be identified. Then, according to the position recognition result, the visual measurement volume of the target liquid corresponding to the target area in the centrifugal image is calculated, and then the liquid volume corresponding to the target area is predicted according to the visual measurement volume.
[0082] In this way, by introducing a deep learning model, the key features of the target area can be automatically extracted from complex centrifugal images, achieving high-precision recognition of these features. Compared with traditional image processing algorithms, deep learning models are less sensitive to environmental changes such as light, shadows, and sidewall artifacts, and the recognition results are more stable and reliable. This high-precision recognition capability provides a solid foundation for subsequent applications such as cell separation and volume measurement. At the same time, in order to further improve the stability and reliability of recognition, this technical solution also innovatively combines the visual measurement volume output by the deep learning model to predict the liquid volume, so as to improve the stability and accuracy of the data. This improvement in data processing and fusion capabilities makes this technical solution more reliable in complex and changeable practical application scenarios.
[0083] In a possible manner, the location recognition model is a YOLO network model.
[0084] For example, in the present disclosure, the YOLOv8 network model can be used for transfer learning to obtain a location recognition model. Of course, other network models for target detection can also be used for training, and the present disclosure embodiment does not specifically limit this.
[0085] Among the possible approaches, the location recognition model training method includes:
[0086] Collecting a centrifugal scene image of the centrifugal liquid, the centrifugal scene image includes a centrifugal state image and a background image;
[0087] Performing image preprocessing on the centrifugal scene image to obtain centrifugal image data;
[0088] Annotating the centrifugal image data to mark the location of the target area with a bounding box on the centrifugal state image;
[0089] Based on the centrifugal image data after data annotation, the initial model is trained for multiple rounds of iterations to obtain a position recognition model.
[0090] For example, we can first collect video data containing target objects such as gas areas, red blood cell areas, cup wall areas, etc. These data should cover different application scenarios (such as PBMC separation of whole blood, PBMC separation of single blood, etc.) and possible environmental conditions (such as lighting changes, background complexity, etc.) to ensure the diversity and representativeness of the data so that the trained model can be generalized to unseen scenes.
[0091] The video file is then decomposed into individual image frames, and blurred, repeated or irrelevant image frames in the image data are removed to reduce the annotation workload and improve data quality. In this way, the screened image frames are organized into a data set to be annotated to obtain centrifugal image data.
[0092] Next, you can use open source or professional annotation tools to manually annotate the centrifuge cup area in the image, the interface position between the gas and the liquid, and the location of one or more stratified areas in the centrifuged liquid to obtain the annotated data. In addition, you can also add some background images to increase the generalization ability of the model, especially for those areas that may appear in the target image but do not need to be detected.
[0093] It should be understood that the YOLO network model usually requires a specific data format to specify the category, location, and size of the target in the image. Therefore, a conversion tool can be used to convert the labeled data into the format required by the YOLO network model.
[0094] Finally, the model is trained using the labeled data, and the model structure or training parameters are adjusted according to the training results. Multiple rounds of iterative training are performed until satisfactory performance is achieved, thereby obtaining the location recognition model of the present disclosure.
[0095] It is worth noting that background images usually refer to images that do not contain the target or object of interest. These images are used to help the training model distinguish between the foreground (i.e., the part containing the above target object) and the background (i.e., the part not containing the above target object).
[0096] Therefore, when preparing the training data set for the location recognition model, in addition to collecting images containing the above-mentioned recognition objects, attention should also be paid to collecting a sufficient number of background images. These background images should cover as many backgrounds and scenes as possible that may appear in actual applications to ensure that the trained model can perform well in actual applications.
[0097] By introducing background images that do not contain the target object, the training dataset can be enriched, allowing the model to learn more information about background features. The diversity and variability in background images (such as lighting conditions, texture, color, etc.) can help the model learn more robust feature representations, thereby improving its generalization ability on new scenes and unseen data.
[0098] Possible ways to predict the liquid volume corresponding to the target area based on visually measured volume include:
[0099] Determine the initial volume of the target liquid based on visual volume measurements;
[0100] Acquiring pump speed information of a peristaltic pump for conveying a target liquid;
[0101] Based on Kalman filtering, prediction is made based on the initial volume and pump speed information to obtain the predicted volume of the target liquid in the next time step;
[0102] determining a target predicted volume based on the predicted volume and a first visual measurement of the target fluid at a next time step;
[0103] According to the target predicted volume, the liquid volume corresponding to the target area is determined.
[0104] It should be understood that the time step refers to the discretization of the time domain into a series of small time intervals for numerical solution or analysis. In the time integration method or numerical simulation, the time step represents the step size of each solver in time to calculate the system state at the next time point.
[0105] When using the Kalman filter for state estimation, each time step includes a prediction step and an update step. The prediction step predicts the state at the next moment based on the system's state transition model. The update step updates the state estimate based on the actual measurement data and observation model. By continuously iterating these steps, the Kalman filter can continuously improve its estimation accuracy.
[0106] For example, first, the initial volume of the target liquid can be determined based on the visual measurement volume and other factors that may affect the actual volume of the target liquid, and the current pump speed information can be obtained by reading the controller or sensor of the peristaltic pump. In the present disclosure, the Kalman filter uses the linear change of the liquid volume over time to predict the future state, that is, based on the initial volume and pump speed information, the predicted volume of the target liquid in the next time step is predicted, and the state covariance matrix is updated to reflect the uncertainty.
[0107] The Kalman filter also considers measurement noise and process noise, and optimizes the prediction value by weighting and updating the equation, which will not be elaborated in the embodiments of the present disclosure.
[0108] Then, at the next time step, the liquid volume is measured again using the visual measurement method to obtain the first visual measurement result of the target liquid at the next time step. The Kalman filter will combine the predicted volume and the first visual measurement result to calculate a more accurate target predicted volume through its update equation, which is the target predicted volume corresponding to this round of prediction.
[0109] According to the target predicted volume, determining the liquid volume corresponding to the target area can be based on a preset liquid volume value requirement and / or other conditions to determine whether the target predicted volume meets the prediction requirements. If the target predicted volume meets the prediction requirements, the target predicted volume is determined as the liquid volume corresponding to the target area. If the target predicted volume does not meet the prediction requirements, the target predicted volume is determined as the initial volume of the next round of prediction to perform the next round of prediction, and so on.
[0110] It is worth noting that in the scenario of the present disclosure, the peristaltic pump moves at a uniform speed, so the liquid volume theoretically changes linearly, but the present disclosure is not limited to uniform speed scenarios, and is also applicable to non-uniform speed scenarios.
[0111] Specifically, based on the above method, for the current time step, the predicted volume for the current time step can be predicted based on the pump speed information and the target predicted volume obtained in the previous round of prediction. For the first round of prediction, the prediction can be made based on the pump speed information and the initial volume determined based on the visual measurement volume.
[0112] The pseudo code of Kalman filtering in the present disclosure may be as follows:
[0113] State variable definition:
[0114] x t =V t
[0115] Among them, x t is the state variable at time t (i.e., the current time step), V t is the target prediction volume corresponding to the target area at time t (i.e., the current time step).
[0116] Prediction Steps:
[0117] (1) Using pump speed data to predict status
[0118] V t|t-1 =V t-1 +ΔV
[0119] Among them, V t|t-1 V is the predicted volume at time t (i.e., the current time step) obtained by predicting the pump speed information at time t-1 (i.e., the previous time step) and time t, t-1is the target prediction volume of the previous time step (if this is the first round of prediction, then V t-1 is the initial volume), and ΔV is the change in liquid volume calculated based on the pump speed (for example, the volume of liquid pumped out per second).
[0120] (2) Prediction state covariance matrix
[0121] P t|t-1 =P t-1 +Q
[0122] Among them, P t|t-1 is the predicted state covariance matrix at time t-1, P t-1 is the state covariance matrix at time t-1, and Q is the process noise covariance matrix.
[0123] Update steps:
[0124] (1) Calculate the Kalman gain K at time t t :
[0125]
[0126] Where R is the measurement noise covariance.
[0127] (2) Update status:
[0128] V t =V t|t-1 +K t (Z t -V t|t-1 )
[0129] Among them, Z t is the visual measurement volume at time t.
[0130] (3) Update the state covariance matrix P t :
[0131] P t =(1-K t ) t|t-1
[0132] In a possible manner, before determining the liquid volume corresponding to the target area according to the target predicted volume, the method further includes:
[0133] Determine whether the target prediction volume meets a preset iteration stop condition;
[0134] When the target predicted volume does not meet the iteration stop condition, the initial volume is updated according to the target predicted volume;
[0135] Predicting based on the updated initial volume to update the predicted volume of the target liquid;
[0136] The target predicted volume is updated based on the updated predicted volume and a second visual measurement of the target fluid at the next time step.
[0137] For example, the iteration stop condition can be set according to the actual prediction requirements. If the target prediction volume meets the prediction requirements, the iteration can be stopped. If the target prediction volume does not meet the prediction requirements, the next round of prediction will continue until the target prediction volume meets the prediction requirements.
[0138] Specifically, when the target predicted volume does not meet the iteration stop condition, the initial volume is updated according to the target predicted volume as the initial volume for the next round of prediction. At the same time, the current pump speed information of the peristaltic pump is reacquired, and then the prediction is continued according to the reacquired pump speed information and the updated initial volume to update the predicted volume of the target liquid. Then, based on the updated predicted volume and the second visual measurement result of the target liquid at the next time step, the target predicted volume is updated, and so on, which will not be elaborated in detail in the present disclosure.
[0139] In a possible manner, determining a target predicted volume based on the predicted volume and a first visual measurement result of the target liquid at a next time step comprises:
[0140] Determining whether the first visual measurement result meets a preset usability condition;
[0141] In the case that the first visual measurement result does not meet the usable condition, determining the predicted volume as the target predicted volume;
[0142] The applicable condition is that the first visual measurement result contains a first visual measurement volume of the target liquid in the next time step, and the first visual measurement volume is within a preset liquid volume range.
[0143] For example, first determine whether the first visual measurement result contains the first visual measurement volume of the target liquid at the next time step. If the first visual measurement volume is contained, determine whether the first visual measurement volume value is within a preset liquid volume range. If the first visual measurement volume is within the preset liquid volume range, determine that the first visual measurement result meets the availability condition, otherwise, it is considered that the first visual measurement result does not meet the availability condition. If the first visual measurement result does not meet the availability condition, determine the predicted volume as the target predicted volume.
[0144] Specifically, based on the above pseudo code, if the visual measurement result does not meet the availability conditions, the existing prediction volume V is directly used. t|t-1 As the target prediction volume V t .
[0145] Among the possible methods, they also include:
[0146] When the first visual measurement result meets the availability condition, calculating the difference value between the predicted volume and the first visual measurement volume;
[0147] According to the difference value, determine whether the predicted volume meets the preset error condition;
[0148] When the predicted volume does not satisfy the error condition, the predicted volume is determined as the target predicted volume;
[0149] When the predicted volume meets the error condition, data fusion is performed based on the first visual measurement result and the predicted volume to obtain the target predicted volume.
[0150] For example, the difference between the predicted volume and the visually measured volume may be calculated, and then it may be determined whether the difference between the predicted volume and the visually measured volume is within an acceptable range according to a preset error condition.
[0151] If the first visual measurement result meets the availability conditions and the difference value does not meet the error conditions, it means that there is a significant deviation between the predicted volume and the first visual measurement result. At this time, the first visual measurement result can be ignored and the existing predicted volume can be directly used as the target predicted volume to avoid introducing inaccurate data. If the first visual measurement result meets the availability conditions and the difference value meets the error conditions, it means that the difference value is within an acceptable range. The predicted volume and the visual measurement volume can be fused by data fusion to obtain the target predicted volume.
[0152] Specifically, based on the above pseudo code, if the predicted volume V t|t-1 and visual measurement volume Z t If the difference between the two does not meet the error condition, the predicted volume V t|t-1 Directly assigned to the target prediction volume V t If the predicted volume V t|t-1 and visual measurement volume Z t If the difference between them meets the error condition, the update state formula V can be used t =V t|t-1 +K T (Z T -V T|t-1 ), the predicted volume and the visual measurement volume are fused to obtain the target predicted volume.
[0153] Through this technical solution, by improving the Kalman filter, when the visual recognition system fails to give a valid result or the recognition result differs greatly from the Kalman filter prediction value, the prediction value can be automatically used as the final result, thereby avoiding system crash or erroneous operation caused by visual recognition failure. This double-security design makes this technical solution more reliable and efficient in practical applications.
[0154] In a possible manner, calculating the visual measurement volume of the target liquid corresponding to the target area in the centrifugal image according to the position recognition result includes:
[0155] Determine the target area position of the target area according to the position recognition result;
[0156] According to the position of the target area and the area position of the centrifuge cup, a target pixel distance from the target area to the inner wall of the centrifuge cup is calculated;
[0157] Obtaining a quadratic fitting function corresponding to the centrifugal cup, where the quadratic fitting function is used to characterize a functional mapping relationship between a liquid volume of the liquid in the centrifugal cup and a pixel distance;
[0158] The visually measured volume of the target liquid is calculated based on the target pixel distance and the quadratic fitting function.
[0159] For example, Figure 3 A schematic diagram of a position recognition result involved in the embodiment of the present disclosure is shown in FIG. Figure 3 As shown in the figure, the position recognition results respectively draw the gas area and bounding box, the centrifuge cup area and bounding box. Gas 0.99 means that the confidence of the gas area given by the position recognition model is 0.99, and cup 0.95 means that the confidence of the centrifuge cup area given by the position recognition model is 0.95. Based on the position recognition results, the bounding box of the gas area and the bounding box of the centrifuge cup area can be determined, and the rightmost x-direction coordinate of the bounding box of the gas area is marked as x gas , mark the rightmost x-coordinate of the centrifuge cup area as x cup , so the target pixel distance x delta =x cup –x gas .
[0160] It is known that the centrifuge cup can be approximated as a cylinder with an outer radius of r. o , the inner radius is r i , if the actual radial distance from the center of the centrifugal cup to the gas edge r1 = s*x delta , where s is a constant related to the camera magnification. Once the centrifuge cup mold is determined, r o and r i are all constants. The actual liquid volume of the target liquid must be x after simple mathematical derivation. delta A quadratic function of .
[0161] Therefore, for a new target pixel distance x value, the corresponding y value can be calculated using the fitted quadratic function to determine the visually measured volume or volume change of the target liquid.
[0162] Possible ways to obtain the quadratic fitting function corresponding to the centrifuge cup include:
[0163] Acquire a calibration experimental data set, the calibration experimental data set including a plurality of data points between liquid volumes and pixel distances corresponding to the centrifugal cup;
[0164] The calibration experimental data set is fitted by the least squares method to obtain a quadratic fitting function.
[0165] For example, in order to simplify the problem, we can directly establish a mapping relationship between a pixel distance x and a volume V (or a volume change ΔV) through calibration, rather than directly calculating the volume of the annular region.
[0166] First, use the least squares method or other regression methods to fit the quadratic function y=ax 2 +bx+c, where x represents the pixel distance x mentioned above delta , y represents the liquid volume or the change in liquid volume. Then, through the calibration experiment, multiple groups of calibration experiment data sets are obtained, namely, the pixel distance x and the corresponding volume V (or volume change ΔV) data points [x1, V1], [x2, V2], ..., [xN, VN]. The values of a, b, and c can be calculated by least squares fitting and stored in the database for use. Of course, other methods can also be used to obtain the quadratic fitting function corresponding to the centrifugal cup, which is not limited to the embodiments of the present disclosure.
[0167] Reference Figure 4 , Figure 4 This is a structural block diagram of a liquid volume prediction device involved in an embodiment of the present disclosure. Based on the same inventive concept as the above embodiment, the device includes:
[0168] An acquisition module 10 is used to acquire an image to be identified, wherein the image to be identified is an image of centrifugal liquid in a centrifugal cup;
[0169] A position recognition module 20, configured to input the image to be recognized into a pre-trained position recognition model to obtain a position recognition result corresponding to the image to be recognized, wherein the position recognition result includes bounding box information for marking a target area in the image to be recognized;
[0170] A calculation module 30, configured to calculate the visual measurement volume of the target liquid corresponding to the target area according to the position recognition result;
[0171] The prediction module 40 is used to predict the liquid volume corresponding to the target area according to the visually measured volume.
[0172] Optionally, the prediction module 40 is used to:
[0173] determining an initial volume of the target liquid based on the visually measured volume;
[0174] Acquiring pump speed information of a peristaltic pump used to transport the target liquid;
[0175] Based on Kalman filtering, prediction is performed according to the initial volume and the pump speed information to obtain a predicted volume of the target liquid at the next time step;
[0176] determining a target predicted volume based on the predicted volume and a first visual measurement of the target liquid at a next time step;
[0177] The liquid volume corresponding to the target area is determined according to the target predicted volume.
[0178] Optionally, the prediction module 40 is used to:
[0179] Before determining the liquid volume corresponding to the target area according to the target predicted volume, determining whether the target predicted volume satisfies a preset iteration stop condition;
[0180] When the target predicted volume does not satisfy the iteration stop condition, updating the initial volume according to the target predicted volume;
[0181] Performing prediction according to the updated initial volume to update the predicted volume of the target liquid;
[0182] The target predicted volume is updated based on the updated predicted volume and a second visual measurement result of the target liquid at a next time step.
[0183] Optionally, the prediction module 40 is used to:
[0184] Determining whether the first visual measurement result meets a preset usability condition;
[0185] In a case where the first visual measurement result does not satisfy the availability condition, determining the predicted volume as the target predicted volume;
[0186] The available condition is that the first visual measurement result contains a first visual measurement volume of the target liquid at the next time step, and the first visual measurement volume is within a preset liquid volume range.
[0187] Optionally, the prediction module 40 is used to:
[0188] When the first visual measurement result satisfies the availability condition, calculating a difference value between the predicted volume and the first visual measurement volume;
[0189] Determining whether the predicted volume meets a preset error condition according to the difference value;
[0190] In a case where the predicted volume does not satisfy the error condition, determining the predicted volume as the target predicted volume;
[0191] When the predicted volume satisfies the error condition, data fusion is performed according to the first visual measurement result and the predicted volume to obtain the target predicted volume.
[0192] Optionally, the calculation module 30 is used for:
[0193] Determining a target area position of the target area according to the position recognition result;
[0194] Calculating a target pixel distance from the target area to the inner wall of the centrifuge cup according to the target area position and the area position of the centrifuge cup;
[0195] Obtaining a quadratic fitting function corresponding to the centrifugal cup, wherein the quadratic fitting function is used to characterize a functional mapping relationship between a liquid volume and a pixel distance of the liquid in the centrifugal cup;
[0196] The visual measurement volume of the target liquid is calculated according to the target pixel distance and the quadratic fitting function.
[0197] Optionally, the calculation module 30 is used for:
[0198] Acquire a calibration experimental data set, wherein the calibration experimental data set includes data points between multiple groups of liquid volumes and pixel distances corresponding to the centrifugal cup;
[0199] The calibration experimental data set is fitted by the least square method to obtain the quadratic fitting function.
[0200] Optionally, the training method of the location recognition model includes:
[0201] Collecting a centrifugal scene image of the centrifugal liquid, wherein the centrifugal scene image includes a centrifugal state image and a background image;
[0202] Performing image preprocessing on the centrifugal scene image to obtain centrifugal image data;
[0203] Performing data annotation on the centrifugal image data to mark the area position of the target area with a bounding box on the centrifugal state image;
[0204] Based on the centrifugal image data after data annotation, the initial model is trained for multiple rounds of iterations to obtain the position recognition model.
[0205] Optionally, the location recognition model is a YOLO network model.
[0206] It should be noted that, since the steps executed by the device of this embodiment are the same as the steps of the aforementioned method embodiment, its specific implementation method and the technical effects that can be achieved can refer to the aforementioned embodiment and will not be repeated here.
[0207] In addition, in one embodiment, an embodiment of the present disclosure further provides an electronic device, which includes a processor, a memory, and a computer program stored in the memory, and the computer program implements the steps of the method in the aforementioned embodiment when executed by the processor.
[0208] In addition, in one embodiment, an embodiment of the present disclosure further provides a computer storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the method in the aforementioned embodiment are implemented.
[0209] In some embodiments, the computer readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or various devices including one or any combination of the above memories. The computer may be various computing devices including intelligent terminals and servers.
[0210] In some embodiments, executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.
[0211] As an example, executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file that stores other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions).
[0212] By way of example, executable instructions may be deployed to be executed on one computing device, or on multiple computing devices located at one site, or on multiple computing devices distributed across multiple sites and interconnected by a communication network.
[0213] It should be noted that, in this article, the terms "include", "may include" or any other variations thereof are intended to cover non-exclusive inclusions, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.
[0214] The serial numbers of the above-mentioned embodiments of the present disclosure are only for description and do not represent the advantages or disadvantages of the embodiments.
[0215] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present disclosure, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as a read-only memory / random access memory, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a multimedia terminal device (which can be a mobile phone, a computer, a television receiver, or a network device, etc.) to execute the methods described in each embodiment of the present disclosure.
[0216] The above descriptions are only optional embodiments of the present disclosure, and are not intended to limit the patent scope of the present disclosure. All equivalent structural transformations made using the contents of the present disclosure and the drawings under the inventive concept of the present disclosure, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present disclosure.
Claims
1. A method for predicting liquid volume, It is characterized in that include: Acquire an image to be identified, where the image to be identified is an image of centrifugal liquid in a centrifugal cup; Inputting the image to be identified into a pre-trained position recognition model to obtain a position recognition result corresponding to the image to be identified, wherein the position recognition result includes bounding box information for marking a target area in the image to be identified; Calculating the visual measurement volume of the target liquid corresponding to the target area according to the position recognition result; The liquid volume corresponding to the target area is predicted according to the visually measured volume.
2. The method according to claim 1, It is characterized in that The predicting the liquid volume corresponding to the target area according to the visually measured volume comprises: determining an initial volume of the target liquid based on the visually measured volume; Acquiring pump speed information of a peristaltic pump used to transport the target liquid; Based on Kalman filtering, prediction is performed according to the initial volume and the pump speed information to obtain a predicted volume of the target liquid at the next time step; determining a target predicted volume based on the predicted volume and a first visual measurement of the target liquid at a next time step; The liquid volume corresponding to the target area is determined according to the target predicted volume.
3. The method according to claim 2, It is characterized in that Before determining the liquid volume corresponding to the target area according to the target predicted volume, the method further includes: Determining whether the target prediction volume satisfies a preset iteration stop condition; When the target predicted volume does not satisfy the iteration stop condition, updating the initial volume according to the target predicted volume; Performing prediction according to the updated initial volume to update the predicted volume of the target liquid; The target predicted volume is updated based on the updated predicted volume and a second visual measurement result of the target liquid at a next time step.
4. The method according to claim 2, It is characterized in that The step of determining a target predicted volume based on the predicted volume and a first visual measurement result of the target liquid at a next time step comprises: Determining whether the first visual measurement result meets a preset usability condition; In a case where the first visual measurement result does not satisfy the availability condition, determining the predicted volume as the target predicted volume; The available condition is that the first visual measurement result contains a first visual measurement volume of the target liquid at the next time step, and the first visual measurement volume is within a preset liquid volume range.
5. The method according to claim 4, It is characterized in that Also includes: When the first visual measurement result satisfies the availability condition, calculating a difference value between the predicted volume and the first visual measurement volume; Determining whether the predicted volume meets a preset error condition according to the difference value; In a case where the predicted volume does not satisfy the error condition, determining the predicted volume as the target predicted volume; When the predicted volume satisfies the error condition, data fusion is performed according to the first visual measurement result and the predicted volume to obtain the target predicted volume.
6. The method according to claim 1, It is characterized in that The step of calculating the visual measurement volume of the target liquid corresponding to the target area according to the position recognition result includes: Determining a target area position of the target area according to the position recognition result; Calculating a target pixel distance from the target area to the inner wall of the centrifuge cup according to the target area position and the area position of the centrifuge cup; Obtaining a quadratic fitting function corresponding to the centrifugal cup, wherein the quadratic fitting function is used to characterize a functional mapping relationship between a liquid volume and a pixel distance of the liquid in the centrifugal cup; The visual measurement volume of the target liquid is calculated according to the target pixel distance and the quadratic fitting function.
7. The method according to claim 6, It is characterized in that The step of obtaining a quadratic fitting function corresponding to the centrifugal cup includes: Acquire a calibration experimental data set, wherein the calibration experimental data set includes data points between multiple groups of liquid volumes and pixel distances corresponding to the centrifugal cup; The calibration experimental data set is fitted by the least square method to obtain the quadratic fitting function.
8. The method according to claim 1, It is characterized in that The training method of the location recognition model includes: Acquiring a centrifugal scene image of the centrifugal liquid, wherein the centrifugal scene image includes a centrifugal liquid image and a background image; Performing image preprocessing on the centrifugal scene image to obtain centrifugal scene image data; Performing data annotation on the centrifugal scene image data to mark the area position of the target area with a bounding box on the centrifugal liquid image; Based on the centrifugal scene image data after data annotation, the initial model is trained for multiple rounds of iterations to obtain the position recognition model.
9. The method according to any one of claims 1 to 8, It is characterized in that The location recognition model is a YOLO network model.
10. A liquid volume prediction device, It is characterized in that include: An acquisition module, used for acquiring an image to be identified, wherein the image to be identified is an image of centrifugal liquid in a centrifugal cup; A position recognition module, used for inputting the image to be recognized into a pre-trained position recognition model to obtain a position recognition result corresponding to the image to be recognized, wherein the position recognition result includes bounding box information for marking a target area in the image to be recognized; A calculation module, used for calculating the visual measurement volume of the target liquid corresponding to the target area according to the position recognition result; A prediction module is used to predict the liquid volume corresponding to the target area according to the visual measurement volume.
