Intelligent liquid medicine preparation equipment and method for department of hepatobiliary medicine

Through the deep learning technology, keyframe sampling and color feature extraction of drug solution reaction videos is solved, and the problem that the existing technology cannot monitor the drug solution reaction is realized, and the real-time pH value of drug solution is automatically obtained and complex reaction process is realized. Intelligent processing.

CN119015124AInactive Publication Date: 2024-11-26HENAN UNIV OF CHINESE MEDICINE

Patent Information

Application Number
CN202411023971.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-29
Publication Date
2024-11-26
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

The existing automatic dispensing technology cannot effectively monitor the chemical reaction between the drug after preparation and reagents, and rely on predefined models and algorithms to adapt to complex or non-standardized drug solution reaction patterns.

Method used

The reaction state monitoring video of the drug liquid collected by the camera is used to keyframe sampling and color feature extraction. Through the reaction color timing context-related features of the reaction state keyframe time series during the entire reaction process, the medicine is automatically obtained. Real-time pH of liquid.

Benefits of technology

It improves the intelligence and automation of the processing, can adapt to the video data of drug liquid reactions under different types and conditions, automatically understand and capture dynamic changes in complex drug liquid reactions, and ensure good results in different environments.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119015124A_ABST
    Figure CN119015124A_ABST
Patent Text Reader

Abstract

The invention relates to the field of intelligent liquid medicine preparation, and particularly discloses intelligent liquid medicine preparation equipment and method for the department of hepatobiliary internal medicine. Key frame sampling and color feature extraction are carried out on a liquid medicine reaction state monitoring video collected by a camera by adopting a deep learning-based technology; and liquid medicine configuration information of the real-time PH value of the liquid medicine is automatically obtained based on the reaction color time sequence context correlation characteristics of the time sequence of the reaction state key frame in the whole reaction process. Through the mode, dynamic changes in the complex liquid medicine reaction process can be automatically understood and captured, a feature extraction algorithm does not need to be manually designed, the intelligent degree and the automation level of processing are improved, meanwhile, the method can adapt to liquid medicine reaction video data under different types and conditions, and the method is suitable for large-scale popularization and application. Therefore, a good effect can be achieved in different environments.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of intelligent drug solution configuration, and more specifically, to an intelligent drug solution configuration device and method for hepatobiliary internal medicine. Background Art

[0002] Since humans began to be treated by intravenous infusion, the method of mixing and preparing drugs has not undergone fundamental changes. The usual practice is to dissolve the drug in a solvent and then add it to the basic infusion solution, such as adding the therapeutic drug to glucose injection, sodium chloride injection, or glucose sodium chloride injection to prepare the final drug-added infusion.

[0003] With the rapid development of intelligent technology, automatic dispensing technology has emerged. This technology automatically completes the dispensing process through a robotic arm, significantly reducing the workload of clinical dispensing and improving the accuracy of dispensing. However, it is found in actual applications that the existing automatic dispensing technology mainly solves the problem of using machines instead of manual repetitive operations, which only reduces the labor intensity of dispensing personnel to a certain extent, but still cannot monitor the chemical reaction between the drug and the reagent after the drug is prepared. In response to the above technical problems, Chinese patent CN 115015250A discloses an intelligent liquid medicine detection method, which automatically guides the liquid medicine to flow into the reagent container to react with the reagent by detecting the request information triggered by the liquid medicine, and uses the image processing device and the data analysis module to collect and analyze the reaction images respectively, to quickly obtain the reaction information of the liquid medicine, such as the reagent color change curve and the color change value, which helps to accurately determine the analysis results of the liquid medicine, such as pH and osmotic pressure.

[0004] However, the above-mentioned prior art relies on predefined models and algorithms to identify and correct reaction images. This method usually requires pre-defined rules for image processing and analysis, such as color thresholds, shape recognition, etc., which limits the system's ability to adapt to new situations because it may not be able to recognize drug solution reaction patterns that are not included in the predefined model. In addition, the use of fixed algorithms for image recognition and correction, such as edge detection, feature matching, etc., may not work well when encountering complex or non-standardized images.

[0005] Therefore, an optimized intelligent configuration solution for hepatobiliary medicine liquid is desired. Summary of the invention

[0006] In order to solve the above technical problems, the present application is proposed. The embodiment of the present application provides an intelligent configuration device and method for hepatobiliary medicine liquid, which uses deep learning-based technology to perform key frame sampling and color feature extraction on the monitoring video of the reaction state of the medicine liquid collected by the camera, so as to automatically obtain the real-time pH value of the medicine liquid based on the time series of the reaction state key frame and the reaction color temporal context association characteristics in the entire reaction process. The medicine liquid configuration information. In this way, it is possible to automatically understand and capture the dynamic changes in the complex medicine liquid reaction process without manually designing feature extraction algorithms, thereby improving the intelligence and automation level of the processing, and at the same time can adapt to medicine liquid reaction video data under different types and conditions, so as to achieve good results in different environments.

[0007] According to one aspect of the present application, a method for intelligent configuration of hepatobiliary medicine liquid is provided, which comprises:

[0008] In response to receiving a request for preparing a drug solution, controlling the control valves of the first drug solution sampling component and the second drug solution sampling component to open so that the first drug solution material and the second drug solution material flow to the drug solution preparation container, wherein the first drug solution material and the second drug solution material react in the drug solution preparation container;

[0009] Collect monitoring video of drug liquid reaction status through camera;

[0010] The video analysis module processes the reaction status monitoring video to obtain the drug solution configuration information.

[0011] In the above-mentioned intelligent configuration method of hepatobiliary internal medicine, the video analysis module processes the reaction state monitoring video to obtain the medicine configuration information, including: performing key frame sampling on the reaction state monitoring video to obtain a time series of reaction state monitoring key frames; extracting the color features of each reaction state monitoring key frame in the time series of the reaction state monitoring key frames to obtain a time series of reaction state color feature vectors; inputting the time series of the reaction state color feature vectors into a semantic association enhancement module based on feature energy level metric associated radiation to obtain a time series of enhanced reaction state color feature vectors; inputting the time series of enhanced reaction state color feature vectors into a reaction state temporal context encoder to obtain a reaction state color temporal context association feature; based on the reaction state color temporal context association feature, obtaining the medicine configuration information, the medicine configuration information is used to represent the real-time pH value of the medicine.

[0012] In the above-mentioned intelligent configuration method of hepatobiliary internal medicine liquid, the color features of each reaction state monitoring key frame in the time series of the reaction state monitoring key frames are extracted to obtain a time series of reaction state color feature vectors, including: using a color histogram to extract the color features of each reaction state monitoring key frame in the time series of the reaction state monitoring key frames to obtain a time series of the reaction state color feature vectors.

[0013] In the above-mentioned intelligent configuration method for hepatobiliary internal medicine liquid, the time series of the reaction state color feature vector is input into a semantic association enhancement module based on feature energy level measurement association radiation to obtain the time series of enhanced reaction state color feature vectors, including: for the time series of the reaction state color feature vector, the feature energy level coefficients of each reaction state color feature vector in the time series of the reaction state color feature vector are calculated to obtain a sequence of reaction state color feature energy level coefficients; the sequence of reaction state color feature energy level coefficients is arranged as a reaction state color feature energy level coefficient input vector; one-dimensional convolution encoding is performed on the reaction state color feature energy level coefficient input vector to obtain a reaction state color feature energy level radiation association vector; point convolution encoding and probabilistic mapping are performed on the reaction state color feature energy level radiation association vector to obtain a reaction state color feature energy level radiation association weight vector; using the reaction state color feature energy level radiation association weight vector as the weight vector, weighted processing and residual processing are performed on the time series of the reaction state color feature vector to obtain the time series of the enhanced reaction state color feature vector.

[0014] In the above-mentioned intelligent configuration method for hepatobiliary internal medicine liquid, for the time series of the reaction state color feature vector, the characteristic energy level coefficients of each reaction state color feature vector in the time series of the reaction state color feature vector are calculated to obtain a sequence of reaction state color feature energy level coefficients, including: extracting the characteristic maximum value of each position in the reaction state color feature vector to obtain the reaction state color feature maximum value; respectively calculating the mean and variance of the reaction state color feature vector to obtain the reaction state color feature mean and the reaction state color feature variance; adding the reaction state color feature variance and the preset hyperparameter The obtained value is multiplied by four to obtain a first energy level coefficient of the reaction state color feature; the square of the difference between the maximum value of the reaction state color feature and the mean value of the reaction state color feature is calculated to obtain a reaction state color feature difference value; the modulated reaction state color feature variance obtained by multiplying the reaction state color feature variance by two is added to the value obtained by multiplying the preset hyperparameter by two and the reaction state color feature difference value to obtain a second energy level coefficient of the reaction state color feature; the first energy level coefficient of the reaction state color feature is divided by the second energy level coefficient of the reaction state color feature to obtain the reaction state color feature energy level coefficient.

[0015] In the above-mentioned intelligent configuration method for hepatobiliary internal medicine liquid, the reaction state color feature energy level radiation association vector is point convolution encoded and probabilistically mapped to obtain the reaction state color feature energy level radiation association weight vector, including: point convolution encoding the reaction state color feature energy level radiation association vector to obtain the reaction state color feature energy level radiation convolution association vector; inputting the reaction state color feature energy level radiation convolution association vector into a sigmoid function for probabilistic mapping to obtain the reaction state color feature energy level radiation association weight vector.

[0016] In the above-mentioned intelligent configuration method of hepatobiliary internal medicine liquid, the reaction state color feature energy level radiation associated weight vector is used as the weight vector, and the time series of the reaction state color feature vector is weighted and residual processed to obtain the time series of the enhanced reaction state color feature vector, including: using the characteristic value of each position in the reaction state color feature energy level radiation associated weight vector as the weight value, performing vector-by-vector point multiplication on the time series of the reaction state color feature vector and adding the time series of the reaction state color feature vector to obtain the time series of the enhanced reaction state color feature vector.

[0017] In the above-mentioned intelligent configuration method for hepatobiliary internal medicine liquid, the time series of the enhanced reaction state color feature vector is input into the reaction state temporal context encoder to obtain the reaction state color temporal context association feature, including: inputting the time series of the enhanced reaction state color feature vector into a reaction state temporal context encoder based on a Bi-LSTM model to obtain a reaction state color temporal context association feature vector as the reaction state color temporal context association feature.

[0018] In the above-mentioned intelligent configuration method of hepatobiliary internal medicine, the medicine configuration information is obtained based on the reaction state color timing context association feature, and the medicine configuration information is used to represent the real-time pH value of the medicine, including: inputting the reaction state color timing context association feature vector into a decoder-based medicine configuration information generator to obtain the medicine configuration information, and the medicine configuration information is used to represent the real-time pH decoding value of the medicine.

[0019] According to another aspect of the present application, there is also provided an intelligent configuration device for hepatobiliary medicine liquid, wherein the intelligent configuration device for hepatobiliary medicine liquid is used to execute the intelligent configuration method for hepatobiliary medicine liquid as described above.

[0020] Compared with the prior art, the intelligent configuration device and method for hepatobiliary medicine provided by the present application uses deep learning-based technology to perform key frame sampling and color feature extraction on the monitoring video of the reaction state of the medicine collected by the camera, thereby automatically obtaining the real-time pH value of the medicine based on the reaction color time sequence context association features of the time series of the reaction state key frame in the entire reaction process. In this way, the dynamic changes in the complex medicine reaction process can be automatically understood and captured without manually designing feature extraction algorithms, which improves the intelligence and automation level of the processing, and can adapt to different types and conditions of medicine reaction video data, so as to achieve good results in different environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0022] Figure 1 This is a flow chart of the intelligent configuration method for hepatobiliary medicine solution according to an embodiment of the present application.

[0023] Figure 2 The present invention is a flowchart of the video analysis module in the intelligent configuration method of hepatobiliary medicine solution according to the embodiment of the present application processing the reaction state monitoring video to obtain the medicine solution configuration information.

[0024] Figure 3 This is a schematic diagram of the architecture in which the video analysis module in the intelligent configuration method for hepatobiliary internal medicine according to an embodiment of the present application processes the reaction state monitoring video to obtain medicine configuration information.

[0025] Figure 4 The present invention is a flowchart of inputting the time series of the reaction state color feature vector into the semantic association enhancement module based on the feature energy level measurement associated radiation in the intelligent configuration method for hepatobiliary internal medicine liquid according to the embodiment of the present application to obtain the time series of the enhanced reaction state color feature vector. DETAILED DESCRIPTION

[0026] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0027] The way drugs are mixed and prepared has not changed significantly since humans began using intravenous therapy. Traditionally, drugs are prepared by dissolving them in a liquid and then adding them to a basic infusion solution such as dextrose, saline, or a mixture of these to prepare the final infusion drug.

[0028] With the rapid development of intelligent technology, automatic dispensing technology has emerged, which uses a robotic arm to automatically complete the drug mixing process, which not only reduces the burden on medical personnel, but also improves the accuracy of dispensing. However, in actual applications, although automatic dispensing technology reduces manual operations, it still cannot effectively monitor the chemical reactions that may occur between drugs and reagents after mixing.

[0029] To solve the above technical problems, Chinese patent CN 115015250A proposes an intelligent liquid medicine detection method, which automatically guides the liquid medicine to react with the reagent by detecting the request information of the liquid medicine, and uses image processing and data analysis technology to collect and analyze the images of the reaction process. This method can quickly obtain key information of the liquid medicine reaction, such as color change curves and color change values, which is very helpful for accurately judging the properties of the liquid medicine, such as pH value and osmotic pressure.

[0030] However, the above-mentioned prior art still relies on preset models and algorithms to identify and correct reaction images. This requires the system to define image processing rules before operation, such as color boundaries and shape recognition, which limits the system's adaptability to unknown situations because it may not be able to recognize those liquid reaction patterns that are not in the preset model. At the same time, using fixed algorithms for image recognition and correction, such as edge detection and feature matching, may not be effective when facing complex or irregular images.

[0031] Based on this, the present application proposes an intelligent configuration method for hepatobiliary medicine liquid. Figure 1 FIG. 1 is a flow chart of a method for intelligently configuring a hepatobiliary medicine solution according to an embodiment of the present application. Figure 1 As shown, according to the intelligent configuration method of hepatobiliary medicine liquid in accordance with the embodiment of the present application, the method includes: S110, in response to receiving a medicine liquid configuration request, controlling the control valves of the first medicine liquid sampling component and the second medicine liquid sampling component to open so that the first medicine liquid material and the second medicine liquid material flow to the medicine configuration container, wherein the first medicine liquid material and the second medicine liquid material react in the medicine configuration container; S120, collecting a monitoring video of the reaction status of the medicine liquid through a camera; S130, a video analysis module processes the reaction status monitoring video to obtain medicine liquid configuration information.

[0032] Specifically, in step S110, in response to receiving the liquid medicine configuration request, the control valves of the first liquid medicine sampling component and the second liquid medicine sampling component are controlled to open so that the first liquid medicine material and the second liquid medicine material flow to the medicine configuration container, wherein the first liquid medicine material and the second liquid medicine material react in the medicine configuration container. It should be understood that by automatically guiding the liquid medicine to react with the reagent, allowing the medicine to react in the medicine configuration container can ensure that the liquid medicine materials are mixed in a predetermined ratio, which is crucial to the efficacy and safety of the medicine, and also helps to improve the stability and solubility of the medicine.

[0033] Specifically, in step S120, a monitoring video of the reaction state of the drug solution is collected by a camera. Accordingly, the monitoring video of the reaction state of the drug solution refers to the use of a camera to record the reaction of the drug during the preparation process, including visually observable phenomena such as mixing, dissolution, precipitation, and color change. Based on this, in order to be able to more carefully and timely discover and record any changes in the drug solution during the preparation process of the drug solution, so as to help optimize the drug solution configuration information, in the technical solution of the present application, a monitoring video of the reaction state of the drug solution is collected by a camera.

[0034] Specifically, in step S130, the video analysis module processes the reaction state monitoring video to obtain the liquid medicine configuration information, which performs key frame sampling and color feature extraction on the liquid medicine reaction state monitoring video captured by the camera based on deep learning technology, thereby automatically obtaining the liquid medicine configuration information of the real-time pH value of the liquid medicine based on the reaction color time sequence context association features of the time series of the reaction state key frame in the entire reaction process. In this way, the dynamic changes in the complex liquid medicine reaction process can be automatically understood and captured without manually designing feature extraction algorithms, which improves the intelligent degree and automation level of the processing, and can adapt to liquid medicine reaction video data under different types and conditions, so as to achieve good results in different environments.

[0035] Figure 2 The present invention is a flowchart of the video analysis module in the intelligent configuration method of hepatobiliary medicine solution according to the embodiment of the present application processing the reaction state monitoring video to obtain the medicine solution configuration information. Figure 3 The schematic diagram of the structure of the video analysis module in the intelligent configuration method of hepatobiliary medicine liquid according to the embodiment of the present application processes the reaction state monitoring video to obtain the medicine liquid configuration information. Figure 2 and Figure 3As shown, the video analysis module processes the reaction state monitoring video to obtain the liquid medicine configuration information, including: S131, sampling the key frames of the reaction state monitoring video to obtain the time series of the reaction state monitoring key frames; S132, extracting the color features of each reaction state monitoring key frame in the time series of the reaction state monitoring key frames to obtain the time series of the reaction state color feature vector; S133, inputting the time series of the reaction state color feature vector into the semantic association enhancement module based on the feature energy level metric associated radiation to obtain the time series of enhanced reaction state color feature vector; S134, inputting the time series of the enhanced reaction state color feature vector into the reaction state temporal context encoder to obtain the reaction state color temporal context association feature; S135, based on the reaction state color temporal context association feature, obtaining the liquid medicine configuration information, the liquid medicine configuration information is used to represent the real-time pH value of the liquid medicine.

[0036] In step S131, the reaction state monitoring video is key frame sampled to obtain a time sequence of reaction state monitoring key frames. Accordingly, considering that the reaction state monitoring video usually contains a large number of frames, which include key time sequence images related to this task, and also include unimportant background information. Based on this, in order to capture and extract the reaction state key frames related to the liquid medicine configuration information, in the technical solution of the present application, the reaction state monitoring video is key frame sampled to obtain a time sequence of reaction state monitoring key frames. It should be understood that key frames usually represent key moments or significant changes in the video, so by analyzing these reaction state monitoring key frames, the characteristics of the liquid medicine reaction state, such as color change, bubble generation, etc., can be more effectively extracted. In particular, in a specific embodiment of the present application, the reaction state monitoring video can be key frame sampled at a predetermined sampling frequency to obtain a time sequence of the reaction state monitoring key frames.

[0037] In step S132, the color features of each reaction state monitoring key frame in the time sequence of the reaction state monitoring key frame are extracted to obtain the time sequence of the reaction state color feature vector. Specifically, in the embodiment of the present application, the color features of each reaction state monitoring key frame in the time sequence of the reaction state monitoring key frame are extracted to obtain the time sequence of the reaction state color feature vector, including: using a color histogram to extract the color features of each reaction state monitoring key frame in the time sequence of the reaction state monitoring key frame to obtain the time sequence of the reaction state color feature vector. It should be understood that the color of each reaction state monitoring key frame in the time sequence of the reaction state monitoring key frame will change during the reaction process. In order to better focus on and capture the color feature information of each reaction state monitoring key frame, in the technical solution of the present application, a color histogram is used to extract the color features of each reaction state monitoring key frame in the time sequence of the reaction state monitoring key frame to obtain the time sequence of the reaction state color feature vector. It is worth mentioning that the color histogram can intuitively represent the distribution of colors in the image. That is, by extracting the color histogram of each reaction state monitoring key frame, the relative proportion and distribution of various colors in the drug solution reaction process at each moment can be clearly seen, which is very useful for analyzing the color changes of the drug solution reaction state and helps to generate the real-time pH value of the drug solution more accurately.

[0038] In step S133, the time series of the reaction state color feature vector is input into the semantic association enhancement module based on the feature energy level measurement associated radiation to obtain the time series of the enhanced reaction state color feature vector. Accordingly, it is considered that the reaction state color features at different moments in the time series of the reaction state color feature vector have different degrees of semantic information of color feature changes. Therefore, in order to understand and analyze the reaction state color features at different moments in more detail, and further process and optimize the original reaction state color feature vector to obtain a richer and more distinctive feature representation, in the technical solution of the present application, the time series of the reaction state color feature vector is input into the semantic association enhancement module based on the feature energy level measurement associated radiation to obtain the time series of the enhanced reaction state color feature vector.

[0039] Specifically, the semantic association enhancement module based on the characteristic energy level measurement association radiation first calculates the characteristic energy level coefficient of each reaction state color feature vector in the time series of the reaction state color feature vector, and the energy distribution of each reaction state color feature can be quantified, which helps to understand the activity and information content of the color feature in describing the reaction state of the liquid. In particular, the characteristic energy level coefficient is closely related to the statistical characteristics (mean, variance) of the scale and distribution of each reaction state color feature vector, which reflects the density of the color feature semantic information contained in each reaction state color feature vector, and its radiation effect on the semantic information of other feature vectors in the adjacent area. Then, one-dimensional convolution coding is used to process the reaction state color feature energy level coefficient input vector obtained by arranging each characteristic energy level coefficient to capture the local dependence between adjacent elements in the input vector. Then point convolution coding is used to further enhance the expression ability of the spatial features of the vector after one-dimensional convolution coding, and the vector obtained after point convolution is probabilistically mapped using the sigmoid function to generate the reaction state color feature energy level radiation association weight vector. Finally, the reaction state color feature energy level radiation associated weight vector is used as a weight to perform weighted and residual processing on the time series of the reaction state color feature vector, so as to dynamically adjust the importance of different reaction state color features in describing the reaction state of the drug solution, to capture the key reaction state color feature information, and thus obtain the time series of the enhanced reaction state color feature vector.

[0040] Specifically, Figure 4 The flowchart of the method for intelligent configuration of hepatobiliary medicine liquid according to the embodiment of the present application is to input the time series of the reaction state color feature vector into the semantic association enhancement module based on the feature energy level measurement association radiation to obtain the time series of the enhanced reaction state color feature vector. Figure 4As shown, the time series of the reaction state color feature vector is input into the semantic association enhancement module based on the feature energy level measurement association radiation to obtain the time series of the enhanced reaction state color feature vector, including: S1331, for the time series of the reaction state color feature vector, the feature energy level coefficients of each reaction state color feature vector in the time series of the reaction state color feature vector are calculated to obtain a sequence of reaction state color feature energy level coefficients; S1332, the sequence of reaction state color feature energy level coefficients is arranged as a reaction state color feature energy level coefficient input vector; S1333, one-dimensional convolution encoding is performed on the reaction state color feature energy level coefficient input vector to obtain a reaction state color feature energy level radiation association vector; S1334, point convolution encoding and probabilistic mapping are performed on the reaction state color feature energy level radiation association vector to obtain a reaction state color feature energy level radiation association weight vector; S1335, using the reaction state color feature energy level radiation association weight vector as the weight vector, weighting processing and residual processing are performed on the time series of the reaction state color feature vector to obtain the time series of the enhanced reaction state color feature vector.

[0041] More specifically, in an embodiment of the present application, for the time series of the reaction state color feature vectors, the characteristic energy level coefficients of each reaction state color feature vector in the time series of the reaction state color feature vectors are calculated to obtain a sequence of reaction state color feature energy level coefficients, including: extracting the characteristic maximum value of each position in the reaction state color feature vector to obtain the reaction state color feature maximum value; respectively calculating the mean and variance of the reaction state color feature vector to obtain the reaction state color feature mean and the reaction state color feature variance; multiplying the value obtained by adding the reaction state color feature variance and a preset hyperparameter by four to obtain a first energy level coefficient of the reaction state color feature; calculating the square of the difference between the maximum value of the reaction state color feature and the mean value of the reaction state color feature to obtain a reaction state color feature difference value; adding the modulated reaction state color feature variance obtained by multiplying the reaction state color feature variance by two, the value obtained by multiplying the preset hyperparameter by two, and the reaction state color feature difference value to obtain a second energy level coefficient of the reaction state color feature; dividing the first energy level coefficient of the reaction state color feature by the second energy level coefficient of the reaction state color feature to obtain the reaction state color feature energy level coefficient.

[0042] More specifically, in an embodiment of the present application, the reaction state color feature energy level radiation association vector is point convolution encoded and probabilistically mapped to obtain a reaction state color feature energy level radiation association weight vector, including: point convolution encoding the reaction state color feature energy level radiation association vector to obtain a reaction state color feature energy level radiation convolution association vector; inputting the reaction state color feature energy level radiation convolution association vector into a sigmoid function for probabilistic mapping to obtain the reaction state color feature energy level radiation association weight vector.

[0043] More specifically, in an embodiment of the present application, the reaction state color feature energy level radiation associated weight vector is used as the weight vector, and the time series of the reaction state color feature vector is weighted and residual processed to obtain the time series of the enhanced reaction state color feature vector, including: using the characteristic value of each position in the reaction state color feature energy level radiation associated weight vector as the weight value, performing vector-by-vector point multiplication on the time series of the reaction state color feature vector and adding the time series of the reaction state color feature vector to obtain the time series of the enhanced reaction state color feature vector.

[0044] In the embodiment of the present application, specifically, the time series of the reaction state color feature vector is input into the semantic association enhancement module based on the feature energy level measurement associated radiation to obtain the time series of the enhanced reaction state color feature vector, which is used to: use the semantic association enhancement module based on the feature energy level measurement associated radiation to process the time series of the reaction state color feature vector with the following enhancement formula to obtain the time series of the enhanced reaction state color feature vector; wherein the enhancement formula is:

[0045] X={x1,x2,...,x k ,...,x n}

[0046]

[0047] V e =(e1; e2; ...; e k ;...;e n )

[0048] V f =Conv 1×l (V e )

[0049] V c =Sigmoid(Conv 1×1 (V f ))

[0050] X'=X⊙Vc +X

[0051] Where X is the time series of the color feature vector of the reaction state, x k and x n are the kth and nth reaction state color feature vectors in the time series of the reaction state color feature vector, n represents the number of feature vectors in the time series of the reaction state color feature vector, μ i and σ 2 i are the mean and variance of the i-th reaction state color feature vector in the time series of the reaction state color feature vector, max(x i ) is the maximum value of the features at each position in the i-th reaction state color feature vector, ε is the preset hyperparameter, e i is the characteristic energy level coefficient of the color feature vector of the i-th reaction state, e k and e n are the characteristic energy level coefficients of the kth and nth reaction state color feature vectors, V e is the reaction state color feature energy level coefficient input vector, Conv 1×l is a one-dimensional convolutional code, l is the scale of the one-dimensional convolution kernel, V f is the reaction state color characteristic energy level radiation correlation vector, Conv 1×1 (·) is the point convolutional coding, Sigmoid is the Sigmoid function, V c is the energy level radiation associated weight vector of the reaction state color feature, ⊙ represents the point multiplication by position, and X' is the time series of the enhanced reaction state color feature vector.

[0052] In step S134, the time series of the reinforced reaction state color feature vector is input into the reaction state temporal context encoder to obtain the reaction state color temporal context association feature. Specifically, in an embodiment of the present application, the time series of the reinforced reaction state color feature vector is input into the reaction state temporal context encoder to obtain the reaction state color temporal context association feature, including: the time series of the reinforced reaction state color feature vector is input into the reaction state temporal context encoder based on the Bi-LSTM model to obtain the reaction state color temporal context association feature vector as the reaction state color temporal context association feature. It should be understood that the context association relationship between the reinforced reaction state color features is different between different time spans of the time series of the reinforced reaction state color feature vector. In particular, considering that Bi-LSTM is a special type of recurrent neural network, it combines the characteristics of the long short-term memory network (LSTM) and the bidirectional processing capability, and is designed to solve the problems of gradient vanishing and gradient explosion problems encountered by traditional recurrent neural networks when processing long sequence data. Specifically, Bi-LSTM increases the ability of bidirectional data processing, so that the network can learn information from both directions (forward and backward) of the sequence at the same time. That is, since the Bi-LSTM model can process forward (past) and backward (future) information at the same time, it has a good perception ability for understanding and capturing long-term and short-term dependencies of different time lengths in the reaction state of the liquid medicine, which is particularly important for understanding the dynamic changes of the reaction state of the liquid medicine. Therefore, in the technical solution of the present application, the time series of the enhanced reaction state color feature vector is input into the reaction state temporal context encoder based on the Bi-LSTM model to capture and extract the context-related information about the reaction state color change at different time scales, thereby obtaining the reaction state color temporal context-related feature vector.

[0053] In step S135, based on the reaction state color temporal context association feature, the liquid configuration information is obtained, and the liquid configuration information is used to represent the real-time pH value of the liquid. Specifically, in the embodiment of the present application, based on the reaction state color temporal context association feature, the liquid configuration information is obtained, and the liquid configuration information is used to represent the real-time pH value of the liquid, including: inputting the reaction state color temporal context association feature vector into a liquid configuration information generator based on a decoder to obtain the liquid configuration information, and the liquid configuration information is used to represent the real-time pH decoding value of the liquid. That is, the reaction state color temporal context association feature obtained by temporal context association using the time series of the enhanced reaction state color feature vector is classified and processed, so as to automatically obtain the liquid configuration information of the real-time pH value of the liquid. In this way, the dynamic changes in the complex liquid reaction process can be automatically understood and captured, without manually designing the feature extraction algorithm, improving the intelligent degree and automation level of the processing, and can adapt to liquid reaction video data under different types and conditions, so as to achieve good results in different environments.

[0054] It should be understood that here, considering that each of the reinforced reaction state color feature vectors in the time series of the reinforced reaction state color feature vectors respectively represents the reinforced color histogram features of the characteristic energy level measurement associated radiation of each reaction state monitoring key frame in the time series of the reaction state monitoring key frame. When the time series of the reinforced reaction state color feature vectors is input into the reaction state temporal context encoder based on the Bi-LSTM model, the reaction state temporal context encoder based on the Bi-LSTM model uses a forget gate and a memory gate to perform context semantic association encoding on the time series of the reinforced reaction state color feature vectors, but considering that each of the reinforced reaction state color feature vectors in the time series of the reinforced reaction state color feature vectors has an unbalanced correspondence ratio under the fine-grained distribution of the sequence, this will cause the obtained reaction state color temporal context association feature vector to have an extreme decoding offset relative to the decoder, thereby improving the accuracy of the case closing result.

[0055] Based on this, preferably, the reaction state color temporal context associated feature vector is input into a decoder-based liquid configuration information generator to obtain the liquid configuration information, including: cascading the time series of the reinforced reaction state color feature vector into a reinforced reaction state color coding cascade feature vector; calculating the first norm and second norm of the mean vector of the reinforced reaction state color coding cascade feature vector and the reaction state color temporal context associated feature vector; calculating the weighted sum of the inverse of the square root of the second norm and the first norm, and performing a point multiplication with the point addition sum vector of the reinforced reaction state color coding cascade feature vector and the reaction state color temporal context associated feature vector to obtain a first reaction state color temporal context associated correction subvector; and adding the reinforced reaction state color coding cascade feature vector and the Performing a dot product of the dot product vector of the reaction state color timing context association feature vector and the square root of the length of the reaction state color timing context association feature vector to obtain a second reaction state color timing context association correction subvector; calculating a weighted sum of the first reaction state color timing context association correction subvector and the second reaction state color timing context association correction subvector to obtain a reaction state color timing context association correction vector; performing a dot multiplication of the reaction state color timing context association correction vector and the reaction state color timing context association feature vector to obtain a corrected reaction state color timing context association feature vector; inputting the corrected reaction state color timing context association feature vector into the decoder-based liquid configuration information generator to obtain the liquid configuration information.

[0056] The calculation process of the reaction state color temporal context association correction vector is expressed as follows:

[0057]

[0058] Wherein, V1 and V2 are the color coding cascade feature vector of the reinforcement reaction state and the temporal contextual feature vector of the reaction state color, respectively. μ is the mean vector of the reinforcement reaction state color coding cascade feature vector and the reaction state color temporal context association feature vector, ‖·‖1 is the first norm of the feature vector, ‖·‖2 is the second norm of the feature vector, is the position point addition, ⊙ is the position point multiplication, L is the length of the reaction state color temporal contextual association feature vector, L 1 / 2 is the square root of the length of the reaction state color temporal context associated feature vector, and α and β are weighted sum weights as hyperparameters, and V' is the reaction state color temporal context associated correction vector.

[0059] Therefore, based on the mean vector norm between the time series of the reinforced reaction state color feature vector and the reaction state color temporal context associated feature vector, the constrained representation of the structured foreground and background distinction of the supermanifold of the time series of the reinforced reaction state color feature vector and the reaction state color temporal context associated feature vector is used to model the feature-level key correspondence between the feature vectors of the time series of the reinforced reaction state color feature vector and the reaction state color temporal context associated feature vector, and adjust the global correlation relationship between the corresponding features, so as to make positive fine-grained correspondence suggestions through the unbalanced proportion control between the corresponding feature values ​​of the time series of the reinforced reaction state color feature vector and the reaction state color temporal context associated feature vector, so as to avoid extreme decoding imbalance between the feature vectors of the time series of the reinforced reaction state color feature vector and the reaction state color temporal context associated feature vector by focusing on the focus.

[0060] In this way, by correcting the reaction state color timing context association feature vector through the reaction state color timing context association correction vector, the accuracy of the liquid configuration information obtained by inputting the reaction state color timing context association feature vector into the liquid configuration information generator based on the decoder can be improved. In this way, the dynamic changes in the complex liquid reaction process can be automatically understood and captured without manually designing feature extraction algorithms, which improves the intelligence and automation level of processing, and can adapt to liquid reaction video data under different types and conditions, so as to achieve good results in different environments.

[0061] In summary, the intelligent configuration method for hepatobiliary medicine based on the embodiment of the present application is explained, which uses deep learning-based technology to perform key frame sampling and color feature extraction on the monitoring video of the reaction state of the medicine collected by the camera, so as to automatically obtain the real-time pH value of the medicine based on the reaction color time series contextual association features of the reaction state key frame in the entire reaction process. In this way, the dynamic changes in the complex medicine reaction process can be automatically understood and captured without manually designing feature extraction algorithms, which improves the intelligence and automation level of the processing, and can adapt to different types and conditions of medicine reaction video data, so as to achieve good results in different environments.

[0062] Specifically, the present application also provides an intelligent configuration device for hepatobiliary medicine liquid, and the intelligent configuration device for hepatobiliary medicine liquid is used to execute the intelligent configuration method for hepatobiliary medicine liquid as described above.

[0063] The basic principles of the present application are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present application are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present application. In addition, the specific details of the above application are only for the purpose of illustration and ease of understanding, rather than limitation, and the above details do not limit the present application to the use of the above specific details to be implemented.

[0064] In this application, words such as "including", "comprising", "having", etc. are open words, meaning "including but not limited to", and can be used interchangeably therewith. The words "or" and "and" used herein mean the words "and / or" and can be used interchangeably therewith, unless the context clearly indicates otherwise. The word "such as" used herein means the phrase "such as but not limited to", and can be used interchangeably therewith.

[0065] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present application. Therefore, the present application is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features of the present application.

[0066] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present application to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. A method for intelligently configuring a hepatobiliary medicine solution, characterized in that: include: In response to receiving a request for preparing a drug solution, controlling the control valves of the first drug solution sampling component and the second drug solution sampling component to open so that the first drug solution material and the second drug solution material flow to the drug solution preparation container, wherein the first drug solution material and the second drug solution material react in the drug solution preparation container; Collect monitoring video of drug liquid reaction status through camera; The video analysis module processes the reaction status monitoring video to obtain the drug solution configuration information.

2. The intelligent configuration method for hepatobiliary medicine according to claim 1, characterized in that: The video analysis module processes the reaction status monitoring video to obtain the drug solution configuration information, including: Performing key frame sampling on the reaction state monitoring video to obtain a time sequence of reaction state monitoring key frames; Extracting the color features of each reaction state monitoring key frame in the time series of the reaction state monitoring key frames to obtain a time series of reaction state color feature vectors; Inputting the time series of the reaction state color feature vector into a semantic association enhancement module based on feature energy level measurement association radiation to obtain the time series of enhanced reaction state color feature vector; Inputting the time series of the reinforcement reaction state color feature vector into a reaction state temporal context encoder to obtain a reaction state color temporal context association feature; Based on the reaction state color temporal context association feature, the liquid medicine configuration information is obtained, and the liquid medicine configuration information is used to represent the real-time pH value of the liquid medicine.

3. The intelligent configuration method for hepatobiliary medicine according to claim 2, characterized in that: Extracting the color features of each reaction state monitoring key frame in the time series of the reaction state monitoring key frames to obtain a time series of reaction state color feature vectors, including: using a color histogram to extract the color features of each reaction state monitoring key frame in the time series of the reaction state monitoring key frames to obtain a time series of the reaction state color feature vectors.

4. The intelligent configuration method for hepatobiliary medicine according to claim 3, characterized in that: Inputting the time series of the reaction state color feature vector into a semantic association enhancement module based on feature energy level measurement association radiation to obtain the time series of enhanced reaction state color feature vector, including: For the time series of the reaction state color feature vectors, calculating the characteristic energy level coefficient of each reaction state color feature vector in the time series of the reaction state color feature vectors to obtain a sequence of reaction state color feature energy level coefficients; Arranging the sequence of the reaction state color characteristic energy level coefficients into a reaction state color characteristic energy level coefficient input vector; Performing one-dimensional convolution encoding on the reaction state color feature energy level coefficient input vector to obtain a reaction state color feature energy level radiation correlation vector; Performing point convolution encoding and probabilistic mapping on the reaction state color feature energy level radiation association vector to obtain a reaction state color feature energy level radiation association weight vector; The reaction state color feature energy level radiation associated weight vector is used as a weight vector, and weighted processing and residual processing are performed on the time series of the reaction state color feature vector to obtain the time series of the enhanced reaction state color feature vector.

5. The intelligent configuration method for hepatobiliary medicine according to claim 4, characterized in that: For the time series of the reaction state color feature vectors, calculating the characteristic energy level coefficient of each reaction state color feature vector in the time series of the reaction state color feature vectors to obtain a sequence of reaction state color feature energy level coefficients, including: Extracting the maximum value of the feature at each position in the reaction state color feature vector to obtain the maximum value of the reaction state color feature; Respectively calculating the mean and variance of the reaction state color feature vector to obtain a reaction state color feature mean and a reaction state color feature variance; The value obtained by adding the reaction state color feature variance and the preset hyperparameter is multiplied by four to obtain the first energy level coefficient of the reaction state color feature; Calculating the square of the difference between the maximum value of the reaction state color feature and the mean value of the reaction state color feature to obtain a reaction state color feature difference value; The reaction state color feature variance is multiplied by two to obtain a modulated reaction state color feature variance, the preset super parameter is multiplied by two to obtain a value, and the reaction state color feature difference value is added to obtain a reaction state color feature second energy level coefficient; The reaction state color characteristic first energy level coefficient is divided by the reaction state color characteristic second energy level coefficient to obtain the reaction state color characteristic energy level coefficient.

6. The intelligent configuration method for hepatobiliary medicine according to claim 5, characterized in that: Performing point convolution encoding and probabilistic mapping on the reaction state color feature energy level radiation association vector to obtain a reaction state color feature energy level radiation association weight vector, including: Performing point convolution encoding on the reaction state color feature energy level radiation association vector to obtain a reaction state color feature energy level radiation convolution association vector; The reaction state color feature energy level radiation convolution association vector is input into a sigmoid function for probabilistic mapping to obtain the reaction state color feature energy level radiation association weight vector.

7. The intelligent configuration method for hepatobiliary medicine according to claim 6, characterized in that: The reaction state color feature energy level radiation associated weight vector is used as the weight vector, and the time series of the reaction state color feature vector is weighted and residual processed to obtain the time series of the enhanced reaction state color feature vector, including: using the characteristic value of each position in the reaction state color feature energy level radiation associated weight vector as the weight value, performing vector-by-vector point multiplication on the time series of the reaction state color feature vector and adding the time series of the reaction state color feature vector to obtain the time series of the enhanced reaction state color feature vector.

8. The intelligent configuration method for hepatobiliary medicine according to claim 7, characterized in that: Inputting the time series of the reinforced reaction state color feature vector into a reaction state temporal context encoder to obtain a reaction state color temporal context associated feature, including: inputting the time series of the reinforced reaction state color feature vector into a reaction state temporal context encoder based on a Bi-LSTM model to obtain a reaction state color temporal context associated feature vector as the reaction state color temporal context associated feature.

9. The intelligent configuration method for hepatobiliary medicine according to claim 8, characterized in that: Based on the reaction state color temporal context association feature, the medicine liquid configuration information is obtained, and the medicine liquid configuration information is used to represent the real-time pH value of the medicine liquid, including: inputting the reaction state color temporal context association feature vector into a decoder-based medicine liquid configuration information generator to obtain the medicine liquid configuration information, and the medicine liquid configuration information is used to represent the real-time pH decoding value of the medicine liquid.

10. An intelligent device for dispensing liquid medicine for hepatobiliary medicine, characterized in that: The intelligent configuration device for hepatobiliary medicine liquid is used to execute the intelligent configuration method for hepatobiliary medicine liquid described in claims 1-9.

Citation Information

Patent Citations

  • Intelligent detection method and system for liquid medicine

    CN115015250A

Cited By

  • Shared split type oxygen supply terminal machine and control method thereof

    CN118903625A