Monitoring method, device and system for traditional Chinese medicine extraction process and storage medium

Abstract

The invention provides a monitoring method, device and system for a traditional Chinese medicine extraction process and a storage medium, and the method comprises the steps: selecting sample data from video data based on an active learning sampling strategy of space-time uncertainty; a deep neural network model combining a convolutional neural network model and a Transform model is constructed; and constructing a knowledge distillation model, extracting knowledge in the trained deep neural network model by using the knowledge distillation model, and transferring the knowledge in the Transform model to a lightweight student model to obtain a lightweight video monitoring model. According to the monitoring method, device and system for the traditional Chinese medicine extraction process and the storage medium, the method, device and system can be better suitable for the resource-limited industrial environment, efficient and accurate production state monitoring can be achieved in the resource-limited industrial environment, and support is provided for state recognition and decision making in the production process.

Description

Technical Field

[0001] The present invention belongs to the field of machine vision technology, and in particular relates to a monitoring method, device, system and storage medium for a traditional Chinese medicine extraction process. Background Art

[0002] Traditional Chinese medicine extraction is the primary link in the production stage of traditional Chinese medicine. It is a key step that affects the transfer of active ingredients from medicinal materials to traditional Chinese medicine extracts. It is one of the core processes that determine the clinical efficacy and quality consistency of the final drug. However, the extraction process of traditional Chinese medicine is a dynamic process. Complex process parameters and multi-dimensional influencing factors cause unstable quality control. On the other hand, the lack of a mature online detection system for the extraction process and a process data analysis system has led to difficulties in controlling the extraction process of traditional Chinese medicine. In summary, it is necessary to develop process analysis technology, clarify the laws of mass transfer, construct a process quality prediction model, establish a quality control system for the extraction process of traditional Chinese medicine, and break through the bottleneck in the development of precise extraction of traditional Chinese medicine.

[0003] The quality of Chinese medicine decoction is directly related to the efficacy. Li Shizhen, a medical scientist in the Ming Dynasty, said: "When taking decoctions, even if the ingredients are specialized and the preparation is in accordance with the law, if the person who decocts the medicine is reckless and hasty, the water and fire are not good, and the temperature is not appropriate, then the medicine will be ineffective." It can be seen that the control of temperature plays an important role in the efficacy of decoctions. Temperature can be divided into gentle fire and strong fire, which refer to the intensity of the boiling of the liquid. The boiling state of the liquid is divided into slight boiling, medium boiling and high boiling, and slight boiling and high boiling represent gentle and strong fire decoction respectively. However, the current control of temperature in Chinese medicine decoction still remains at the simple description of the boiling state of the liquid, and there is no clear judgment indicator between gentle fire and strong fire. Modern Chinese medicine extraction uses stainless steel extraction tanks to decoct medicine. Since it cannot directly reflect the boiling state of the liquid, improper control of temperature may affect important indicators such as the concentration of the active substance and the rate of the paste. Therefore, it is urgent to standardize the temperature with more scientific definitions and clear indicator parameters, in order to provide a basis for the digitization of the boiling state and the precise control of temperature during the extraction process. Summary of the invention

[0004] In view of this, the present invention aims to propose a monitoring method, device, system and storage medium for the traditional Chinese medicine extraction process, so as to solve the problem that the accuracy of identifying the boiling state in the existing traditional Chinese medicine decoction extraction process is poor, which is not conducive to the precise control of the heat and thus affects the extraction effect of the traditional Chinese medicine.

[0005] To achieve the above object, the technical solution of the present invention is achieved as follows:

[0006] In a first aspect, an embodiment of the present invention provides a monitoring method for a traditional Chinese medicine extraction process, comprising:

[0007] Acquire the collected liquid surface boiling video, and pre-process the liquid surface boiling video to obtain video data;

[0008] Selecting sample data from the video data based on an active learning sampling strategy of spatial-temporal uncertainty;

[0009] Constructing a deep neural network model that combines a convolutional neural network model and a Transformer model, and training the deep neural network model using the sample data to obtain a trained deep neural network model;

[0010] Constructing a knowledge distillation model, and using the knowledge distillation model to extract knowledge from the trained deep neural network model, transferring the knowledge in the Transformer model to the lightweight student model, and obtaining a lightweight video monitoring model;

[0011] The lightweight video monitoring model is used to monitor and identify the liquid level during the extraction process of traditional Chinese medicine.

[0012] Further, the acquiring of the collected video of the liquid surface boiling and preprocessing of the video of the liquid surface boiling to obtain video data include:

[0013] Get the collected video of liquid surface boiling;

[0014] The liquid surface boiling video is subjected to frame extraction, image enhancement, size adjustment, and normalization processing in sequence to obtain video data.

[0015] Furthermore, the active learning sampling strategy based on space-time uncertainty selects sample data from the video data, including:

[0016] evaluating spatial uncertainty by calculating entropy of classification probabilities of video frames in the video data;

[0017] evaluating temporal uncertainty by calculating an average entropy of all frames of a video sequence in the video data;

[0018] Taking a weighted sum of the spatial uncertainty and the temporal uncertainty to obtain a comprehensive uncertainty score of the video data;

[0019] According to the comprehensive uncertainty score, active learning sampling is achieved by presetting performance thresholds and uncertainty thresholds to obtain sample data; wherein, when the model performance reaches the preset performance threshold or the comprehensive uncertainty score is lower than the preset uncertainty threshold, the learning process is stopped.

[0020] Furthermore, the construction of a deep neural network model combining a convolutional neural network model and a Transformer model, and training the deep neural network model using the sample data to obtain a trained deep neural network model includes:

[0021] Construct a deep neural network model that combines a convolutional neural network model and a Transformer model; wherein the convolutional neural network model uses a MobileNet model to extract features from video frames to capture local features in the image; and the Transformer model uses a self-attention mechanism to capture temporal dependencies and changes between frames;

[0022] The deep neural network model is trained using the sample data, and the classification accuracy of the deep neural network model is optimized through a back propagation algorithm and a minimization loss function to obtain a deep neural network model after training.

[0023] Furthermore, the knowledge distillation model is constructed, and the knowledge in the deep neural network model after the training is extracted by using the knowledge distillation model, and the knowledge in the Transformer model is transferred to the lightweight student model to obtain a lightweight video monitoring model, including:

[0024] Constructing a teacher model, the teacher model includes a feature extraction layer, a temporal modeling layer, and an output layer; wherein the feature extraction layer uses a convolutional neural network to extract features from video frames to capture spatial features; the temporal modeling layer uses a Transformer model to capture temporal dependencies between frames through a self-attention mechanism; the output layer outputs prediction results by mapping temporal features to classification space;

[0025] Construct a student model, which includes a feature extraction layer, a temporal modeling layer, a TCN layer, a GRU layer, and an output layer; wherein the feature extraction layer uses a lightweight convolutional neural network to extract frame-level features; the temporal modeling layer combines the TCN layer and the GRU layer to capture the temporal dependencies in the video sequence; the TCN layer uses a temporal convolutional network to extract local temporal features in the temporal dimension through one-dimensional convolution to improve the parallel processing capability of the model; the GRU layer uses a gated recurrent unit to capture the long-range dependencies of the sequence by using a recurrent neural network to further process the temporal features; the output layer outputs the prediction results by mapping the temporal features to the classification space through a fully connected layer;

[0026] Based on the hard target loss function and the soft target loss function, the learning effect of the student model is optimized by adjusting the distillation temperature parameter and the weight parameter to improve the accuracy of the student model and reduce the computational complexity of the student model.

[0027] Furthermore, based on the hard target loss function and the soft target loss function, the learning effect of the student model is optimized by adjusting the distillation temperature parameter and the weight parameter to improve the accuracy of the student model and reduce the computational complexity of the student model, including:

[0028] The hard target loss function uses the cross entropy loss function, and the formula is:

[0029]

[0030] Among them, y is the true label, is the class probability distribution predicted by the model;

[0031] The soft target loss function uses the Kullback-Leibler divergence to compare the softened outputs of the teacher model and the student model. The formula is:

[0032] FLoss soft =T 2 ·KLD iv (logSoftmax(S / T),Softmax(T / T))

[0033] Where T is the temperature parameter, which is used to soften the output probability distribution;

[0034] The hard target loss function and the soft target loss function are weighted and summed to obtain the total loss function, which is:

[0035] Loss = α × Loss soft +(1-α)×Loss hard

[0036] Among them, α is the weight parameter;

[0037] Based on the total loss function, the learning effect of the student model is optimized by adjusting the distillation temperature parameter and the weight parameter α to improve the accuracy of the student model and reduce the computational complexity of the student model.

[0038] In a second aspect, an embodiment of the present invention further provides a monitoring device, including:

[0039] An acquisition module, used for acquiring the collected liquid surface boiling video, and preprocessing the liquid surface boiling video to obtain video data;

[0040] A screening module, used for selecting sample data from the video data based on an active learning sampling strategy of spatial-temporal uncertainty;

[0041] A construction module is used to construct a deep neural network model that combines a convolutional neural network model and a Transformer model, and train the deep neural network model using the sample data to obtain a deep neural network model after training;

[0042] A distillation module is used to construct a knowledge distillation model, and use the knowledge distillation model to extract knowledge from the deep neural network model after training, transfer the knowledge in the Transformer model to the lightweight student model, and obtain a lightweight video monitoring model;

[0043] A deployment module is used to monitor and identify the liquid level during the extraction process of traditional Chinese medicine using the lightweight video monitoring model.

[0044] In a third aspect, an embodiment of the present invention further provides a monitoring system, including:

[0045] one or more processors;

[0046] A storage device for storing one or more programs;

[0047] A camera, used to collect images;

[0048] A display, used for displaying image recognition results;

[0049] When the one or more programs are executed by the one or more processors, the one or more processors implement the monitoring method for the traditional Chinese medicine extraction process provided in the above embodiment.

[0050] In a fourth aspect, an embodiment of the present invention further provides a storage medium comprising computer executable instructions, which, when executed by a computer processor, are used to execute a monitoring method for a traditional Chinese medicine extraction process as provided in the above embodiment.

[0051] Compared with the prior art, the monitoring method, device, system and storage medium for the traditional Chinese medicine extraction process described in the present invention have the following advantages:

[0052] The monitoring method, device, system and storage medium for the traditional Chinese medicine extraction process described in the present invention can be better adapted to resource-constrained industrial environments. By combining the active learning method and deep learning technology of spatial-temporal uncertainty sampling, it can selectively participate in the annotation of high-value samples, thereby significantly improving the accuracy of the model while reducing the number of training samples. In addition, by processing video data through a hybrid architecture of convolutional neural networks and Transformers, and using knowledge distillation technology to lightweight the model, efficient and accurate production status monitoring can be achieved in resource-constrained industrial environments, providing support for status recognition and decision-making in the production process. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The accompanying drawings constituting a part of the present invention are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the accompanying drawings:

[0054] Figure 1 A schematic flow chart of a monitoring method for a traditional Chinese medicine extraction process according to Embodiment 1 of the present invention;

[0055] Figure 2 This is a flow chart of an active learning sampling strategy in a monitoring method for a traditional Chinese medicine extraction process according to Embodiment 1 of the present invention;

[0056] Figure 3 This is a schematic diagram of the structure of a deep neural network model in a monitoring method for a traditional Chinese medicine extraction process according to Embodiment 1 of the present invention;

[0057] Figure 4 This is a schematic diagram of the structure of a monitoring device according to Embodiment 2 of the present invention;

[0058] Figure 5 A schematic diagram of the structure of a system provided in Embodiment 3 of the present invention. DETAILED DESCRIPTION

[0059] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0060] Embodiment 1

[0061] Figure 1 The flowchart of a monitoring method for a Chinese medicine extraction process according to the first embodiment of the present invention is shown in FIG. This monitoring method can be used to identify boiling during the extraction process of Chinese medicine products and can be operated efficiently in resource-constrained industrial environments. Figure 1 , this monitoring method specifically includes the following steps:

[0062] Step 101: Acquire the collected video of liquid surface boiling, and pre-process the video of liquid surface boiling to obtain video data.

[0063] In the field of machine vision, video data provides significant advantages for classification and target recognition tasks. Videos provide continuous time information, which enhances the model's ability to understand behaviors and events in complex scenes. They can also capture the movement trajectory and speed of objects, helping to analyze movement trends. In addition, factors such as noise, lighting changes, and obstacles may affect the target recognition accuracy of a single image. However, video data mitigates these effects by integrating information from multiple time points, improving the reliability and accuracy of the model. Therefore, this embodiment uses the boiling video of the liquid surface as the data source to train the model.

[0064] In the actual application process, an industrial camera can be installed above the important extraction equipment to collect high-definition video data of the boiling liquid surface in real time at a rate of 30 frames per second. Through the TCP / IP protocol, the camera transmits the video data through Ethernet connection and transmits the collected high-definition video stream to the industrial computer in the central control room for subsequent video processing to improve the video quality to meet the needs of model training.

[0065] Specifically, a large number of collected liquid surface boiling videos can be first obtained, and then the liquid surface boiling videos are subjected to frame extraction, image enhancement, resizing, and normalization processing in sequence to obtain video data. Exemplarily, frame extraction: extract continuous image frames from a video stream. Image enhancement: adjust brightness and contrast to reduce the impact of lighting changes. Resizing: uniformly adjust the image to the size required for model input (for example, 128×72 pixels). Normalization: normalize pixel values ​​to meet the input requirements of the model.

[0066] Step 102: Select sample data from the video data based on an active learning sampling strategy with spatial-temporal uncertainty.

[0067] In real industrial scenarios, data often has the characteristics of high sampling frequency but low information density. Data is unevenly distributed in the sample space and lacks diversity. Model training on all available data may lead to confusion and performance degradation. Therefore, selecting high-value information from massive video data is crucial to improving model accuracy. Active learning actively screens or generates the most valuable samples through appropriate strategies, and then lets experts in the field annotate these labeled samples and add them to the training data set. Finally, the model is trained on the updated training set. Therefore, introducing active learning strategies in deep learning can complete model training with very few samples.

[0068] Different from deep learning, which tries every possible means to design a model with high-performance feature extraction capabilities by using manual or automatic methods. Active learning starts with a data set, mainly by designing sophisticated query rules to select the best samples from unlabeled data sets and query their labels, trying to reduce the labeling cost. Therefore, the design of query rules is crucial to the performance of active learning. In order to meet the requirements of video tasks, this embodiment introduces a spatiotemporal uncertainty sampling (STUS) strategy based on traditional uncertainty sampling. This strategy is more suitable for processing time series data such as videos. The sample data that is most conducive to model improvement is selected by utilizing the space-time uncertainty sampling strategy.

[0069] Specifically, Figure 2 This is a flow chart of an active learning sampling strategy in a monitoring method for a traditional Chinese medicine extraction process according to Embodiment 1 of the present invention. Figure 2 , active learning sampling can be performed according to the following steps:

[0070] Step 1021 : Evaluate spatial uncertainty by calculating the entropy of classification probabilities of video frames in the video data.

[0071] For example, the spatial uncertainty can be evaluated by first calculating the entropy of the classification probability of the video frame. For a video frame, if its predicted probability for category C is p i , then the spatial uncertainty U of the frame S The calculation formula is:

[0072] Step 1022: Evaluate temporal uncertainty by calculating the average entropy of all frames in the video sequence in the video data.

[0073] For example, assuming that a video contains T frames, the temporal uncertainty U of the video is t The calculation formula is:

[0074] Step 1023: perform weighted summation of the spatial uncertainty and the temporal uncertainty to obtain a comprehensive uncertainty score of the video data.

[0075] For example, the comprehensive uncertainty score U of the video data is c It is the weighted sum of spatial uncertainty and temporal uncertainty, and is calculated as:

[0076] U c (video) = α × mean (U s )+β×U t

[0077] Among them, α and β are weight parameters used to balance the contribution of spatial uncertainty and temporal uncertainty.

[0078] Step 1024: According to the comprehensive uncertainty score, active learning sampling is achieved by presetting the performance threshold and the uncertainty threshold to obtain sample data; wherein, when the model performance reaches the preset performance threshold or the comprehensive uncertainty score is lower than the preset uncertainty threshold, the learning process is stopped. In actual application, the system automatically determines the termination condition of the learning process by presetting the performance threshold and the uncertainty threshold. When the model performance reaches the performance threshold or the uncertainty is lower than the uncertainty threshold, the learning process is stopped.

[0079] In actual application, the hyperparameter configuration of the active learning process can be set as follows: α and β are both 0.5. The number of samples for each iteration query is set to 10, and the sample pool size is 1600. The confidence threshold is set to 0.9, and the maximum number of iterations is 100. Those skilled in the art can use the above active learning sampling strategy to select high-value video sample data that is conducive to model improvement from a large amount of video data, which is conducive to improving the training effect of the deep neural network model.

[0080] Step 103: construct a deep neural network model that combines a convolutional neural network model and a Transformer model, and use the sample data to train the deep neural network model to obtain a trained deep neural network model.

[0081] Specifically, Figure 3 This is a schematic diagram of the structure of a deep neural network model in a monitoring method for a traditional Chinese medicine extraction process according to Embodiment 1 of the present invention. Figure 3 First, a deep neural network model combining a convolutional neural network model and a Transformer model can be constructed; wherein the convolutional neural network model uses a MobileNet model to extract features from video frames to capture local features in the image; the Transformer model uses a self-attention mechanism to capture the temporal dependencies and changes between frames; secondly, the deep neural network model can be trained using the sample data, and the classification accuracy of the deep neural network model can be optimized through a back propagation algorithm and minimizing the loss function to obtain a deep neural network model after training.

[0082] By adopting a hybrid architecture combining convolutional neural networks (CNN) and Transformer, the spatiotemporal features in video data can be better processed. The CNN part uses the MobileNet model to extract features from video frames and effectively capture local features in images, such as shape, texture, and color. The Transformer part uses the self-attention mechanism to capture the temporal dependencies and changes between frames.

[0083] In actual application, assuming that the input frame is X, the feature extracted by the convolutional neural network is expressed as: F = MobileNet(X). For the input feature F, the calculation formula of the Transformer self-attention mechanism is: Among them, Q, K, and V are the matrices of query, key, and value respectively, and d k is the dimension of the key. Through this mechanism, the model can effectively classify video sequences.

[0084] Step 104: construct a knowledge distillation model, and use the knowledge distillation model to extract knowledge from the deep neural network model after training, transfer the knowledge in the Transformer model to the lightweight student model, and obtain a lightweight video monitoring model.

[0085] In order to deploy deep learning models in resource-constrained industrial environments, this embodiment also uses knowledge distillation technology to transfer the knowledge in the complex Transformer model to a lightweight student model. The student model is combined with a temporal convolutional network (TCN) and a gated recurrent unit (GRU) to effectively process video sequence data.

[0086] Specifically, a teacher model is first constructed, which includes a feature extraction layer, a temporal modeling layer, and an output layer; wherein the feature extraction layer uses a convolutional neural network to extract features from video frames to capture spatial features; the temporal modeling layer uses a Transformer model to capture the temporal dependencies between frames through a self-attention mechanism; the output layer outputs prediction results by mapping temporal features to classification space.

[0087] Secondly, a student model is constructed, which includes a feature extraction layer, a temporal modeling layer, a TCN layer, a GRU layer, and an output layer; wherein the feature extraction layer uses a lightweight convolutional neural network to extract frame-level features; the temporal modeling layer combines the TCN layer and the GRU layer to capture the temporal dependencies in the video sequence; the TCN layer uses a temporal convolutional network to extract local temporal features in the temporal dimension through one-dimensional convolution to improve the parallel processing capability of the model; the GRU layer uses a gated recurrent unit to capture the long-range dependencies of the sequence by utilizing a recurrent neural network to further process the temporal features; the output layer outputs the prediction results by mapping the temporal features to the classification space through a fully connected layer.

[0088] Finally, based on the hard target loss function and the soft target loss function, the learning effect of the student model is optimized by adjusting the distillation temperature parameter and the weight parameter to improve the accuracy of the student model and reduce the computational complexity of the student model.

[0089] In actual application, the knowledge distillation described in this embodiment combines hard target and soft target loss functions, and optimizes the learning effect of the student model by adjusting the distillation temperature parameters and weight parameters, so that it has lower computational complexity while maintaining high accuracy.

[0090] Specifically, the hard target loss function adopts the cross entropy loss function, and the formula is:

[0091]

[0092] Among them, y is the true label, is the class probability distribution predicted by the model;

[0093] The soft target loss function uses the Kullback-Leibler divergence to compare the softened outputs of the teacher model and the student model. The formula is:

[0094] FLoss soft =T 2 ·KLD iv (logSoftmax(S / T),Softmax(T / T))

[0095] Where T is the temperature parameter, which is used to soften the output probability distribution;

[0096] The hard target loss function and the soft target loss function are weighted and summed to obtain the total loss function, which is:

[0097] Loss = α × Loss soft +(1-α)×Loss hard

[0098] Among them, α is the weight parameter.

[0099] Based on the total loss function, the learning effect of the student model is optimized by adjusting the distillation temperature parameter and the weight parameter α to improve the accuracy of the student model and reduce the computational complexity of the student model. Exemplarily, in the knowledge distillation process, the hyperparameter definition can refer to the following settings: the distillation temperature is set to 2.0, alpha is 0.7, and the process is iterated 200 times. In addition, those skilled in the art can also adjust the hyperparameter definition according to actual conditions to further optimize the learning effect of the student model.

[0100] Step 105: Use the lightweight video monitoring model to monitor and identify the liquid level during the traditional Chinese medicine extraction process.

[0101] In actual application, the lightweight video monitoring model described in this embodiment can be deployed in an actual industrial environment to process multiple video streams in real time and in parallel. Specifically, the industrial camera is installed above the extraction equipment to collect video signals and transmit them to the industrial computer in the central control room through the TCP / IP protocol. The video signal is transmitted to the industrial computer after being converged through the network switch. The model runs on the computer and uses the OpenCV module for real-time analysis. The feature extraction of the video stream is completed by MobileNet, the sequence classification is processed by the GRU model, and the classification results are finally transmitted synchronously to the workshop display to provide real-time decision support for engineers.

[0102] By deploying a lightweight video monitoring model, first, real-time analysis can be performed: the model classifies each video stream in real time to determine whether the liquid level is in a boiling state. Second, the result output can be achieved: the classification results are transmitted to the workshop display through the network for engineers to refer to. In addition, parallel processing can be supported to ensure efficient operation in a multi-camera environment. For example, the simulation experiment can be implemented in PyTorch and run on an NVIDIA GeForce RTX 3080Ti GPU. Among them, the student model can be deployed on an industrial computer with an NVIDIA GeForce RTX 4060 GPU.

[0103] A monitoring method for a traditional Chinese medicine extraction process described in this embodiment includes: acquiring a collected video of a boiling liquid surface, and preprocessing the video of the boiling liquid surface to obtain video data; selecting sample data from the video data based on an active learning sampling strategy of spatial-temporal uncertainty; constructing a deep neural network model that combines a convolutional neural network model and a Transformer model, and training the deep neural network model using the sample data to obtain a trained deep neural network model; constructing a knowledge distillation model, and using the knowledge distillation model to extract knowledge from the trained deep neural network model, transferring the knowledge from the Transformer model to a lightweight student model to obtain a lightweight video monitoring model; and using the lightweight video monitoring model to monitor and identify the liquid surface during the traditional Chinese medicine extraction process.

[0104] Compared with the prior art, the monitoring method for the extraction process of traditional Chinese medicine described in this embodiment can be better applied to resource-constrained industrial environments. By combining the active learning method of spatial-temporal uncertainty sampling and deep learning technology, it can selectively participate in the annotation of high-value samples, thereby significantly improving the accuracy of the model while reducing the number of training samples. In addition, by processing video data through a hybrid architecture of convolutional neural networks and Transformers, and using knowledge distillation technology to lightweight the model, efficient and accurate production status monitoring can be achieved in resource-constrained industrial environments, providing support for state recognition and decision-making in the production process.

[0105] Embodiment 2

[0106] Figure 4 This is a schematic diagram of the structure of a monitoring device according to Embodiment 2 of the present invention, see Figure 4 , such monitoring device comprises:

[0107] The acquisition module 201 is used to acquire the collected liquid surface boiling video and pre-process the liquid surface boiling video to obtain video data.

[0108] The screening module 202 is used to select sample data from the video data based on an active learning sampling strategy of spatial-temporal uncertainty.

[0109] The construction module 203 is used to construct a deep neural network model that combines a convolutional neural network model and a Transformer model, and train the deep neural network model using the sample data to obtain a deep neural network model after training.

[0110] The distillation module 204 is used to construct a knowledge distillation model, and use the knowledge distillation model to extract the knowledge in the deep neural network model after the training is completed, transfer the knowledge in the Transformer model to the lightweight student model, and obtain a lightweight video monitoring model.

[0111] The deployment module 205 is used to monitor and identify the liquid level during the extraction process of traditional Chinese medicine using the lightweight video monitoring model.

[0112] The monitoring device described in this embodiment is optimized based on the above-mentioned embodiment, and can execute the monitoring method provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0113] Embodiment 3

[0114] Figure 5 A schematic diagram of the structure of a system provided in Embodiment 3 of the present invention; Figure 5 A block diagram of an exemplary terminal system suitable for implementing embodiments of the present invention is shown. Figure 5 The terminal system shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0115] like Figure 5 As shown, the terminal 12 is in the form of a general purpose computing device. The components of the terminal 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 connecting different system components (including the system memory 28 and the processing unit 16).

[0116] Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. By way of example, these architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an Enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0117] Terminal 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by terminal 12, including volatile and non-volatile media, removable and non-removable media.

[0118] System memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. Terminal 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 may be used to read and write non-removable, non-volatile magnetic media ( Figure 5 not shown, usually called a "hard drive"). Although Figure 5 Not shown in the figure, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present invention.

[0119] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in the memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. The program modules 42 generally perform the functions and / or methods of the embodiments described herein.

[0120] The terminal 12 may also communicate with one or more external devices 14 (e.g., keyboards, pointing devices, displays 24, etc.), may communicate with one or more devices that enable a user to interact with the terminal 12, and / or may communicate with any device that enables the terminal 12 to communicate with one or more other computing devices (e.g., network cards, modems, etc.). Such communication may be performed via an input / output (I / O) interface 22. Furthermore, the terminal 12 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 20. As shown, the network adapter 20 communicates with other modules of the terminal 12 via a bus 18. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the terminal 12, including, but not limited to, microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0121] The processing unit 16 executes various functional applications and data processing by running the programs stored in the system memory 28, such as implementing the monitoring method for the traditional Chinese medicine extraction process provided by the embodiment of the present invention.

[0122] Embodiment 4

[0123] Embodiment 4 of the present invention further provides a storage medium comprising computer executable instructions, which, when executed by a computer processor, are used to execute a monitoring method for a traditional Chinese medicine extraction process as described in any one of the above embodiments.

[0124] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0125] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0126] Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0127] Computer program code for performing the operations of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0128] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A monitoring method for a Chinese medicine extraction process, characterized in that: include: Acquire the collected liquid surface boiling video, and pre-process the liquid surface boiling video to obtain video data; Selecting sample data from the video data based on an active learning sampling strategy of spatial-temporal uncertainty; Constructing a deep neural network model that combines a convolutional neural network model and a Transformer model, and training the deep neural network model using the sample data to obtain a trained deep neural network model; Constructing a knowledge distillation model, and using the knowledge distillation model to extract knowledge from the trained deep neural network model, transferring the knowledge in the Transformer model to the lightweight student model, and obtaining a lightweight video monitoring model; The lightweight video monitoring model is used to monitor and identify the liquid level during the traditional Chinese medicine extraction process.

2. The method according to claim 1, characterized in that The step of acquiring the collected liquid surface boiling video and preprocessing the liquid surface boiling video to obtain video data includes: Get the collected video of liquid surface boiling; The liquid surface boiling video is subjected to frame extraction, image enhancement, size adjustment, and normalization processing in sequence to obtain video data.

3. The method according to claim 1, characterized in that The active learning sampling strategy based on space-time uncertainty selects sample data from the video data, including: evaluating spatial uncertainty by calculating entropy of classification probabilities of video frames in the video data; evaluating temporal uncertainty by calculating an average entropy of all frames of a video sequence in the video data; Taking a weighted sum of the spatial uncertainty and the temporal uncertainty to obtain a comprehensive uncertainty score of the video data; According to the comprehensive uncertainty score, active learning sampling is achieved by presetting performance thresholds and uncertainty thresholds to obtain sample data; wherein, when the model performance reaches the preset performance threshold or the comprehensive uncertainty score is lower than the preset uncertainty threshold, the learning process is stopped.

4. The method according to claim 1, characterized in that: The constructing of a deep neural network model combining a convolutional neural network model and a Transformer model, and training the deep neural network model using the sample data to obtain a trained deep neural network model includes: Construct a deep neural network model that combines a convolutional neural network model and a Transformer model; wherein the convolutional neural network model uses a MobileNet model to extract features from video frames to capture local features in the image; and the Transformer model uses a self-attention mechanism to capture temporal dependencies and changes between frames; The deep neural network model is trained using the sample data, and the classification accuracy of the deep neural network model is optimized through a back propagation algorithm and a minimization loss function to obtain a deep neural network model after training.

5. The method according to claim 1, characterized in that The knowledge distillation model is constructed, and the knowledge distillation model is used to extract the knowledge in the deep neural network model after the training is completed, and the knowledge in the Transformer model is transferred to the lightweight student model to obtain a lightweight video monitoring model, including: Constructing a teacher model, the teacher model includes a feature extraction layer, a temporal modeling layer, and an output layer; wherein the feature extraction layer uses a convolutional neural network to extract features from video frames to capture spatial features; the temporal modeling layer uses a Transformer model to capture temporal dependencies between frames through a self-attention mechanism; the output layer outputs prediction results by mapping temporal features to classification space; Construct a student model, which includes a feature extraction layer, a temporal modeling layer, a TCN layer, a GRU layer, and an output layer; wherein the feature extraction layer uses a lightweight convolutional neural network to extract frame-level features; the temporal modeling layer combines the TCN layer and the GRU layer to capture the temporal dependencies in the video sequence; the TCN layer uses a temporal convolutional network to extract local temporal features in the temporal dimension through one-dimensional convolution to improve the parallel processing capability of the model; the GRU layer uses a gated recurrent unit to capture the long-range dependencies of the sequence by using a recurrent neural network to further process the temporal features; the output layer outputs the prediction results by mapping the temporal features to the classification space through a fully connected layer; Based on the hard target loss function and the soft target loss function, the learning effect of the student model is optimized by adjusting the distillation temperature parameter and the weight parameter to improve the accuracy of the student model and reduce the computational complexity of the student model.

6. The method according to claim 5, characterized in that The method of optimizing the learning effect of the student model by adjusting the distillation temperature parameter and the weight parameter based on the hard target loss function and the soft target loss function to improve the accuracy of the student model and reduce the computational complexity of the student model includes: The hard target loss function uses the cross entropy loss function, and the formula is: Among them, y is the true label, is the class probability distribution predicted by the model; The soft target loss function uses the Kullback-Leibler divergence to compare the softened outputs of the teacher model and the student model. The formula is: FLoss soft =T 2 ·KLD iv (logSoftmax(S / T),Softmax(T / T)) Where T is the distillation temperature parameter, which is used to soften the output probability distribution; The hard target loss function and the soft target loss function are weighted and summed to obtain the total loss function, which is: Loss=α×Loss soft +(1-a)×Loss hard Among them, α is the weight parameter; Based on the total loss function, the learning effect of the student model is optimized by adjusting the distillation temperature parameter and the weight parameter α to improve the accuracy of the student model and reduce the computational complexity of the student model.

7. A monitoring device, characterized in that: include: An acquisition module, used for acquiring the collected liquid surface boiling video, and preprocessing the liquid surface boiling video to obtain video data; A screening module, used for selecting sample data from the video data based on an active learning sampling strategy of spatial-temporal uncertainty; A construction module is used to construct a deep neural network model that combines a convolutional neural network model and a Transformer model, and train the deep neural network model using the sample data to obtain a deep neural network model after training; A distillation module is used to construct a knowledge distillation model, and use the knowledge distillation model to extract knowledge from the deep neural network model after training, transfer the knowledge in the Transformer model to the lightweight student model, and obtain a lightweight video monitoring model; A deployment module is used to monitor and identify the liquid level during the extraction process of traditional Chinese medicine using the lightweight video monitoring model.

8. A system, characterized in that: include: one or more processors; A storage device for storing one or more programs; A camera, used to collect images; A display, used for displaying image recognition results; When the one or more programs are executed by the one or more processors, the one or more processors implement the monitoring method for the traditional Chinese medicine extraction process as described in any one of claims 1-6.

9. A storage medium comprising computer executable instructions, wherein the computer executable instructions, when executed by a computer processor, are used to execute the monitoring method for a traditional Chinese medicine extraction process as described in any one of claims 1 to 6.

Images