Low temperature medicine adding method and system for plaster making

Abstract

A low-temperature medicine adding method and system for plaster production, which acquires state monitoring video of a dan oil mixture in a predetermined time period collected by a camera; uses artificial intelligence technology based on deep learning to mine state time sequence dynamic change feature information of the dan oil mixture in the state monitoring video of the dan oil mixture, to evaluate the timing of dropping the dan oil into water, thereby achieving precise control of the timing of dropping the dan oil, and improving the production efficiency and product quality of the plaster.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent medicine adding, and more particularly, to a low-temperature medicine adding method and system for plaster preparation. BACKGROUND

[0002] The traditional Chinese medicine processing method of "cold oil frying" refers to frying medicinal materials in vegetable oil at low temperature to form a layer of black oxide skin on the surface to enhance the medicinal properties.

[0003] At present, the cold oil frying method has a long history and rich experience, and has certain guarantee in the preparation quality, so the cold oil frying method is used in the preparation process of some plasters to improve the medicinal effect and stability of medicinal materials in the plasters. However, in the actual preparation process, the traditional judgment method depends on professional experience and naked eye observation, and cannot accurately control the time of adding oil, resulting in low consistency and yield of the products, and the traditional cold oil frying method has the problems of long operation time, high energy consumption, and high risk of fire.

[0004] Therefore, an optimized low-temperature medicine adding scheme for plaster preparation is expected. SUMMARY

[0005] In order to solve the above technical problems, the present application is proposed. The embodiments of the present application provide a low-temperature medicine adding method and system for plaster preparation, which acquires state monitoring video of a predetermined time period of a medicine oil mixture collected by a camera; uses artificial intelligence technology based on deep learning to mine state time sequence dynamic change feature information of the medicine oil mixture in the state monitoring video of the medicine oil mixture, so as to judge and evaluate the time of adding the medicine oil into water, thereby accurately controlling the time of adding the medicine oil, and improving the production efficiency and product quality of the plaster.

[0006] In a first aspect, a low-temperature medicine adding method for plaster preparation is provided, which includes:

[0007] acquiring state monitoring video of a predetermined time period of a medicine oil mixture collected by a camera;

[0008] extracting a plurality of state monitoring key frames from the state monitoring video of the medicine oil mixture;

[0009] passing the plurality of state monitoring key frames through a convolutional neural network model containing a deep and shallow feature fusion module to obtain a plurality of medicine oil mixture state feature matrices;

[0010] calculating a transition matrix between each adjacent two medicine oil mixture state feature matrices in the plurality of medicine oil mixture state feature matrices to obtain a plurality of transition matrices;

[0011] obtaining a classification feature map by using a convolutional neural network model using a three-dimensional convolution kernel after the plurality of transition matrices are aggregated into a three-dimensional input tensor; and

[0012] obtaining a classification result by using a classifier on the classification feature map, the classification result being used to represent whether the Dan oil can be dropped into water at a current time point.

[0013] In the low-temperature medicine adding method for plaster production, the plurality of state monitoring key frames are input into a convolutional neural network model comprising a deep and shallow feature fusion module to obtain a plurality of Dan oil mixture state feature matrices, including: extracting a shallow feature map from a shallow layer of the convolutional neural network model comprising the deep and shallow feature fusion module; extracting a deep feature map from a deep layer of the convolutional neural network model comprising the deep and shallow feature fusion module; using a deep and shallow feature fusion module of the convolutional neural network model comprising the deep and shallow feature fusion module to fuse the shallow feature map and the deep feature map to obtain a fused feature map; and performing global mean pooling on the fused feature map along the channel dimension to obtain the plurality of Dan oil mixture state feature matrices.

[0014] In the low-temperature medicine adding method for plaster production, the transition matrix between every two adjacent Dan oil mixture state feature matrices in the plurality of Dan oil mixture state feature matrices is calculated to obtain a plurality of transition matrices, including: calculating the transition matrix between every two adjacent Dan oil mixture state feature matrices in the plurality of Dan oil mixture state feature matrices to obtain a plurality of transition matrices using the following transition formula; wherein the transition formula is:

[0015]

[0016] wherein, and denote every two adjacent Dan oil mixture state feature matrices in the plurality of Dan oil mixture state feature matrices, denote the plurality of transition matrices, denote matrix multiplication.

[0017] In the low-temperature medicine adding method for plaster production, the plurality of transition matrices are aggregated into a three-dimensional input tensor, and then a convolutional neural network model using a three-dimensional convolution kernel is used to obtain a classification feature map, including: the convolutional neural network model using the three-dimensional convolution kernel performs three-dimensional convolution processing, mean pooling processing and nonlinear activation processing on the three-dimensional input tensor in the forward transmission of a layer to obtain the classification feature map from the output of the last layer of the convolutional neural network model using the three-dimensional convolution kernel, wherein the input of the first layer of the convolutional neural network model using the three-dimensional convolution kernel is the three-dimensional input tensor.

[0018] The aforementioned low-temperature drug addition method for plaster preparation further includes training the convolutional neural network model containing a shallow and deep feature fusion module, the convolutional neural network model using a three-dimensional convolutional kernel, and the classifier. Training the convolutional neural network model containing the shallow and deep feature fusion module, the convolutional neural network model using a three-dimensional convolutional kernel, and the classifier includes: acquiring training data, which includes a state monitoring video of a mixture of cinnabar and oil over a predetermined time period, and the actual value of whether cinnabar and oil can be dripped into water at the current time point; extracting multiple training state monitoring keyframes from the state monitoring video of the mixture of cinnabar and oil; and passing the multiple training state monitoring keyframes through the convolutional neural network model containing the shallow and deep feature fusion module to obtain multiple training cinnabar and oil mixtures. The process involves: generating a state feature matrix for the mixture; calculating the transition matrix between every two adjacent state feature matrices of the mixture to obtain multiple training transition matrices; aggregating the multiple training transition matrices into a training 3D input tensor and passing it through the convolutional neural network model using 3D convolutional kernels to obtain a training classification feature map; performing Fourier-like scale domain probability correction on the training classification feature map to obtain an optimized training classification feature map; passing the optimized training classification feature map through the classifier to obtain a classification loss function value; and using the classification loss function value as the loss function value and backpropagation via gradient descent to train the convolutional neural network model containing a deep and shallow feature fusion module, the convolutional neural network model using 3D convolutional kernels, and the classifier.

[0019] In the aforementioned low-temperature drug addition method for plaster preparation, the optimized training classification feature map is obtained by performing Fourier-like scale-domain probability correction on the training classification feature map, including: performing Fourier-like scale-domain probability correction on the training classification feature map using the following optimization formula; wherein, the optimization formula is:

[0020]

[0021] in, It is the first training classification feature map Location feature value , and These are the height, width, and number of channels of the trained classification feature map, respectively. , and These are hyperparameters used for scaling. This indicates the calculation of the natural exponential function value raised to the power of the numerical value. It is the first optimized training classification feature map The characteristic value of the location.

[0022] In the low-temperature medicine adding method for plaster production, the optimized training classification feature map is input into the classifier to obtain a classification loss function value, including: the classifier processes the optimized training classification feature map according to a classification formula to generate a classification result, wherein the classification formula is:

[0023] wherein represents projecting the optimized training classification feature map into a vector, is a weight matrix, represents a bias matrix; and a cross-entropy value between the classification result and a true value is calculated as the classification loss function value.

[0024] In a second aspect, a low-temperature medicine adding system for plaster production is provided, comprising:

[0025] a video acquisition module configured to acquire a state monitoring video of a Dan oil mixture collected by a camera within a predetermined time period;

[0026] a key frame acquisition module configured to extract a plurality of state monitoring key frames from the state monitoring video of the Dan oil mixture;

[0027] a deep and shallow fusion module configured to input the plurality of state monitoring key frames into a convolutional neural network model comprising a deep and shallow feature fusion module to obtain a plurality of Dan oil mixture state feature matrices;

[0028] a transition matrix calculation module configured to calculate a transition matrix between each adjacent two Dan oil mixture state feature matrices in the plurality of Dan oil mixture state feature matrices to obtain a plurality of transition matrices;

[0029] a three-dimensional convolution module configured to aggregate the plurality of transition matrices into a three-dimensional input tensor, and then input the three-dimensional input tensor into a convolutional neural network model using a three-dimensional convolution kernel to obtain a classification feature map; and

[0030] a Dan oil control result generation module configured to input the classification feature map into a classifier to obtain a classification result, wherein the classification result is used to indicate whether Dan oil can be dropped into water at a current time point.

[0031] ​​In the low-temperature medicine adding system for plaster production, the deep-shallow fusion module comprises: a shallow layer extraction unit configured to extract a shallow layer feature map from a shallow layer of the convolutional neural network model comprising the deep-shallow feature fusion module; a deep layer extraction unit configured to extract a deep layer feature map from a deep layer of the convolutional neural network model comprising the deep-shallow feature fusion module; a fusion unit configured to fuse the shallow layer feature map and the deep layer feature map to obtain a fused feature map using a deep-shallow feature fusion module of the convolutional neural network model comprising the deep-shallow feature fusion module; and a pooling unit configured to perform global mean pooling on the fused feature map along a channel dimension to obtain the plurality of Dan oil mixture state feature matrices.

[0032] In the low-temperature medicine adding system for plaster production, the transition matrix calculation module is configured to calculate a transition matrix between each two adjacent Dan oil mixture state feature matrices in the plurality of Dan oil mixture state feature matrices using a transition formula to obtain a plurality of transition matrices, wherein the transition formula is:

[0033]

[0034] wherein, and denote each two adjacent Dan oil mixture state feature matrices in the plurality of Dan oil mixture state feature matrices, denote the plurality of transition matrices, denote matrix multiplication.

[0035] Compared with the prior art, the low-temperature medicine adding method and system for plaster production provided by the present application acquire a state monitoring video of a Dan oil mixture in a predetermined time period collected by a camera; artificial intelligence technology based on deep learning is used to mine state time sequence dynamic change feature information about the Dan oil mixture in the state monitoring video of the Dan oil mixture, so as to evaluate the timing of dropping the Dan oil into water, thereby realizing accurate control of the Dan oil dropping timing and improving the production efficiency and product quality of the plaster. BRIEF DESCRIPTION OF DRAWINGS

[0036] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0037] Figure 1 The scene schematic diagram of the low-temperature medicine adding method for plaster production according to the embodiments of the present application.

[0038] Figure 2Flow chart of the low-temperature medicine adding method for plaster making according to the embodiment of the present application.

[0039] Figure 3 Architecture schematic diagram of the low-temperature medicine adding method for plaster making according to the embodiment of the present application.

[0040] Figure 4 Flow chart of the sub-step of step 130 in the low-temperature medicine adding method for plaster making according to the embodiment of the present application.

[0041] Figure 5 Flow chart of the sub-step of step 170 in the low-temperature medicine adding method for plaster making according to the embodiment of the present application.

[0042] Figure 6 Block diagram of the low-temperature medicine adding system for plaster making according to the embodiment of the present application. DETAILED DESCRIPTION

[0043] The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of the present application.

[0044] Unless otherwise defined, all technical and scientific terms used in the embodiments of the present application have the same meanings as those commonly understood by those skilled in the art to which the present application belongs. The terms used in the present application are only for the purpose of describing the specific embodiments of the present application, and are not intended to limit the scope of the present application.

[0045] In the description of the embodiments of the present application, it should be noted that unless otherwise specified and limited, the term "connection" should be understood broadly, for example, it can be an electrical connection, or a connection between two elements, or a direct connection, or an indirect connection through an intermediate medium. Those of ordinary skill in the art can understand the specific meaning of the above-mentioned term according to the specific circumstances.

[0046] It should be noted that the terms "first", "second", and "third" in the embodiments of the present application are only to distinguish similar objects, and do not represent a specific order of the objects. Understandably, "first", "second", and "third" can be interchanged in specific order or sequence as allowed. It should be understood that the objects distinguished by "first", "second", and "third" can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein.

[0047] As described above, in the actual preparation process of the plaster, the traditional judgment method for the timing of dropping the oil dan relies on professional experience and naked eye observation, which cannot be accurately controlled, resulting in low consistency and yield of the product, and the traditional cold oil dan method has the problems of long operation time, high energy consumption, and high risk of causing fire. Therefore, an optimized low-temperature medicine adding scheme for plaster making is expected.

[0048] Specifically, in the technical solution of the present application, a low-temperature medicine adding method for plaster making is proposed, which comprises the following steps: first step: powdering, crushing the prepared traditional Chinese medicine to 100 meshes or more, and weighing according to the formula proportion; second step: frying dan, sieving the yellow dan, and frying the yellow dan with a small fire until it is dry and has no moisture; third step: oiling, adding tung oil according to the ratio of one dan to two oils, especially, the ratio here is different due to different seasons, for example, summer: 500ml:240g; winter: 500ml:150g; spring and autumn: 500ml:210g; fourth step: cold oil dan, heating the tung oil to about 150°C with a small fire, then slowly adding the yellow dan, and continuously stirring it clockwise with a camphor branch to make it uniform; when the oil dan is fused and black, the white smoke is gone, and the blue smoke is just starting, the oil dan is dropped into water, and then popped out of the water surface; fifth step: medicine adding, turn off the fire and continue to stir clockwise, when the temperature drops to a certain degree (about 150°C), add the prepared medicine powder to the plaster, and continuously stir clockwise to make the plaster and medicine one; sixth step: removing fire toxin, pour the prepared plaster into a pottery basin filled with clean water, soak for 3-7 days, and change the water several times a day to remove the fire toxin; seventh step: spreading, heat the plaster that has removed the fire toxin to about 80-100 degrees, melt and stir it evenly, then mix in the fine medicine and volatile medicine powder, and then spread it. Especially, when spreading, take a bamboo chopstick, pick up an appropriate amount of plaster that has not cooled down, and according to the size of the paper cloth, point the bamboo chopstick at the center of the paper cloth, and spread it clockwise for one turn. Preferably, in the technical solution of the present application, slow heating is used to slow down the saponification reaction speed, to avoid high-temperature reaction and cause fire accidents; low-temperature medicine adding can effectively avoid the destruction of the effective components in the medicinal materials; adding fine medicine and volatile traditional Chinese medicine before spreading the plaster can avoid unnecessary loss.

[0049] Correspondingly, considering the actual process of making plaster, the timing of dropping oil Dan is the key node of preparing plaster. The traditional judgment method is based on professional experience and visual observation, which cannot accurately control the consistency and yield of the product. Therefore, in the technical solution of the present application, it is expected to judge whether to drop Dan oil by the state change of the Dan oil mixture. The state change of the Dan oil mixture can be detected by analyzing the state monitoring video of the Dan oil mixture. However, since there is a large amount of information in the state monitoring video, and the state time sequence change characteristics of the Dan oil mixture are small-scale subtle change characteristic information in the video, it is difficult to accurately capture, resulting in inaccurate timing of Dan oil dropping, affecting the efficacy and efficiency. That is, in this process, the difficulty lies in how to mine the state time sequence dynamic change characteristic information of the Dan oil mixture in the state monitoring video of the Dan oil mixture, so as to judge and evaluate the timing of dropping the Dan oil into water, so as to accurately control the timing of dropping the Dan oil, and improve the production efficiency and product quality of the plaster.

[0050] In recent years, deep learning and neural networks have been widely used in computer vision, natural language processing, text signal processing and other fields. The development of deep learning and neural networks provides a new solution for mining state time sequence dynamic change characteristic information of the Dan oil mixture in the state monitoring video of the Dan oil mixture.

[0051] Specifically, in the technical solution of the present application, first, the state monitoring video of the Dan oil mixture in a predetermined time period is collected by a camera. Then, considering that the state change characteristics of the Dan oil mixture in the state monitoring video of the Dan oil mixture can be represented by the difference between adjacent monitoring frames in the personnel state monitoring video, that is, the state change of the Dan oil mixture is represented by the image representation of adjacent image frames. However, considering that the difference between adjacent frames in the state monitoring video is small, there is a lot of data redundancy, therefore, in order to reduce the amount of calculation, and avoid the adverse effects of data redundancy on detection, the state monitoring video is key frame sampled at a predetermined sampling frequency, so as to extract a plurality of state monitoring key frames from the state monitoring video of the Dan oil mixture. Here, it is worth mentioning that the sampling frequency can be adjusted based on the application requirements of the actual scene, rather than the default value.

[0052] Then, the convolutional neural network model with excellent performance in implicit feature extraction of images is used for feature mining of the plurality of state monitoring key frames. In particular, in order to more accurately detect the state feature change of the mixture, more attention should be paid to the shallow features such as color and texture of the mixture in the state monitoring key frames when extracting the hidden features of the mixture in the state monitoring key frames, in order to accurately judge whether the turpentine can be added into water. These shallow features are important for the state detection of the mixture, and the shallow features will become blurred or even submerged by noise as the depth of the convolutional neural network increases during encoding. Therefore, in the technical solution of the present application, a convolutional neural network model containing a deep and shallow feature fusion module is used to process the state monitoring key frames to obtain a plurality of turpentine mixture state feature matrices. It can be understood that compared with the standard convolutional neural network model, the convolutional neural network model according to the present application can retain the shallow and deep features of the mixture in the state monitoring key frames, so as to not only make the feature information more abundant, but also retain the features of different depths, so as to improve the accuracy of the state detection of the mixture. At the same time, the structure of the deep neural network is often complex, and a large amount of sample data is needed for training and adjustment. The training time of the deep network is longer, and it is easy to overfit. Therefore, in the design of the neural network model, the combination of shallow and deep networks is usually adopted, and through deep and shallow feature fusion, the complexity and overfitting risk of the network can be reduced to a certain extent, while the feature extraction ability and generalization ability of the model are improved.

[0053] Further, since the state implicit features of the turpentine mixture have dynamic change feature information in the time dimension, and the dynamic change feature is weak in the explicit degree based on the time sequence globally, in the technical solution of the present application, the transition matrix between each adjacent two turpentine mixture state feature matrices in the plurality of turpentine mixture state feature matrices is calculated to represent the state change mode feature of the turpentine mixture at adjacent two time points, thereby obtaining a plurality of transition matrices. It can be understood that when making the plaster, the addition of turpentine needs to master the timing of addition, and the turpentine needs to be added into water gradually to fully release the medicinal properties and avoid overheating. Therefore, when adding medicine at low temperature, the state of the turpentine mixture needs to be judged to determine when to add the turpentine into water. By calculating the transition matrix between the plurality of turpentine mixture state feature matrices, the state change relationship of the turpentine mixture can be better captured, so as to determine when to add the turpentine into water.

[0054] Then, considering that the state change pattern features of the dan oil mixture at each two adjacent time points have dynamic variation rules in the time dimension as a whole, that is, there is a dynamic correlation relationship between the state change pattern features of the dan oil mixture at each two adjacent time points. Therefore, in the technical solution of the present application, the plurality of transition matrices are aggregated into a three-dimensional input tensor, and then the convolutional neural network model using a three-dimensional convolution kernel is used for feature extraction to extract the correlation feature distribution information of the state change pattern features of the dan oil mixture at each two adjacent time points in the time dimension, thereby obtaining a classification feature map. That is, the convolutional neural network model using a three-dimensional convolution kernel can capture the three-dimensional convolution kernel-based correlation of the state change of the dan oil mixture in the time dimension and the spatial dimension.

[0055] Then, the classification feature map is further classified by a classifier to obtain a classification result for indicating whether the dan oil can be dropped into water at the current time point. That is, in the technical solution of the present application, the labels of the classifier include that the dan oil can be dropped into water at the current time point (first label) and that the dan oil cannot be dropped into water at the current time point (second label), wherein the classifier determines which classification label the classification feature map belongs to by using a soft-max function. It is worth noting that the first label p1 and the second label p2 here do not contain human-set concepts. In fact, during the training process, the computer model does not have the concept of “whether the dan oil can be dropped into water at the current time point”, it only has two classification labels and outputs the probabilities of the feature under the two classification labels, that is, the sum of p1 and p2 is one. Therefore, the classification result of whether the dan oil can be dropped into water at the current time point is actually converted into a binary classification probability distribution conforming to the natural law by using the classification label, and the physical meaning of the natural probability distribution of the label is used in essence, rather than the language text meaning of “whether the dan oil can be dropped into water at the current time point”. It should be understood that in the technical solution of the present application, the classification label of the classifier is the control strategy label of whether the dan oil can be dropped into water at the current time point, so that the control judgment of the dan oil dropped into water can be performed based on the classification result after the classification result is obtained, thereby realizing accurate control of the dropping time of the dan oil, improving the production efficiency and product quality of the plaster.

[0056] Correspondingly, in the technical solution of the present application, the transition matrix between each two adjacent Dan oil mixture state feature matrices is used to represent the state change pattern feature of the Dan oil mixture at adjacent two time points. Therefore, after the plurality of transition matrices are aggregated into a three-dimensional input tensor, the convolutional neural network model using a three-dimensional convolution kernel can capture the three-dimensional convolution kernel-based correlation of the state change of the Dan oil mixture in the time dimension and the spatial dimension. Based on this, in the technical solution of the present application, if the overall feature learning correlation of the convolutional neural network model in three dimensions can be obtained, the expression effect of the overall classification feature map can be further improved, so as to improve the accuracy of the classification result obtained by the classifier through the classification feature map.

[0057] Based on this, in the training process, for the classification feature map a class Fourier scale domain probability correction is performed, which is specifically represented as:

[0058]

[0059] wherein, is the feature value of the th position of the classification feature map , , and are the height, width and channel number of the classification feature map , and , and are hyperparameters for scale adjustment.

[0060] Here, the class Fourier scale domain probability correction considers the homology between the high-dimensional feature distribution and the scale domain where it is located. The potential distribution correlation in the homologous space can be captured based on the low-rank constraint of the scale space through the class Fourier sparse low-rank transformation of the scale space, so as to realize the joint spatial feature learning with scale coherence of the feature in the training process of the convolutional neural network model using a three-dimensional convolution kernel, so as to improve the learning correlation of the three-dimensional convolution kernel in the overall spatial scale, thereby improving the expression effect of the obtained classification feature map in the overall three dimensions, and improving the accuracy of the classification result obtained by the classifier through the classification feature map. In this way, it can be judged and controlled in real time whether the Dan oil is dropped into water, so as to realize accurate control of the oil Dan dropping time, improve the production efficiency and product quality of the plaster.

[0061] Figure 1 is a scene schematic diagram of a low-temperature medicine adding method for plaster making according to an embodiment of the present application. As Figure 1As shown, in this application scenario, firstly, the mixture of dan-oil is acquired by a camera over a predetermined time period (e.g., as shown in the image). Figure 1 The status monitoring video of M (as shown) (e.g., such as Figure 1 (as shown in C); then, the acquired status monitoring video is input to a server deployed with a low-temperature dosing algorithm for plaster making (e.g., as shown in C). Figure 1 In the illustrated S), the server is able to process the status monitoring video based on a low-temperature dosing algorithm for plaster making to generate a classification result indicating whether the dan oil can be dripped into the water at the current time point.

[0062] After introducing the basic principles of this application, various non-limiting embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0063] In one embodiment of this application, Figure 2 This is a flowchart of a low-temperature drug addition method for making plasters according to an embodiment of this application. Figure 2 As shown, a low-temperature drug addition method 100 for plaster preparation according to an embodiment of this application includes: 110, acquiring a state monitoring video of a mixture of cinnamon oil and scallion oil collected by a camera over a predetermined time period; 120, extracting multiple state monitoring key frames from the state monitoring video of the mixture of cinnamon oil and scallion oil; 130, passing the multiple state monitoring key frames through a convolutional neural network model including a deep and shallow feature fusion module to obtain multiple state feature matrices of the mixture of cinnamon oil and scallion oil; 140, calculating a transition matrix between every two adjacent state feature matrices of the mixture of cinnamon oil and scallion oil to obtain multiple transition matrices; 150, aggregating the multiple transition matrices into a three-dimensional input tensor and then using a convolutional neural network model with a three-dimensional convolutional kernel to obtain a classification feature map; and 160, passing the classification feature map through a classifier to obtain a classification result, the classification result being used to indicate whether the cinnamon oil can be dripped into water at the current time point.

[0064] Figure 3 This is a schematic diagram of the architecture of a low-temperature drug addition method for making plasters according to an embodiment of this application. Figure 3As shown, in the network architecture, first, a state monitoring video of the Dan oil mixture in a predetermined time period collected by a camera is obtained; then, a plurality of state monitoring key frames are extracted from the state monitoring video of the Dan oil mixture; next, the plurality of state monitoring key frames are passed through a convolutional neural network model containing a deep and shallow feature fusion module to obtain a plurality of Dan oil mixture state feature matrices; then, a transition matrix between each adjacent two Dan oil mixture state feature matrices in the plurality of Dan oil mixture state feature matrices is calculated to obtain a plurality of transition matrices; next, the plurality of transition matrices are aggregated into a three-dimensional input tensor and then passed through a convolutional neural network model using a three-dimensional convolution kernel to obtain a classification feature map; and finally, the classification feature map is passed through a classifier to obtain a classification result, which is used to indicate whether the Dan oil can be dropped into water at the current time point.

[0065] Specifically, in step 110, a state monitoring video of the Dan oil mixture in a predetermined time period collected by a camera is obtained. As described above, in the actual preparation process of the plaster, the traditional judgment method for the timing of dropping the oil Dan relies on professional experience and naked eye observation, which cannot accurately control the product consistency and yield, and the traditional cold oil Dan method has the problems of long operation time, high energy consumption, and high risk of causing fire.

[0066] Specifically, in step 120, a plurality of state monitoring key frames are extracted from the state monitoring video of the Dan oil mixture. Next, considering that the state change characteristics of the Dan oil mixture in the state monitoring video of the Dan oil mixture can be represented by the difference between adjacent monitoring frames in the personnel state monitoring video, that is, the state change of the Dan oil mixture is represented by the image representation of adjacent image frames. However, considering that the difference between adjacent frames in the state monitoring video is small, there is a lot of data redundancy, therefore, in order to reduce the calculation amount and avoid the adverse effects of data redundancy on detection, the state monitoring video is sampled at a predetermined sampling frequency to extract a plurality of state monitoring key frames from the state monitoring video of the Dan oil mixture. Here, it is worth mentioning that the sampling frequency can be adjusted based on the application requirements of the actual scene, rather than a default value.

[0067] Specifically, in step 130, the plurality of state monitoring key frames are processed by a convolutional neural network model comprising a shallow and deep feature fusion module to obtain a plurality of Dan oil mixture state feature matrices. Then, the convolutional neural network model with excellent performance in extracting the hidden features of images is used to mine the features of the plurality of state monitoring key frames. In particular, when extracting the hidden features of the mixture in the respective state monitoring key frames, in order to more accurately detect the state feature change of the mixture and accurately determine whether the Dan oil can be added to the water, more attention should be paid to the shallow features such as color and texture of the mixture in the respective state monitoring key frames, which are important for the state detection of the mixture. However, when the convolutional neural network is encoded, the shallow features will become blurred or even submerged by noise as the depth increases.

[0068] Therefore, in the technical solution of the present application, the respective state monitoring key frames are processed by a convolutional neural network model comprising a shallow and deep feature fusion module to obtain a plurality of Dan oil mixture state feature matrices. It can be understood that, compared with a standard convolutional neural network model, the convolutional neural network model according to the present application can retain the shallow and deep features of the mixture in the respective state monitoring key frames, so as to not only make the feature information more abundant, but also retain the features of different depths, thereby improving the accuracy of the state detection of the mixture. At the same time, the structure of a deep neural network is often complex and requires a large amount of sample data for training and adjustment. The training time of a deep network is relatively long and is prone to overfitting. Therefore, in the design of a neural network model, a combination of shallow and deep networks is usually adopted, and through shallow and deep feature fusion, the complexity and overfitting risk of the network can be reduced to a certain extent, while the feature extraction ability and generalization ability of the model are improved.

[0069] Figure 4 For the flowchart of the sub-step of step 130 in the low-temperature medicine adding method for plaster making according to the embodiment of the present application, as shown in Figure 4 the plurality of state monitoring key frames are processed by a convolutional neural network model comprising a shallow and deep feature fusion module to obtain a plurality of Dan oil mixture state feature matrices, comprising: 131, extracting a shallow feature map from a shallow layer of the convolutional neural network model comprising a shallow and deep feature fusion module; 132, extracting a deep feature map from a deep layer of the convolutional neural network model comprising a shallow and deep feature fusion module; 133, using a shallow and deep feature fusion module of the convolutional neural network model comprising a shallow and deep feature fusion module to fuse the shallow feature map and the deep feature map to obtain a fused feature map; and 134, performing global mean pooling on the fused feature map along the channel dimension to obtain the plurality of Dan oil mixture state feature matrices.

[0070] It should be understood that, compared with a standard convolutional neural network model, the convolutional neural network model according to the present application can retain the shallow features and deep features of the plurality of state monitoring key frames, so as to not only make the feature information more abundant, but also retain the features of different depths, so as to improve the accuracy of extraction of the plurality of state monitoring key frames. At the same time, the structure of a deep neural network is often complex, and a large amount of sample data is needed for training and adjustment. The training time of a deep network is long, and overfitting is easy to occur. Therefore, in the design of a neural network model, a combination of shallow networks and deep networks is usually adopted, and through deep and shallow feature fusion, the complexity of the network and the risk of overfitting can be reduced to a certain extent, and the feature extraction ability and generalization ability of the model can be improved.

[0071] Specifically, in step 140, a transition matrix between each two adjacent dan oil mixture state feature matrices in the plurality of dan oil mixture state feature matrices is calculated to obtain a plurality of transition matrices. Further, since the state implicit features of the dan oil mixture have dynamic change characteristic information in the time dimension, and the dynamic change characteristics are weak in the degree of explicitness based on the global time sequence,

[0072] Therefore, in the technical solution of the present application, a transition matrix between each two adjacent dan oil mixture state feature matrices in the plurality of dan oil mixture state feature matrices is calculated to represent the state change mode features of the dan oil mixture at two adjacent time points, thereby obtaining a plurality of transition matrices. It should be understood that when making a plaster, the addition of dan oil needs to master the timing of addition, and the dan oil needs to be added drop by drop into water to fully release the medicinal properties and avoid overheating. Therefore, when adding medicine at low temperature, it is necessary to judge when to add dan oil into water according to the state of the dan oil mixture. By calculating the transition matrix between the plurality of dan oil mixture state feature matrices, the state change relationship of the dan oil mixture can be better captured, so as to judge when to add dan oil into water.

[0073] The transition matrix between each two adjacent dan oil mixture state feature matrices in the plurality of dan oil mixture state feature matrices is calculated to obtain a plurality of transition matrices, including: calculating the transition matrix between each two adjacent dan oil mixture state feature matrices in the plurality of dan oil mixture state feature matrices by using the following transition formula to obtain a plurality of transition matrices; wherein the transition formula is:

[0074]

[0075] wherein, and represent the transition matrix between each two adjacent dan oil mixture state feature matrices in the plurality of dan oil mixture state feature matrices, representing the plurality of transition matrices, representing matrix multiplication.

[0076] Specifically, in step 150, the plurality of transition matrices are aggregated into a three-dimensional input tensor, and then a convolutional neural network model using a three-dimensional convolution kernel is used to obtain a classification feature map. Then, considering that the state change pattern features of the dan oil mixture at every two adjacent time points have dynamic change rules in the time dimension as a whole, that is, there is a dynamic correlation relationship between the state change pattern features of the dan oil mixture at every two adjacent time points.

[0077] Therefore, in the technical solution of the present application, the plurality of transition matrices are aggregated into a three-dimensional input tensor, and then a convolutional neural network model using a three-dimensional convolution kernel is used for feature extraction to extract the correlation feature distribution information of the state change pattern features of the dan oil mixture at every two adjacent time points in the time dimension, thereby obtaining a classification feature map. That is, the convolutional neural network model using a three-dimensional convolution kernel can capture the three-dimensional convolution kernel-based correlation of the state change of the dan oil mixture in the time dimension and the spatial dimension.

[0078] A convolutional neural network (CNN) is an artificial neural network that has a wide range of applications in image recognition and other fields. The convolutional neural network can include an input layer, a hidden layer, and an output layer, wherein the hidden layer can include a convolution layer, a pooling layer, an activation layer, and a fully connected layer, etc. The previous layer performs corresponding operations according to the input data and outputs the operation results to the next layer. After the initial data is input and undergoes multiple layers of operations, a final result is obtained.

[0079] The convolutional neural network model uses a convolution kernel as a feature filtering factor and has very excellent performance in image local feature extraction. Compared with traditional image feature extraction algorithms based on statistics or feature engineering, the convolutional neural network model has stronger feature extraction generalization ability and fitting ability.

[0080] Specifically, in step 160, the classification feature map is classified by a classifier to obtain a classification result, which is used to represent whether the dan oil can be dropped into water at the current time point. Then, the classification feature map is further classified by the classifier to obtain a classification result representing whether the dan oil can be dropped into water at the current time point. That is, in the technical solution of the present application, the labels of the classifier include that the dan oil can be dropped into water at the current time point (first label) and that the dan oil cannot be dropped into water at the current time point (second label), wherein the classifier determines which classification label the classification feature map belongs to by using a soft-max function.

[0081] It is worth noting that the first label p1 and the second label p2 here do not contain the concept set by human beings. In fact, during the training process, the computer model does not have the concept of “whether the current time point can drop the oil into the water”, but only has two classification labels and outputs the probability of the feature under the two classification labels, that is, the sum of p1 and p2 is one. Therefore, the classification result of whether the current time point can drop the oil into the water is actually converted into a binary classification probability distribution conforming to the law of nature, and the physical meaning of the natural probability distribution of the label is actually used, rather than the language text meaning of “whether the current time point can drop the oil into the water”.

[0082] It can be understood that in the technical solution of the present application, the classification label of the classifier is the control strategy label of whether the current time point can drop the oil into the water, so that after obtaining the classification result, the control judgment of whether the oil can be dropped into the water at the current time point can be performed based on the classification result, thereby realizing accurate control of the oil dropping time, improving the production efficiency and product quality of the plaster.

[0083] The classification feature map is input into the classifier to obtain a classification result, and the classification result is used to represent whether the current time point can drop the oil into the water, including: expanding the classification feature map into a classification feature vector according to a row vector or a column vector; using a plurality of fully connected layers of the classifier to perform fully connected coding on the classification feature vector to obtain a coded classification feature vector; and inputting the coded classification feature vector into a Softmax classification function of the classifier to obtain the classification result.

[0084] The low-temperature medicine adding method for plaster production further includes training the convolutional neural network model including the deep and shallow feature fusion module, the convolutional neural network model using the three-dimensional convolution kernel, and the classifier. Figure 5 The flowchart of the sub-step of step 170 in the low-temperature medicine adding method for plaster production according to the embodiment of the present application is as follows: Figure 5As shown, the training of the convolutional neural network model comprising the deep and shallow feature fusion module, the convolutional neural network model using a three-dimensional convolution kernel and the classifier comprises: 171, obtaining training data, wherein the training data comprises a state training monitoring video of the Dan oil mixture in a predetermined time period, and a real value of whether the Dan oil can be dropped into water at the current time point; 172, extracting a plurality of training state monitoring key frames from the state training monitoring video of the Dan oil mixture; 173, passing the plurality of training state monitoring key frames through the convolutional neural network model comprising the deep and shallow feature fusion module to obtain a plurality of training Dan oil mixture state feature matrices; 174, calculating a transition matrix between each adjacent two training Dan oil mixture state feature matrices in the plurality of training Dan oil mixture state feature matrices to obtain a plurality of training transition matrices; 175, aggregating the plurality of training transition matrices into a training three-dimensional input tensor and then passing the training three-dimensional input tensor through the convolutional neural network model using a three-dimensional convolution kernel to obtain a training classification feature map; 176, performing a class Fourier scale domain probability correction on the training classification feature map to obtain an optimized training classification feature map; 177, passing the optimized training classification feature map through the classifier to obtain a classification loss function value; and 178, taking the classification loss function value as a loss function value and training the convolutional neural network model comprising the deep and shallow feature fusion module, the convolutional neural network model using a three-dimensional convolution kernel and the classifier through the gradient descent back propagation.

[0085] Correspondingly, in the technical solution of the present application, the transition matrix between each adjacent two Dan oil mixture state feature matrices is used to represent the state change mode feature of the Dan oil mixture at adjacent two time points. Therefore, after aggregating the plurality of transition matrices into a three-dimensional input tensor, the convolutional neural network model using a three-dimensional convolution kernel can capture the three-dimensional convolution kernel-based correlation of the state change of the Dan oil mixture in the time sequence dimension and the spatial dimension. Based on this, in the technical solution of the present application, if the overall feature learning correlation of the convolutional neural network model in three dimensions can be improved, the expression effect of the classification feature map as a whole can be further improved, so as to improve the accuracy of the classification result obtained by the classifier through the classification feature map.

[0086] Based on this, in the training process, the classification feature map is corrected in a class Fourier scale domain probability, which is specifically represented as: the training classification feature map is corrected in a class Fourier scale domain probability by using the following optimization formula to obtain the optimized training classification feature map; wherein the optimization formula is:

[0087]

[0088] wherein, is the first feature values of the position, , and are height, width and channel number of the training classification feature map respectively, and , and are hyperparameters for scale adjustment, represents calculating the natural exponential function value of a number to the power, is the feature value of the i-th position of the optimized training classification feature map.

[0089] Here, the class Fourier scale domain probability correction considers the homology of high-dimensional feature distribution and the scale domain where it is located. The potential distribution correlation under the homology space can be captured based on the low-rank constraint of the scale space through the class Fourier sparse low-rank transformation of the scale space. Therefore, in the training process of the convolutional neural network model using the three-dimensional convolution kernel, the joint spatial feature learning with scale coherence of the feature whole is realized while obtaining the local correlation representation of the feature value based on the three-dimensional convolution kernel. The learning correlation degree of the convolutional neural network model of the three-dimensional convolution kernel under the whole spatial scale is improved, so as to improve the expression effect of the obtained classification feature map in the whole three dimensions, and the accuracy of the classification result obtained by the classifier. In this way, it can be accurately judged whether to drop the oil into the water in real time, so as to realize the precise control of the oil dropping time, improve the production efficiency and product quality of the plaster.

[0090] Further, the optimized training classification feature map is processed by the classifier to obtain a classification loss function value, including: the classifier processes the optimized training classification feature map according to the following classification formula to generate a classification result, wherein the classification formula is: wherein represents projecting the optimized training classification feature map into a vector, to are weight matrices, to represent bias matrices; and calculating the cross-entropy value between the classification result and the true value as the classification loss function value.

[0091] ​In summary, the low-temperature medicine adding method 100 for plaster production based on the embodiments of the present application is illustrated, which acquires state monitoring videos of the Dan oil mixture in a predetermined time period collected by a camera; uses artificial intelligence technology based on deep learning to mine state time sequence dynamic change feature information of the Dan oil mixture in the state monitoring videos of the Dan oil mixture, so as to evaluate the timing of dropping the Dan oil into water, thereby realizing accurate control of the Dan oil dropping timing and improving the production efficiency and product quality of the plaster.

[0092] In an embodiment of the present application, Figure 6 The block diagram of the low-temperature medicine adding system for plaster production according to the embodiments of the present application is shown in FIG. 2. As shown in FIG. 2, the low-temperature medicine adding system 200 for plaster production according to the embodiments of the present application includes a video acquisition module 210 configured to acquire state monitoring videos of the Dan oil mixture in a predetermined time period collected by a camera. Figure 6

[0093] A key frame acquisition module 220 is configured to extract a plurality of state monitoring key frames from the state monitoring videos of the Dan oil mixture.

[0094] A deep and shallow fusion module 230 is configured to obtain a plurality of Dan oil mixture state feature matrices by using a convolutional neural network model containing a deep and shallow feature fusion module to process the plurality of state monitoring key frames. A transition matrix calculation module 240 is configured to calculate a transition matrix between each adjacent two Dan oil mixture state feature matrices in the plurality of Dan oil mixture state feature matrices to obtain a plurality of transition matrices. A three-dimensional convolution module 250 is configured to obtain a classification feature map by using a convolutional neural network model with a three-dimensional convolution kernel to aggregate the plurality of transition matrices into a three-dimensional input tensor. A Dan oil control result generation module 260 is configured to obtain a classification result by using a classifier to process the classification feature map, wherein the classification result is used to indicate whether the Dan oil can be dropped into water at a current time point.

[0095] In a specific example, in the low-temperature medicine adding system for plaster production described above, the deep and shallow fusion module includes a shallow layer extraction unit configured to extract a shallow layer feature map from a shallow layer of the convolutional neural network model containing a deep and shallow feature fusion module; a deep layer extraction unit configured to extract a deep layer feature map from a deep layer of the convolutional neural network model containing a deep and shallow feature fusion module; a fusion unit configured to use a deep and shallow feature fusion module of the convolutional neural network model containing a deep and shallow feature fusion module to fuse the shallow layer feature map and the deep layer feature map to obtain a fusion feature map; and a pooling unit configured to perform global mean pooling on the fusion feature map along the channel dimension to obtain the plurality of Dan oil mixture state feature matrices.

[0096] ​In one specific example, in the low-temperature medicine adding system for plaster production described above, the transition matrix calculation module is configured to: calculate a transition matrix between every two adjacent Dan oil mixture state feature matrices in the plurality of Dan oil mixture state feature matrices using a transition formula to obtain a plurality of transition matrices; wherein the transition formula is:

[0097]

[0098] wherein, and denote every two adjacent Dan oil mixture state feature matrices in the plurality of Dan oil mixture state feature matrices, denote the plurality of transition matrices, denote matrix multiplication.

[0099] In one specific example, in the low-temperature medicine adding system for plaster production described above, the three-dimensional convolution module is configured to: the convolutional neural network model using a three-dimensional convolution kernel performs three-dimensional convolution processing, mean pooling processing, and nonlinear activation processing on the three-dimensional input tensor based on the three-dimensional convolution kernel in the forward transmission of the layer to obtain the classification feature map from the output of the last layer of the convolutional neural network model using the three-dimensional convolution kernel, wherein the input of the first layer of the convolutional neural network model using the three-dimensional convolution kernel is the three-dimensional input tensor.

[0100] In one specific example, in the low-temperature medicine adding system for plaster production described above, the training module for training the convolutional neural network model comprising the deep-shallow feature fusion module, the convolutional neural network model using a three-dimensional convolution kernel, and the classifier is further included; wherein the training module comprises: a training video acquisition unit configured to acquire training data, the training data comprising a state training monitoring video of a Dan oil mixture in a predetermined time period, and a real value of whether the current time point is capable of dropping Dan oil into water; a training key frame acquisition unit configured to extract a plurality of training state monitoring key frames from the state training monitoring video of the Dan oil mixture; a training deep-shallow fusion unit configured to pass the plurality of training state monitoring key frames through the convolutional neural network model comprising the deep-shallow feature fusion module to obtain a plurality of training Dan oil mixture state feature matrices; a training transition matrix calculation unit configured to calculate a transition matrix between every two adjacent training Dan oil mixture state feature matrices in the plurality of training Dan oil mixture state feature matrices to obtain a plurality of training transition matrices; and a training three-dimensional convolution unit configured to aggregate the plurality of training transition matrices into a training three-dimensional input tensor and then pass the training three-dimensional input tensor through the convolutional neural network model using a three-dimensional convolution kernel to obtain a training classification feature map.

[0101] The training optimization unit is used to perform Fourier-like scale domain probability correction on the training classification feature map to obtain an optimized training classification feature map; the loss function value calculation unit is used to pass the optimized training classification feature map through the classifier to obtain a classification loss function value; and the training unit is used to train the convolutional neural network model containing the deep and shallow feature fusion module, the convolutional neural network model using three-dimensional convolutional kernels, and the classifier using the classification loss function value as the loss function value and backpropagation through gradient descent.

[0102] In a specific example, in the aforementioned low-temperature drug addition system for plaster production, the training optimization unit is configured to: perform Fourier-like scale-domain probability correction on the training classification feature map using the following optimization formula to obtain the optimized training classification feature map; wherein, the optimization formula is:

[0103]

[0104] in, It is the first training classification feature map Location feature value , and These are the height, width, and number of channels of the trained classification feature map, respectively. , and These are hyperparameters used for scaling. This indicates the calculation of the natural exponential function value raised to the power of the numerical value. It is the first optimized training classification feature map The characteristic value of the location.

[0105] In a specific example, in the aforementioned low-temperature drug addition system for plaster production, the loss function value calculation unit includes: a classification subunit, used by the classifier to process the optimized training classification feature map using the following classification formula to generate a classification result, wherein the classification formula is:

[0106] ,in This means projecting the optimized training classification feature map into a vector. to This is the weight matrix. to The bias matrix is ​​represented by a sub-unit for calculating the cross-entropy value between the classification result and the true value as the classification loss function value.

[0107] Here, those skilled in the art will understand that the specific functions and operations of each unit and module in the aforementioned low-temperature drug addition system for plaster production have been referenced above.Figures 1 to 5 The low-temperature medicine adding method for plaster manufacturing is described in detail in the description of the low-temperature medicine adding method for plaster manufacturing, and thus, the repeated description will be omitted.

[0108] The present application also provides a computer program product including instructions that, when executed, cause an apparatus to perform operations corresponding to those in the above-described methods.

[0109] In an embodiment of the present application, a computer-readable storage medium storing a computer program for executing the above-described method is also provided.

[0110] It should be understood that embodiments of the present application can be implemented in the form of methods, systems or computer program products. Therefore, embodiments of the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk memory, CD-ROM, optical memory and the like) including computer-usable program code.

[0111] The methods, systems and computer program products of embodiments of the present application are described by flowcharts and / or block diagrams in the drawings. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and combinations of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, a special purpose computer, an embedded processor or other programmable data processing apparatus to produce a machine, so that the instructions, which are executed via the processor of the computer or other programmable data processing apparatus, generate a means for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus that implements the functions specified in the flowcharts and / or block diagrams.

[0112] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including instruction apparatus, which implements the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus that implements the functions specified in the flowcharts and / or block diagrams.

[0113] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus, so that a series of operation steps are performed on the computer or other programmable data processing apparatus to produce a computer-implemented process, so that the instructions executed on the computer or other programmable data processing apparatus provide a means for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1steps of a function specified in the one block or multiple blocks.

[0114] The above describes the basic principles of the present application in combination with specific embodiments, but it needs to be pointed out that the advantages, benefits, effects and the like mentioned in the present application are only examples and not limitations, and these advantages, benefits, effects and the like cannot be considered as necessary for each embodiment of the present application. In addition, the above disclosed specific details are only for the purpose of example and understanding, and not for the purpose of limitation, and the above details do not limit the present application to be necessarily implemented with the above specific details.

[0115] The block diagrams of the devices, apparatuses, equipment, systems involved in the present application are only illustrative examples and are not intended to require or imply the connection, arrangement, configuration as shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, equipment, systems can be connected, arranged, configured in any manner. Words such as "include", "contain", "have" and the like are open-ended words, which mean "including but not limited to", and can be used interchangeably. The words "or" and "and" used herein mean the word "and / or", and can be used interchangeably unless the context clearly indicates otherwise. The word "such as" used herein means the phrase "such as but not limited to", and can be used interchangeably.

[0116] It also needs to be pointed out that in the devices, equipment and methods of the present application, each component or each step can be decomposed and / or recombined. These decompositions and / or recombinations should be considered as equivalent solutions of the present application.

[0117] The above description of the disclosed aspects is provided so that any person skilled in the art can make or use the present application. Various modifications to these aspects will be apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of the present application. Therefore, the present application is not intended to be limited to the aspects shown herein, but is intended to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0118] Finally, it should be noted that, in the description above, relative terms such as first and second, etc. are used merely to distinguish one entity or action from another, without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises", "comprising", or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0119] The foregoing description has been presented for the purposes of illustration and description. Furthermore, the description is not intended to limit the embodiments of the application to the form disclosed herein. Although various example aspects and embodiments have been discussed above, those of ordinary skill in the art will appreciate a variety of modifications, alternatives, permutations, additions, and sub-combinations, which fall within the scope of the application.

Claims

1. A low temperature medicine adding method for plaster making, characterized by, The method comprises: acquiring a state monitoring video of the turpentine mixture collected by a camera for a predetermined time period; extracting a plurality of state monitoring key frames from the state monitoring video of the turpentine mixture; passing the plurality of state monitoring key frames through a convolutional neural network model comprising a deep-shallow feature fusion module to obtain a plurality of turpentine mixture state feature matrices; calculating a transition matrix between each adjacent two of the plurality of turpentine mixture state feature matrices to obtain a plurality of transition matrices; passing the plurality of transition matrices through a convolutional neural network model using a three-dimensional convolution kernel after being aggregated into a three-dimensional input tensor to obtain a classification feature map; and passing the classification feature map through a classifier to obtain a classification result, wherein the classification result is used to indicate whether the turpentine can be dropped into water at the current time point. The method comprises: extracting a shallow layer feature map from a shallow layer of the convolutional neural network model comprising a deep-shallow feature fusion module; extracting a deep layer feature map from a deep layer of the convolutional neural network model comprising a deep-shallow feature fusion module; fusing the shallow layer feature map and the deep layer feature map using a deep-shallow feature fusion module of the convolutional neural network model comprising a deep-shallow feature fusion module to obtain a fused feature map; and performing global mean pooling on the fused feature map along the channel dimension to obtain the plurality of turpentine mixture state feature matrices. The method comprises: calculating a transition matrix between each adjacent two of the plurality of turpentine mixture state feature matrices to obtain a plurality of transition matrices, comprising: calculating a transition matrix between each adjacent two of the plurality of turpentine mixture state feature matrices to obtain a plurality of transition matrices using the following transition formula: wherein the transition formula is: wherein, and denotes each adjacent pair of the plurality of dammar oil mixture state feature matrices, denotes the plurality of transition matrices, denotes matrix multiplication.

2. The low temperature medicine adding method for plaster manufacturing according to claim 1, wherein The method comprises:

3. The low-temperature medicine adding method for plaster manufacturing according to claim 2, characterized by, performing three-dimensional convolution processing, mean pooling processing, and nonlinear activation processing on the three-dimensional input tensor based on a three-dimensional convolution kernel in the forward propagation of the convolutional neural network model using a three-dimensional convolution kernel to obtain the classification feature map from the output of the last layer of the convolutional neural network model using a three-dimensional convolution kernel, wherein the input of the first layer of the convolutional neural network model using a three-dimensional convolution kernel is the three-dimensional input tensor. The method further comprises training the convolutional neural network model comprising a deep-shallow feature fusion module, the convolutional neural network model using a three-dimensional convolution kernel, and the classifier. The method comprises: acquiring training data, wherein the training data comprises state training monitoring videos of the turpentine mixture for a predetermined time period, and a true value of whether the turpentine can be dropped into water at the current time point; extracting a plurality of training state monitoring key frames from the state training monitoring videos of the turpentine mixture; passing the plurality of training state monitoring key frames through the convolutional neural network model comprising the deep and shallow feature fusion module to obtain a plurality of training turpentine mixture state feature matrices; calculating a transition matrix between each adjacent two of the plurality of training turpentine mixture state feature matrices to obtain a plurality of training transition matrices; passing the plurality of training transition matrices through the convolutional neural network model using a three-dimensional convolution kernel after the plurality of training transition matrices are aggregated into a three-dimensional input tensor to obtain a training classification feature map; performing class Fourier scale domain probability correction on the training classification feature map to obtain an optimized training classification feature map; passing the optimized training classification feature map through the classifier to obtain a classification loss function value; and training the convolutional neural network model comprising the deep and shallow feature fusion module, the convolutional neural network model using a three-dimensional convolution kernel, and the classifier by taking the classification loss function value as a loss function value and performing back propagation of gradient descent.

4. The low temperature medicine adding method for plaster manufacturing according to claim 3, wherein performing class Fourier scale domain probability correction on the training classification feature map to obtain an optimized training classification feature map, comprising: performing class Fourier scale domain probability correction on the training classification feature map to obtain the optimized training classification feature map according to the following optimization formula: wherein the optimization formula is: wherein, is a feature value of a (i, j)th position of the training classification feature map, , , , are a height, a width, and a number of channels of the training classification feature map, respectively, and , , are hyperparameters for scale adjustment, represents a calculation of a natural exponential function value with a number as a power, is a feature value of a (i, j)th position of the optimized training classification feature map. ​ 5. The low temperature medicine adding method for plaster manufacturing according to claim 4, wherein passing the optimized training classification feature map through the classifier to obtain a classification loss function value, comprising: the classifier processing the optimized training classification feature map according to the following classification formula to generate a classification result, wherein the classification formula is: wherein denotes projecting the optimized training classification feature map into a vector, to is a weight matrix, to denotes a bias matrix; and calculating a cross-entropy value between the classification result and a true value as the classification loss function value.

6. A low temperature medicine adding system for plaster manufacturing, characterized by comprising: comprising: a video acquisition module configured to acquire a state monitoring video of a turpentine mixture collected by a camera for a predetermined time period; a key frame acquisition module configured to extract a plurality of state monitoring key frames from the state monitoring video of the turpentine mixture; a deep and shallow fusion module configured to pass the plurality of state monitoring key frames through a convolutional neural network model comprising a deep and shallow feature fusion module to obtain a plurality of turpentine mixture state feature matrices; a transition matrix calculation module configured to calculate a transition matrix between each adjacent two of the plurality of turpentine mixture state feature matrices to obtain a plurality of transition matrices; a three-dimensional convolution module configured to pass the plurality of transition matrices through a convolutional neural network model using a three-dimensional convolution kernel after the plurality of transition matrices are aggregated into a three-dimensional input tensor to obtain a classification feature map; and a turpentine control result generation module configured to pass the classification feature map through a classifier to obtain a classification result, the classification result being used to indicate whether the turpentine can be dropped into water at a current time point. the deep and shallow fusion module comprises: a shallow extraction unit configured to extract a shallow feature map from a shallow layer of the convolutional neural network model comprising the deep and shallow feature fusion module; a deep extraction unit configured to extract a deep feature map from a deep layer of the convolutional neural network model comprising the deep and shallow feature fusion module; ​ a fusion unit configured to fuse the shallow feature map and the deep feature map using a deep-shallow feature fusion module of the convolutional neural network model to obtain a fused feature map; and a pooling unit configured to perform global mean pooling on the fused feature map along a channel dimension to obtain the plurality of balsam mixture state feature matrices; the transition matrix calculation module is configured to: calculate a transition matrix between each two adjacent balsam mixture state feature matrices in the plurality of balsam mixture state feature matrices according to a transition formula to obtain a plurality of transition matrices; wherein the transition formula is: wherein, and denotes each adjacent pair of dammar oil mixture state feature matrices in the plurality of dammar oil mixture state feature matrices, denotes the plurality of transition matrices, denotes matrix multiplication.

Images