An intelligent traditional Chinese medicine prescription configuration method, device and computer storage medium

Abstract

The application provides a smart traditional Chinese medicine prescription configuration method and device and a computer storage medium. Through picture collection of traditional Chinese medicinal materials, the name and efficacy of the traditional Chinese medicinal materials can be conveniently queried, and a corresponding prescription can be intelligently prepared according to the existing traditional Chinese medicinal materials. For a user who is not familiar with the pharmacology of traditional Chinese medicine, a symptomatic prescription can be conveniently configured by the user. Through a first recognition model, multiple medicinal material objects in the collected picture can be recognized at the same time. A first recognition model is set for each recognized object. After the characteristics of the object are input into the first recognition model composed of a neural network model, all medicinal material information in the picture is obtained. The obtained medicinal material information is input into a second recognition model. The second recognition model is used to analyze the prescription characteristics of the traditional Chinese medicinal material object, to obtain a probability value of a prescription type corresponding to the traditional Chinese medicinal material object, and to take a type with a probability value greater than a preset value as a prescription type of the object to be analyzed.

Description

Technical Field

[0001] This invention belongs to the field of information retrieval, specifically involving image recognition of Chinese medicinal materials, and on this basis, the configuration of intelligent Chinese medicinal prescriptions based on machine learning. Background Technology

[0002] Currently, knowledge of traditional Chinese medicine (TCM) herbs is not effectively popularized among the general public. Herbs typically obtained from hospital prescriptions or purchased from TCM markets are often packaged together in a mixed manner, and their appearance, properties, and other details are unknown to the average person. Whether herbs are lost or surplus, non-TCM professionals cannot make their own judgments, nor do they know what prescriptions can be made with these herbs, their efficacy, or the diseases they can treat. Therefore, after bringing herbs home, ordinary people usually do not check them themselves, often resulting in omissions or insufficient quantities of herbs during the preparation process, thus preventing the medicine from achieving its full effect. Summary of the Invention

[0003] In view of the shortcomings of the prior art, the present invention proposes an intelligent method for preparing traditional Chinese medicine prescriptions, an electronic device, and a computer storage medium. The purpose is to take pictures of the obtained Chinese medicinal materials, upload them, identify them, and output information such as the name and properties of the Chinese medicinal materials based on the identification results. Then, based on this, an usable prescription is automatically prepared, and the name, efficacy, and other information of the corresponding prescription are returned to the user.

[0004] The detailed technical solution of this invention is as follows:

[0005] S1. Identify the image uploaded by the user, wherein the image includes one or more Chinese medicinal materials; perform preprocessing and processing on the image, including signal enhancement, smoothing, and filtering;

[0006] S2. The image is identified, and image segmentation and multi-object detection are performed. The multi-object detection uses the YOLOv3 algorithm to obtain each target object in the image. Each target object corresponds to each Chinese medicinal herb object in the image.

[0007] S3. Extract the features of each target object, input the features of the target object into the first recognition model corresponding to the target object, wherein the number of target objects is the same as the number of first recognition models, and output the recognition result based on all the first recognition models, wherein the recognition result corresponds to one or more Chinese medicinal materials information; when the output recognition result is multiple Chinese medicinal materials information, the same Chinese medicinal materials information is clustered to remove duplicates;

[0008] S4. When the number of types of Chinese medicinal materials output in step S3 is greater than or equal to the preset value M, step S5 is executed. When the number of types of Chinese medicinal materials output is less than the preset value M, the Chinese medicinal materials information in the recognition result is directly output to the user, and a prompt is made that the prescription cannot be configured.

[0009] S5. Input the identified information of various Chinese medicinal materials into the second recognition model to obtain the probability that the information of the various Chinese medicinal materials can be combined into a prescription type. If the probability is greater than a set threshold, output the corresponding prescription type to the user. The prescription type includes prescription name, information of Chinese medicinal materials, proportion of medicinal materials, and efficacy of prescription.

[0010] Furthermore, before step S1 is executed, a knowledge graph database of Chinese medicinal materials is built in the background database to collect the shape, name, efficacy of various Chinese medicinal materials, as well as the name, ingredients, ratio, and efficacy of various Chinese medicine prescriptions.

[0011] Furthermore, during step S1, when recognizing images uploaded by the user, if the user uploads multiple images, it is possible to select that after all multiple images have been processed and multiple objects in the multiple images have been obtained, the multiple objects are then uniformly input into the first recognition model, and the recognition result is uniformly output.

[0012] Furthermore, in step S2, the image segmentation uses a histogram-based adaptive thresholding method to segment the foreground and background of the image. The threshold for each pixel is determined by a neighborhood window centered on itself, using the median mean or Gaussian convolution as the threshold.

[0013] Furthermore, in step S3, the features include color, shape, area, edge gear features, etc.

[0014] Furthermore, in step S3, the features of each target object are extracted and input into the first recognition model corresponding to the target object. The recognition result includes the name and efficacy of the target object.

[0015] The specific construction method of the first recognition model is as follows: a convolutional neural network is constructed, four training images of each medicinal material at the same resolution (overall, local, front, and back sides) are collected, medicinal material classification labels are set for the training images, the four training images of each medicinal material are preprocessed, the four preprocessed training images of each medicinal material are used as a set of input values, the four training images are input into the convolutional neural network in parallel, the medicinal material classification labels are used as the output values ​​of the convolutional neural network, and the convolutional neural network is trained to obtain a trained convolutional neural network model; the convolutional neural network model structure includes a convolutional layer, a pooling layer, and a fully connected layer, the pooling layer is connected to the convolutional layer, the pooling layer is connected to the fully connected layer, and each output node on the final fully connected layer is the medicinal material attribute probability of the recognized medicinal material object;

[0016] The convolutional neural network model includes convolutional layer 1, pooling layer 1, convolutional layer 2, pooling layer 2, convolutional layer 3, convolutional layer 4, pooling layer 3, fully connected layer 1, and fully connected layer 2. Convolutional layer 1 has a kernel size of 11×11, 3 color channels, 64 kernels, and a stride of 4×4. Each 4×4 region of the image is sampled only once, with a horizontal and vertical interval of 4. Each sample uses an 11×11 kernel. Pooling layer 1 has a pooling size of 3×3 and a sampling stride of 2×2. The convolutional layer... Layer 2 has a 5×5 kernel size, 64 input channels, 192 kernels, and a stride of 1; Pooling layer 2 has a 3×3 pooling size and a 2×2 sampling stride; Convolutional layer 3 has a 3×3 kernel size, 192 input channels, 384 kernels, and a stride of 1; Convolutional layer 4 has a 3×3 kernel size, 384 input channels, 256 kernels, and a stride of 1; Pooling layer 3 has a 3×3 pooling size and a 2×2 sampling stride.

[0017] The calculation formula for the convolutional layer is as follows:

[0018]

[0019] Where l is the number of layers, M is the convolutional kernel that connects the features of layer l to the features of layer (l-1). l-1 The input features selected for the (l-1)th layer is the bias, and f is the nonlinear activation function.

[0020] The pooling layer compresses pixels the size of a filter into a single pixel, compressing the feature map size and extracting the main features. Average pooling is then used to extract the average value of the features within the window. The average pooling formula is as follows:

[0021]

[0022] in, For the j-th output feature, For input features, M i It is the set of features selected from the (l-1)th layer feature map that make up the output features of the lth layer.

[0023] Fully connected layer: Transforms the input feature map matrix into a column vector, then multiplies it by weights, which must include all pixels, to obtain the classification result. A softmax classifier is used for feature recognition and classification. The softmax loss function is defined as follows:

[0024]

[0025] Where θ represents the training parameters of the neural network, and X... j Let y represent the j-th sample. j This represents the probability of the prescription type corresponding to the j-th sample.

[0026] Furthermore, in step S5, the identified information on the various Chinese medicinal herbs is input into the second recognition model. The specific steps for obtaining the prescription using the second recognition model are as follows:

[0027] A. Sample training: Collect prescriptions made of different Chinese medicinal materials in different proportions as samples, and label each sample with the prescription name as a sample label;

[0028] B. Feature extraction: Extract the ingredients and proportions of the prescription for each sample, and combine them to obtain the features of each sample;

[0029] C. Constructing a neural network: Construct a neural network with a Softmax classifier as the output layer;

[0030] Specifically, the following steps are taken: Define a Softmax loss function, use the ingredients and proportions of each prescription as sample data, train the neural network, output the probability value of each sample corresponding to each prescription type, update the training parameters of the neural network each time training, and use the training parameters that minimize the Softmax loss function as the final parameters to obtain the prescription analysis model.

[0031] D. Model Application: Obtain the Chinese medicinal material object output by the first recognition model, analyze the prescription characteristics of the Chinese medicinal material object using the second recognition model, obtain the probability value of the prescription type corresponding to the Chinese medicinal material object, and take the type with a probability value greater than a preset value as the prescription type of the object to be analyzed.

[0032] In step S5, if the probability value of the prescription type corresponding to the Chinese medicinal material object is lower than a preset threshold, a corresponding prompt message will be output to the user, such as "Sorry, your medicinal materials cannot be configured to generate the corresponding prescription".

[0033] An electronic device includes a processor, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to execute the intelligent traditional Chinese medicine prescription configuration method of steps S1-S5 by the one or more processors.

[0034] A computer storage medium for storing a computer program to execute the method for preparing an intelligent traditional Chinese medicine prescription in steps S1-S5.

[0035] Compared with traditional methods for identifying medicinal materials, this invention creatively proposes the following:

[0036] By collecting images of Chinese medicinal herbs, users can easily search for their names, effects, and other information. Based on the available Chinese medicinal herbs, corresponding prescriptions can be intelligently formulated. For users who are not familiar with the pharmacology of Chinese medicine, they can easily formulate prescriptions that are appropriate for their symptoms.

[0037] The first recognition model can simultaneously identify multiple medicinal objects in the collected image. The first recognition model is set for each object to be identified. After the features of the object are input into the first recognition model composed of a neural network model, the information of all medicinal objects in the image can be obtained.

[0038] The obtained medicinal material information is then input into the second recognition model. The second recognition model is used to analyze the prescription characteristics of the Chinese medicinal material object to obtain the probability value of the prescription type corresponding to the Chinese medicinal material object. The type with a probability value greater than a preset value is taken as the prescription type of the object to be analyzed.

[0039] By combining multi-object detection in image recognition with machine learning algorithms, it can not only quickly and accurately identify one or more images of Chinese medicinal herbs simultaneously, but also automatically configure usable prescriptions through machine learning algorithms. This allows users to make their own judgments when herbs are missing or the dosage is incorrect, correct the errors in time, and avoid taking the herbs directly if they are incorrectly matched. Alternatively, when there are surplus Chinese medicinal herbs at home, they can also make timely judgments and reuse the remaining herbs to avoid waste. Attached Figure Description

[0040] Figure 1 This is a flowchart of the intelligent traditional Chinese medicine prescription preparation method of the present invention. Detailed Implementation

[0041] The present application will now be described clearly and completely with reference to the embodiments and accompanying drawings.

[0042] The specific method includes the following steps:

[0043] S1. Identify the image uploaded by the user, wherein the image includes one or more Chinese medicinal materials; perform preprocessing and processing on the image, including signal enhancement, smoothing, and filtering;

[0044] Furthermore, before this step is executed, a knowledge graph database of Chinese medicinal materials is built in the background database to collect the shape, name, and efficacy of various Chinese medicinal materials, as well as the name, ingredients, proportions, and efficacy of various Chinese medicine prescriptions.

[0045] Furthermore, in this step, when recognizing images uploaded by the user, if the user uploads multiple images, it is possible to select that after all multiple images have been processed and multiple objects in the multiple images have been obtained, the multiple objects are then uniformly input into the first recognition model, and the recognition result is uniformly output.

[0046] S2. The image is identified, and image segmentation and multi-object detection are performed. The multi-object detection uses the YOLOv3 algorithm to obtain each target object in the image. Each target object corresponds to each Chinese medicinal herb object in the image.

[0047] Furthermore, in this step, the image segmentation uses a histogram-based adaptive thresholding method to segment the foreground and background of the image. The threshold for each pixel is determined by a neighborhood window centered on itself, using the median mean or Gaussian convolution as the threshold.

[0048] S3. Extract the features of each target object, input the features of the target object into the first recognition model corresponding to the target object, wherein the number of target objects is the same as the number of first recognition models, and output the recognition result based on all the first recognition models, wherein the recognition result corresponds to one or more Chinese medicinal materials information; when the output recognition result is multiple Chinese medicinal materials information, the same Chinese medicinal materials information is clustered to remove duplicates;

[0049] Furthermore, in this step, the features include color, shape, area, edge gear features, etc.

[0050] Furthermore, in this step, the features of each target object are extracted and input into the first recognition model corresponding to the target object. The recognition result includes the name and efficacy of the target object.

[0051] The specific construction method of the first recognition model is as follows: a convolutional neural network is constructed, four training images of each medicinal material at the same resolution (overall, local, front, and back sides) are collected, medicinal material classification labels are set for the training images, the four training images of each medicinal material are preprocessed, the four preprocessed training images of each medicinal material are used as a set of input values, the four training images are input into the convolutional neural network in parallel, the medicinal material classification labels are used as the output values ​​of the convolutional neural network, and the convolutional neural network is trained to obtain a trained convolutional neural network model; the convolutional neural network model structure includes a convolutional layer, a pooling layer, and a fully connected layer, the pooling layer is connected to the convolutional layer, the pooling layer is connected to the fully connected layer, and each output node on the final fully connected layer is the medicinal material attribute probability of the recognized medicinal material object;

[0052] The convolutional neural network model includes convolutional layer 1, pooling layer 1, convolutional layer 2, pooling layer 2, convolutional layer 3, convolutional layer 4, pooling layer 3, fully connected layer 1, and fully connected layer 2. Convolutional layer 1 has a kernel size of 11×11, 3 color channels, 64 kernels, and a stride of 4×4. Each 4×4 region of the image is sampled only once, with a horizontal and vertical interval of 4. Each sample uses an 11×11 kernel. Pooling layer 1 has a pooling size of 3×3 and a sampling stride of 2×2. The convolutional layer... Layer 2 has a 5×5 kernel size, 64 input channels, 192 kernels, and a stride of 1; Pooling layer 2 has a 3×3 pooling size and a 2×2 sampling stride; Convolutional layer 3 has a 3×3 kernel size, 192 input channels, 384 kernels, and a stride of 1; Convolutional layer 4 has a 3×3 kernel size, 384 input channels, 256 kernels, and a stride of 1; Pooling layer 3 has a 3×3 pooling size and a 2×2 sampling stride.

[0053] The calculation formula for the convolutional layer is as follows:

[0054]

[0055] Where l is the number of layers, M is the convolutional kernel that connects the features of layer l to the features of layer (l-1). l-1 The input features selected for the (l-1)th layer is the bias, and f is the nonlinear activation function.

[0056] The pooling layer compresses pixels the size of a filter into a single pixel, compressing the feature map size and extracting the main features. Average pooling is then used to extract the average value of the features within the window. The average pooling formula is as follows:

[0057]

[0058] in, For the j-th output feature, For input features, M i It is the set of features selected from the (l-1)th layer feature map that make up the output features of the lth layer.

[0059] Fully connected layer: Transforms the input feature map matrix into a column vector, then multiplies it by weights, which must include all pixels, to obtain the classification result. A softmax classifier is used for feature recognition and classification. The softmax loss function is defined as follows:

[0060]

[0061] Where θ represents the training parameters of the neural network, and X... j Let y represent the j-th sample. j This represents the probability of the prescription type corresponding to the j-th sample.

[0062] S4. When the number of types of Chinese medicinal materials output in step S3 is greater than or equal to the preset value M, step S5 is executed. When the number of types of Chinese medicinal materials output is less than the preset value M, the Chinese medicinal materials information in the recognition result is directly output to the user, and a prompt is made that the prescription cannot be configured.

[0063] S5. Input the identified information of the various Chinese medicinal materials into the second recognition model to obtain the probability that the information of the various Chinese medicinal materials can be combined into a prescription type. If the probability is greater than a set threshold, output the corresponding prescription type to the user. The prescription type includes prescription name, information of Chinese medicinal materials, proportion of medicinal materials, and efficacy of prescription.

[0064] Furthermore, in this step, the identified information on the various Chinese medicinal herbs is input into the second recognition model. The specific steps for obtaining the prescription using the second recognition model are as follows:

[0065] A. Sample training: Collect prescriptions made of different Chinese medicinal materials in different proportions as samples, and label each sample with the prescription name as a sample label;

[0066] B. Feature extraction: Extract the ingredients and proportions of the prescription for each sample, and combine them to obtain the features of each sample;

[0067] C. Constructing a neural network: Construct a neural network with a Softmax classifier as the output layer;

[0068] Specifically, the following steps are taken: Define a Softmax loss function, use the ingredients and proportions of each prescription as sample data, train the neural network, output the probability value of each sample corresponding to each prescription type, update the training parameters of the neural network each time training, and use the training parameters that minimize the Softmax loss function as the final parameters to obtain the prescription analysis model.

[0069] D. Model Application: Obtain the Chinese medicinal material object output by the first recognition model, analyze the prescription characteristics of the Chinese medicinal material object using the second recognition model, obtain the probability value of the prescription type corresponding to the Chinese medicinal material object, and take the type with a probability value greater than a preset value as the prescription type of the object to be analyzed.

[0070] Furthermore, in this step, if the probability value of the prescription type corresponding to the Chinese medicinal material object is lower than a preset threshold, a corresponding prompt message will be output to the user, such as "Sorry, your medicinal materials cannot be configured into the corresponding prescription".

[0071] And an electronic device, including a processor, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors as described in steps S1-S5, and a computer storage medium for configuring intelligent traditional Chinese medicine prescriptions, for storing computer programs to execute the above-described configuration method based on intelligent traditional Chinese medicine prescriptions.

[0072] Those skilled in the art can make various modifications and variations to this invention without departing from its scope.

Claims

1. A method for preparing intelligent traditional Chinese medicine prescriptions, comprising the following steps: S1. Identify the image uploaded by the user, wherein the image includes one or more Chinese medicinal materials; perform preprocessing and processing on the image, including signal enhancement, smoothing, and filtering; S2. The image is identified, and image segmentation and multi-object detection are performed; wherein, the multi-object detection uses the YOLOv3 algorithm to obtain each target object in the image, and each target object corresponds to each Chinese medicinal herb object in the image; S3. Extract the features of each target object, input the features of the target object into the first recognition model corresponding to the target object, wherein the number of target objects is the same as the number of first recognition models, and output the recognition result based on all the first recognition models, wherein the recognition result corresponds to one or more Chinese medicinal materials information; when the output recognition result is multiple Chinese medicinal materials information, the same Chinese medicinal materials information is clustered to remove duplicates; S4. When the number of types of Chinese medicinal materials output in step S3 is greater than or equal to the preset value M, step S5 is executed. When the number of types of Chinese medicinal materials output is less than the preset value M, the Chinese medicinal materials information in the recognition result is directly output to the user, and a prompt is made that the prescription cannot be configured. S5. Input the identified information of various Chinese medicinal materials into the second recognition model to obtain the probability that the information of the various Chinese medicinal materials can be combined into a prescription type. If the probability is greater than a set threshold, output the corresponding prescription type to the user. The prescription type includes prescription name, information of Chinese medicinal materials, proportion of medicinal materials, and efficacy of prescription. The specific steps for obtaining prescriptions using the second recognition model are as follows: A. Sample training: Collect prescriptions composed of different Chinese medicinal materials in different proportions as samples, and label each sample with the prescription name as a sample label; B. Feature extraction: Extract the prescription components and proportions of each sample, and combine them to obtain the features of each sample; C. Neural network construction: Construct a neural network with a Softmax classifier as the output layer; specifically: define a Softmax loss function, use the prescription components and proportions of each sample as sample data, train the neural network, output the probability value of each sample corresponding to each prescription type, update the training parameters of the neural network each time training, and use the training parameters that minimize the Softmax loss function as the final parameters to obtain the prescription analysis model; D. Model application: Obtain the Chinese medicinal material objects output by the first recognition model, use the second recognition model to analyze the prescription features of the Chinese medicinal material objects, obtain the probability value of the prescription type corresponding to the Chinese medicinal material objects, and take the type with a probability value greater than a preset value as the prescription type of the object to be analyzed.

2. According to the method described in claim 1, before step S1 is executed, a knowledge graph database of Chinese medicinal materials is built in the background database to collect the shape, name, efficacy of various Chinese medicinal materials, as well as the name, ingredients, proportions and efficacy of various Chinese medicine prescriptions.

3. According to the method of claim 1, in step S1, when recognizing the images uploaded by the user, if the user uploads multiple images, it is possible to select that after all the multiple images have been processed and multiple objects in the multiple images have been obtained, the multiple objects are then uniformly input into the first recognition model and the recognition result is uniformly output.

4. The method according to claim 1, wherein in step S2, the image segmentation employs a histogram-based adaptive thresholding method to segment the foreground and background of the image, wherein, The threshold for each pixel is determined by a neighborhood window centered on itself, using the median mean or Gaussian convolution as the threshold.

5. The method according to claim 1, wherein in step S3, the features include color, shape, area, and edge gear features.

6. The method according to claim 1, in step S3, the features of each target object are extracted and the features of the target object are input into the first recognition model corresponding to the target object, and the recognition result includes the name and efficacy of the target object; The specific construction method of the first recognition model is as follows: Convolutional neural network is constructed, four training images of each medicinal material at the same resolution (overall, local, front, and back) are collected, medicinal material classification labels are set for the training images, the four training images of each medicinal material are preprocessed, the four preprocessed training images of each medicinal material are used as a set of input values, the four training images are input into the convolutional neural network in parallel, the medicinal material classification labels are used as the output values ​​of the convolutional neural network, and the convolutional neural network is trained to obtain a trained convolutional neural network model; the convolutional neural network model structure includes a convolutional layer, a pooling layer, and a fully connected layer, the pooling layer is connected to the convolutional layer, the pooling layer is connected to the fully connected layer, and each output node on the final fully connected layer is the probability of the medicinal material attribute of the recognized medicinal material object; The convolutional neural network model includes convolutional layer 1, pooling layer 1, convolutional layer 2, pooling layer 2, convolutional layer 3, convolutional layer 4, pooling layer 3, fully connected layer 1, and fully connected layer 2. Convolutional layer 1 has a kernel size of 11×11, 3 color channels, 64 kernels, and a stride of 4×4. Each 4×4 region of the image is sampled only once, with a horizontal and vertical interval of 4. Each sample uses an 11×11 kernel. Pooling layer 1 has a pooling size of 3×3 and a sampling stride of 2×2. The convolutional layer... Layer 2 has a 5×5 kernel size, 64 input channels, 192 kernels, and a stride of 1; Pooling layer 2 has a 3×3 pooling size and a 2×2 sampling stride; Convolutional layer 3 has a 3×3 kernel size, 192 input channels, 384 kernels, and a stride of 1; Convolutional layer 4 has a 3×3 kernel size, 384 input channels, 256 kernels, and a stride of 1; Pooling layer 3 has a 3×3 pooling size and a 2×2 sampling stride. in, The formula for calculating convolutional layers is shown below: Where l is the number of layers, M is the convolutional kernel that connects the features of layer l to the features of layer (l-1). l-1 The input features selected for the (l-1)th layer Here, f is the bias, and f is the nonlinear activation function; The pooling layer compresses pixels the size of a filter into a single pixel, compressing the feature map size and extracting the main features. Average pooling is then used to extract the average value of the features within the window. The average pooling formula is as follows: in, For the j-th output feature, For input features, M i The set of features selected from the (l-1)th layer feature map that constitutes the output features of the lth layer. Fully connected layer: Transforms the input feature map matrix into a column vector, then multiplies it by weights, which must include all pixels to obtain the classification result; a softmax classifier is used for feature recognition and classification.

7. According to the method of claim 4, in step S5, if the probability value of the prescription type corresponding to the Chinese medicinal material object is lower than a preset threshold, then a corresponding prompt message is output to the user.

8. An electronic device comprising a processor, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor as described in any one of claims 1-7.

9. A computer storage medium for storing a computer program to execute the intelligent traditional Chinese medicine prescription preparation method as described in any one of claims 1-7.

Images