Traditional Chinese medicine teacher following learning assisting method and system, electronic equipment and storage medium

Abstract

The invention provides a traditional Chinese medicine teacher-following learning assistance method and system, electronic equipment and a storage medium. The method comprises the following steps: identifying and displaying each medicine name in a prescription image through an OCR technology; in response to a selection operation aiming at the name of the target medicine, searching all processing methods of the target medicine and the medicine efficacy of the target medicine under each processing method in the medicine processing knowledge graph, and displaying; and in response to a selection operation for the target processing method, searching and displaying the drug characteristic and instruction data of the target drug under the target processing method in the drug characteristic and instruction database. In a teacher following learning process, through carrying out OCR identification on a prescription image corresponding to a prescription set by a teacher and inquiring a drug processing knowledge graph and a drug characteristic and use instruction database, drugs in the prescription and a processing method, efficacy, characteristics and use instructions thereof can be displayed in an associated manner by taking the prescription as a main body; and a real-time, comprehensive and systematic learning effect is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of medicinal material data processing, and in particular, to a method, a system, an electronic device and a storage medium for assisting in learning from a traditional Chinese medicine master. Background Art

[0002] When following a master in a traditional Chinese medicine outpatient clinic, it is difficult for students to communicate instantaneously with the teacher regarding the students' confusing questions. Students are unable to systematically master and review drugs and prescriptions, cannot comprehensively master the dosage and taboos of drugs, and it is also difficult to comprehensively learn modern pharmacological research and the experience of famous traditional Chinese medicine doctors. After the outpatient clinic, due to time and various other factors, some confusions are easily forgotten. Therefore, there is an urgent need for a Chinese medicine learning system that can provide instant and comprehensive learning to improve the clinical ability and learning efficiency of traditional Chinese medicine practitioners. Summary of the Invention

[0003] In view of this, the purpose of the present invention is to provide a method, a system, an electronic device and a storage medium for assisting in learning from a traditional Chinese medicine master, so as to achieve instant, comprehensive and systematic learning and improve the learning effect.

[0004] In a first aspect, the present invention provides a method for assisting in learning from a traditional Chinese medicine master, which is applied to a client for assisting in learning from a traditional Chinese medicine master, and includes: Obtain a prescription image taken during the process of learning from a master; Identify the names of each drug in the prescription image through OCR technology and display the names of each drug; In response to a selection operation for at least one drug name among the names of each drug, determine at least one drug name as the target drug name; Search in a pre-constructed knowledge graph of drug processing for all processing methods of the target drug corresponding to the target drug name and the drug efficacy of the target drug under each processing method, and display all processing methods of the target drug and the drug efficacy of the target drug under each processing method; In response to a selection operation for at least one processing method among all processing methods of the target drug, determine at least one processing method as the target processing method; Search in a pre-constructed database of drug characteristics and usage instructions for the drug characteristics and usage instruction data of the target drug under the target processing method, and display the drug characteristics and usage instruction data of the target drug under the target processing method; the drug characteristics and usage instruction data at least include drug nature and flavor, meridian tropism, common dosage, medication taboos, decoction method, and whether it is both a medicine and a food.

[0005] Optionally, identifying the names of each drug in the prescription image through OCR technology includes: Perform image preprocessing on the prescription image to obtain a binary image; Use the bwlabel function to label connected components in the binary image; each connected component corresponds to a potential character; Use the regionprops function to identify the attributes of each connected component; the attributes include the bounding box and the centroid; Locate each character based on the centroid of each connected component, and segment the characters using the bounding box of each connected component; Input the segmented characters into a pre-trained character recognition model to obtain the character recognition result; Post-process the character recognition result to correct incorrect characters, and finally obtain each drug name.

[0006] Optionally, the image preprocessing of the prescription image to obtain a binary image includes: Adjust the brightness and contrast of the prescription image to obtain a first intermediate image; Remove noise in the first intermediate image based on a filter to obtain a second intermediate image; Perform skew correction on the second intermediate image to obtain a third intermediate image; Perform binary processing on the third intermediate image to obtain a binary image.

[0007] Optionally, the attributes of the connected component also include the area; after using the regionprops function to identify the attributes of each connected component, it further includes: Compare the area of each connected component with a preset area threshold respectively, and filter out the connected components with an area smaller than the preset area threshold.

[0008] Optionally, the construction process of the drug processing knowledge graph includes: Obtain a dataset related to drug processing methods; Perform data preprocessing on the dataset; the data preprocessing at least includes data cleaning, formatting, deduplication, error correction, and filling missing values; Use NLP technology to identify each entity in the preprocessed dataset and establish the relationships between each entity; the entities at least include the name of the medicinal material, the name of the disease, the processing method, and the drug efficacy; Construct a drug processing knowledge graph based on each entity and the relationships between each entity.

[0009] Optionally, the auxiliary method for traditional Chinese medicine follow-up learning provided by the present invention further includes: In response to the disease name input in the search box, search for classical diagnostic experience data and modern diagnostic experience data related to the disease name, and display the classical diagnostic experience data and modern diagnostic experience data.

[0010] Optionally, the auxiliary method for traditional Chinese medicine follow-up learning provided by the present invention further includes: In response to a target prescription entered in the search box, search for prescriptions in a pre-constructed prescription database that contain at least one of the drugs in the target prescription; Based on the drug names in the found prescriptions and the drug names in the target prescription, calculate the matching degree between each prescription and the target prescription; Screen out prescriptions with a matching degree not less than a preset matching threshold from each prescription as similar prescriptions to the target prescription, and display each similar prescription of the target prescription.

[0011] In a second aspect, the present invention provides a TCM teacher-following learning assistance system, which is applied to a TCM teacher-following learning assistance client and includes: An acquisition unit for acquiring the prescription image taken during the teacher-following learning process; An identification unit for identifying each drug name in the prescription image by OCR technology and displaying each drug name; A first search unit for, in response to a selection operation on at least one of the drug names, determining at least one drug name as a target drug name; searching in a pre-constructed drug processing knowledge graph for all processing methods of the target drug corresponding to the target drug name and the drug efficacy of the target drug under each processing method, and displaying all processing methods of the target drug and the drug efficacy of the target drug under each processing method; A second search unit for, in response to a selection operation on at least one of all the processing methods of the target drug, determining at least one processing method as a target processing method; searching in a pre-constructed drug property and usage instruction database for the drug property and usage instruction data of the target drug under the target processing method, and displaying the drug property and usage instruction data of the target drug under the target processing method; the drug property and usage instruction data at least includes drug nature and flavor, meridian tropism, common dosage, medication taboos, decocting method, and whether it is both a medicine and a food.

[0012] In a third aspect, the present invention provides an electronic device, including a processor, a communication interface, a memory, and a communication bus; wherein, the processor, the communication interface, and the memory complete communication with each other through the communication bus; The memory is used for storing a computer program; The processor is used for implementing the above-mentioned TCM teacher-following learning assistance method when executing the computer program stored on the memory.

[0013] In a fourth aspect, the present invention provides a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed by a processor, the above-mentioned TCM teacher-following learning assistance method is implemented.

[0014] The TCM apprenticeship learning assistance method, system, electronic device and storage medium provided by the embodiments of the present invention can, during the apprenticeship learning process, take pictures of the prescriptions written by the teacher at any time. By performing OCR recognition on the captured prescription images and querying the drug processing knowledge graph and the database of drug properties and usage instructions, it is possible to take the prescription as the main body and associate and display the drugs in the prescription with their processing methods, efficacy, properties and usage instructions, so as to achieve a real-time, comprehensive and systematic learning effect.

[0015] To make the above objects, features and advantages of the present invention more obvious and understandable, the following specifically enumerates preferred embodiments and, in conjunction with the accompanying drawings, gives a detailed description as follows. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the accompanying drawings required for use in the embodiments. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as limiting the scope. For those of ordinary skill in the art, without creative efforts, other related drawings can also be obtained based on these drawings.

[0017] Figure 1 Shows a flowchart of a TCM apprenticeship learning assistance method provided by an embodiment of the present invention; Figure 2 Shows a schematic diagram of the character recognition process of the network structure based on the Chinese character recognition model and the non-Chinese character recognition model provided by an embodiment of the present invention; Figure 3 Shows a schematic diagram of the structure of a TCM apprenticeship learning assistance system provided by an embodiment of the present invention; Figure 4 Shows a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0018] To make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only some of the embodiments of the present invention, rather than all of them. Usually, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the present invention to be protected, but only represents the selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0019] Considering that most of the traditional Chinese medicine learning-related software utilizes the aggregation of multiple single knowledge databases and realizes simple learning of single knowledge through simple search, it cannot effectively combine the diagnosed diseases, prescriptions, and processing methods, etc., and cannot achieve comprehensive and systematic learning, resulting in poor learning effects.

[0020] Based on this, the embodiments of the present invention provide a method for assisting in learning from a traditional Chinese medicine master, which will be described below through embodiments. The embodiments of the present invention provide a method for assisting in learning from a traditional Chinese medicine master, which is applied to a client for assisting in learning from a traditional Chinese medicine master, as Figure 1 shown. This method includes the following steps: Step S101: Obtain the prescription image taken during the process of learning from a master.

[0021] In the embodiments of the present invention, the client for assisting in learning from a traditional Chinese medicine master can be installed in a mobile terminal. By obtaining the camera permission of the mobile terminal, the client for assisting in learning from a traditional Chinese medicine master can call the camera of the mobile terminal to take the prescription opened by the master according to the patient's disease during the process of learning from a master, and thus the prescription image corresponding to the prescription can be obtained.

[0022] In a feasible implementation manner, during the process of calling the camera of the mobile terminal to take a picture of the prescription, the learner can manually adjust the shooting area range to limit the position area of the medicine in the prescription, so as to be able to take a prescription image with a simple background and accurate position, and further improve the efficiency and accuracy of subsequent medicine recognition.

[0023] Step S102: Identify the names of each medicine in the prescription image through OCR (Optical Character Recognition) technology and display the names of each medicine.

[0024] The specific recognition process will be described in the following embodiments and will not be elaborated here.

[0025] In a feasible implementation manner, after identifying the names of each medicine in the prescription image through OCR technology, the wrongly recognized medicine names can also be manually corrected to ensure the accuracy of the medicine names.

[0026] Step S103: In response to a selection operation for at least one medicine name among the names of each medicine, determine the at least one medicine name as the target medicine name.

[0027] In the embodiments of the present invention, the learner can perform a selection operation such as clicking or circle selection on the medicine names to be learned among the displayed names of each medicine to realize the selection of each medicine to be learned. The client for assisting in learning from a traditional Chinese medicine master identifies the names of each medicine selected by the learner as the target medicine names according to the learner's selection operation.

[0028] Step S104: Search for all the processing methods of the target drug corresponding to the target drug name and the drug efficacy of the target drug under each processing method in the pre-constructed drug processing knowledge graph, and display all the processing methods of the target drug and the drug efficacy of the target drug under each processing method.

[0029] In this step, after the TCM follow-up learning assistance client identifies each selected drug name as the target drug name according to the learner's selection operation, it automatically searches for all the processing methods of the target drug corresponding to the target drug name and the drug efficacy of the target drug under each processing method in the pre-constructed drug processing knowledge graph, and displays them. From the perspective of the learner, by selecting the drug name to be learned, the learner can obtain all the processing methods of the drug to be learned and the drug efficacy of the drug under each processing method. The operation is simple and a relatively comprehensive learning effect can be achieved.

[0030] The drug efficacy of the same drug under different processing methods has different emphases. For example, raw Ephedra has a strong effect of dispersing the lung qi and relieving the exterior syndrome. After being honey-fried, the dispersing effect is mild and the effect of relieving cough and asthma is enhanced; raw Rehmannia glutinosa is cold in nature and has the effect of cooling blood, while prepared Rehmannia glutinosa is warm in nature and has the effect of tonifying blood; raw Pollen Typhae has the effect of promoting blood circulation and removing stasis, while stir-fried with bran until charred can stop bleeding. The content of various TCM learning books is extensive, and the content results directly retrieved by traditional retrieval software are uneven. It is necessary to screen reliable information from a large amount of information, resulting in low learning efficiency and poor learning results. In the embodiments of the present application, through the pre-constructed drug processing knowledge graph, the transformation from multi-source heterogeneous data to multi-dimensional connections between carriers of different processing methods of various drugs and the drug efficacy of various drugs under different processing methods can be realized. Therefore, learners do not need to consult various TCM learning books, and by querying the drug processing knowledge graph through the TCM follow-up learning assistance client, they can realize a systematic and comprehensive learning of different processing methods of various drugs and the drug efficacy of various drugs under different processing methods. The operation is simple, the efficiency is high, and the learning results are good.

[0031] In a feasible implementation manner, the construction process of the drug processing knowledge graph includes the following steps: Step A: Obtain a data set related to drug processing methods.

[0032] In this step, a data set related to drugs, their processing methods, and drug efficacy can be obtained from various data sources such as various books on TCM theory (such as classic works "Huangdi Neijing", "Treatise on Febrile and Miscellaneous Diseases", etc.), network resources (such as TCM-related forums, databases, and academic papers, etc.).

[0033] Specifically, for non-digital resources such as books, OCR technology can be used for identification and conversion; for network resources, data collection can be used to obtain them.

[0034] Step B: Perform data preprocessing on the dataset.

[0035] In the embodiment of the present invention, data preprocessing at least includes data cleaning, formatting, deduplication, error correction, and filling missing values, etc. Through data preprocessing, unstructured data is converted into structured data.

[0036] Step C: Use NLP (Natural Language Processing) technology to identify each entity in the preprocessed dataset and establish the relationships between each entity.

[0037] In this step, NLP technology can be used to automatically identify entities such as the names of medicinal materials, disease names, processing methods, and drug effects, and establish the relationships between each entity, such as "Astragalus membranaceus is used to treat qi deficiency", "The steaming method can enhance the blood-tonifying effect of Rehmannia glutinosa", etc.

[0038] Step D: Construct a drug processing knowledge graph based on each entity and the relationships between each entity.

[0039] In this step, first select a suitable knowledge representation model, such as RDF (Resource Description Framework), OWL (Web Ontology Language), etc. to describe each entity and the relationships between each entity, and select a corresponding database system, such as the graph database Neo4j, the document database MongoDB, etc. for storage. Then design the architecture of the drug processing knowledge graph, including nodes (representing entities) and edges (representing relationships). In an example, a graphical interface tool can be used to assist in the design.

[0040] After the drug processing knowledge graph is constructed, corresponding application programs can be developed, such as query interfaces, recommendation systems, etc., so that the TCM follow-up learning assistance client can quickly access and utilize the information therein.

[0041] In addition, when displaying all the processing methods of the target drug and the drug effects of the target drug under each processing method, all the processing methods of the target drug and the drug effects of the target drug under each processing method can be displayed in a sequential manner.

[0042] In the embodiments of the present invention, by organically associating and displaying the medicine, its processing method and the efficacy of the medicine, taking traditional Chinese medicine processing as an important main body and reflecting it in medicine learning, learners can learn about the medicine more comprehensively and systematically. At the same time, instant learning can be realized, taking into account both learning efficiency and comprehensiveness.

[0043] Step S105: In response to a selection operation for at least one processing method among all the processing methods of the target medicine, determine the at least one processing method as the target processing method.

[0044] In the embodiments of the present invention, learners can perform a selection operation in any way such as clicking or selecting the processing methods to be learned among all the processing methods of the displayed target medicine, so as to select the processing methods to be learned. The auxiliary client for traditional Chinese medicine apprentice learning can identify each processing method selected by the learner as the target processing method according to the learner's selection operation.

[0045] Step S106: Search for the medicine characteristics and usage instructions data of the target medicine under the target processing method in the pre-constructed medicine characteristics and usage instructions database, and display the medicine characteristics and usage instructions data of the target medicine under the target processing method.

[0046] In this step, the medicine characteristics and usage instructions data at least include the nature and flavor of the medicine, meridian tropism, common dosage, medication taboos (such as the eighteen incompatible medicaments and the nineteen medicaments with mutual restraint), decocting methods, and whether it is both medicine and food.

[0047] In a feasible embodiment, if it is both medicine and food, a recommended daily consumption method can be generated and displayed together with the medicine characteristics and usage instructions data.

[0048] In the embodiments of the present invention, when displaying the medicine characteristics and usage instructions data of the target medicine under the target processing method, the medicine characteristics and usage instructions data can be displayed in a preset presentation manner. Among them, the preset presentation manner is at least one of the following: graphic form, table form, and text form.

[0049] In the embodiments of the present invention, taking the prescription opened by the teacher in each apprentice learning process as the main body, and associating and displaying the medicines in the prescription with their processing methods and efficacy, a real-time, comprehensive and systematic learning effect is achieved.

[0050] In addition, in order to improve the recognition accuracy of the medicine names in the prescription image, in the embodiments of the present application, the steps of identifying each medicine name in the prescription image by OCR technology are as follows: Step S102A: Perform image preprocessing on the prescription image to obtain a binary image.

[0051] In a feasible implementation, the image preprocessing of the prescription image to obtain a binary image includes: Adjust the brightness and contrast of the prescription image to obtain a first intermediate image. Among them, by adjusting the brightness and contrast of the image, the difference between characters and the background can be highlighted.

[0052] Remove the noise in the first intermediate image based on a filter to obtain a second intermediate image. In an example, the filter may adopt a Gaussian blur filter or a median filter.

[0053] Perform skew correction on the second intermediate image to obtain a third intermediate image. Among them, through skew correction, a more standard image can be provided, which is convenient for subsequent processing.

[0054] Perform binary processing on the third intermediate image to obtain a binary image.

[0055] Convert the prescription image into black and white through binary processing to distinguish the foreground (characters) from the background and simplify subsequent operations.

[0056] Step S102B: Use the bwlabel function to label the connected components in the binary image; among them, each connected component corresponds to a potential character.

[0057] Among them, when labeling the connected components, the 8-neighboring method can be used to find the connected components. The 8-neighboring method means that each pixel point is considered to be connected to its surrounding eight pixel points, including the neighbors in the vertical, horizontal, and diagonal directions. The following are the basic steps to implement this process: The first step: Preparation work Mark the characters in the binary prescription image as the foreground (represented by white, that is, the pixel value is 255), and the background is black (the pixel value is 0).

[0058] The second step: Initialization Create a label map with the same size as the prescription image, initialized to 0, for storing the connected component number to which each pixel point belongs. At the same time, prepare a counter variable for assigning unique connected component numbers to different connected components.

[0059] The third step: Traverse the image Scan the binarized prescription image line by line starting from the upper left corner. For each pixel: If the pixel belongs to the background (pixel value is 0), skip it. If the pixel belongs to the foreground (pixel value is not 0), check whether the 8 adjacent pixels of this pixel have been labeled. If there are labeled adjacent pixels for this pixel, label this pixel with the connected component number corresponding to the adjacent pixel; if there are no labeled adjacent pixels for this pixel, assign a new connected component number to the new connected component represented by this pixel and update the counter.

[0060] Step 4, Output the result After completing the above steps, the label map contains the connected component numbers of all connected components, and each non-zero value represents a specific connected component.

[0061] Among them, in the third step, during the first traversal, it may occur that two adjacent connected components are assigned different connected component numbers, but these two adjacent connected components belong to the same connected component. Therefore, after the first traversal, a second traversal can be performed to resolve the above conflicts. In a feasible implementation, a mapping table can be created to record equivalent connected component numbers, and then the entire binarized prescription image is traversed again, and the connected component numbers of each pixel are updated according to the mapping table.

[0062] If searching pixel by pixel, the efficiency is too low. To accelerate the connected component recognition process, in a feasible implementation, the same pixels are merged.

[0063] Specifically, first, assign a unique identifier to each pixel in the binarized prescription image, and form an initial set based on the identifiers of each pixel in the binarized prescription image; for each pixel in the binarized prescription image, check the 8 adjacent pixels (including the four directions of up, down, left, and right) of this pixel. If the adjacent pixels have the same value (for example, they are all foreground pixels), use the union operation of the union-find set to merge them. After all merges are completed, traverse the image again, use the find operation of the union-find set to determine the final connected component to which each pixel belongs, and assign a new label to it. By this method, the time for connected component labeling is significantly reduced, and the processing efficiency is improved.

[0064] Step S102C: Use the regionprops function to identify the attributes of each connected region; among them, the attributes of the connected region include the bounding box and the centroid.

[0065] Step S102D: Locate each character based on the centroid of each connected region, and segment the characters using the bounding box of each connected region.

[0066] The positioning and segmentation of a single character are achieved through steps S102B - S102D. Accurately segmenting a single character is a key step in improving the accuracy of OCR recognition. In the embodiments of the present invention, characters are segmented by marking connected components.

[0067] In a specific example, the code for positioning and segmenting characters by marking connected components is as follows: % Read the image and convert it to a binary image I = imread('text.png'); % Assume there is an image file named 'text.png'; BW = imbinarize(rgb2gray(I)); % Convert the image to grayscale and then perform binarization; % Perform connected component analysis CC = bwconncomp(BW, 8); % Use 8 - neighborhood to find connected components; % Obtain the attributes of each connected component stats = regionprops(CC, 'BoundingBox', 'Area', 'Centroid'); % Display the original image imshow(I); hold on; % Draw the bounding box of each connected component on the original image for idx = 1:length(stats) rectangle('Position',stats(idx).BoundingBox,'EdgeColor','r','LineWidth', 2); plot(stats(idx).Centroid(1),stats(idx).Centroid(2),'b*'); end hold off.

[0068] Step S102E: Input the segmented characters into a pre - trained character recognition model to obtain the character recognition result.

[0069] In the embodiments of the present invention, the character recognition model can use CNN (Convolutional Neural Network), RNN (Recurrent Neural Network), LSTM (Long Short-Term Memory network), and their combined models, etc.

[0070] Since Chinese characters (Chinese characters) and other characters (such as Latin letters, numbers, punctuation marks, etc.) are very different in shape and structure. In the embodiments of the present invention, a pre-trained Chinese character recognition model and a non-Chinese character recognition model are used to recognize Chinese characters and non-Chinese characters respectively. In a feasible implementation, as Figure 2 shown, both the Chinese character recognition model and the non-Chinese character recognition model adopt a combination of a convolutional neural network and a long short-term memory network.

[0071] Specifically, first, the segmented characters are further segmented into Chinese characters and non-Chinese characters.

[0072] In this step, Chinese characters refer to the Chinese characters defined in Unicode, and their range is usually between u4e00 - u9fff; non-Chinese characters are all characters other than Chinese characters, including but not limited to Latin letters, numbers, punctuation marks, and characters in other languages, etc.

[0073] In a feasible implementation, Chinese characters can be extracted through a regular expression. For example, the regular expression is hanzi_pattern = re.compile(r'[\u4e00-\u9fff]+'), and the rest are non-Chinese characters.

[0074] Then, taking non-Chinese characters as the boundary, the outline of Chinese characters is cropped to obtain the first picture set, and the outline of non-Chinese characters is cropped to obtain the second picture set.

[0075] In an example, for example, the prescription is "2g of abalone shell, 3g of oyster"; then the first picture set after cropping includes Picture 1 and Picture 2. The character in Picture 1 is "stone abalone shell"; the character in Picture 2 is "oyster"; the second picture set includes Picture 3 and Picture 4. The character in Picture 3 is "2g", and the character in Picture 4 is "3g".

[0076] Secondly, the pictures in the first picture set and the second picture set are respectively formatted to make the picture sizes the same.

[0077] In an example, for example, the size of each picture is adjusted to 64x960.

[0078] Since the sizes of the cropped images vary, it is not convenient for model processing. Therefore, to facilitate model processing, the sizes of the cropped images are adjusted to be the same, making it convenient for subsequent unified model processing.

[0079] After that, the formatted first image set is input into a pre-trained Chinese character recognition model to obtain Chinese character recognition results, and the formatted second image set is input into a pre-trained non-Chinese character recognition model to obtain non-Chinese character recognition results.

[0080] In this step, the network structures of the Chinese character recognition model and the non-Chinese character recognition model are the same, except for the training data samples. This network structure includes a convolutional layer and a recurrent layer. As Figure 2 shown, the first image and the second image of size 64x960x1 are respectively input into the Chinese character recognition model and the non-Chinese character recognition model. After passing through 4 convolutional layers to extract spatial dimension features (such as the shape and position of characters), 64 feature maps of size 4x60 are obtained; these feature maps are converted into 60 sequences of features of size 4x64 through the reshape function and input into the recurrent network to extract the dependencies between characters in the sequence features, so as to better understand the structure of the entire sentence or paragraph. In one example, this recurrent network can adopt an LSTM neural network; the beam search algorithm is used to select the result with the highest probability.

[0081] In another embodiment, if the recognition result of a certain Chinese character or non-Chinese character is inaccurate, the image where the character is located can be separately re-input into the model for recognition to correct the final recognition result.

[0082] Finally, the Chinese character recognition results and the non-Chinese character recognition results are fused and the final character recognition result is output.

[0083] In this step, during fusion, it is necessary to combine the corresponding relationship between Chinese characters and non-Chinese characters in the original prescription image for fusion to ensure the correctness of the information.

[0084] In the embodiments of the present invention, by separately recognizing Chinese characters and non-Chinese characters, it is because Chinese characters (Chinese characters) and other characters (such as Latin letters, numbers, punctuation marks, etc.) are very different in shape and structure. Using different models can optimize according to the characteristics of their respective characters, improve the recognition accuracy. Especially for the recognition of handwritten Chinese characters, separate processing can better adapt to the characteristics of handwritten Chinese characters, thus ensuring more accurate recognition results. In addition, separately recognizing Chinese characters and non-Chinese characters also helps to reduce the complexity of the model and improve the calculation efficiency. Because if all types of characters are put into the same model for training, it may lead to an overly large model that is difficult to train, and at the same time increases the unnecessary calculation burden.

[0085] Step S102F: Post-process the character recognition results to correct incorrect characters, and finally obtain each drug name.

[0086] In this step, NLP technology can be used to correct the character recognition results in terms of grammar and semantics, so that not only formal or formatting errors can be corrected, but also accurate corrections can be made from the aspects of grammar and semantics, thereby improving the accuracy of character recognition.

[0087] Based on the above embodiments, the attributes of the connected components also include area. After using the regionprops function to identify the attributes of each connected component, the area of each connected component can be compared with a preset area threshold respectively, and the connected components with an area smaller than the preset area threshold can be filtered.

[0088] If the area of a certain connected component is smaller than the preset area threshold, it is considered that the connected component is an isolated pixel or a small-scale pixel group. In the embodiments of the present invention, by filtering out the pixel points that obviously do not belong to any connected component, the burden of subsequent character recognition can be reduced, and thus the efficiency of character recognition can be improved.

[0089] Based on the above embodiments, the TCM apprentice learning assistance method provided by the embodiments of the present invention further includes: Step S107: In response to the disease name input in the search box, search for classical diagnostic experience data and modern diagnostic experience data related to the disease name, and display the classical diagnostic experience data and modern diagnostic experience data.

[0090] In one example, classical diagnostic experience data and modern diagnostic experience data related to the disease name can be collected from the Internet.

[0091] Through this embodiment, the treatment plan for this case can be comprehensively learned from two aspects: the treatment experience of ancient famous TCM doctors and the treatment prescriptions of modern famous doctors for this case, providing more comprehensive and systematic learning data for learners.

[0092] In another feasible implementation, in the display of the search results for a certain disease, content such as Western medicine clinical manifestations, diagnosis, treatment plans, etc. can also be added, as well as the TCM clinical syndromes and treatment principles and methods under this disease, and content such as disease epidemiology can be added.

[0093] Based on the above embodiments, the TCM apprentice learning assistance method provided by the embodiments of the present invention further includes: Step S108: In response to the target prescription input in the search box, search for prescriptions containing at least one drug in the target prescription in the pre-constructed prescription database.

[0094] In this step, it is necessary to pre - establish a formula database that contains multiple traditional Chinese medicine formulas and their constituent drugs. Each formula in the formula database should record information such as its name, constituent drugs, and their proportions. A relational database (such as MySQL) or a non - relational database (such as MongoDB) can be used to store the relevant data of each formula in the formula database.

[0095] Step S109: Based on the drug names in the found formulas and the drug names in the target formula, calculate the matching degree between each formula and the target formula.

[0096] In this step, the drug lists in the two formulas can be directly compared, and the proportion of the number of common drugs to the total number of drugs is calculated as the matching degree.

[0097] Step S110: Screen out the formulas with a matching degree not less than the preset matching threshold from each formula as the similar formulas of the target formula, and display each similar formula of the target formula.

[0098] In the embodiment of the present invention, by using the formula database to retrieve and display the similar formulas of the target formula that the learner needs to learn, the learner can learn more comprehensive and systematic traditional Chinese medicine knowledge and improve the learning effect.

[0099] Based on the same inventive concept, the embodiment of the present invention provides a traditional Chinese medicine learning - from - master auxiliary system. As Figure 3 shown, the traditional Chinese medicine learning - from - master auxiliary system provided by the embodiment of the present invention includes: An acquisition unit 201, configured to acquire the prescription image taken during the process of learning from a master. An identification unit 202, configured to identify each drug name in the prescription image through OCR technology and display each drug name. A first search unit 203, configured to, in response to a selection operation for at least one drug name among each drug name, determine at least one drug name as the target drug name; search for all the processing methods of the target drug corresponding to the target drug name and the drug efficacy of the target drug under each processing method in the pre - constructed drug processing knowledge graph, and display all the processing methods of the target drug and the drug efficacy of the target drug under each processing method. A second search unit 204, configured to, in response to a selection operation for at least one processing method among all the processing methods of the target drug, determine at least one processing method as the target processing method; search for the drug characteristics and usage instruction data of the target drug under the target processing method in the pre - constructed drug characteristics and usage instruction database, and display the drug characteristics and usage instruction data of the target drug under the target processing method; the drug characteristics and usage instruction data at least include drug nature and flavor, meridian tropism, common dosage, medication taboos, decoction methods, and whether it is both a medicine and a food.

[0100] In an alternative embodiment, the recognition unit 202 is specifically configured to perform image preprocessing on the prescription image to obtain a binary image; label connected regions in the binary image using the bwlabel function; each connected region corresponds to a potential character; identify the attributes of each connected region using the regionprops function; the attributes include a bounding box and a centroid; locate each character based on the centroid of each connected region, and segment the characters using the bounding boxes of each connected region; input the segmented characters into a pre-trained character recognition model to obtain a character recognition result; post-process the character recognition result to correct incorrect characters, and finally obtain each drug name.

[0101] In an alternative embodiment, the recognition unit 202 is specifically configured to adjust the brightness and contrast of the prescription image to obtain a first intermediate image; remove noise from the first intermediate image based on a filter to obtain a second intermediate image; perform skew correction on the second intermediate image to obtain a third intermediate image; perform binary processing on the third intermediate image to obtain a binary image.

[0102] In an alternative embodiment, the attributes of the connected region further include an area; the recognition unit 202 is further configured to compare the area of each connected region with a preset area threshold respectively, and filter out the connected regions with an area smaller than the preset area threshold.

[0103] In an alternative embodiment, the traditional Chinese medicine apprentice learning assistance system provided by the embodiment of the present invention further includes: A knowledge graph construction unit 205, configured to obtain a dataset related to drug processing methods; perform data preprocessing on the dataset; the data preprocessing at least includes data cleaning, formatting, deduplication, error correction, and filling missing values; identify each entity in the preprocessed dataset using NLP technology, and establish relationships between each entity; the entities at least include medicinal material names, disease names, processing methods, and drug effects; construct a drug processing knowledge graph based on each entity and the relationships between each entity.

[0104] In an alternative embodiment, the traditional Chinese medicine apprentice learning assistance system provided by the present invention further includes: A third search unit 206, configured to search for classical diagnosis experience data and modern diagnosis experience data related to the disease name in response to the disease name input in the search box, and display the classical diagnosis experience data and the modern diagnosis experience data.

[0105] In an alternative embodiment, the traditional Chinese medicine apprentice learning assistance system provided by the present invention further includes: The fourth search unit 207 is configured to, in response to a target prescription input in the search box, search for prescriptions containing at least one drug in the target prescription in a pre-constructed prescription database; calculate the matching degree between each prescription and the target prescription based on the drug names in the found prescriptions and the drug names in the target prescription; screen out the prescriptions with a matching degree not less than a preset matching threshold from each prescription as the similar prescriptions of the target prescription, and display each similar prescription of the target prescription.

[0106] Based on the same technical concept, an embodiment of the present invention further provides an electronic device, as Figure 4 shown, including a processor 301, a communication interface 302, a memory 303, and a communication bus 304. Among them, the processor 301, the communication interface 302, and the memory 303 complete communication with each other through the communication bus 304.

[0107] The memory 303 is used to store a computer program; The processor 301 is configured to implement the above-mentioned traditional Chinese medicine follow-up learning assistance method when executing the computer program stored on the memory 303.

[0108] The communication bus mentioned in the above electronic device may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of convenience of representation, only a thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0109] The communication interface is used for communication between the above electronic device and other devices.

[0110] The memory may include a Random Access Memory (RAM), and may also include a Non-Volatile Memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located far from the aforementioned processor.

[0111] The above-mentioned processor may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0112] The TCM apprentice learning assistance method provided by the embodiments of the present invention may also be implemented as a computer program product. The computer program product includes program code, and the instructions included in the program code can be used to execute the above-mentioned TCM apprentice learning assistance method. For specific implementation, reference can be made to the method embodiments, which will not be elaborated here.

[0113] The TCM apprentice learning assistance system provided by the embodiments of the present invention may be specific hardware on a device or software or firmware installed on the device, etc. For the system provided by the embodiments of the present invention, its implementation principle and the technical effects produced are the same as those of the foregoing method embodiments. For the sake of brief description, for the parts not mentioned in the system embodiments, reference can be made to the corresponding content in the foregoing method embodiments. Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the systems and units described above can all refer to the corresponding processes in the above method embodiments, which will not be elaborated here.

[0114] In the embodiments provided by the present invention, it should be understood that the disclosed systems and methods can be implemented in other ways. The system embodiments described above are only illustrative. For example, the division of the units is only a logical function division, and there may be other division methods in actual implementation. For another example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some communication interfaces. The indirect coupling or communication connection of the devices or units can be in an electrical, mechanical or other forms.

[0115] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0116] In addition, each functional unit in the embodiments provided by the present invention may be integrated into a processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit.

[0117] If the above-mentioned function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0118] It should be noted that similar reference numerals and letters represent similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. In addition, the terms "first", "second", "third", etc. are only used for distinguishing descriptions and cannot be understood as indicating or implying relative importance.

[0119] Finally, it should be noted that the above-mentioned embodiments are only specific implementation manners of the present invention, used to illustrate the technical solutions of the present invention, rather than limiting it. The protection scope of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that any person skilled in the technical field of the present invention can still modify the technical solutions described in the foregoing embodiments, or can easily think of changes, or make equivalent replacements for some of the technical features; and these modifications, changes, or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention. All should be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. A method for assisting students to learn from a teacher in traditional Chinese medicine, characterized in that: Applied to a TCM learning assistant client, the method includes: Obtain prescription images taken during the learning process; Recognize the names of the drugs in the prescription image by using OCR technology, and display the names of the drugs; In response to a selection operation on at least one of the drug names, determining the at least one drug name as a target drug name; Searching for all the preparation methods of the target drug corresponding to the target drug name and the drug efficacy of the target drug under each preparation method in a pre-constructed drug preparation knowledge graph, and displaying all the preparation methods of the target drug and the drug efficacy of the target drug under each preparation method; In response to a selection operation of at least one processing method among all processing methods of the target drug, determining the at least one processing method as a target processing method; The drug properties and instructions for use data of the target drug under the target preparation method are searched in a pre-constructed drug properties and instructions database, and the drug properties and instructions for use data of the target drug under the target preparation method are displayed; the drug properties and instructions for use data at least include drug properties, meridians, common dosage, contraindications, decoction methods and whether the drug is food of the same origin.

2. The method according to claim 1, characterized in that The identifying of each drug name in the prescription image by using OCR technology includes: Performing image preprocessing on the prescription image to obtain a binary image; Using the bwlabel function to mark connected domains in the binary image; each connected domain corresponds to a potential character; Using the regionprops function to identify the properties of each connected domain; the properties include a bounding box and a centroid; Locate each character based on the centroid of each connected domain, and segment the characters using the bounding box of each connected domain; Input the segmented characters into the pre-trained character recognition model to obtain the character recognition results; The character recognition results are post-processed to correct erroneous characters, and finally the names of the drugs are obtained.

3. The method according to claim 2, characterized in that The performing image preprocessing on the prescription image to obtain a binary image comprises: Adjusting the brightness and contrast of the prescription image to obtain a first intermediate image; removing noise in the first intermediate image based on a filter to obtain a second intermediate image; performing tilt correction on the second intermediate image to obtain a third intermediate image; The third intermediate image is binarized to obtain a binarized image.

4. The method according to claim 2, characterized in that: The properties of the connected domain also include area; after using the regionprops function to identify the properties of each connected domain, it also includes: The area of ​​each connected domain is compared with a preset area threshold, and the connected domains whose area is smaller than the preset area threshold are filtered.

5. The method according to claim 1, characterized in that The construction process of the drug preparation knowledge graph includes: Obtain data sets related to drug preparation methods; Performing data preprocessing on the data set; the data preprocessing at least includes data cleaning, formatting, deduplication, error correction and missing value filling; Using NLP technology to identify entities in the preprocessed data set and establish relationships between entities; the entities at least include medicinal material names, disease names, processing methods, and drug efficacy; The drug preparation knowledge graph is constructed based on various entities and the relationships between various entities.

6. The method according to claim 1, characterized in that Also includes: In response to a disease name input in the search box, classical diagnostic experience data and modern diagnostic experience data related to the disease name are searched, and the classical diagnostic experience data and the modern diagnostic experience data are displayed.

7. The method according to claim 1, characterized in that Also includes: In response to a target prescription input in the search box, searching a pre-built prescription database for a prescription containing at least one drug in the target prescription; Based on the drug names in each of the found prescriptions and the drug names in the target prescription, calculating the matching degree between each of the prescriptions and the target prescription; Prescriptions with a matching degree not less than a preset matching threshold are screened out from the various prescriptions as similar prescriptions of the target prescription, and the various similar prescriptions of the target prescription are displayed.

8. A TCM learning assistance system, characterized in that: Applied to the auxiliary client of TCM learning, the system includes: An acquisition unit, used to acquire prescription images taken during the learning process; A recognition unit, used to recognize the names of the drugs in the prescription image by OCR technology and display the names of the drugs; A first search unit is used for, in response to a selection operation on at least one of the drug names, determining the at least one drug name as a target drug name; searching for all preparation methods of the target drug corresponding to the target drug name and the drug efficacy of the target drug under each preparation method in a pre-constructed drug preparation knowledge graph, and displaying all preparation methods of the target drug and the drug efficacy of the target drug under each preparation method; The second search unit is used to respond to the selection operation of at least one of all the preparation methods for the target drug, determine the at least one preparation method as the target preparation method; search for the drug properties and instructions data of the target drug under the target preparation method in a pre-constructed drug properties and instructions database, and display the drug properties and instructions data of the target drug under the target preparation method; the drug properties and instructions data at least include drug properties, meridians, common dosage, contraindications, decoction method and whether it is medicine and food of the same origin.

9. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus; wherein the processor, the communication interface and the memory communicate with each other through the communication bus; The memory is used to store computer programs; The processor is used to implement the method according to any one of claims 1 to 7 when executing the computer program stored in the memory.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images