Traditional Chinese medicine prescription mixed reality visual learning method and device and wearable equipment

Abstract

The invention relates to a traditional Chinese medicine prescription mixed reality visual learning method and device and wearable equipment, and the method comprises the steps: obtaining visual field information captured by a user through the wearable equipment when the user wears the wearable equipment; under the condition that it is detected that the target area in the visual field information contains multiple medicinal materials, medicinal material information of the multiple medicinal materials is obtained, medicinal material views of the multiple medicinal materials and a prescription view based on the multiple medicinal materials are obtained, and the medicinal material information is overlaid to the target area in the visual field information to be displayed. And overlapping the medicinal material view and the prescription view to a non-target area in the visual field information for display. In this way, a user can visually understand information such as medicinal material names or medicinal material attributes of the medicinal materials through the medicinal material information superposed in the visual field information, and the user can quickly learn the attributes of the medicinal materials and the medicinal materials contained in the prescription through the medicinal material view and the prescription view superposed in the visual field information; and the learning efficiency can be improved to a great extent.

Description

Technical Field

[0001] The present disclosure relates to the technical field of visualization of traditional Chinese medicine prescriptions, and particularly to a method, device and wearable device for visualizing the learning of traditional Chinese medicine prescriptions in mixed reality. Background Art

[0002] Learning traditional Chinese medicine prescriptions is one of the core contents of traditional Chinese medicine education. Analyzing the prescription composition rules of traditional Chinese medicine prescriptions is of great significance and value for using traditional Chinese medicine to treat different diseases in the actual diagnosis and treatment process. If the rules can be intuitively displayed, it will help learners learn more quickly.

[0003] However, the traditional methods for learning traditional Chinese medicine prescriptions mainly include using textbooks, websites or applications to transmit the composition information of prescriptions. This makes it difficult for learners to comprehensively understand the compatibility rules of prescriptions and compare the similarities and differences between different prescriptions, resulting in low learning efficiency for learners. Summary of the Invention

[0004] The present disclosure provides a method, device and wearable device for visualizing the learning of traditional Chinese medicine prescriptions in mixed reality.

[0005] According to a first aspect of the present disclosure, there is provided a method for visualizing the learning of traditional Chinese medicine prescriptions in mixed reality, the method comprising:

[0006] When a user wears a wearable device, obtaining the visual field information captured by the user through the wearable device;

[0007] When it is detected that the target area in the visual field information contains multiple medicinal materials, respectively obtaining the medicinal material information of the multiple medicinal materials, and obtaining the medicinal material views of the multiple medicinal materials and the prescription views based on the multiple medicinal materials; wherein, the medicinal material views are generated based on the medicinal material attributes of the multiple medicinal materials, and the prescription views include multiple prescriptions composed of the multiple medicinal materials;

[0008] Overlaying the medicinal material information on the target area in the visual field information for display, and overlaying the medicinal material views and the prescription views on the non-target area in the visual field information for display.

[0009] According to a second aspect of the present disclosure, there is provided a device for visualizing the learning of traditional Chinese medicine prescriptions in mixed reality, the device comprising:

[0010] A visual field information acquisition module, configured to obtain the visual field information captured by the user through the wearable device when the user wears the wearable device;

[0011] The medicinal material information and view acquisition module is used to respectively acquire the medicinal material information of the multiple medicinal materials when it is detected that the target area in the visual field information contains multiple medicinal materials, and acquire the medicinal material views of the multiple medicinal materials and the formula views based on the multiple medicinal materials; wherein, the medicinal material views are generated based on the medicinal material attributes of the multiple medicinal materials, and the formula views include multiple formulas composed of the multiple medicinal materials.

[0012] The mixed reality module is used to superimpose the medicinal material information on the target area in the visual field information for display, and superimpose the medicinal material views and the formula views on the non-target area in the visual field information for display.

[0013] According to the third aspect of the present disclosure, there is provided a wearable device, which includes a mixed reality helmet. The mixed reality helmet may include: a memory and a processor. A computer program is stored on the memory, and when the processor executes the program, the method described above is implemented.

[0014] According to the fourth aspect of the present disclosure, there is provided a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the above method of the present disclosure is implemented.

[0015] According to the fifth aspect of the present disclosure, there is provided a computer program product, including a computer program, and when the computer program is executed by a processor, the above method of the present disclosure is implemented.

[0016] The traditional Chinese medicine formula mixed reality visualization learning method, device and wearable device provided by the embodiments of the present disclosure can acquire the visual field information captured by the user through the wearable device when the user wears the wearable device. When it is detected that the target area in the visual field information contains multiple medicinal materials, the medicinal material information of the multiple medicinal materials can be respectively acquired, and the medicinal material views of the multiple medicinal materials and the formula views based on the multiple medicinal materials can be acquired. The medicinal material information is superimposed on the target area in the visual field information for display, and the medicinal material views and the formula views are superimposed on the non-target area in the visual field information for display. In this way, the user can intuitively understand information such as the medicinal material name or medicinal material attributes of the medicinal materials through the medicinal material information superimposed on the visual field information, and through the medicinal material views and the formula views superimposed on the visual field information, the user can quickly learn the attributes of the medicinal materials and the medicinal materials included in the formula, thereby greatly improving the learning efficiency. Description of the Drawings

[0017] In the following description of the exemplary embodiments in conjunction with the drawings, more details, features and advantages of the present disclosure are disclosed. In the drawings:

[0018] Figure 1Effect diagram of physical medicinal material panel image based on QR code positioning and superimposed virtual information provided by an exemplary embodiment of the present disclosure;

[0019] Figure 2 Effect diagram of medicinal material view provided by an exemplary embodiment of the present disclosure;

[0020] Figure 3 Effect diagram of prescription view provided by an exemplary embodiment of the present disclosure;

[0021] Figure 4 Effect diagram of physical medicinal material interaction illustration provided by an exemplary embodiment of the present disclosure;

[0022] Figure 5A Effect diagram of drug view during interaction provided by an exemplary embodiment of the present disclosure;

[0023] Figure 5B Effect diagram of prescription view during interaction provided by an exemplary embodiment of the present disclosure;

[0024] Figure 5C Effect diagram of prescription view during interaction provided by another exemplary embodiment of the present disclosure;

[0025] Figure 6 Flowchart of traditional Chinese medicine prescription mixed reality visualization learning method provided by an exemplary embodiment of the present disclosure;

[0026] Figure 7 Schematic block diagram of functional modules of traditional Chinese medicine prescription mixed reality visualization learning device provided by an exemplary embodiment of the present disclosure;

[0027] Figure 8 Structural block diagram of an electronic device provided by an exemplary embodiment of the present disclosure;

[0028] Figure 9 Structural block diagram of a computer system provided by an exemplary embodiment of the present disclosure. Detailed implementation manners

[0029] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although some embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. On the contrary, these embodiments are provided to more thoroughly and completely understand the present disclosure. It should be understood that the accompanying drawings and embodiments of the present disclosure are only for exemplary purposes and are not used to limit the protection scope of the present disclosure.

[0030] It should be understood that the various steps described in the method embodiments of the present disclosure can be executed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this regard.

[0031] As used herein, the term "including" and its variations are open-ended, that is, "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description. It should be noted that the concepts such as "first", "second", etc. mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0032] It should be noted that the modifications of "one" and "a plurality" mentioned in the present disclosure are illustrative rather than restrictive. Those skilled in the art should understand that unless otherwise clearly specified in the context, it should be understood as "one or more".

[0033] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only for illustrative purposes and are not used to limit the scope of these messages or information.

[0034] It can be understood that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, usage scopes, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0035] For example, when responding to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested by the user will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, an application program, a server, or a storage medium that performs the operations of the technical solutions of the present disclosure according to the prompt message.

[0036] As an optional but non-limiting implementation manner, the manner of sending a prompt message to the user in response to receiving an active request from the user may be, for example, in the form of a pop-up window, and the prompt message may be presented in text in the pop-up window. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device. It can be understood that the above process of notifying and obtaining the user's authorization is only illustrative and does not constitute a limitation on the implementation manner of the present disclosure, and other manners that meet relevant laws and regulations can also be applied to the implementation manner of the present disclosure.

[0037] Embodiments of the present disclosure relate to a method for learning traditional Chinese medicine prescriptions based on MR (Mixed Reality) technology. Through a method for learning traditional Chinese medicine prescriptions that can interact with real medicinal material samples and virtual charts, it aims to improve the interactivity, immersion, and learning efficiency of learning traditional Chinese medicine prescriptions through MR technology.

[0038] Therefore, in order to provide learners with a more immersive and interactive learning experience, the embodiments combine real medicinal material samples with virtual data. Learners interact with actual medicinal material samples by wearing wearable devices. Among them, the wearable device can be a head-mounted MR device such as a HoloLens2 holographic glasses, and the embodiments are not limited thereto.

[0039] Specifically, in the embodiments, a variety of medicinal materials can be placed on the medicinal material panel. For example, the medicinal material panel can be divided into multiple sub-regions, and a certain amount of medicinal material samples or specimens can be placed in each sub-region. In this way, when the above-mentioned medicinal material panel containing a variety of medicinal materials appears in the field of view of the wearable device during learning by the learner, the wearable device can identify the various medicinal materials on the medicinal material panel, obtain the medicinal material information of each medicinal material on the medicinal material panel, and superimpose the identified medicinal material information on the corresponding medicinal materials on the medicinal material panel for virtual and real display. For example, the medicinal material information can be the name of the corresponding medicinal material, or it can also include the medicinal material attributes of the medicinal material, etc., and the embodiments are not limited thereto. The medicinal material information can be superimposed on the field of view of the wearable device in the form of virtual text to realize the combination of virtual visual text information and actual medicinal materials.

[0040] In the embodiments, the wearable device can be used to obtain an image of the medicinal material panel, extract features from the image of the medicinal material panel to obtain the feature data of each medicinal material in the medicinal material panel, and identify the medicinal material information of each medicinal material in the medicinal material panel by identifying the feature data.

[0041] In the embodiments, identifiers can also be set at the corresponding positions of each medicinal material in the medicinal material panel. Different medicinal materials correspond to different identifiers. In this way, the wearable device obtains the medicinal material information of each medicinal material in the medicinal material panel by obtaining the image data of the medicinal material panel and identifying the identifiers in the image data. Among them, the identifier is not limited to being implemented by means of numbers, characters, or two-dimensional codes, etc.

[0042] In the embodiment, a two-dimensional code may also be set on the medicinal material panel. For example, the two-dimensional code may be a QR (Quick Response Code) code or the like. When the wearable device scans and identifies the two-dimensional code, four vertices of the boundary of the two-dimensional code in the real three-dimensional space can be obtained, the normal vector of the plane where the two-dimensional code is located can be obtained, and the positions of the four vertices in the coordinate system corresponding to the wearable device can be calculated. By obtaining the relative position relationship between the two-dimensional code and each medicinal material in the real three-dimensional space, the positions of each medicinal material in the real three-dimensional space can be converted into the three-dimensional virtual space of the wearable device, so that the medicinal material information can be superimposed on the visual field of the wearable device, realizing the function of superimposing virtual information on the physical medicinal materials.

[0043] In this way, identification and position tracking can be performed through the two-dimensional code. By positioning the physical medicinal material panel and its size in the actual physical space, the function of superimposing virtual data on the physical medicinal materials can be realized, helping learners understand the relationship between the attributes of the medicinal materials and the prescriptions. Specifically, reference can be made to Figure 1 as shown Figure 1 which is the effect diagram of the physical medicinal material panel image based on two-dimensional code positioning and superimposing virtual information provided by the embodiment of the present disclosure. Among them, Figure 1 the two-dimensional code in

[0044] can be displayed in the form of a physical picture, and can also be displayed through an electronic display screen. Figure 1 Based on

[0045] as shown, after the coordinate position in the real three-dimensional space is converted into the coordinate position in the coordinate system corresponding to the wearable device through two-dimensional code identification by the wearable device, the embodiment can displace and rotate the blue virtual frame and the white virtual text, so that the blue virtual frame and the white virtual text are aligned with the plane where the two-dimensional code is located, and superimposed on the real medicinal material panel to realize the function of learners interacting with the physical medicinal materials. Figure 1 It should be noted that

[0046] In the embodiments provided by the present disclosure, on the basis of displaying virtual text on the above physical medicinal materials, a medicinal material view and a prescription view can also be generated based on the above-mentioned various medicinal materials, and the medicinal material view and the prescription view are also superimposed on the visual field information obtained by the learner through the wearable device, so as to realize the combination of real medicinal materials and more virtual data.

[0047] In the embodiments, in the process of generating the medicinal material view, since medicinal materials usually include multiple dimensions of drug attributes, such as four natures, five flavors, and meridians entered by the drugs and other multi-dimensional attributes. To facilitate the learner to quickly understand the medicinal material attribute information of each medicinal material, the multi-dimensional attributes of the above-mentioned medicinal materials can be processed by dimensionality reduction, for example, reducing from multi-dimensional to two-dimensional, and visualizing with a scatter plot. The attributes of the medicinal materials are represented by the position and color of the drugs in the two-dimensional space, and a data-driven color coding method is used to design the scatter colors to help the learner understand the basic characteristics of the medicinal materials.

[0048] For example, by obtaining the medicinal material information of various medicinal materials in the above-mentioned medicinal material panel, calculating the attribute distances between the medicinal materials, and based on the attribute distances, obtaining several specific medicinal materials among the various medicinal materials, and the attribute distances between the several specific medicinal materials and other medicinal materials are greater than the threshold. And set the several specific medicinal materials at different positions in the two-dimensional space and set them to different colors. In this way, the attribute distances between the remaining medicinal materials among the above-mentioned various medicinal materials and the several specific medicinal materials can be calculated respectively. If the attribute distance between a certain medicinal material and a certain specific medicinal material is relatively close, the medicinal material can be set at a position relatively close to the specific medicinal material in the two-dimensional space and set to a similar color. Specifically, reference can be made to Figure 2 as shown Figure 2 which is the effect diagram of the medicinal material view provided by the embodiments of the present disclosure.

[0049] In the embodiments, the above multi-dimensional drug attributes can be dimensionally reduced based on UMAP (Uniform Manifold Approximation and Projection, a visualization method based on non-linear dimensionality reduction), and data-driven color coding can be calculated. That is, representative colors can be designed according to traditional Chinese medicine principles, and representative medicinal materials can be specified from various medicinal materials, that is, the above specific medicinal materials. By representing colors using the sRGB (standard RGB, standard red-green-blue color space) color space, converting to the CIECAM02-UCS (CIE color appearance model 02-uniform color space, a uniform color space based on the CIECAM02 color appearance model), performing RBF (Radial basis function) interpolation, then converting the processed image to the sRGB color space, and generating a continuous two-dimensional color map, the colors of the above-specified representative medicinal materials can be obtained, and based on the attribute distances between the medicinal materials, the colors corresponding to each medicinal material can be obtained. To save the storage space of the wearable device and reduce the calculation time, the colors corresponding to each medicinal material can be pre-calculated, and the colors corresponding to each medicinal material are stored in the wearable device in the form of RGB values.

[0050] In the embodiments, as Figure 3 shown, Figure 3 is the effect diagram of the prescription view provided by the embodiments of the present disclosure. In the process of generating the prescription view, the medicinal material composition of the prescription and the relationship between prescriptions can be displayed in the form of an icicle plot. Each column of the icicle plot represents a prescription, and the "sovereign drug" in the prescription is located at the bottom and is represented in blue, and other medicinal materials are arranged above it. The arrangement of the "sovereign drug" and other drugs is based on the similarity of the medicinal material attributes, which helps learners quickly understand the relationship between the medicinal materials and their positions in the prescription. As Figure 3 shown, in the embodiments, the layout of the icicle plot can be performed according to the medicinal material attributes using the optimal similarity sequence sorting algorithm:

[0051] (1) Sort the "sovereign drugs" of all prescriptions to obtain the horizontal layout of the icicle plot. For example, the sorting can be performed according to the medicinal material attributes of the "sovereign drugs", and the medicinal material attributes between adjacent "sovereign drugs" are the closest.

[0052] (2) Perform optimal similarity sorting on the remaining drugs in each column according to the medicinal material attributes.

[0053] (3) Starting from the second column from the left, check the arrangement order of the non-"sovereign drugs" in this column and the left column. If a drug in the left column appears in this column, insert this medicinal material at the same vertical position as in the left column, and the order of other drugs remains unchanged to obtain the vertical layout of the icicle plot.

[0054] In the embodiment, to reduce the calculation time, the above process can also be pre-calculated, and the arrangement result of the medicinal materials is stored in the wearable device. Here, the color of the medicinal materials uses the corresponding color in the medicinal material view, and the alpha value is set to achieve a semi-transparent effect. Among them, the alpha value represents the transparency of the color, ranging from 0 to 1, where 0 represents completely transparent and 1 represents completely opaque.

[0055] In the embodiment provided by the present disclosure, after obtaining the drug view and the prescription view, the drug view Figure 2 and the No. prescription view can be respectively superimposed on the visual field information obtained through the wearable device, so that learners can learn and understand more about the medicinal materials and prescriptions.

[0056] Based on the above embodiment, when the learner selects a physical medicinal material, for example, when selecting a certain medicinal material in it through operations such as finger clicking, a virtual curve can be generated, and the medicinal material is connected to the corresponding medicinal material in the prescription view through the virtual curve. Specifically, reference can be made to Figure 1 as shown in Figure 4 which is Figure 4 the effect diagram of the interaction of the physical medicinal materials provided by the embodiment of the present disclosure.

[0057] In the embodiment, a quadratic Bezier curve can be used to generate the virtual curve, and the medicinal materials in the medicinal materials and the corresponding medicinal materials in the prescription view are connected through the virtual curve. Specifically, the virtual curve can be calculated by the following formula (1).

[0058] B(t) = (1 - t) 2 P0 + 2t(1 - t)P1 + t 2 P2, t ∈ [0, 1] (1)

[0059] Among them, B(t) represents the virtual curve, P0 is the three-dimensional space coordinate of the center of gravity of the physical medicinal material frame in the wearable device, P1 is the center of gravity of the corresponding element in the prescription ice cone diagram (also the three-dimensional space coordinate of the wearable device), P2 is the intersection point of the perpendicular bisector of the line segment P0P1 and the XY plane (i.e., the plane where the medicinal material panel is located), and t is the curve parameter.

[0060] Combined with Figure 4 as shown in, in the physical medicinal material panel, when the learner clicks on the medicinal material on the panel with a finger, a curve will connect it to the corresponding medicinal material in the prescription view. At the same time, the prescription text containing this medicinal material in the prescription view turns yellow. For example, when clicking on the "White Peony Root" specimen of the physical medicinal material, the curve connects all the "White Peony Root" cubes in the prescription view, and all the prescriptions containing "White Peony Root", such as "Eight Precious Decoction", "Complete Belt Decoction", and "Four Substances Decoction", turn yellow.

[0061] In the embodiments provided by the present disclosure, when a learner clicks on a medicinal material in the medicinal material view with a finger, the prescriptions that can be formed by this medicinal material will be displayed below the medicinal material view. For example, as Figure 5A shown, when the learner clicks on "ginseng" and "atractylodes macrocephala" in the drug view, the text below shows that these two medicinal materials may form "Sijunzi Decoction", "Shenling Baizhu Powder", "Buzhong Yiqi Decoction", etc.). At the same time, the medicinal materials corresponding to the prescription view and the relevant prescription information will be highlighted. As Figure 5B shown, "ginseng" and "atractylodes macrocephala" are framed in blue, and the text of the prescriptions they may form turns yellow. Among them, Figure 5A is the effect diagram of the drug view during interaction in the embodiments provided by the present disclosure, Figure 5B is the effect diagram of the prescription view during interaction provided by the embodiments of the present disclosure.

[0062] As Figure 5C shown, Figure 5C is the effect diagram of the prescription view during interaction provided by another embodiment of the present disclosure. In the prescription view, when the user clicks on the medicinal material cube in the ice pick diagram with a finger, the corresponding medicinal material will change color, and the prescriptions that this medicinal material may form will also be displayed below the view. Combining Figure 5C shown, when clicking on "ginseng", the text below shows that "ginseng" as the monarch drug may form "Shengmai Powder", "Sijunzi Decoction", "Babao Decoction" and "Shenling Baizhu Powder".

[0063] Based on the above embodiments, in the embodiments of the present disclosure, when a learner wears a wearable device such as a HoloLens2 holographic glasses, the drug information, as well as the medicinal material view and the prescription view, can be superimposed on the visual field information of the wearable device, realizing the combination of virtual information and physical objects, enabling the learner to more intuitively understand and learn traditional Chinese medicine prescriptions, and through the above-mentioned interaction based on real medicinal materials and virtual charts, the learning efficiency of the learner can be greatly improved.

[0064] To verify the effectiveness of the system, an experimental example designed and implemented a controlled experiment, comparing the mixed reality traditional Chinese medicine prescription learning system in the experimental example with traditional tables based on text and two-dimensional visualization methods based on web pages. The experiment invited 30 subjects, none of whom had background knowledge in the field of traditional Chinese medicine prescriptions. In the experiment, the subjects used HoloLens2 holographic glasses for interaction and needed to complete three tasks: herbal medicine recognition, prescription composition analysis, and prescription memory. The experimental data covered task performance (including accuracy and response time) and user experience, and focused on evaluating the task completion efficiency, accuracy, and user preference under different learning modes.

[0065] The experimental results show that, compared with text-based and web-based learning methods, the mixed reality-based learning method in the embodiments exhibits shorter response times in the formula learning task and has no significant difference in task accuracy compared with traditional methods. In terms of user experience, the mixed reality learning method shows the highest system usability and the lowest task load, effectively reducing the cognitive burden on users, so that learners can complete the formula learning task in a more efficient and relaxed manner.

[0066] This experiment fully verifies the effectiveness of the embodiments of the present disclosure in improving the formula learning effect and user experience. In the experiment, the subjects' performance in the formula memory task is not inferior to that of traditional methods, which proves that by combining the multi-dimensional attributes of medicinal materials with the visualization method of formula composition, the embodiments can help learners comprehensively understand the compatibility rules of formulas and the attributes of medicinal materials; by using the portable HoloLens2 holographic glasses, the embodiments can break through the limitations of traditional teaching venues; at the same time, the system usability feedback from the subjects also proves that the immersive and interactive design based on mixed reality significantly enhances the learners' sense of participation and learning interest, and comprehensively improves the efficiency and experience of traditional Chinese medicine formula learning. In short, the embodiments of the present disclosure provide an innovative and practically applicable solution for traditional Chinese medicine education.

[0067] Based on the above embodiments, the embodiments of the present disclosure also provide a method for visualizing the learning of traditional Chinese medicine formulas in mixed reality, as Figure 6 shown, the method may include the following steps:

[0068] In step S610, when the user wears the wearable device, the visual field information captured by the user through the wearable device is obtained.

[0069] In the embodiments, the user may include the learners in the above embodiments, and the wearable device includes devices such as HoloLens2 holographic glasses, and the embodiments are not limited thereto.

[0070] In step S620, when it is detected that the target area in the visual field information contains multiple medicinal materials, the medicinal material information of the multiple medicinal materials is respectively obtained, and the medicinal material views of the multiple medicinal materials and the formula views based on the composition of the multiple medicinal materials are obtained. Among them, the medicinal material views are generated based on the medicinal material attributes of the multiple medicinal materials, and the formula views include multiple formulas based on the composition of the multiple medicinal materials.

[0071] In an embodiment, for example, a variety of medicinal materials can be placed on a medicinal material panel. When the visual field information captured by the wearable device by the user includes the medicinal material panel, the medicinal material information of each medicinal material in the medicinal material panel can be obtained. The medicinal material information can be the name or attributes of the medicinal material, etc. In addition, the area where the medicinal material panel is located can be used as a target area, that is, the target area is an area containing a variety of medicinal materials. Among them, the methods for obtaining the medicinal material information of each medicinal material in the medicinal material panel can include, but are not limited to, the following methods:

[0072] (1) Identify the medicinal material information of each medicinal material in the medicinal material panel by means of image recognition.

[0073] For example, through feature extraction, extract the feature information of each medicinal material, and compare the feature information with a feature template containing medicinal material information to determine the medicinal material information of each medicinal material.

[0074] (2) Add identifiers near each medicinal material.

[0075] For example, identifiers can be added near the corresponding medicinal materials on the medicinal material panel respectively. Different medicinal materials correspond to different identifiers, and each identifier corresponds to the medicinal material information of the medicinal material. In this way, the medicinal material information corresponding to each medicinal material can be obtained through the identifier of the medicinal material.

[0076] (3) Determine the medicinal material information according to the positions of each medicinal material on the medicinal material panel.

[0077] In an embodiment, the medicinal material panel can be divided into multiple sub-regions, and each sub-region can be determined to hold the corresponding medicinal material. In this way, by obtaining the positions of each sub-region on the medicinal material panel, the medicinal material information of each medicinal material can be determined.

[0078] In this way, by obtaining the medicinal material information of a variety of medicinal materials, a medicinal material view and a prescription view can be generated or pre-generated based on the medicinal material information.

[0079] In an embodiment, it is assumed that there are M kinds of medicinal materials in the target area. In this way, a medicinal material view can be generated according to the M kinds of medicinal materials.

[0080] Specifically, the medicinal material attributes corresponding to the M kinds of medicinal materials can be obtained, and based on the medicinal material attributes, the attribute distances between the M kinds of medicinal materials can be calculated. Among them, the attribute distance is inversely related to the similarity degree of the medicinal material attributes between the medicinal materials, that is, for medicinal materials with similar medicinal material attributes, the attribute distance between the medicinal materials is smaller, and vice versa.

[0081] Obtain N kinds of medicinal materials from the M kinds of medicinal materials. Among them, the attribute distances between the N kinds of medicinal materials are greater than a threshold value. Both M and N are positive integers, and M > N. For example, N kinds of medicinal materials with relatively large differences in medicinal material attributes can be obtained from the M kinds of medicinal materials.

[0082] Distribute N kinds of medicinal materials in a two-dimensional coordinate system for display. Among them, the N kinds of medicinal materials are respectively displayed in different colors, and the positions of the N kinds of medicinal materials in the two-dimensional coordinate system are different. In the embodiment, the N kinds of medicinal materials can be distributed as far as possible in the limited space in the two-dimensional coordinate system.

[0083] Based on the positions of the N kinds of medicinal materials in the two-dimensional coordinate system and the attribute distances between the M kinds of medicinal materials, determine the positions and displayed colors of the M kinds of medicinal materials in the two-dimensional coordinate system to obtain a medicinal material view.

[0084] In the embodiment, first determine different positions and different colors in the two-dimensional space for the medicinal materials with relatively large differences in attributes among the N kinds of medicinal materials. Among them, the distances between the N kinds of medicinal materials are relatively large, and the colors are quite different. In this way, by calculating the attribute distances between the remaining medicinal materials among the M kinds of medicinal materials and the N kinds of medicinal materials respectively, the position distribution and colors of the remaining medicinal materials in the two-dimensional space can be determined. Among them, the position distribution and colors of the remaining medicinal materials in the two-dimensional space are also relatively close to the distances and colors of the medicinal materials with the smallest attribute distances to them. The greater the attribute distance between the M kinds of medicinal materials, the greater the difference in the distances between the positions in the two-dimensional space, and the greater the difference in colors. And the smaller the attribute distance between two kinds of medicinal materials, the smaller the distance in the two-dimensional space, and the smaller the difference in colors.

[0085] In the embodiment, when obtaining a prescription view composed of multiple kinds of medicinal materials, multiple kinds of medicinal materials in the target area can be traversed to obtain multiple prescriptions that can be composed of multiple kinds of medicinal materials. In this way, the multiple prescriptions can be arranged and displayed according to their medicinal properties, and the medicinal material information included in each prescription can be displayed to obtain a prescription view. Among them, in the prescription view, the ice cone diagram can be laid out based on the optimal similarity sequence sorting algorithm in the above embodiment. For specific details, reference can be made to the above implementation, which will not be elaborated here.

[0086] In step S630, the medicinal material information is superimposed and displayed on the target area in the visual field information, and the medicinal material view and the prescription view are superimposed and displayed on the non-target area in the visual field information.

[0087] In the embodiments, the medicinal material information, the medicinal material view, and the formula view of the medicinal materials belong to virtual data, while the medicinal materials are real physical objects. Through the wearable device, the user can superimpose the virtual data onto the physical object in the visual field information to achieve virtual combination. Among them, by superimposing the medicinal material information onto the target area in the visual field information for display, the medicinal material information of the medicinal materials can be displayed in areas such as near each medicinal material, so that the user can intuitively understand the medicinal material names or medicinal material attributes of each medicinal material, facilitating the user's learning. In addition, by superimposing the medicinal material view and the formula view onto the non-target area in the visual field information, in the visual field seen by the user through the wearable device, since the medicinal materials are located in the target area of the visual field, while the medicinal material view and the formula view are located in the non-target area of the visual field, the medicinal material view and the formula view will not block the medicinal materials. And by displaying the medicinal material view and the formula view in the non-target area, the user can better understand the attributes of the medicinal materials and the formulas that can be combined, which is beneficial to the user's traditional Chinese medicine learning.

[0088] The traditional Chinese medicine formula mixed reality visualization learning method provided by the embodiments of the present disclosure can obtain the visual field information captured by the user through the wearable device when the user wears the wearable device. When it is detected that the target area in the visual field information contains multiple medicinal materials, the medicinal material information of the multiple medicinal materials can be obtained respectively, and the medicinal material views of the multiple medicinal materials and the formula view based on the multiple medicinal materials can be obtained. The medicinal material information is superimposed onto the target area in the visual field information for display, and the medicinal material view and the formula view are superimposed onto the non-target area in the visual field information for display. In this way, the user can intuitively understand information such as the medicinal material names or medicinal material attributes of the medicinal materials through the medicinal material information superimposed onto the visual field information, and through the medicinal material view and the formula view superimposed onto the visual field information, the user can quickly learn the attributes of the medicinal materials and the medicinal materials included in the formula, thereby greatly improving the learning efficiency.

[0089] Based on the above embodiments, in another embodiment provided by the present disclosure, the method may further include the following steps:

[0090] In step S640, obtain the selection operation of the user on the target medicinal material placed in the target area.

[0091] In step S650, highlight the target medicinal material in the medicinal material view and the formula view.

[0092] As Figure 4 shown, when it is detected that the user performs a selection operation such as clicking on the target medicinal material in the target area by means of a finger or the like, the target medicinal material can be highlighted in the medicinal material view and the formula view. For example, when the user clicks on the medicinal material "white peony root", the "white peony root" in the medicinal material view will be highlighted, and the "white peony root" in the formula view will be added with a highlighted border to highlight it.

[0093] In the embodiment, when the user clicks on the target medicinal material in the medicinal material view, the medicinal material view will highlight the target medicinal material, and the possible prescriptions that can be formed may also be displayed below the medicinal material view.

[0094] Similarly, when the user clicks on the target medicinal material in the prescription view, the target medicinal material can be highlighted, or the original display color of the target medicinal material can be changed. At the same time, the prescriptions that can be formed based on the target medicinal material can be displayed below the prescription view. That is, obtain the user's selection operation on the target medicinal material in the prescription view; obtain the target prescription information containing the target medicinal material, and display the target prescription information in the visual field information. For example, in the prescription view, by clicking on "Ginseng", the text below shows that "Ginseng" as the monarch drug may form "Shengmai Powder", "Four Gentlemen Decoction", "Eight Precious Decoction", and "Shenling Baizhu Powder".

[0095] In the embodiment, when obtaining the user's selection operation on the target medicinal material placed in the target area and highlighting the target medicinal material in the medicinal material view and the prescription view, the position of the target medicinal material in the target area can also be obtained, and the position of the target medicinal material in the prescription view can be obtained. Thus, based on the positions of the target medicinal material in the target area and the prescription view respectively, a target curve can be generated, and the target curve can be superimposed on the visual field information for display.

[0096] Specifically, the first position of the target medicinal material in the target area can be obtained, and the second position of the target medicinal material in the prescription view can be obtained. Generate a target curve connecting the first position and the second position, and superimpose the target curve on the visual field information for display. Among them, since there may be multiple prescriptions in the prescription view, and there may be X prescriptions among the multiple prescriptions that all contain the target medicinal material, and there will also be X second positions. Therefore, there will be X target curves generated, and the X curves can all be connected from the first position to the X second positions. Wherein, X is a positive integer.

[0097] For example, it can be combined with Figure 4 As shown, the target medicinal material can be "White Peony Root". "White Peony Root" is in the target area, that is, at the first position on the medicinal material panel, and at multiple different second positions in the prescription view. When the user clicks on the "White Peony Root" specimen of the medicinal material, the multiple generated target curves can connect all the "White Peony Root" cubes in the prescription view, and all the prescriptions containing "White Peony Root", "Eight Precious Decoction", "Complete Belt Decoction", and "Four Substances Decoction" all turn yellow to highlight. Among them, the target curve can be generated based on Equation (1) in the above embodiment, which will not be elaborated here.

[0098] Based on the above embodiment, in another embodiment provided by the present disclosure, the above target medicinal material can include at least two medicinal materials, and the method can further include the following steps:

[0099] In step S660, it is determined whether the target medicinal materials can form the target prescription.

[0100] In the embodiment, if the target medicinal materials include at least two medicinal materials, it indicates that there may be medicinal materials with opposite medicinal properties among the at least two medicinal materials, or a certain medicinal material is missing. Then, they cannot be combined into the corresponding prescription.

[0101] Therefore, when the target medicinal materials can form the target prescription, step S670 is executed. When the target medicinal materials cannot form the target prescription, step S680 is executed.

[0102] In step S670, the prescription information of the target prescription is displayed.

[0103] In step S680, the reason information for not being able to form the target prescription is displayed. Among them, the reason information includes: the target prescription still lacks the corresponding medicinal materials or there are medicinal materials with opposite medicinal property relationships among the target medicinal materials.

[0104] In the embodiment, the user can, based on their understanding of the medicinal properties of the medicinal materials or according to the medicinal material view, try to combine several medicinal materials in the medicinal material panel into a prescription. When the combination is successful, the prescription information of the corresponding prescription is displayed, and when the combination fails, the reason information is given. By continuously summarizing, it can prompt the user to continuously learn and more quickly and effectively master the medicinal property information and prescription information of the medicinal materials, which is beneficial to improving the learning efficiency of the user.

[0105] Based on the above embodiment, in another embodiment provided by the present disclosure, the above target area may further include a QR code image. The QR code image and various medicinal materials in the medicinal material panel are in the same plane in the real space and have a determined relative position. When the medicinal material information is superimposed and displayed in the target area of the visual field information, the coordinate position of the QR code image in the real space and the normal vector of the plane where the QR code image is located can be obtained. Based on the coordinate position and normal vector of the QR code image in the real space, the coordinate position of the QR code image in the virtual space corresponding to the wearable device is obtained. Based on the coordinate position and relative position of the QR code image in the virtual space, the positions of various medicinal materials in the virtual space are obtained. Based on the positions of various medicinal materials in the virtual space, a display layer containing medicinal material information is added to the target area. Among them, each medicinal material information in the display layer corresponds to the positions of various medicinal materials in the virtual space respectively.

[0106] In the embodiment, it can be combined with Figure 4As shown, by obtaining the four vertices of the boundary of the QR code in the real space, obtaining the normal vector of the plane where the QR code is located, and calculating the positions of the four vertices in the coordinate system corresponding to the wearable device, by obtaining the relative position relationship between the QR code and each medicinal material in the medicinal material panel in the real three-dimensional space, the positions of each medicinal material in the real three-dimensional space can be converted into the virtual space of the wearable device, so that the medicinal material information can be superimposed on the field of view of the wearable device, realizing the function of superimposing virtual information on the physical medicinal material specimens. Among them, the display layer can be a transparent display layer. By adding medicinal material information on the display layer and setting the medicinal material information at the corresponding positions in the virtual space, the medicinal material information can be displayed on the physical medicinal materials based on the virtual text information.

[0107] In the case of dividing each functional module according to the corresponding functions, the embodiments of the present disclosure provide a traditional Chinese medicine formula mixed reality visualization learning device, and the traditional Chinese medicine formula mixed reality visualization learning device can be a server, a terminal or a chip applied to the server. Figure 7 It is a schematic block diagram of the functional modules of the traditional Chinese medicine formula mixed reality visualization learning device provided by an exemplary embodiment of the present disclosure. As Figure 7 shown, the traditional Chinese medicine formula mixed reality visualization learning device includes:

[0108] A field of view information acquisition module 10, configured to acquire the field of view information captured by the user through the wearable device when the user wears the wearable device;

[0109] A medicinal material information and view acquisition module 20, configured to respectively acquire the medicinal material information of the multiple medicinal materials and acquire the medicinal material views of the multiple medicinal materials and the formula views based on the multiple medicinal materials when it is detected that the target area in the field of view information contains multiple medicinal materials; wherein, the medicinal material views are generated based on the medicinal material attributes of the multiple medicinal materials, and the formula views include multiple formulas composed of the multiple medicinal materials;

[0110] A mixed reality module 30, configured to superimpose the medicinal material information on the target area in the field of view information for display, and superimpose the medicinal material views and the formula views on the non-target area in the field of view information for display.

[0111] In another embodiment provided by the present disclosure, the device further includes:

[0112] Obtaining a selection operation of the user on the target medicinal material placed in the target area;

[0113] Highlighting the target medicinal material in the medicinal material views and the formula views.

[0114] In yet another embodiment provided by the present disclosure, the device further includes a target curve generation module, which is specifically configured to:

[0115] Obtain the first position of the target medicinal material in the target area, and obtain the second position of the target medicinal material in the prescription view;

[0116] Generate a target curve connecting the first position and the second position, and superimpose the target curve on the field of view information for display.

[0117] In yet another embodiment provided by the present disclosure, the target medicinal material includes at least two kinds of medicinal materials; the device further includes:

[0118] A prescription judgment module, configured to judge whether the target medicinal materials can form a target prescription;

[0119] A prescription information display module, configured to display the prescription information of the target prescription when the target medicinal materials can form a target prescription;

[0120] When the target medicinal materials cannot form a target prescription, display the reason information for not being able to form a target prescription; wherein, the reason information includes: the target prescription still lacks corresponding medicinal materials or there are medicinal materials with opposite medicinal material attributes among the target medicinal materials.

[0121] In yet another embodiment provided by the present disclosure, the device further includes:

[0122] A selection operation acquisition module, configured to acquire the selection operation of the user on the target medicinal materials in the prescription view;

[0123] A target prescription information display module, configured to acquire the target prescription information including the target medicinal materials, and display the target prescription information in the field of view information.

[0124] In yet another embodiment provided by the present disclosure, the target area further includes a QR code image, and the QR code image and the multiple medicinal materials are in the same plane in the real space and have a determined relative position; the mixed reality module is specifically configured to:

[0125] Obtain the coordinate position of the QR code image in the real space and the normal vector of the plane where the QR code image is located;

[0126] Based on the coordinate position of the QR code image in the real space and the normal vector, obtain the coordinate position of the QR code image in the virtual space corresponding to the wearable device;

[0127] Based on the coordinate position of the QR code image in the virtual space and the relative position, obtain the positions of the multiple medicinal materials in the virtual space;

[0128] Based on the positions of the multiple medicinal materials in the virtual space, add a display layer containing medicinal material information to the target area; wherein, each piece of medicinal material information in the display layer corresponds to the position of the multiple medicinal materials in the virtual space respectively.

[0129] In another embodiment provided by the present disclosure, the multiple medicinal materials include M kinds of medicinal materials; the medicinal material information and view acquisition module is specifically used for:

[0130] Obtain the medicinal material attributes corresponding to the M kinds of medicinal materials respectively;

[0131] Based on the medicinal material attributes, calculate the attribute distances between the M kinds of medicinal materials; wherein, the attribute distance is inversely correlated with the similarity degree of the medicinal material attributes between the medicinal materials.

[0132] Obtain N kinds of medicinal materials from the M kinds of medicinal materials; wherein, the attribute distances between the N kinds of medicinal materials are greater than a threshold value, and both M and N are positive integers, and M > N.

[0133] Distribute the N kinds of medicinal materials to be displayed in a two-dimensional coordinate system; wherein, the N kinds of medicinal materials are respectively displayed in different colors, and the positions of the N kinds of medicinal materials in the two-dimensional coordinate system are different.

[0134] Based on the positions of the N kinds of medicinal materials in the two-dimensional coordinate system and the attribute distances between the M kinds of medicinal materials, determine the positions and displayed colors of the M kinds of medicinal materials in the two-dimensional coordinate system to obtain a medicinal material view.

[0135] In another embodiment provided by the present disclosure, the medicinal material information and view acquisition module is specifically used for:

[0136] Traverse the multiple medicinal materials to obtain multiple prescriptions that can be composed of the multiple medicinal materials;

[0137] Arrange and display the multiple prescriptions according to their medicinal properties, and display the medicinal material information included in each prescription to obtain a prescription view.

[0138] The visualization learning device for traditional Chinese medicine prescriptions provided by the embodiments of the present disclosure can obtain the visual field information captured by the user through the wearable device when the user wears the wearable device. When it is detected that the target area in the visual field information contains various medicinal materials, the medicinal material information of the various medicinal materials can be obtained respectively, and the medicinal material views of the various medicinal materials and the prescription views based on the various medicinal materials can be obtained. The medicinal material information is superimposed on the target area in the visual field information for display, and the medicinal material views and the prescription views are superimposed on the non-target area in the visual field information for display. In this way, the user can intuitively understand information such as the name or properties of the medicinal materials through the medicinal material information superimposed on the visual field information, and through the medicinal material views and the prescription views superimposed on the visual field information, the user can quickly learn the properties of the medicinal materials and the medicinal materials included in the prescription, thereby greatly improving the learning efficiency.

[0139] The embodiments of the present disclosure also provide an electronic device, which may specifically be the above-mentioned wearable device. The wearable device may include a mixed reality helmet, etc. The wearable device may include: at least one processor; a memory for storing executable instructions of the at least one processor; wherein, the at least one processor is configured to execute the instructions to implement the above-mentioned method disclosed by the embodiments of the present disclosure.

[0140] Figure 8 It is a schematic structural diagram of an electronic device provided by an exemplary embodiment of the present disclosure. As Figure 8 shown, the electronic device 1800 includes at least one processor 1801 and a memory 1802 coupled to the processor 1801. The processor 1801 can execute the corresponding steps in the above-mentioned method disclosed by the embodiments of the present disclosure.

[0141] The above-mentioned processor 1801 can also be referred to as a central processing unit (CPU). It can be an integrated circuit chip with the ability to process signals. Each step in the above methods disclosed in the embodiments of the present disclosure can be completed by the integrated logic circuit in the hardware of the processor 1801 or instructions in the form of software. The above-mentioned processor 1801 can be a general-purpose processor, a digital signal processor (DSP), an ASIC (Application Specific Integrated Circuit), an FPGA (field-programmable gate array), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of the present disclosure can be directly implemented by the hardware decoding processor, or implemented by a combination of the hardware and software modules in the decoding processor. The software module can be located in the memory 1802, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, and other well-known storage media in the art. The processor 1801 reads the information in the memory 1802 and completes the steps of the above methods in combination with its hardware.

[0142] In addition, when various operations / processes according to the present disclosure are implemented by software and / or firmware, a program constituting the software can be installed from a storage medium or a network into a computer system having a dedicated hardware structure, such as Figure 9 the computer system 1900 shown. When various programs are installed in the computer system, it can execute various functions, including the functions described above, etc. Figure 9 It is a block diagram of the structure of a computer system provided by an exemplary embodiment of the present disclosure.

[0143] The computer system 1900 is intended to represent various forms of digital electronic computer devices, such as, a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, a cellular phone, a smart phone, a wearable device, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are only examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0144] As Figure 9As shown, computer system 1900 includes a computing unit 1901 that can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1902 or a computer program loaded from a storage unit 1908 into a random access memory (RAM) 1903. In the RAM 1903, various programs and data required for the operation of the computer system 1900 can also be stored. The computing unit 1901, the ROM 1902, and the RAM 1903 are connected to each other via a bus 1904. An input / output (I / O) interface 1905 is also connected to the bus 1904.

[0145] Multiple components in the computer system 1900 are connected to the I / O interface 1905, including: an input unit 1906, an output unit 1907, a storage unit 1908, and a communication unit 1909. The input unit 1906 can be any type of device capable of inputting information into the computer system 1900. The input unit 1906 can receive input digital or character information and generate key signal inputs related to user settings and / or function controls of the electronic device. The output unit 1907 can be any type of device capable of presenting information and can include, but is not limited to, a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The storage unit 1908 can include, but is not limited to, magnetic disks and optical disks. The communication unit 1909 allows the computer system 1900 to exchange information / data with other devices via a network such as the Internet and can include, but is not limited to, a modem, a network card, an infrared communication device, a wireless communication transceiver, and / or a chipset, such as a BluetoothTM device, a WiFi device, a WiMax device, a cellular communication device, and / or the like.

[0146] The computing unit 1901 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 1901 executes the various methods and processes described above. For example, in some embodiments, the above-described methods disclosed in the embodiments of the present disclosure can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 1908. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device via the ROM 1902 and / or the communication unit 1909. In some embodiments, the computing unit 1901 can be configured to execute the above-described methods disclosed in the embodiments of the present disclosure by any other appropriate means (e.g., by means of firmware).

[0147] An embodiment of the present disclosure also provides a computer-readable storage medium. When instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the above methods disclosed in the embodiments of the present disclosure.

[0148] The computer-readable storage medium in the embodiments of the present disclosure may be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. The above computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any suitable combination of the above. More specifically, the above computer-readable storage medium may include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above.

[0149] The above computer-readable medium may be included in the above electronic device; or may exist separately without being assembled into the electronic device.

[0150] An embodiment of the present disclosure also provides a computer program product, including a computer program, wherein when the computer program is executed by a processor, the above methods disclosed in the embodiments of the present disclosure are implemented.

[0151] In the embodiments of the present disclosure, computer program code for performing the operations of the present disclosure may be written in one or more programming languages or combinations thereof. The above programming languages include, but are not limited to, object-oriented programming languages such as C# or C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, executed as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network (including a local area network (LAN) or a wide area network (WAN)), or may be connected to an external computer.

[0152] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a portion of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that, in some alternative implementations, the functions noted in the blocks may occur in a different order than that noted in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0153] The modules, components, or units described in the embodiments of the present disclosure can be implemented in software or in hardware. In some cases, the names of the modules, components, or units do not constitute a limitation on the modules, components, or units themselves.

[0154] The functions described above can be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary hardware logic components that can be used include: Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), Systems on a Chip (SOCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0155] The above description is only some embodiments of the present disclosure and an explanation of the applied technical principles. Those skilled in the art should understand that the scope of the disclosure involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosure concept. For example, the technical solutions formed by mutually replacing the above features with the (but not limited to) technical features having similar functions disclosed in the present disclosure.

[0156] Although some specific embodiments of the present disclosure have been described in detail by way of examples, those skilled in the art should understand that the above examples are only for the purpose of illustration and not for limiting the scope of the present disclosure. Those skilled in the art should understand that the above embodiments can be modified without departing from the scope and spirit of the present disclosure. The scope of the present disclosure is defined by the appended claims.

Claims

1. A mixed reality visualization learning method for traditional Chinese medicine prescriptions, characterized in that: The method comprises: When a user wears a wearable device, obtaining visual field information captured by the user through the wearable device; In the case where it is detected that the target area in the field of view information contains multiple medicinal materials, medicinal material information of the multiple medicinal materials is obtained respectively, and a medicinal material view of the multiple medicinal materials and a prescription view based on the composition of the multiple medicinal materials are obtained; wherein the medicinal material view is generated based on the medicinal material properties of the multiple medicinal materials, and the prescription view contains multiple prescriptions based on the composition of the multiple medicinal materials; The medicinal material information is superimposed on the target area in the visual field information for display, and the medicinal material view and the prescription view are superimposed on the non-target area in the visual field information for display.

2. The method according to claim 1, characterized in that The method further comprises: Acquiring a selection operation of the user on a target medicinal material placed in the target area; The target medicinal material in the medicinal material view and the prescription view is highlighted.

3. The method according to claim 2, characterized in that The method further comprises: Acquire a first position of the target medicinal material in the target area, and acquire a second position of the target medicinal material in the prescription view; A target curve connecting the first position and the second position is generated, and the target curve is superimposed on the field of view information for display.

4. The method according to claim 2, characterized in that: The target medicinal materials include at least two medicinal materials; the method further includes: Determining whether the target medicinal material can constitute a target prescription; When the target medicinal material can constitute a target prescription, displaying prescription information of the target prescription; Alternatively, when the target medicinal materials cannot constitute the target prescription, the reason information for not being able to constitute the target prescription is displayed; wherein the reason information includes: the target prescription still lacks corresponding medicinal materials or there are medicinal materials with opposite medicinal properties among the target medicinal materials.

5. The method according to claim 1, characterized in that: The method further comprises: Obtaining a user's selection operation on a target medicinal material in the prescription view; Target prescription information including the target medicinal material is acquired, and the target prescription information is displayed in the field of view information.

6. The method according to claim 1, characterized in that The target area also includes a two-dimensional code image, and the two-dimensional code image and the plurality of medicinal materials are located in the same plane in real space and have a determined relative position; The step of superimposing the medicinal material information on the target area in the visual field information for display includes: Obtaining the coordinate position of the two-dimensional code image in real space and the normal vector of the plane where the two-dimensional code image is located; Based on the coordinate position of the two-dimensional code image in the real space and the normal vector, obtaining the coordinate position of the two-dimensional code image in the virtual space corresponding to the wearable device; Based on the coordinate position of the two-dimensional code image in the virtual space and the relative position, obtaining the positions of the plurality of medicinal materials in the virtual space; Based on the positions of the multiple medicinal materials in the virtual space, a display layer containing medicinal material information is added to the target area; wherein each medicinal material information in the display layer corresponds to the position of the multiple medicinal materials in the virtual space.

7. The method according to claim 1, characterized in that The multiple medicinal materials include M kinds of medicinal materials; and obtaining the medicinal material view of the multiple medicinal materials includes: Obtaining medicinal material properties corresponding to the M kinds of medicinal materials respectively; Based on the medicinal material properties, calculating the property distance between the M kinds of medicinal materials; wherein the property distance is inversely correlated with the degree of similarity of the medicinal material properties between the medicinal materials; Obtaining N kinds of medicinal materials from the M kinds of medicinal materials; wherein the attribute distance between the N kinds of medicinal materials is greater than a threshold, M and N are both positive integers, and M>N; Distribute the N kinds of medicinal materials in a two-dimensional coordinate system for display; wherein the N kinds of medicinal materials are displayed in different colors, and the positions of the N kinds of medicinal materials in the two-dimensional coordinate system are different; Based on the positions of the N medicinal materials in the two-dimensional coordinate system and the attribute distances between the M medicinal materials, the positions and displayed colors of the M medicinal materials in the two-dimensional coordinate system are determined to obtain a medicinal material view.

8. The method according to claim 1, characterized in that: Obtain a prescription view based on the multiple medicinal materials, including: Traversing the multiple medicinal materials to obtain multiple prescriptions that can be composed based on the multiple medicinal materials; The multiple prescriptions are arranged and displayed according to their medicinal properties, and the medicinal material information contained in each prescription is displayed to obtain a prescription view.

9. A mixed reality visualization learning device for traditional Chinese medicine prescriptions, characterized in that: The device comprises: A visual field information acquisition module, used to acquire visual field information captured by the user through the wearable device when the user wears the wearable device; A medicinal material information and view acquisition module, for respectively acquiring medicinal material information of the plurality of medicinal materials, and acquiring medicinal material views of the plurality of medicinal materials and prescription views based on the plurality of medicinal materials when it is detected that the target area in the field of view information contains a plurality of medicinal materials; wherein the medicinal material view is generated based on medicinal material properties of the plurality of medicinal materials, and the prescription view contains a plurality of prescriptions based on the plurality of medicinal materials; The mixed reality module is used to superimpose the medicinal material information on the target area in the visual field information for display, and to superimpose the medicinal material view and the prescription view on the non-target area in the visual field information for display.

10. A wearable device, comprising a mixed reality helmet, characterized in that: include: at least one processor; a memory for storing the at least one processor-executable instruction; The at least one processor is configured to execute the instructions to implement the method according to any one of claims 1 to 8.

Images