Clinical teaching method and system based on pulmonary clinical images

Abstract

The present invention provides a clinical teaching method and system based on lung clinical images, which relates to the field of clinical teaching technology. The method includes: step S1: obtaining a plurality of plane images of lung clinical images, recorded as a first image set; step S2: constructing a three-dimensional lung model according to the plane images in the first image set; step S3: obtaining clinical teaching information and converting it into annotation information, wherein the annotation information includes part information; step S4: adding the annotation information in the form of a label to the position corresponding to the part information on the three-dimensional lung model; step S5: displaying the labeled three-dimensional lung model to the user through an AR device or a VR device. The present invention establishes a three-dimensional lung model and interactively displays information on various parts of the lung, thereby achieving better clinical teaching effects.

Description

Technical Field

[0001] The present invention relates to the technical field of clinical teaching, and in particular to a clinical teaching method and system based on lung clinical images. Background Art

[0002] The clinical teaching of lung clinical images is based on the basic structure of the lungs and the construction of a three-dimensional lung model, and through this three-dimensional lung model, a better teaching method is achieved.

[0003] Augmented reality (AR) technology is a technology that cleverly integrates virtual information with the real world. It widely uses multimedia, 3D modeling, real-time tracking and registration, intelligent interaction, sensing and other technical means to simulate computer-generated text, images, 3D models, music, video and other virtual information and apply them to the real world. The two types of information complement each other, thus achieving "enhancement" of the real world. Virtual reality (VR) technology can bring more immersion.

[0004] Currently, most existing clinical teaching methods use graphics or physical models. Graphic teaching cannot create a sense of reality and has a poor sensory effect. The physical model is only a reproduction of the lung structure. It can only better reflect the visual effect during the teaching process, but cannot present information about each part. Therefore, it still cannot achieve good teaching results during the learning process. Summary of the invention

[0005] In view of the defects in the prior art, the purpose of the present invention is to provide a clinical teaching method and system based on lung clinical images, establish a three-dimensional model of the lungs, and interactively display information of various parts of the lungs, so as to achieve better clinical teaching effects.

[0006] According to a clinical teaching method and system based on lung clinical images provided by the present invention, the scheme is as follows:

[0007] In a first aspect, a clinical teaching method based on lung clinical images is provided, the method comprising:

[0008] Acquire multiple planar images of lung clinical images, recorded as a first image set;

[0009] constructing a three-dimensional lung model according to the planar images in the first image set;

[0010] Acquire clinical teaching information and convert it into annotation information, wherein the annotation information includes location information;

[0011] adding the annotation information in the form of a label to a position corresponding to the part information on the three-dimensional lung model;

[0012] The labeled three-dimensional model of the lungs is displayed to the user through an AR device or a VR device.

[0013] Preferably, in the three-dimensional lung model, the initial display state of the label is a hidden state.

[0014] Preferably, the method further comprises:

[0015] Detecting an interactive operation between a user and the three-dimensional lung model;

[0016] According to the interactive operation, the display state of the label corresponding to the operated part of the operated lung three-dimensional model is modified to the display state.

[0017] Preferably, the interactive operation includes: rendering different parts of the three-dimensional lung model in different colors, setting multiple trigger points in each color area, and when it is detected that the user's fingers in the virtual environment touch two trigger points in a color area at the same time, it is considered that the part corresponding to the color area is selected by the user and operated;

[0018] The multiple users include one or more user groups, and one user group includes a first user, a second user, and multiple third users; in the user group, different parts of the three-dimensional lung model are rendered in different colors, wherein the same part is displayed in a first color to the first user and the second user in the user group, and in a second color to the third user in the user group, and the first color is different from the second color; the first color is set by the first user in the user group after the finger of the first user in the user group touches multiple trigger points of the same part in the virtual environment at the same time, and the first user in the user group designates a user as the second user in the user group to display the set first color to the second user, and designates other users as the third user in the user group to display a second color different from the set first color to the third user; wherein the multiple users form different user groups, each user group has its own user roles such as the first user, the second user, and the third user, and different users serve as the same and / or different user roles in each user group, the first colors in each user group are different from each other, and the second colors in each user group are the same.

[0019] Preferably, the method further comprises:

[0020] Recording step: recording the user's interactive operations in the virtual environment, and generating an operation trajectory according to the sequence of the user's interactive operations, wherein the trajectory is superimposed and displayed on the three-dimensional lung model.

[0021] In a second aspect, a clinical teaching system based on lung clinical images is provided, the system comprising:

[0022] Module M1: acquiring multiple planar images of lung clinical images, recorded as the first image set;

[0023] Module M2: constructing a three-dimensional lung model based on the planar images in the first image set;

[0024] Module M3: Acquire clinical teaching information and convert it into annotation information, wherein the annotation information includes location information;

[0025] Module M4: adding the annotation information in the form of a label to a position corresponding to the part information on the three-dimensional lung model;

[0026] Module M5: Displaying a labeled three-dimensional lung model to the user through an AR device or a VR device.

[0027] Preferably, in the three-dimensional lung model, the initial display state of the label is a hidden state.

[0028] Preferably, the system further comprises:

[0029] Module M6: detecting the interaction operation between the user and the three-dimensional lung model;

[0030] Module M7: According to the interactive operation, the display state of the label corresponding to the operated part of the operated three-dimensional lung model is modified to the display state.

[0031] Preferably, the interactive operation includes: rendering different parts of the three-dimensional lung model in different colors, setting multiple trigger points in each color area, and when it is detected that the user's fingers in the virtual environment touch two trigger points in a color area at the same time, it is considered that the part corresponding to the color area is selected by the user and operated;

[0032] The multiple users include one or more user groups, and one user group includes a first user, a second user, and multiple third users; in the user group, different parts of the three-dimensional lung model are rendered in different colors, wherein the same part is displayed in a first color to the first user and the second user in the user group, and in a second color to the third user in the user group, and the first color is different from the second color; the first color is set by the first user in the user group after the finger of the first user in the user group touches multiple trigger points of the same part in the virtual environment at the same time, and the first user in the user group designates a user as the second user in the user group to display the set first color to the second user, and designates other users as the third user in the user group to display a second color different from the set first color to the third user; wherein the multiple users form different user groups, each user group has its own user roles such as the first user, the second user, and the third user, and different users serve as the same and / or different user roles in each user group, the first colors in each user group are different from each other, and the second colors in each user group are the same.

[0033] Preferably, the system further comprises:

[0034] Recording module: records the user's interactive operations in the virtual environment, and generates an operation trajectory according to the sequence of the user's interactive operations, wherein the trajectory is superimposed on the three-dimensional lung model.

[0035] Compared with the prior art, the present invention has the following beneficial effects:

[0036] 1. Compared with the traditional teaching method of graphics or physical models, the three-dimensional lung model constructed in the present invention can further enhance the visual sensory effect and make the overall model closer to the structure of the real lung;

[0037] 2. By combining the information of various parts of the lungs with the three-dimensional model of the lungs and displaying the information in an interactive form, the learning interest of clinical teaching can be enhanced. At the same time, the display of information can deepen the learning impression and improve the overall clinical teaching effect.

[0038] 3. Multiple users form different user groups. Different users can play the same and / or different user roles in each user group, which helps to realize the shared operation display of multiple people on a model at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0040] Figure 1 It is a schematic diagram of the overall process of the present invention. DETAILED DESCRIPTION

[0041] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several changes and improvements can also be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0042] The embodiment of the present invention provides a clinical teaching method based on lung clinical images. Figure 1 As shown, first, it is necessary to obtain multiple plane images of lung clinical images, record the multiple plane images obtained as a first image set, and then construct an overall lung three-dimensional model based on the multiple plane images in the first image set.

[0043] Acquire clinical teaching information, and convert the acquired clinical teaching information into annotation information, wherein the annotation information includes part information. In this embodiment, the actual acquired clinical information is distinguished into different parts according to the lung structure, and the annotation information corresponds to each different part one by one.

[0044] The annotation information is added in the form of labels to the positions corresponding to the location information on the constructed three-dimensional lung model. In the three-dimensional lung model, the initial display state of the label is hidden. Under normal visual effects, the label cannot be seen and only the overall structure of the lung can be seen.

[0045] The 3D lung model with labels is displayed to the user through an AR device or a VR device, and the display of the 3D lung model through the AR device or the VR device makes the display of the 3D lung model more realistic. During use, the user is detected to perform interactive operations with the 3D lung model, and after the interaction, the display state of the label corresponding to the operated part of the 3D lung model can be changed to the display state.

[0046] During the interaction, different parts of the three-dimensional lung model are rendered in different colors, and multiple trigger points are set in each color area. When it is detected that the user's fingers in the virtual environment touch two trigger points of a color area at the same time, it is considered that the part corresponding to the color area is selected by the user and operated. At this time, the part information displayed on the label can be seen.

[0047] Finally, the user's interactive operations in the virtual environment are recorded, and an operation trajectory is generated according to the sequence of the user's interactive operations, wherein the trajectory is superimposed on the three-dimensional lung model.

[0048] In a preferred example, the multiple users include one or more user groups, and one user group includes a first user, a second user, and multiple third users; in the user group, different parts of the three-dimensional lung model are rendered in different colors, wherein the same part is displayed in a first color to the first user and the second user in the user group, and in a second color to the third user in the user group, and the first color is different from the second color; the first color may be red, and the second color may be gray. The first color is set by the first user in the user group after the finger of the first user in the virtual environment touches multiple trigger points of the same part at the same time (for example, it can be set by the first user clicking, or it can be a default setting, such as the default setting is red), and the first user in the user group designates a user as the second user in the user group to display the set first color to the second user, and designates other users as the third users in the user group to display a second color different from the set first color to the third user (the same part is still displayed in gray to other users as the third users); wherein multiple users form different user groups, each user group has its own user roles such as the first user, the second user, and the third user, different users serve as the same and / or different user roles in each user group, the first colors in each user group are different from each other, and the second colors in each user group are the same. For example, a user is the user role of the first user in user groups A and B, the user role of the second user in user group C, and the user role of the third user in other user groups D, E, and F. Since the first colors in each user group are different and the second colors in each user group are the same, the user sees the first colors red, yellow, and blue in user groups A, B, and C, respectively, and the second color gray in user groups D, E, and F on the same lung three-dimensional model. It is particularly preferred that when different first colors are used to display the same part in different user groups, the part is displayed to the first user and the second user in these user groups as a color pattern of color bars arranged alternately between the different first colors. For example, for the same part, the first color of user group A is red and the first color of user group B is blue, then the first user and the second user in user A and user B see the part as a color pattern of red and blue bars arranged alternately. In a teaching scenario, the first user is the teacher, the second user is the student who asks questions, and the remaining students are the user role of the third user. In another teaching scenario, the first user and the second user in each user group are played by two students in the user group who are discussing with each other, and the teacher and the remaining students play the role of the third user.In yet another teaching scenario, a user group in which a teacher and a questioning student are the first user and the second user coexist in a user group in which two discussion students are the first user and the second user.

[0049] The embodiment of the present invention provides a clinical teaching method based on lung clinical images. Constructing a three-dimensional lung model can further enhance the visual sensory effect, and displaying information in an interactive form can enhance the learning interest of clinical teaching. At the same time, the display of information can deepen the learning impression, thereby improving the overall clinical teaching effect.

[0050] The present invention also provides a clinical teaching system based on lung clinical images, which can be implemented by those skilled in the art through the steps of the clinical teaching method based on lung clinical images, that is, the clinical teaching method based on lung clinical images can be understood as a preferred implementation of the clinical teaching system based on lung clinical images. The clinical teaching system based on lung clinical images includes:

[0051] Module M1: acquiring multiple planar images of lung clinical images, recorded as the first image set;

[0052] Module M2: constructing a three-dimensional lung model based on the planar images in the first image set;

[0053] Module M3: Acquire clinical teaching information and convert it into annotation information, wherein the annotation information includes location information;

[0054] Module M4: adding the annotation information in the form of a label to a position corresponding to the part information on the three-dimensional lung model;

[0055] Module M5: Displaying a labeled three-dimensional lung model to the user through an AR device or a VR device.

[0056] In the three-dimensional lung model, an initial display state of the label is a hidden state.

[0057] The system further comprises:

[0058] Module M6: detecting the interaction operation between the user and the three-dimensional lung model;

[0059] Module M7: According to the interactive operation, the display state of the label corresponding to the operated part of the operated three-dimensional lung model is modified to the display state.

[0060] The interactive operation includes: rendering different parts of the lung three-dimensional model into different colors, setting multiple trigger points in each color area, and when it is detected that the user's fingers in the virtual environment touch two trigger points in a color area at the same time, it is considered that the part corresponding to the color area is selected by the user and operated;

[0061] The multiple users include one or more user groups, and one user group includes a first user, a second user, and multiple third users; in the user group, different parts of the three-dimensional lung model are rendered in different colors, wherein the same part is displayed in a first color to the first user and the second user in the user group, and in a second color to the third user in the user group, and the first color is different from the second color; the first color is set by the first user in the user group after the finger of the first user in the user group touches multiple trigger points of the same part in the virtual environment at the same time, and the first user in the user group designates a user as the second user in the user group to display the set first color to the second user, and designates other users as the third user in the user group to display a second color different from the set first color to the third user; wherein the multiple users form different user groups, each user group has its own user roles such as the first user, the second user, and the third user, and different users serve as the same and / or different user roles in each user group, the first colors in each user group are different from each other, and the second colors in each user group are the same.

[0062] The system further comprises:

[0063] Recording module: records the user's interactive operations in the virtual environment, and generates an operation trajectory according to the sequence of the user's interactive operations, wherein the trajectory is superimposed on the three-dimensional lung model.

[0064] Those skilled in the art know that, in addition to realizing the system and its various devices, modules, and units provided by the present invention in a purely computer-readable program code, it is entirely possible to realize the same functions in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system and its various devices, modules, and units provided by the present invention can be considered as a hardware component, and the devices, modules, and units included therein for realizing various functions can also be regarded as structures within the hardware component; the devices, modules, and units for realizing various functions can also be regarded as both software modules for realizing the method and structures within the hardware component.

[0065] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essence of the present invention. In the absence of conflict, the embodiments of the present application and the features in the embodiments can be combined with each other arbitrarily.

Claims

1. A clinical teaching method based on lung clinical images, characterized in that: The method comprises: Step S1: acquiring a plurality of planar images of clinical lung images, recorded as a first image set; Step S2: constructing a three-dimensional lung model according to the planar images in the first image set; Step S3: Acquire clinical teaching information and convert it into annotation information, wherein the annotation information includes location information; Step S4: adding the annotation information in the form of a label to a position corresponding to the part information on the three-dimensional lung model; Step S5: Displaying the labeled three-dimensional lung model to the user through an AR device or a VR device; Step S6: detecting the interaction operation between the user and the three-dimensional lung model; Step S7: according to the interactive operation, modifying the display state of the label corresponding to the operated part of the operated lung three-dimensional model to a display state; The interactive operation includes: rendering different parts of the lung three-dimensional model into different colors, setting multiple trigger points in each color area, and when it is detected that the user's fingers in the virtual environment touch two trigger points in a color area at the same time, it is considered that the part corresponding to the color area is selected by the user and operated; The multiple users include one or more user groups, and one user group includes a first user, a second user, and multiple third users; in the user group, different parts of the three-dimensional lung model are rendered in different colors, wherein the same part is displayed in a first color to the first user and the second user in the user group, and in a second color to the third user in the user group, and the first color is different from the second color; the first color is set by the first user in the user group after the finger of the first user in the user group touches multiple trigger points of the same part in the virtual environment at the same time, and the first user in the user group designates a user as the second user in the user group to display the set first color to the second user, and designates other users as the third user in the user group to display a second color different from the set first color to the third user; wherein the multiple users form different user groups, each user group has its own user roles such as the first user, the second user, and the third user, and different users serve as the same and / or different user roles in each user group, the first colors in each user group are different from each other, and the second colors in each user group are the same; When different user groups use different first colors to display the same part, the part is displayed to the first user and the second user in these user groups as a color pattern of alternately arranged color strips formed by the different first colors.

2. The method according to claim 1, characterized in that In the three-dimensional lung model, an initial display state of the label is a hidden state.

3. The method according to claim 1, characterized in that The method further comprises: Recording step: recording the user's interactive operations in the virtual environment, and generating an operation trajectory according to the sequence of the user's interactive operations, wherein the trajectory is superimposed and displayed on the three-dimensional lung model.

4. A clinical teaching system based on lung clinical images, characterized in that: The system comprises: Module M1: acquiring multiple planar images of lung clinical images, recorded as the first image set; Module M2: constructing a three-dimensional lung model based on the planar images in the first image set; Module M3: Acquire clinical teaching information and convert it into annotation information, wherein the annotation information includes location information; Module M4: adding the annotation information in the form of a label to a position corresponding to the part information on the three-dimensional lung model; Module M5: Displaying a labeled three-dimensional lung model to a user through an AR device or a VR device; Module M6: detecting the interaction operation between the user and the three-dimensional lung model; Module M7: according to the interactive operation, modifying the display state of the label corresponding to the operated part of the operated three-dimensional lung model to a display state; The interactive operation includes: rendering different parts of the lung three-dimensional model into different colors, setting multiple trigger points in each color area, and when it is detected that the user's fingers in the virtual environment touch two trigger points in a color area at the same time, it is considered that the part corresponding to the color area is selected by the user and operated; The multiple users include one or more user groups, and one user group includes a first user, a second user, and multiple third users; in the user group, different parts of the three-dimensional lung model are rendered in different colors, wherein the same part is displayed in a first color to the first user and the second user in the user group, and in a second color to the third user in the user group, and the first color is different from the second color; the first color is set by the first user in the user group after the finger of the first user in the user group touches multiple trigger points of the same part in the virtual environment at the same time, and the first user in the user group designates a user as the second user in the user group to display the set first color to the second user, and designates other users as the third user in the user group to display a second color different from the set first color to the third user; wherein the multiple users form different user groups, each user group has its own user roles such as the first user, the second user, and the third user, and different users serve as the same and / or different user roles in each user group, the first colors in each user group are different from each other, and the second colors in each user group are the same; When different user groups use different first colors to display the same part, the part is displayed to the first user and the second user in these user groups as a color pattern of alternately arranged color strips formed by the different first colors.

5. The system according to claim 4, characterized in that In the three-dimensional lung model, an initial display state of the label is a hidden state.

6. The system according to claim 4, characterized in that The system further comprises: Recording module: records the user's interactive operations in the virtual environment, and generates an operation trajectory according to the sequence of the user's interactive operations, wherein the trajectory is superimposed on the three-dimensional lung model.

Images