Twin space design method and device for antenatal ultrasound teaching and use method

Abstract

The invention discloses a twin space design method and device for antenatal ultrasound teaching and a use method. The twin space design method for antenatal ultrasound teaching comprises the following steps: generating antenatal ultrasound three-dimensional volume data; establishing a virtual abdomen space of the pregnant woman; generating fetus 3D model data according to the antenatal ultrasonic three-dimensional volume data; translating the 3D model data of the fetus to the virtual abdomen space of the pregnant woman so as to generate virtual 3D data of the pregnant woman; and establishing a simulation probe in a coordinate system where the virtual pregnant woman 3D data is located, and associating the simulation probe with a probe with a space positioning sensor, thereby forming a twin space for antenatal ultrasound teaching. According to the prenatal ultrasound teaching twin space established by the invention, the abdomen position of a pregnant woman can be simulated for scanning, and a real ultrasound image can be fed back in real time according to the angle rotation and position change of the probe, so that the dependence on the real pregnant woman in the training process is avoided, and the operation skill training is realized.

Description

Technical Field

[0001] The present application relates to the field of twin technology, and in particular to a twin space design method for prenatal ultrasound teaching, a twin space design device for prenatal ultrasound teaching, and a method for using the twin space for prenatal ultrasound teaching. Background Art

[0002] Computer-based virtual ultrasound simulator, also known as immersive technology, refers to the use of three-dimensional graphics generation technology, sensor interaction technology and high-resolution display technology to carry out virtual training.

[0003] In recent years, this type of virtual ultrasound teaching system has made a series of progress, and corresponding commercial products have been launched, such as Canada's CAEVimedix, Russia's Medvision, and China's Guidong Medical Education DF-CS8000.

[0004] Its working principle is based on point RFID recognition logic technology, combining the software information case system with hardware PC and simulation probe positioning devices for virtual abdominal and virtual heart ultrasound training. When the simulation probe scans the correct position, a simulation image is given. Through the software system, ultrasound image scanning exercises of different parts are realized, so that students can learn the use of ultrasound equipment, image case recognition, etc., solving the problem that it is impossible to practice with real patients in large quantities in clinical ultrasound teaching. However, the existing virtual ultrasound simulator system mainly focuses on the simulation of abdominal and cardiac ultrasound images. The cross-sectional images are relatively simple, lacking real ultrasound images and lack of realism. In terms of resolution, image quality, and real-time performance, it cannot reach the level of actual ultrasound examinations.

[0005] Therefore, it is hoped that a technical solution can be provided to solve or at least alleviate the above-mentioned deficiencies of the prior art. Summary of the invention

[0006] The purpose of the present invention is to provide a twin space design method for prenatal ultrasound teaching to solve at least one of the above-mentioned technical problems.

[0007] The present invention provides the following scheme:

[0008] According to one aspect of the present invention, a twin space design method for prenatal ultrasound teaching is provided, and the twin space design method for prenatal ultrasound teaching comprises:

[0009] Generate prenatal ultrasound three-dimensional volume data;

[0010] Establishing a virtual abdominal space for pregnant women;

[0011] generating fetal 3D stereoscopic model data according to the prenatal ultrasound three-dimensional volume data;

[0012] Translating the fetus 3D stereoscopic model data to the virtual abdominal space of the pregnant woman to generate virtual pregnant woman 3D data;

[0013] A simulation probe is established in the coordinate system where the virtual pregnant woman's 3D data is located and is associated with a probe having a spatial positioning sensor, thereby forming a twin space for prenatal ultrasound teaching.

[0014] Optionally, generating prenatal ultrasound three-dimensional volume data includes:

[0015] Collect images during pregnancy;

[0016] The pregnancy images are identified by an image segmentation algorithm to obtain a region of interest and mark the region of interest, thereby obtaining prenatal ultrasound three-dimensional volume data, which includes a plurality of consecutive pregnancy images with marked regions of interest.

[0017] Optionally, the establishing of a virtual abdominal space of a pregnant woman comprises:

[0018] Obtaining an actual pregnant woman phantom;

[0019] Modeling the pregnant woman's body membrane to obtain a standard pregnant woman's virtual abdominal space;

[0020] Obtaining an obstacle point cloud database, wherein the obstacle point cloud database includes at least one type of obstacle point cloud information;

[0021] Obstacle point cloud information is randomly added to the initial model of the pregnant woman, so as to obtain at least one non-standard pregnant woman's virtual abdominal space.

[0022] Optionally, generating fetal 3D stereoscopic model data according to the prenatal ultrasound three-dimensional volume data includes:

[0023] Acquire a pathological logic database, wherein the pathological logic database includes at least one pathological condition information and a distorted ultrasound structure point of interest corresponding to each pathological condition information;

[0024] Acquire the correct distorted ultrasound structure points of interest;

[0025] Obtaining point cloud data of each pregnancy image for marking a region of interest respectively;

[0026] Establishing corresponding pathological three-dimensional models for each pathological condition information respectively through the point cloud data, wherein the distorted ultrasonic structural point of interest corresponding to one pathological condition information is used to establish a pathological three-dimensional model;

[0027] A standard three-dimensional model is established through the point cloud data according to the correctly distorted ultrasonic structural points of interest.

[0028] Optionally, before translating the fetal 3D stereoscopic model data to the pregnant woman's virtual abdominal space to generate virtual pregnant woman 3D data, the prenatal ultrasound teaching twin space design method further includes:

[0029] Get the historical performance information of the user to be used;

[0030] Select a non-standard pregnant woman virtual abdomen space or select the standard pregnant woman virtual abdomen space according to the historical performance information of the user;

[0031] Select a pathological three-dimensional model or the standard three-dimensional model according to the historical performance information of the user to be used; wherein,

[0032] The non-standard pregnant woman virtual abdomen space or the standard pregnant woman virtual abdomen space is selected as the pregnant woman virtual abdomen space for translating the fetus 3D stereoscopic model data;

[0033] The selected pathological three-dimensional model or the selected standard three-dimensional model is used as the 3D stereoscopic model data of the fetus for translation into the virtual abdominal space of the pregnant woman.

[0034] Optionally, translating the fetus 3D stereoscopic model data to the virtual abdominal space of the pregnant woman to generate virtual pregnant woman 3D data includes:

[0035] The fetal 3D stereoscopic model field is affine transformed and registered, and translated to the virtual abdominal space position of the pregnant woman corresponding to the pregnant woman phantom.

[0036] Optionally, identifying the pregnancy image by using an image segmentation algorithm includes:

[0037] Establish an improved Mask R-CNN neural network;

[0038] Training the improved Mask R-CNN neural network to obtain a trained improved Mask R-CNN neural network;

[0039] Obtain images at each trimester;

[0040] Each of the pregnancy images is input into a trained Mask R-CNN neural network to obtain a region of interest for each pregnancy image.

[0041] Optionally, the Mask R-CNN neural network is improved by the following method to obtain the improved Mask R-CNN neural network:

[0042] Improve the Mask R-CNN backbone network: add squeeze-excitation network modules and spatial attention modules to the C2-C5 layers of the ResNet101 structure;

[0043] A bottom-up branch is added on the basis of the original FPN. The D series layer is a newly added fusion path, which is added element by element with the original P series layer mapping. The feature pyramid fuses the high-level and low-level features to retain the bottom-level information. The multi-scale output features of the FPN after the third branch is added are fused, and the information of all scale feature maps is averaged to fully retain its semantic and position information.

[0044] The present application also provides a twin space design device for prenatal ultrasound teaching, the twin space design device for prenatal ultrasound teaching comprising:

[0045] A prenatal ultrasound three-dimensional volume data generation module, wherein the prenatal ultrasound three-dimensional volume data generation module is used to generate prenatal ultrasound three-dimensional volume data;

[0046] A virtual abdominal space establishment module for pregnant women, wherein the virtual abdominal space establishment module for pregnant women is used to establish a virtual abdominal space for pregnant women;

[0047] A fetal 3D stereoscopic model data generation module, the fetal 3D stereoscopic model data generation module is used to generate fetal 3D stereoscopic model data according to the prenatal ultrasound three-dimensional volume data;

[0048] A virtual pregnant woman 3D data generation module, the virtual pregnant woman 3D data generation module is used to translate the fetus 3D stereoscopic model data to the pregnant woman's virtual abdominal space to generate virtual pregnant woman 3D data;

[0049] An association module is used to establish an association between a simulation probe and a probe with a spatial positioning sensor in a coordinate system where the virtual pregnant woman's 3D data is located, thereby forming a twin space for prenatal ultrasound teaching.

[0050] The present application also provides a method for using a twin space for prenatal ultrasound teaching, and the method for using a twin space for prenatal ultrasound teaching includes:

[0051] Import the established twin space for prenatal ultrasound teaching into the Unity 3D engine, simulate the segmentation of the model file at any angle in the engine, assign three-dimensional space coordinate values ​​to the section images cut at any angle and combine them with the Unity engine;

[0052] Obtaining the position and depth of the probe located on the pregnant woman phantom;

[0053] Acquire a scanning plane of the user according to the orientation and depth;

[0054] The scanning plane parameters are passed to the prenatal ultrasound teaching twin space, the fetal 3D stereo model data is spatially cut in the prenatal ultrasound teaching twin space, the intersection of the fetal three-dimensional data field and the scanning plane is calculated, and the ultrasound section images of different angles can be arbitrarily displayed in the prenatal ultrasound teaching twin space, and the mapping relationship from the input data to the twin image is obtained.

[0055] The twin space for prenatal ultrasound teaching established by the twin space design method for prenatal ultrasound teaching of this application can "simulate" the abdominal position scanning of pregnant women, and can provide real ultrasound images in real time according to the rotation angle and position change of the probe, avoiding the dependence on "real pregnant women" during training and realizing operation skill training. In addition, in actual use, this application eliminates the need to manufacture real infant molds, and only requires actual pregnant women phantoms. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 It is a flow chart of the twin space design method for prenatal ultrasound teaching in one embodiment of the present application.

[0057] Figure 2 It is a schematic diagram of the structure of an electronic device in one embodiment of the present application.

[0058] Figure 3 Schematic diagram of a pregnant woman phantom in one embodiment of the present application. DETAILED DESCRIPTION

[0059] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0060] Figure 1 It is a flow chart of the twin space design method for prenatal ultrasound teaching in one embodiment of the present application.

[0061] like Figure 1 The twin space design method for prenatal ultrasound teaching shown includes:

[0062] Step 1: Generate prenatal ultrasound three-dimensional volume data;

[0063] Step 2: Establishing a virtual abdominal space for pregnant women;

[0064] Step 3: generating fetal 3D stereoscopic model data according to the prenatal ultrasound 3D volume data;

[0065] Step 4: translating the fetus 3D stereoscopic model data to the virtual abdominal space of the pregnant woman to generate virtual pregnant woman 3D data;

[0066] Step 5: Establish an association between a simulated probe and a probe with a spatial positioning sensor in the coordinate system where the virtual pregnant woman's 3D data is located, thereby forming a twin space for prenatal ultrasound teaching.

[0067] In this embodiment, generating prenatal ultrasound three-dimensional volume data includes:

[0068] Collect images during pregnancy;

[0069] The pregnancy images are identified by an image segmentation algorithm to obtain a region of interest and mark the region of interest, thereby obtaining prenatal ultrasound three-dimensional volume data, which includes a plurality of consecutive pregnancy images with marked regions of interest.

[0070] In this embodiment, the following method can be used to collect pregnancy images:

[0071] The GE Volusion E10 ultrasound imager and RAB 4-8D transabdominal three-dimensional volumetric ultrasound probe were used, and the probe frequency was 4.0-8.5MHz. The ultrasound examination was performed by two physicians with Beijing prenatal diagnosis certificates and more than 10 years of work experience. The midsagittal section of the fetus was selected, and the fetus was kept in a natural extension state (the long axis of the fetus was 90° with the ultrasound beam, without excessive flexion or extension); the forehead skin, frontal bone, nasal bone, nose tip, rectangular upper palate, mandible, diencephalon, and sagittal plane of the spine were clearly displayed; the top of the fetus and sacrum were clearly displayed; the image was enlarged to 1 / 3-1 / 2 of the screen. The three-dimensional volumetric region of interest covered the entire fetus, the scanning angle was set to 65°, the gain and depth were adjusted until the fetus was clearly displayed, and the entire fetus was scanned slowly to obtain prenatal ultrasound three-dimensional volumetric data.

[0072] In this embodiment, the step of establishing a virtual abdominal space for a pregnant woman includes:

[0073] Get the actual pregnant woman phantom ( Figure 3 shown);

[0074] Modeling the pregnant woman's body membrane to obtain a standard pregnant woman's virtual abdominal space;

[0075] Obtaining an obstacle point cloud database, wherein the obstacle point cloud database includes at least one type of obstacle point cloud information;

[0076] Obstacle point cloud information is randomly added to the initial model of the pregnant woman, so as to obtain at least one non-standard pregnant woman's virtual abdominal space.

[0077] In this embodiment, the present application provides a variety of types of non-standard pregnant woman virtual abdominal spaces. In this way, users can choose according to their own circumstances in actual use. For example, more experienced users can use non-standard pregnant woman virtual abdominal spaces with more obstacles to increase the difficulty, while beginners can use standard pregnant woman virtual abdominal spaces.

[0078] In this embodiment, generating fetal 3D stereoscopic model data according to the prenatal ultrasound three-dimensional volume data includes:

[0079] Acquire a pathological logic database, wherein the pathological logic database includes at least one pathological condition information and a distorted ultrasound structure point of interest corresponding to each pathological condition information;

[0080] Get the correct ultrasound structure points of interest;

[0081] Obtaining point cloud data of each pregnancy image for marking a region of interest respectively;

[0082] Establishing corresponding pathological three-dimensional models for each pathological condition information respectively through the point cloud data, wherein the distorted ultrasonic structural point of interest corresponding to one pathological condition information is used to establish a pathological three-dimensional model;

[0083] A standard three-dimensional model is established through the point cloud data according to the correct ultrasonic structural points of interest.

[0084] In this embodiment, the prenatal ultrasound three-dimensional volume data includes multiple two-dimensional ultrasound images. Point cloud data of each two-dimensional ultrasound image is obtained respectively and attribute information (the attribute information includes ultrasound structure points of interest) is added to each point cloud data, and a 3D stereo model is constructed based on the obtained point cloud data.

[0085] Ultrasound structural points of interest In this embodiment, the ultrasound structural points of interest may include the skull, face, spine, limbs, and various organs in the chest and abdomen.

[0086] In this embodiment, distorting the ultrasonic structural points of interest refers to intentionally distorting the point cloud data of some structures when establishing the model, for example, distorting the point cloud data of the limbs into unreasonable positions, or tampering with the point cloud data of the spine, etc.

[0087] In this embodiment, the obstacle point cloud information may include abnormal hemangioma, high-density shadow area, etc.

[0088] In this embodiment, translating the fetus 3D stereoscopic model data to the virtual abdominal space of the pregnant woman to generate virtual pregnant woman 3D data includes:

[0089] The fetal 3D stereoscopic model field is affine transformed and registered, and translated to the virtual abdominal space position corresponding to the pregnant woman phantom.

[0090] Using this method, each time when establishing the prenatal ultrasound teaching twin space, different non-standard pregnant woman virtual abdominal spaces or standard pregnant woman virtual abdominal spaces and randomly selected standard three-dimensional models and pathological three-dimensional models can be combined, so that different prenatal ultrasound teaching twin spaces can be used for teaching each time.

[0091] In this embodiment, before translating the fetal 3D stereoscopic model data to the pregnant woman's virtual abdominal space to generate virtual pregnant woman 3D data, the prenatal ultrasound teaching twin space design method further includes:

[0092] Get the historical performance information of the user to be used;

[0093] Select a non-standard pregnant woman virtual abdomen space or select the standard pregnant woman virtual abdomen space according to the historical performance information of the user;

[0094] Select a pathological three-dimensional model or the standard three-dimensional model according to the historical performance information of the user to be used; wherein,

[0095] The non-standard pregnant woman virtual abdomen space or the standard pregnant woman virtual abdomen space is selected as the pregnant woman virtual abdomen space for translating the fetus 3D stereoscopic model data;

[0096] The selected pathological three-dimensional model or the selected standard three-dimensional model is used as the 3D stereoscopic model data of the fetus for translation into the virtual abdominal space of the pregnant woman.

[0097] In this way, a prenatal ultrasound teaching twin space can be prepared for users who have previously performed poorly based on their historical performance information, or a prenatal ultrasound teaching twin space can be provided for users who have never been taught before.

[0098] In this embodiment, identifying the pregnancy image by using an image segmentation algorithm includes:

[0099] Establish an improved Mask R-CNN neural network;

[0100] Training the improved Mask R-CNN neural network to obtain a trained improved Mask R-CNN neural network;

[0101] Obtain images at each trimester;

[0102] Each of the pregnancy images is input into a trained Mask R-CNN neural network to obtain a region of interest for each pregnancy image.

[0103] Specifically, a CoreTM i7-8700k 3.7GHz CPU, NVDIARTX 2080s graphics card, and 8GB video memory are used. The software environment is a learning framework based on Keras, supported by Tensorflow, and the data is stored in raw data format. Comprehensive consideration is given to multiple variable factors such as image grayscale brightness uniformity, identification of boundaries between tissues in the inspection site, strong echo / echo / hypoecho / anechoic contrast matching, frame rate real-time, edge enhancement, granularity adjustment, and artifact screening to achieve the best simulation of real practical operation. Multiple preprocessing methods such as up and down flipping, left and right flipping, brightness change, random block attack, and deformation distortion are used to increase the generalization ability of the model. The Mask R-CNN backbone network was improved by adding squeeze-and-excitation networks (SENet) and spatial attention modules (SESNet) to the C2-C5 layers of the ResNet101 structure. A bottom-up branch was added on the basis of the original FPN. The D series layer was a new fusion path, which was added element by element with the original P series layer mapping. The feature pyramid (FPN) fused the high-level and low-level features to retain the bottom-level information. The multi-scale output features of the FPN after the third branch were fused, and the information of all scale feature maps was averaged to fully retain its semantic and position information. The Mask R-CNN model was trained using the prepared dataset, and the model was verified and optimized to ensure that it can effectively and accurately segment obstetric ultrasound images. The performance of the trained Mask R-CNN model in the obstetric ultrasound image segmentation task was evaluated, including accuracy, recall, F1 score and other indicators, and compared with other segmentation algorithms. The digital three-dimensional data field (Cartesian coordinate system) of the prenatal ultrasound image is obtained based on the spatial interpolation technology and stored in the computer.

[0104] In this embodiment, the Mask R-CNN neural network is improved by the following method to obtain the improved Mask R-CNN neural network:

[0105] Improve the Mask R-CNN backbone network: add squeeze-excitation network modules and spatial attention modules to the C2-C5 layers of the ResNet101 structure;

[0106] A bottom-up branch is added on the basis of the original FPN. The D series layer is a newly added fusion path, which is added element by element with the original P series layer mapping. The feature pyramid fuses the high-level and low-level features to retain the bottom-level information. The multi-scale output features of the FPN after the third branch is added are fused, and the information of all scale feature maps is averaged to fully retain its semantic and position information.

[0107] The present application also provides a twin space design device for prenatal ultrasound teaching, which includes a prenatal ultrasound three-dimensional volume data generation module, a pregnant woman virtual abdominal space establishment module, a fetal 3D stereoscopic model data generation module, a virtual pregnant woman 3D data generation module and an association module, wherein:

[0108] The prenatal ultrasound three-dimensional volume data generation module is used to generate prenatal ultrasound three-dimensional volume data;

[0109] The pregnant woman's virtual abdominal space establishment module is used to establish the pregnant woman's virtual abdominal space;

[0110] The fetal 3D stereoscopic model data generation module is used to generate fetal 3D stereoscopic model data according to the prenatal ultrasound three-dimensional volume data;

[0111] The virtual pregnant woman 3D data generation module is used to translate the fetus 3D stereoscopic model data to the virtual abdominal space of the pregnant woman to generate virtual pregnant woman 3D data;

[0112] The association module is used to establish an association between a simulation probe and a probe with a spatial positioning sensor in the coordinate system where the virtual pregnant woman's 3D data is located, thereby forming a twin space for prenatal ultrasound teaching.

[0113] The present application also provides a method for using a twin space for prenatal ultrasound teaching, and the method for using a twin space for prenatal ultrasound teaching includes:

[0114] Import the established twin space for prenatal ultrasound teaching into the Unity 3D engine, simulate the segmentation of the model file at any angle in the engine, assign three-dimensional space coordinate values ​​to the section images cut at any angle and combine them with the Unity engine;

[0115] Obtaining the position and depth of the probe located on the pregnant woman phantom;

[0116] Acquire a scanning plane of the user according to the orientation and depth;

[0117] The scanning plane parameters are passed to the prenatal ultrasound teaching twin space, the fetal 3D stereo model data is spatially cut in the prenatal ultrasound teaching twin space, the intersection of the fetal three-dimensional data field and the scanning plane is calculated, and the ultrasound section images of different angles can be arbitrarily displayed in the prenatal ultrasound teaching twin space, and the mapping relationship from the input data to the twin image is obtained.

[0118] like Figure 2 As shown, the present application also discloses an electronic device, including: a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; a computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the steps of the twin space design method for prenatal ultrasound teaching.

[0119] The present application also provides a computer-readable storage medium, which stores a computer program that can be executed by an electronic device. When the computer program runs on the electronic device, it can implement the steps of the twin space design method for prenatal ultrasound teaching.

[0120] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0121] The electronic device includes a hardware layer, an operating system layer running on the hardware layer, and an application layer running on the operating system. The hardware layer includes hardware such as a central processing unit (CPU), a memory management unit (MMU), and a memory. The operating system can be any one or more computer operating systems that implement electronic device control through a process, such as a Linux operating system, a Unix operating system, an Android operating system, an iOS operating system, or a Windows operating system. In addition, in an embodiment of the present invention, the electronic device can be a handheld device such as a smart phone or a tablet computer, or can be an electronic device such as a desktop computer or a portable computer, which is not particularly limited in the embodiment of the present invention.

[0122] The execution subject of the electronic device control in the embodiment of the present invention may be an electronic device, or a functional module in the electronic device that can call and execute a program. The electronic device may obtain the firmware corresponding to the storage medium. The firmware corresponding to the storage medium is provided by the supplier. The firmware corresponding to different storage media may be the same or different, which is not limited here. After the electronic device obtains the firmware corresponding to the storage medium, the firmware corresponding to the storage medium may be written into the storage medium, specifically, the firmware corresponding to the storage medium may be burned into the storage medium. The process of burning the firmware into the storage medium may be implemented using existing technology, which will not be described in detail in the embodiment of the present invention.

[0123] The electronic device may also obtain a reset command corresponding to the storage medium. The reset command corresponding to the storage medium is provided by the supplier. The reset commands corresponding to different storage media may be the same or different, and are not limited here.

[0124] At this time, the storage medium of the electronic device is a storage medium in which the corresponding firmware is written, and the electronic device can respond to the reset command corresponding to the storage medium in the storage medium in which the corresponding firmware is written, so that the electronic device resets the storage medium in which the corresponding firmware is written according to the reset command corresponding to the storage medium. The process of resetting the storage medium according to the reset command can be implemented by the existing technology and will not be described in detail in the embodiments of the present invention.

[0125] For the convenience of description, the above devices are described in terms of functions and are divided into various units and modules. Of course, when implementing the present application, the functions of each unit and module can be implemented in the same or multiple software and / or hardware.

[0126] Those skilled in the art will appreciate that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those generally understood by those skilled in the art in the art to which the present invention belongs. It should also be understood that terms such as those defined in general dictionaries should be understood to have meanings consistent with the meanings in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless specifically defined.

[0127] For the method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0128] It can be known from the description of the above implementation modes that those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be essentially or partly contributed to the prior art in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes several instructions for enabling a computer device (which can be a personal computer, a server or a network device, etc.) to execute the methods described in the various implementation modes of the present application or certain parts of the implementation modes.

[0129] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A twin space design method for prenatal ultrasound teaching, characterized in that: The twin space design method for prenatal ultrasound teaching includes: Generate prenatal ultrasound three-dimensional volume data; Establishing a virtual abdominal space for pregnant women; generating fetal 3D stereoscopic model data according to the prenatal ultrasound three-dimensional volume data; Translating the fetus 3D stereoscopic model data to the virtual abdominal space of the pregnant woman to generate virtual pregnant woman 3D data; A simulation probe is established in the coordinate system where the virtual pregnant woman's 3D data is located and is associated with a probe having a spatial positioning sensor, thereby forming a twin space for prenatal ultrasound teaching.

2. The twin space design method for prenatal ultrasound teaching according to claim 1, characterized in that: The generating of prenatal ultrasound three-dimensional volume data comprises: Collect images during pregnancy; The pregnancy images are identified by an image segmentation algorithm to obtain a region of interest and mark the region of interest, thereby obtaining prenatal ultrasound three-dimensional volume data, which includes a plurality of consecutive pregnancy images with marked regions of interest.

3. The twin space design method for prenatal ultrasound teaching according to claim 2, characterized in that: The establishment of a virtual abdominal space for pregnant women comprises: Obtaining an actual pregnant woman phantom; Modeling the pregnant woman's body membrane to obtain a standard pregnant woman's virtual abdominal space; Obtaining an obstacle point cloud database, wherein the obstacle point cloud database includes at least one type of obstacle point cloud information; Obstacle point cloud information is randomly added to the initial model of the pregnant woman, so as to obtain at least one non-standard pregnant woman's virtual abdominal space.

4. The twin space design method for prenatal ultrasound teaching according to claim 3, characterized in that: Generating fetal 3D stereoscopic model data according to the prenatal ultrasound three-dimensional volume data includes: Acquire a pathological logic database, wherein the pathological logic database includes at least one pathological condition information and a distorted ultrasound structure point of interest corresponding to each pathological condition information; Get the correct ultrasound structure points of interest; Obtaining point cloud data of each pregnancy image for marking a region of interest respectively; Establishing corresponding pathological three-dimensional models for each pathological condition information respectively through the point cloud data, wherein the distorted ultrasonic structural point of interest corresponding to one pathological condition information is used to establish a pathological three-dimensional model; A standard three-dimensional model is established through the point cloud data according to the correct ultrasonic structural points of interest.

5. The twin space design method for prenatal ultrasound teaching according to claim 4, characterized in that: Before translating the fetal 3D stereoscopic model data to the virtual abdominal space of the pregnant woman to generate virtual 3D data of the pregnant woman, the prenatal ultrasound teaching twin space design method further includes: Get the historical performance information of the user to be used; Select a non-standard pregnant woman virtual abdomen space or select the standard pregnant woman virtual abdomen space according to the historical performance information of the user; Select a pathological three-dimensional model or the standard three-dimensional model according to the historical performance information of the user to be used; wherein, The non-standard pregnant woman virtual abdomen space or the standard pregnant woman virtual abdomen space is selected as the pregnant woman virtual abdomen space for translating the fetus 3D stereoscopic model data; The selected pathological three-dimensional model or the selected standard three-dimensional model is used as the 3D stereoscopic model data of the fetus for translation into the virtual abdominal space of the pregnant woman.

6. The twin space design method for prenatal ultrasound teaching according to claim 5, characterized in that: The step of translating the fetus 3D stereoscopic model data to the virtual abdominal space of the pregnant woman to generate virtual pregnant woman 3D data comprises: The fetal 3D stereoscopic model field is affine transformed and registered, and translated to the virtual abdominal space position of the pregnant woman corresponding to the pregnant woman phantom.

7. The twin space design method for prenatal ultrasound teaching according to claim 6, characterized in that: The identifying of pregnancy images by using an image segmentation algorithm comprises: Establish an improved Mask R-CNN neural network; Training the improved Mask R-CNN neural network to obtain a trained improved Mask R-CNN neural network; Obtain images at each trimester; Each of the pregnancy images is input into a trained Mask R-CNN neural network to obtain a region of interest for each pregnancy image.

8. The twin space design method for prenatal ultrasound teaching according to claim 7, characterized in that: The Mask R-CNN neural network is improved by the following method to obtain the improved Mask R-CNN neural network: Improve the Mask R-CNN backbone network: add squeeze-excitation network modules and spatial attention modules to the C2-C5 layers of the ResNet101 structure; A bottom-up branch is added on the basis of the original FPN. The D series layer is a newly added fusion path, which is added element by element with the original P series layer mapping. The feature pyramid fuses the high-level and low-level features to retain the bottom-level information. The multi-scale output features of the FPN after the third branch is added are fused, and the information of all scale feature maps is averaged to fully retain its semantic and position information.

9. A twin space design device for prenatal ultrasound teaching, characterized in that: The twin space design device for prenatal ultrasound teaching comprises: A prenatal ultrasound three-dimensional volume data generation module, wherein the prenatal ultrasound three-dimensional volume data generation module is used to generate prenatal ultrasound three-dimensional volume data; A virtual abdominal space establishment module for pregnant women, wherein the virtual abdominal space establishment module for pregnant women is used to establish a virtual abdominal space for pregnant women; A fetal 3D stereoscopic model data generation module, the fetal 3D stereoscopic model data generation module is used to generate fetal 3D stereoscopic model data according to the prenatal ultrasound three-dimensional volume data; A virtual pregnant woman 3D data generation module, the virtual pregnant woman 3D data generation module is used to translate the fetus 3D stereoscopic model data to the pregnant woman's virtual abdominal space to generate virtual pregnant woman 3D data; An association module is used to establish an association between a simulation probe and a probe with a spatial positioning sensor in the coordinate system where the virtual pregnant woman's 3D data is located, thereby forming a twin space for prenatal ultrasound teaching.

10. A method for using twin space for prenatal ultrasound teaching, characterized in that: The method for using the twin space for prenatal ultrasound teaching includes: Import the established twin space for prenatal ultrasound teaching into the Unity 3D engine, simulate the segmentation of the model file at any angle in the engine, assign three-dimensional space coordinate values ​​to the section images cut at any angle and combine them with the Unity engine; Obtaining the position and depth of the probe located on the pregnant woman phantom; Acquire a scanning plane of the user according to the orientation and depth; The scanning plane parameters are passed to the prenatal ultrasound teaching twin space, the fetal 3D stereo model data is spatially cut in the prenatal ultrasound teaching twin space, the intersection of the fetal three-dimensional data field and the scanning plane is calculated, and the ultrasound section images of different angles can be arbitrarily displayed in the prenatal ultrasound teaching twin space, and the mapping relationship from the input data to the twin image is obtained.

Images