Interactive obstetrical handheld ultrasonic decision support report generation method based on AI

Abstract

The invention discloses an interactive obstetrical handheld ultrasonic decision support report generation method based on AI. The method comprises the following steps: S1, carrying out fine adjustment on a Flamingo model by adopting a P-Tuning method; s2, learning image features in different stages of the perinatal period; s3, constructing an ultrasonic mode judgment decision tree model; and S4, interactive appearance construction. According to the method, the AI is utilized to assist doctors in performing refined and personalized examination and diagnosis aiming at different pregnancy and birth stages, and specific examination reports and diagnosis and treatment schemes are given, so that the time consumption of obstetrical ultrasonic examination can be greatly shortened, the turnover rate of obstetrical ultrasonic doctors and obstetrical ultrasonic equipment is effectively improved, the popularizing rate of obstetrical ultrasonic examination is further improved, and the obstetrical ultrasonic examination efficiency is improved. Therefore, the occurrence of bad health outcome of pregnant and lying-in women and fetuses is prevented.

Description

Technical Field

[0001] The present invention belongs to the field of medical detection technology, and specifically relates to an AI-based interactive obstetric handheld ultrasound decision support report generation method. Background Art

[0002] Ultrasound is a non-invasive diagnostic tool widely used in obstetrics. Due to its high resolution and real-time performance, it can clearly display the structure and dynamics of the fetus, helping doctors to promptly detect fetal developmental abnormalities and maternal disease risks, and providing an important basis for clinical diagnosis and treatment. In addition, handheld ultrasound devices (POCUS, Point-of-Care Ultrasound) are small and portable, which expands the scope of application of obstetric ultrasound. According to the timing and purpose of the examination, obstetric ultrasound can be divided into prenatal ultrasound, intrapartum ultrasound and postpartum ultrasound. Prenatal ultrasound can be divided into levels I to IV according to the specificity of ultrasound examination needs in different pregnancy periods. Among them, level I ultrasound is the most basic prenatal ultrasound examination, which is usually performed in early pregnancy. The main purpose is to confirm the presence, location, number of fetuses and preliminary assessment of the fetal heartbeat. In addition, the gestational age can be estimated by measuring the size of the fetus to help preliminarily determine the due date. Level II ultrasound is a more comprehensive ultrasound examination, which aims to examine the structure of the fetus in detail, including the head, heart, spine, and abdominal organs. Level II ultrasound is also used to screen for certain common birth defects, such as: preliminary evaluation of Down syndrome and other chromosomal abnormalities, as well as fetal growth and amniotic fluid volume. Level III ultrasound is a more detailed examination performed when a potential problem is found with a level II ultrasound, including a detailed evaluation of the fetal heart or a more in-depth examination of other organs and systems. Level IV ultrasound is the highest level of ultrasound examination, which is mainly used in complex diagnostic situations, such as: a high suspicion of a serious malformation or genetic disease in the fetus. It can be seen that compared with other ultrasound examinations, obstetric ultrasound is more complicated to operate, the results are more difficult to interpret, and the experience and qualifications of doctors are relatively higher. This has limited the popularity and accuracy of ultrasound examinations to a certain extent.

[0003] Artificial Intelligence (AI) has been applied to ultrasound examination, diagnosis, and report generation. However, current technical solutions have not been fully integrated into doctors' daily workflows, and there are problems such as low efficiency. First, the formulation of ultrasound examination plans often relies too much on standardized processes and fails to fully consider the unique physiological characteristics and medical history of each pregnant woman, resulting in a significant deviation between the universality of the examination strategy and the individualized medical needs, which in turn weakens the pertinence and accuracy of the examination results. Secondly, the rigid architecture of the traditional decision tree model lacks the flexibility to cope with the dynamic changes in clinical scenarios, especially in the face of emergency medical conditions or complex pregnancy complications. Its guidance effectiveness is significantly limited, and it cannot provide medical personnel with immediate and context-adapted decision support, thus affecting the timeliness and accuracy of clinical decision-making. In addition, the presentation method and information organization logic of the user interface are also insufficient, which is manifested in the lack of orderliness of information display and low visual recognition, which poses a challenge to the ability of doctors to quickly locate key information under high-pressure environments. Especially in urgent clinical environments, such design defects may lead to misinterpretation or omission of important medical information, thereby affecting the efficiency and safety of diagnosis and treatment. The complex data analysis process places high cognitive demands on doctors, requiring them to invest extra time and energy in interpreting test results. This additional cognitive load not only consumes limited medical resources, but also directly reduces the overall efficiency and quality of medical services. Summary of the invention

[0004] In order to solve the above problems, the present invention proposes an AI-based interactive obstetric handheld ultrasound decision support report generation method.

[0005] The AI-based interactive obstetric handheld ultrasound decision support report generation method of the present invention comprises the following steps:

[0006] S1. Fine-tune the Flamingo model using the P-Tunning method;

[0007] S2. Learning image features at different stages of the perinatal period;

[0008] S3. Constructing an ultrasound mode judgment decision tree model;

[0009] S4. Interactive appearance construction.

[0010] The P-Tunning method is used in S1 to fine-tune the Flamingo model, including the following steps:

[0011] S101. Initialize prefix vector;

[0012] S102. B-ultrasound image and text data are input into the Flamingo model;

[0013] S103. The Flamingo model analyzes input data and generates preliminary diagnostic suggestions and parameter adjustment suggestions;

[0014] S104. Update prefix vector and model parameters;

[0015] S105. Use the Adam optimization algorithm to update the parameters of the Flamingo model.

[0016] The S2 learning of image features at different stages of the perinatal period refers to identifying patterns and features of different stages of the perinatal period by using a deep learning algorithm to identify standard ultrasound images and abnormal ultrasound images before, during, and after delivery based on the specific diagnostic requirements of different stages of the perinatal period.

[0017] The decision tree models in S3 include a conventional prenatal ultrasound decision tree model, a prenatal fetal abnormality screening ultrasound decision tree model, and an intrapartum and postpartum ultrasound decision tree model.

[0018] The conventional prenatal ultrasound decision tree model includes determining whether the pregnant woman is in the prenatal stage, determining whether the pregnant woman has an obstetric emergency, performing a level I obstetric ultrasound examination, and generating a prenatal examination report.

[0019] The prenatal ultrasound decision tree model includes judging whether the pregnant woman is in the prenatal stage, judging whether the pregnant woman has an obstetric emergency, judging whether it is necessary to perform an abnormal fetal structure examination, performing an NT ultrasound examination, judging whether the pregnant woman has completed a level II obstetric ultrasound examination, performing a level II obstetric ultrasound examination, performing a level III obstetric ultrasound examination, judging whether there are six major lethal malformations, judging whether there are structural malformations, and generating a prenatal examination report.

[0020] The prenatal fetal abnormality screening ultrasound decision tree model includes three stages of pregnant women at 6-14 weeks, 18-24 weeks and 28-34 weeks.

[0021] The intrapartum and postpartum ultrasound decision tree model includes determining whether the examinee is in the prenatal stage, determining whether the examinee has entered the intrapartum ultrasound, determining whether the examinee has an obstetric emergency, and generating a prenatal examination report.

[0022] The interactive appearance construction of S4 refers to setting a visual interactive menu for ultrasound mode selection based on a specific ultrasound mode decision tree model for different perinatal stages combined with a knowledge graph of ultrasound examinations for different perinatal stages.

[0023] Updating the prefix vector and model parameters in S104 refers to the difference between the text generated by the model and the real text calculated by the loss function, and calculating the loss gradient and updating the prefix vector and task-specific model parameters through back propagation.

[0024] The beneficial effects of the present invention are:

[0025] 1) The fine-tuned large language model is more professional: The present invention intends to fine-tune the Flamingo model through the P-Tuning method. Compared with the traditional AI technology used in the field of medical imaging, P-Tuning only needs to adjust the corresponding prompt words to quickly and accurately guide the model to generate better multimodal output. At the same time, this fine-tuning method has low requirements for computing resources and is an efficient and flexible fine-tuning solution. After fine-tuning, the large model will have stronger professionalism in obstetric ultrasound examinations and reports.

[0026] 2) The interactive ultrasound examination and reporting system can improve the efficiency of obstetric ultrasound examinations: The present invention intends to design an AI interactive examination and reporting system that is more convenient for examination plan formulation and report generation based on the specific diagnostic requirements of different stages of the perinatal period. Importantly, this interactive ultrasound examination and reporting system covers three perinatal stages, namely, prenatal, intrapartum, and postpartum. It can quickly determine the ultrasound examination mode and the biological indicator parameters that need to be measured based on the information of the pregnant woman, thereby significantly improving the efficiency of obstetric ultrasound examinations.

[0027] 3) Visual AI interactive menu lowers the usage threshold: The present invention intends to design the appearance of a visual AI interactive menu based on the ultrasound mode selection decision tree model. AI gives questions and options for judging scenarios. Doctors only need to make a selection or briefly add details, and AI can assist in completing the ultrasound mode and subsequent parameter settings. It can assist doctors in quickly selecting ultrasound examination items and indicators, and improve the efficiency of subsequent examinations and report generation.

[0028] 4) Generate personalized prenatal examination reports by combining ultrasound examination images and multimodal health information data: Compared with other large models, Flamingo is a multimodal large model that can process multiple data types such as text and images, and is suitable for complex data analysis tasks in the medical field. At the same time, the loss function calculates the difference between the generated text and the real text, which greatly improves the professionalism and work efficiency of the model, thereby assisting in generating more accurate and personalized prenatal examination reports. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a flow chart of fine-tuning the Flamingo model using the P-Tuning method of the present invention.

[0030] Figure 2 It is a model diagram of the routine prenatal ultrasound decision tree of the present invention.

[0031] Figure 3 This is a model diagram of the prenatal ultrasound decision tree for fetal abnormality screening of the present invention.

[0032] Figure 4 It is the model diagram of the intrapartum and postpartum ultrasound decision tree of the present invention.

[0033] Figure 5 It is the large language prompt word of the embodiment of the present invention. DETAILED DESCRIPTION

[0034] Embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings. The embodiments described below with reference to the accompanying drawings are exemplary and intended to be used to explain the present invention, but should not be understood as limiting the present invention.

[0035] The present invention uses a finely tuned large model to assist doctors in quickly formulating ultrasound examination plans and contents based on information such as the pregnancy period and risk factors for adverse outcomes of pregnant women, and the large model intelligently generates personalized prenatal examination reports to improve the efficiency of diagnosis and treatment. The process of the present invention is described in detail below:

[0036] The AI-based interactive obstetric handheld ultrasound decision support report generation method of the present invention comprises the following steps:

[0037] S1. Fine-tune the Flamingo model using the P-Tunning method: In fine-tuning the large model, we use the P-Tuning method to fine-tune the Flamingo model (see Appendix Figure 1 ), the specific implementation steps at the AI ​​algorithm level are described in detail below:

[0038] First, the prefix vectors are initialized. These vectors are randomly initialized or selected from the model's embedding space as learnable parameters. These prefix vectors have a length of 2,000 dimensions, ensuring that different types of data can be effectively represented in multimodal inputs. Ultrasound images and text data (such as maternal medical history, symptom descriptions, etc.) are directly input into the Flamingo model, which processes these multimodal inputs through forward propagation and generates corresponding outputs.

[0039] During the forward propagation, the Flamingo model uses its internal structure to process and analyze the input data to generate preliminary diagnostic suggestions and parameter adjustment suggestions. The model calculates the difference between the generated text and the real text through a loss function, which consists of the language model loss (measured using the cross entropy loss function) and task-specific metrics. Through backpropagation, the loss gradient is calculated and the prefix vector and task-specific model parameters are updated, while keeping most of the parameters of the Flamingo model unchanged. Using the Adam optimization algorithm for parameter updates, the system is able to accelerate convergence while maintaining performance.

[0040] S2. Learning of image features at different stages of the perinatal period: Based on the specific diagnostic requirements of different stages of the perinatal period, standard ultrasound images and abnormal ultrasound images before, during and after delivery are processed through a deep learning algorithm to identify patterns and features at different stages of the perinatal period.

[0041] S3. Constructing a decision tree model for judging ultrasound mode: Input the ultrasound operation guidelines, expert consensus, and ultrasound report data from hospitals around the country into the large language model, and construct a decision tree model for judging ultrasound examination schemes based on the keywords of specific diagnostic requirements at different stages of the perinatal period. The decision tree models include the decision tree model for routine prenatal ultrasound, the decision tree model for prenatal malformation screening ultrasound, and the decision tree model for intrapartum and postpartum ultrasound.

[0042] 1) Decision tree model of routine prenatal ultrasound is as follows Figure 2 As shown in the figure, first, the B-ultrasound doctor determines whether the pregnant woman is in the prenatal stage based on the clinical data. If the judgment is "yes", it is further determined whether the pregnant woman has an obstetric emergency. The large model matches the keywords of obstetric emergencies, such as vaginal bleeding, abdominal pain, trauma, and disappearance of fetal movement. If it is judged as "yes", a level I obstetric ultrasound examination is further performed, and then the fine-tuned large language model intelligently generates a prenatal examination report based on the examination results and clinical data. If it is judged that there is no obstetric emergency, the B-ultrasound doctor will evaluate whether a fetal structural abnormality examination is needed. When the evaluation result is "no" and there is an indication for obstetric examination, a level I obstetric ultrasound examination is performed, and the fine-tuned large language model intelligently generates a prenatal examination report.

[0043] 2) Decision tree for prenatal ultrasound screening is as follows Figure 3 As shown, first, the B-ultrasound doctor determines whether the pregnant woman is in the prenatal stage. If the judgment is "yes", it will further assess whether the pregnant woman has an obstetric emergency. If the judgment is "no", it will assess whether an abnormal fetal structure examination is needed. If the judgment is "yes", the B-ultrasound doctor needs to input the gestational age range of the pregnant woman and determine the next inspection plan with the assistance of the fine-tuned large language model.

[0044] When the pregnant woman's gestational age is between 6 and 14 weeks, an NT ultrasound examination is performed. At the same time, the fine-tuned large language model automatically combines the pregnant woman's age and laboratory test results with the NT ultrasound examination results, analyzes and evaluates the risk of fetal chromosomal abnormalities, and finally intelligently generates a prenatal examination report based on the evaluation results.

[0045] When the gestational age of a pregnant woman is between 18 and 24 weeks, it is determined whether the pregnant woman has completed a level II obstetric ultrasound examination in the past. If the judgment result is "yes", the large model will search for keywords indicating whether there are further obstetric ultrasound examinations, such as follow-up examinations of fetal malformations, follow-up examinations of placenta previa, a history of abnormal fetal pregnancy or embryonic arrest, suspected aneuploidy, chromosomal abnormalities, chromosomal abnormalities of one or both parents, malformations of one or both parents, familial genetic diseases, maternal diabetes, connective tissue diseases, kidney diseases, pregnancy-induced hypertension, viral infections in early pregnancy, congenital heart disease, smoking, alcoholism, early pregnancy NT thickening, in vitro fertilization (IVF), etc. If the judgment result is "yes", the level III obstetric ultrasound examination will be completed, and the main anatomical structures of the fetus will be observed with the assistance of the fine-tuned large language model, and the presence of the six major lethal malformations will be assessed, and then the prenatal examination report will be intelligently generated. If the pregnant woman has not undergone a level II prenatal ultrasound examination before, a level II obstetric ultrasound examination will be performed to determine whether the six major lethal malformations exist, and ultimately a prenatal examination report will be generated.

[0046] When the gestational age of a pregnant woman is between 28 and 34 weeks, a level III obstetric ultrasound examination is performed to observe the anatomical structure of the fetus and determine whether there are structural malformations. Ultimately, the prenatal examination report is intelligently generated by the fine-tuned large language model.

[0047] 3) Decision tree for intrapartum and postpartum ultrasound is as follows Figure 3 As shown, first, the B-ultrasound doctor determines whether the examinee is in the prenatal stage. If the judgment is "no", it is further determined whether the examinee has entered the intrapartum ultrasound. If the judgment is "yes", the fetal position is monitored in real time with the assistance of the fine-tuned language data model, the degree of cervical dilation is confirmed, and the progress of labor is tracked, and finally an intelligent prenatal examination report is generated.

[0048] If the doctor chooses not to perform an intrapartum ultrasound, a postpartum ultrasound examination will be performed to assess whether there are obstetric emergencies (such as intrauterine blood, intra-abdominal or abdominal wall hematoma, postpartum hemorrhage, retained placenta), and then an intelligent prenatal examination report will be generated by the fine-tuned language data model.

[0049] S4. Interactive appearance construction: Based on the decision tree model of specific ultrasound modes in different stages of the perinatal period, combined with the knowledge graph of ultrasound examinations in different stages of the perinatal period, a visual interactive menu is set for ultrasound mode selection, and appropriate examination parameters are recommended to assist doctors in ultrasound mode selection and improve the efficiency of subsequent examinations and report generation.

[0050] Finally, in order to output personalized prenatal examination reports in a unified format, we designed the large language prompts as follows: Figure 5 shown.

[0051] Although the above embodiments have been shown and described, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations on the present invention. Changes, modifications, substitutions and variations of the above embodiments by those of ordinary skill in the art are all within the scope of protection of the present invention.

Claims

1. An AI-based interactive obstetric handheld ultrasound decision support report generation method, characterized in that: The following steps are involved: S1. Fine-tune the Flamingo model using the P-Tunning method; S2. Learning image features at different stages of the perinatal period; S3. Constructing an ultrasound mode judgment decision tree model; S4. Interactive appearance construction.

2. The AI-based interactive obstetric handheld ultrasound decision support report generation method according to claim 1, characterized in that: In S1, the P-Tunning method is used to fine-tune the Flamingo model, including the following steps: S101. Initialize prefix vector; S102. B-ultrasound image and text data are input into the Flamingo model; S103. The Flamingo model analyzes input data and generates preliminary diagnostic suggestions and parameter adjustment suggestions; S104. Update prefix vector and model parameters; S105. Use the Adam optimization algorithm to update the parameters of the Flamingo model.

3. The AI-based interactive obstetric handheld ultrasound decision support report generation method according to claim 1, characterized in that: The S2 learning of image features at different stages of the perinatal period refers to identifying patterns and features of different stages of the perinatal period by using a deep learning algorithm to identify standard ultrasound images and abnormal ultrasound images before, during, and after delivery based on the specific diagnostic requirements of different stages of the perinatal period.

4. The AI-based interactive obstetric handheld ultrasound decision support report generation method according to claim 1, characterized in that: The decision tree models in S3 include a conventional prenatal ultrasound decision tree model, a prenatal fetal abnormality screening ultrasound decision tree model, and an intrapartum and postpartum ultrasound decision tree model.

5. The AI-based interactive obstetric handheld ultrasound decision support report generation method according to claim 4, characterized in that: The conventional prenatal ultrasound decision tree model includes determining whether the pregnant woman is in the prenatal stage, determining whether the pregnant woman has an obstetric emergency, performing a level I obstetric ultrasound examination, and generating a prenatal examination report.

6. The AI-based interactive obstetric handheld ultrasound decision support report generation method according to claim 4, characterized in that: The prenatal ultrasound decision tree model includes judging whether the pregnant woman is in the prenatal stage, judging whether the pregnant woman has an obstetric emergency, judging whether it is necessary to perform an abnormal fetal structure examination, performing an NT ultrasound examination, judging whether the pregnant woman has completed a level II obstetric ultrasound examination, performing a level II obstetric ultrasound examination, performing a level III obstetric ultrasound examination, judging whether there are six major lethal malformations, judging whether there are structural malformations, and generating a prenatal examination report.

7. The AI-based interactive obstetric handheld ultrasound decision support report generation method according to claim 4, characterized in that: The prenatal fetal abnormality screening ultrasound decision tree model includes three stages of pregnant women at 6-14 weeks, 18-24 weeks and 28-34 weeks.

8. The AI-based interactive obstetric handheld ultrasound decision support report generation method according to claim 4, characterized in that: The intrapartum and postpartum ultrasound decision tree model includes determining whether the examinee is in the prenatal stage, determining whether the examinee has entered the intrapartum ultrasound, determining whether the examinee has an obstetric emergency, and generating a prenatal examination report.

9. The AI-based interactive obstetric handheld ultrasound decision support report generation method according to claim 4, characterized in that: The interactive appearance construction of S4 refers to setting a visual interactive menu for ultrasound mode selection based on a decision tree model of specific ultrasound modes in different stages of the perinatal period, combined with a knowledge graph of ultrasound examinations in different stages of the perinatal period.

10. The AI-based interactive obstetric handheld ultrasound decision support report generation method according to claim 2, characterized in that: Updating the prefix vector and model parameters in S104 refers to the difference between the text generated by the model and the real text calculated by the loss function, and calculating the loss gradient and updating the prefix vector and task-specific model parameters through back propagation.

Images