Brain full-automatic 3D ultrasonic examination system guided by robot and LLM model

Abstract

The invention discloses a full-automatic brain 3D ultrasonic examination system guided by a robot and an LLM model, which constructs a closed-loop system integrating intelligent semantic understanding, precise robot control, high-quality image acquisition and intelligent diagnosis report generation through software and hardware deep integration and an intelligent algorithm. Compared with the prior art, the brain ultrasonic examination system has the advantages that the whole process and end-to-end automation from clinical requests to diagnosis reports is achieved, the standardization degree, repeatability, efficiency and accuracy of brain ultrasonic examination are remarkably improved, the workload of doctors is relieved, and an efficient and reliable technical tool is provided for early screening, precise diagnosis and treatment planning of brain diseases.

Description

Technical Field

[0001] This invention belongs to the field of medical imaging technology and intelligent robot technology, specifically relating to a fully automated 3D ultrasound examination system for the brain guided by a robot and an LLM model. Background Technology

[0002] Traditional brain ultrasound scans rely heavily on the experience and skills of the operators, and suffer from low standardization, limited efficiency, and fragmented diagnostic processes. This not only leads to variability in examination results but also affects examination efficiency.

[0003] Although some semi-automatic assisted scanning devices exist, manual intervention is still required for positioning and path planning. Furthermore, they lack intelligent parsing capabilities that directly connect with clinical semantic needs, making it difficult to achieve closed-loop automation of the entire process from examination instructions to diagnostic reports. Summary of the Invention

[0004] To address the aforementioned technical problems, the present invention provides a fully automated 3D ultrasound examination system for the brain guided by a robot and an LLM model, comprising: The central intelligent control module receives structured and / or natural language scanning requests submitted by doctors through the hospital information network. Based on the built-in LLM model, it automatically parses the clinical intent, identifies the target area to be scanned in the clinical intent, and generates a structured task list. According to the "target area" in the task list, it queries the "knowledge base of areas that can be examined by brain ultrasound" to determine whether the request is technically feasible. If the request is for a deep bone area that ultrasound cannot penetrate, it is determined to be invalid. The HIS returns the reason for rejection and recommended alternative examination plan to the requesting doctor. If the request is valid, it generates data to guide the patient's positioning and scanning requirements based on standard anatomical atlas. The patient scheduling and management system is used to acquire data to guide patients to their positions, display patient queue information on the waiting area display screen according to the queue order, and after detecting that a patient has entered the examination room, guide the patient to lie on the adjustable examination bed in a supine position and place their head in the headrest groove through voice and animation on the wall display screen. The adjustable examination bed is used to adjust synchronously according to the patient positioning data and transmit the patient's position information to the central intelligent control module in real time until the patient's position meets the requirements of the patient positioning data. A high-precision 3D structured light vision sensor and a high-precision six-degree-of-freedom robotic arm are used to acquire the patient's head and neck pose information, as well as the point cloud of the patient's face and head and neck. This data is transmitted to a central intelligent control module. The central intelligent control module, based on the patient's body position information, head and neck pose information, and the point cloud of the patient's face and head and neck, runs a fast point cloud registration algorithm to register the acquired point cloud with a pre-stored "standard supine head scan model." It calculates the positional and posture deviation between the two. Based on this deviation, combined with the patient's body position information, the target scanning area, and the scanning requirements, it generates a precise sequence of adjustable examination bed adjustment commands. After the adjustable examination bed completes its adjustment, the results are verified again. To ensure that the error between the patient's actual head posture and the standard model is within a preset threshold, if the threshold is not met, iterative fine-tuning is performed. After the patient's position is calibrated, a 3D surface model of the patient's head is obtained. Combining the structured task list, the geometric and acoustic parameters of the ultrasound probe, and the relevant safety constraints of the prohibited area, the key parameters of the scanning control are calculated through multimodal information fusion and adaptive path planning algorithms. The current base coordinates and initial position of the high-precision robotic arm are read. The key parameters are then adjusted in a final adaptive manner with the 3D surface model of the patient's head. A high-precision, personalized robotic arm scanning trajectory is generated using the robotic arm kinematic model, and the movement is based on the robotic arm scanning trajectory. The professional ultrasonic scanning device is used to drive a high-precision six-degree-of-freedom robotic arm to perform stable and uniform contact scanning along the robotic arm's scanning trajectory under force-position hybrid control. At the same time, the probe contact pressure is monitored and adjusted in real time to ensure image quality. The continuous 2D ultrasound sequence images acquired during the scanning process are transmitted to the central intelligent control module in real time. The diagnostic report display and output module is used by the central intelligent control module to select high-quality frames from continuous 2D ultrasound sequence images using a real-time image quality assessment algorithm based on deep learning. It then uses an improved sparse view 3D reconstruction algorithm (such as an iterative reconstruction algorithm based on deep learning priors) to quickly synthesize high-resolution, isotropic 3D ultrasound images of the brain. The module performs automated quantitative analysis on the 3D ultrasound images of the brain, combines the quantitative analysis data with a medical knowledge base to generate a structured preliminary diagnostic report, obtains the preliminary diagnostic report transmitted from the central intelligent control module, and obtains the ultrasound physician's review result. If the ultrasound physician's review result is approved, the preliminary diagnostic report is sent back to the central intelligent control module, which then returns it to the requesting doctor's workstation through the hospital information network.

[0005] Preferably, the target area for scanning includes a specific brain lobe, a vascular region, or a suspected lesion site.

[0006] Preferably, the automated quantitative analysis includes ventricular volume measurement, midline offset calculation, extraction and parameter calculation of blood flow spectrum of key vessels, and automatic segmentation and feature extraction of abnormal echo regions.

[0007] Preferably, the quantitative analysis data is a structured data summary of a screenshot containing quantitative results and key parameters.

[0008] Preferably, the preliminary diagnostic report includes examination findings, measurement data, abnormality indications, diagnostic impressions, and recommendations for subsequent examinations.

[0009] This invention provides a fully automated 3D ultrasound examination system for the brain guided by a robot and an LLM model. Through deep integration of hardware and software and intelligent algorithms, it constructs a closed-loop system that integrates intelligent semantic understanding, precise robot control, high-quality image acquisition, and intelligent diagnostic report generation. This achieves end-to-end automation of the entire process from clinical request to diagnostic report, significantly improving the standardization, repeatability, efficiency, and accuracy of brain ultrasound examinations, reducing the workload of physicians, and providing an efficient and reliable technical tool for early screening, accurate diagnosis, and treatment planning of brain diseases. Attached Figure Description

[0010] Figure 1 A schematic diagram of the framework of a fully automated 3D ultrasound examination system for the brain guided by a robot and an LLM model, provided in an embodiment of the present invention; Figure 2 This is a flowchart illustrating a fully automated 3D ultrasound examination system for the brain guided by a robot and an LLM model, as provided in an embodiment of the present invention. Detailed Implementation

[0011] The present invention will be further illustrated below with reference to specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. Furthermore, it should be understood that after reading the teachings of this invention, those skilled in the art can make various alterations or modifications to the invention, and these equivalent forms also fall within the scope defined by the appended claims.

[0012] like Figure 1 As shown in the figure, an embodiment of the present invention provides a fully automated 3D ultrasound examination system for the brain guided by a robot and an LLM model, comprising: The central intelligent control module 4, based on a Large Language Model (LLM), includes a high-performance computing platform. It receives structured and / or natural language scanning requests submitted by doctors through the hospital information network. Based on the built-in LLM model, it automatically parses the clinical intent, identifies the target area to be scanned (such as a specific brain lobe, vascular region, or suspected lesion site), and generates a structured task list. According to the "target area" in the task list, it queries the "Brain Ultrasound Examineable Area Knowledge Base" to determine whether the request is technically feasible. If the request targets a deep bone region that ultrasound cannot penetrate, it is deemed invalid. The HIS returns the reason for rejection and recommended alternative examination plan to the requesting doctor. If the request is valid, it generates patient positioning data and scanning requirements based on standard anatomical atlases.

[0013] like Figure 2 As shown in 'a', neurologists submit examination requests (structured and / or natural language scanning requests) through a Hospital Information System (HIS) workstation, which are then transmitted to the system's central intelligent control module via the hospital information network. Examination requests may include structured fields, such as selecting the examination site, and free text descriptions such as "headache to be investigated, focus on the temporal lobe, etc."

[0014] After receiving an examination request, the central intelligent control module first parses the patient's basic information (ID, age, gender) and the examination description text. The examination description text is then fed into the built-in LLM model. For example... Figure 2 As shown in b, the LLM model extracts clinical intent keywords and maps them to the system's internal standard examination protocol library, generating a structured task list. This task list, along with the patient's basic information, is stored in a temporary task database. For example: {Examination mode: B-mode + Color Doppler; Target area: Bilateral temporal lobe cortex and subcortical areas; Focus: Sulci and gyri morphology, blood flow signal intensity; Contrast: Left-right symmetry}. Based on the "target area" in the task list, the system queries the "Brain Ultrasound Examineable Areas Knowledge Base" to determine if the request is technically feasible. If the request targets a deep skeletal area that ultrasound cannot penetrate, it is deemed invalid, and the system automatically returns the reason for refusal and recommended alternative examination plans to the requesting physician via the HIS. If the request is valid, data guiding patient placement and scanning requirements are generated based on standard anatomical atlases.

[0015] The patient scheduling and management system 5 is used to acquire data for guiding patients to their positions. Based on the queue order, it displays patient queue information on a screen in the waiting area. Upon detecting a patient entering the examination room, it guides the patient to lie supine on the adjustable examination bed via voice prompts and animations on the wall display screen, placing the patient's head in the headrest recess. Figure 2 As shown in c in the figure.

[0016] The adjustable examination bed 3 is used to adjust synchronously according to the patient positioning data and transmit the patient's position information to the central intelligent control module in real time until the patient's position meets the requirements of the patient positioning data.

[0017] The adjustable examination bed features multi-degree-of-freedom electric adjustment.

[0018] A high-precision 3D structured light vision sensor and a high-precision six-DOF robotic arm are used to acquire the patient's head and neck pose information, as well as the point cloud data of the patient's face and head and neck, which is then transmitted to the central intelligent control module. Figure 2 As shown in d and e, the central intelligent control module, based on the patient's position information, head and neck pose information, and point clouds of the patient's face and head and neck, runs a fast point cloud registration algorithm to register the acquired point cloud with a pre-stored "standard supine head scan model," calculating the position and posture deviations (ΔX, ΔY, ΔZ, Δα, Δβ) between the two. Based on this deviation (Δγ), combined with patient positioning information, the target scanning area, and scanning requirements, a precise adjustable examination bed adjustment command sequence is generated. After the adjustable examination bed adjustment is completed, it is verified again to ensure that the error between the actual patient head pose and the standard model is within a preset threshold. If the threshold is not met, iterative fine-tuning is performed. After patient positioning calibration, a 3D surface model of the patient's head is obtained. Combining a structured task list, the geometric and acoustic parameters of the ultrasound probe, and safety constraints related to prohibited areas, key parameters for scanning control are calculated through multimodal information fusion and adaptive path planning algorithms. The current base coordinates and initial position of the high-precision robotic arm are read, and the key parameter information is finally adaptively adjusted with the 3D surface model of the patient's head. A high-precision, personalized robotic arm scanning trajectory is generated using the robotic arm kinematic model, and the robotic arm moves based on the scanning trajectory. Figure 2 As shown in f in the figure.

[0019] Specifically, based on the anatomical description in the structured task list, a 3D probability distribution model of the region is retrieved from the digital brain atlas and mapped onto the current patient's head surface model. Considering individual anatomical differences, a 3D spatial range to be covered, i.e., the target body V_target, is defined. On the scalp surface corresponding to V_target, uniform sampling is performed according to the effective field of view of the probe, generating a set of potential probe contact center points {C_i}. At each contact point C_i, the normal direction n_i of the scalp surface is calculated. To ensure optimal sound beam incidence, the normal of the probe's emitting surface needs to be aligned with n_i. Furthermore, considering the reachability and motion smoothness of the robotic arm, n_i is fine-tuned and optimized to form the final probe posture unit {P_i = ... Given a set of nodes (C_i, n_i), a graph model is constructed using the estimated time (considering kinematics) for the robotic arm to move between two poses and the smoothness of the pose change as costs. A heuristic algorithm based on the Traveling Salesman Problem (TSP) or a machine learning-based method is used to find an optimal order to visit all nodes, forming a continuous scanning path. For blood vessels requiring repeated observation, a closed loop path may be planned. The final generated key parameters include: the scanning start point (the first P_i on the path), the sequence of key nodes in the path planning, and the expected pose of the probe at each node. These key parameters are then used to make final adaptive adjustments to the 3D surface model of the patient's head. Using the robotic arm kinematics model, each node P_i is converted into a joint angle vector of the robotic arm's end effector. A fifth-order polynomial trajectory interpolation algorithm is then used to generate a smooth and continuous robotic arm scanning trajectory for each joint, i.e., high-precision robotic arm control commands.

[0020] The geometric and acoustic parameters of an ultrasonic probe include: radius of curvature, scanning plane size, and effective field of view.

[0021] During the movement of the robotic arm: (1) The force sensor of the robotic arm provides real-time feedback on the contact force between the probe and the skin. The central intelligent control module runs a force control closed-loop algorithm to dynamically adjust the position of the robotic arm to maintain a preset constant contact force.

[0022] (2) The ultrasound equipment acquires B-mode images at a fixed frequency and timestamps them.

[0023] (3) The robotic arm reports the real-time pose data of the probe end at a higher frequency and is strictly synchronized with the image timestamp.

[0024] (4) The image data stream and the synchronized pose data stream are transmitted to the buffer of the central intelligent control module in real time through a high-speed interface.

[0025] For example, the adjustable examination bed adjustment command sequence is "headrest rises 15mm, bed tilts 3 degrees to the left", with preset thresholds of translation <2mm and rotation <1°.

[0026] The professional ultrasonic scanning device 1 is used to drive a high-precision six-degree-of-freedom robotic arm to perform stable and uniform contact scanning along the robotic arm's scanning trajectory under force-position hybrid control. Simultaneously, it monitors and adjusts the probe contact pressure in real time to ensure image quality. The continuous 2D ultrasound sequence images acquired during the scanning process are transmitted in real time to the central intelligent control module, such as... Figure 2 As shown in g, h, and i.

[0027] Among them, the professional ultrasonic scanning device includes a dedicated ultrasonic scanning probe and its driving device.

[0028] The diagnostic report display and output module 6 is used by the central intelligent control module to select high-quality frames from continuous 2D ultrasound sequence images using a real-time image quality assessment algorithm based on deep learning, and to quickly synthesize high-resolution, isotropic 3D ultrasound images of the brain using an improved sparse view 3D reconstruction algorithm (such as an iterative reconstruction algorithm based on deep learning priors). Figure 2 As shown in j, automated quantitative analysis of 3D ultrasound images of the brain is performed. The quantitative analysis data is then combined with a medical knowledge base to generate a structured preliminary diagnostic report. This preliminary diagnostic report is then received from the central intelligent control module, and the ultrasound physician's review results are obtained. Figure 2 As shown in k, if the ultrasound physician's review result is approved, the preliminary diagnosis report will be sent back to the central intelligent control module, which will then return it to the applicant doctor's workstation via the hospital information network.

[0029] Specifically, for all 2D images in the buffer, a lightweight quality assessment CNN is run to remove blurred or heavily artifact-ridden frames, retaining high-quality image sequences {I_k} and their corresponding high-precision poses {T_k}. For each retained image I_k, based on its pose T_k, each pixel in the image is mapped to a line in the 3D world coordinate system. A fine 3D voxel grid is defined in the target region. The intensity values ​​of all pixels along a line are accumulated into the voxels it passes through using a distance-weighted backprojection method, resulting in an initial volumetric data V_init filled with noise and stripe artifacts. V_init is then input into a pre-trained 3D image enhancement network. This network employs an encoder-decoder structure and learns using a large amount of paired data during the training phase. Its function is to remove noise, fill in missing parts, and enhance boundaries, outputting a significantly improved volumetric data V_enhanced. To further improve accuracy, an iterative framework can be introduced: minimizing the objective function ||A * V - P||^2 + λ * R(V), where A is the forward projection operator, P is the actual acquired pixel data, R(V) is the regularization term based on the prior of the above deep learning network, and λ is the regularization parameter. Through iterative optimization, the final high-quality 3D ultrasound image of the brain, V_final, is obtained.

[0030] Automated analysis is performed on V_final, using 3D U-Net to segment structures such as the lateral ventricles to generate masks. Based on the masks, volume, maximum transverse diameter, etc., are calculated. If Doppler data is available, blood vessels are automatically located and PSV, EDV, RI, etc. are calculated. The echo intensity distribution of the entire brain parenchyma is calculated, focal hyperechoic or hypoechoic areas are identified, and their coordinates and volumes are marked. Finally, quantitative analysis data is obtained. After combining the quantitative analysis data with a medical knowledge base to generate a structured preliminary diagnostic report, it is displayed along with important 3D image slices and renderings. It is usually displayed on the department's review workstation, and the on-call ultrasound physician is notified for review. The physician can review the report content, view the images, and make necessary modifications or confirmations. After the physician's confirmation, the report is marked as the final version. Through DICOM SR and DICOM image formats, the final report and associated 3D image data are pushed back to the HIS / RIS system via the hospital information network and archived to PACS. The requesting physician can view it immediately on their workstation.

[0031] Simultaneously, the system instructs the patient scheduling and management system to notify the patient that the examination is complete and they can leave the examination room. The examination bed automatically resets, ready to receive the next patient. This concludes a complete fully automated 3D brain ultrasound examination process. Figure 2 As shown in l in the figure.

[0032] The automated quantitative analysis includes, but is not limited to, ventricular volume measurement, midline offset calculation, extraction and parameter calculation of blood flow spectrum of key vessels, and automatic segmentation and feature extraction of abnormal echo regions. The quantitative analysis data is a structured data summary of screenshots with quantitative results and key parameters. The preliminary diagnostic report includes examination findings, measurement data, abnormality indications, diagnostic impressions, and suggestions for subsequent examinations.

[0033] The present invention provides a fully automated 3D ultrasound examination system for the brain guided by a robot and an LLM model. Through deep integration of hardware and software and intelligent algorithms, it constructs a closed-loop system that integrates intelligent semantic understanding, precise robot control, high-quality image acquisition, and intelligent diagnostic report generation. It realizes end-to-end automation of the entire process from clinical request to diagnostic report, significantly improving the standardization, repeatability, efficiency, and accuracy of brain ultrasound examination, reducing the workload of physicians, and providing an efficient and reliable technical tool for early screening, accurate diagnosis, and treatment planning of brain diseases.

Claims

1. A fully automated 3D ultrasound examination system for the brain guided by a robot and an LLM model, characterized in that, include: The central intelligent control module receives structured and / or natural language scanning requests submitted by doctors through the hospital information network. Based on the built-in LLM model, it automatically parses the clinical intent, identifies the target area to be scanned in the clinical intent, and generates a structured task list. According to the "target area" in the task list, it queries the "knowledge base of areas that can be examined by brain ultrasound" to determine whether the request is technically feasible. If the request is for a deep bone area that ultrasound cannot penetrate, it is determined to be invalid. The HIS returns the reason for rejection and recommended alternative examination plan to the requesting doctor. If the request is valid, it generates data to guide the patient's positioning and scanning requirements based on standard anatomical atlas. The patient scheduling and management system is used to acquire data to guide patients to their positions, display patient queue information on the waiting area display screen according to the queue order, and after detecting that a patient has entered the examination room, guide the patient to lie on the adjustable examination bed in a supine position and place their head in the headrest groove through voice and animation on the wall display screen. The adjustable examination bed is used to adjust synchronously according to the patient positioning data and transmit the patient's position information to the central intelligent control module in real time until the patient's position meets the requirements of the patient positioning data. A high-precision 3D structured light vision sensor and a high-precision six-degree-of-freedom robotic arm are used to acquire the patient's head and neck pose information, as well as the point cloud of the patient's face and head and neck. This data is transmitted to a central intelligent control module. The central intelligent control module, based on the patient's body position information, head and neck pose information, and the point cloud of the patient's face and head and neck, runs a fast point cloud registration algorithm to register the acquired point cloud with a pre-stored "standard supine head scan model." It calculates the position and posture deviation between the two. Based on this deviation, combined with the patient's body position information, the target scanning area, and the scanning requirements, it generates a precise sequence of adjustable examination bed adjustment commands. After the adjustable examination bed completes its adjustment, the results are verified again. To ensure that the error between the patient's actual head posture and the standard model is within a preset threshold, if the threshold is not met, iterative fine-tuning is performed. After the patient's position is calibrated, a 3D surface model of the patient's head is obtained. Combining the structured task list, the geometric and acoustic parameters of the ultrasound probe, and the relevant safety constraints of the prohibited area, the key parameters of the scanning control are calculated through multimodal information fusion and adaptive path planning algorithms. The current base coordinates and initial position of the high-precision robotic arm are read. The key parameters are then adjusted in a final adaptive manner with the 3D surface model of the patient's head. A high-precision, personalized robotic arm scanning trajectory is generated using the robotic arm kinematic model, and the movement is based on the robotic arm scanning trajectory. The professional ultrasonic scanning device is used to drive a high-precision six-degree-of-freedom robotic arm to perform stable and uniform contact scanning along the robotic arm's scanning trajectory under force-position hybrid control. At the same time, the probe contact pressure is monitored and adjusted in real time to ensure image quality. The continuous 2D ultrasound sequence images acquired during the scanning process are transmitted to the central intelligent control module in real time. The diagnostic report display and output module is used by the central intelligent control module to select high-quality frames from continuous 2D ultrasound sequence images using a real-time image quality assessment algorithm based on deep learning. It then uses an improved sparse view 3D reconstruction algorithm (such as an iterative reconstruction algorithm based on deep learning priors) to quickly synthesize high-resolution, isotropic 3D ultrasound images of the brain. The module performs automated quantitative analysis on the 3D ultrasound images of the brain, combines the quantitative analysis data with a medical knowledge base to generate a structured preliminary diagnostic report, obtains the preliminary diagnostic report transmitted from the central intelligent control module, and obtains the ultrasound physician's review result. If the ultrasound physician's review result is approved, the preliminary diagnostic report is sent back to the central intelligent control module, which then returns it to the requesting doctor's workstation through the hospital information network.

2. The fully automated 3D ultrasound examination system for the brain guided by a robot and an LLM model as described in claim 1, characterized in that, The target area for scanning includes specific brain lobes, vascular regions, or suspected lesions.

3. The fully automated 3D ultrasound examination system for the brain guided by a robot and an LLM model as described in claim 1, characterized in that, The automated quantitative analysis includes ventricular volume measurement, midline offset calculation, extraction and parameter calculation of blood flow spectrum of key vessels, and automatic segmentation and feature extraction of abnormal echo regions.

4. The fully automated 3D ultrasound examination system for the brain guided by a robot and an LLM model as described in claim 1, characterized in that, The quantitative analysis data is a structured data summary of screenshots containing quantitative results and key parameters.

5. The fully automated 3D ultrasound examination system for the brain guided by a robot and an LLM model as described in claim 1, characterized in that, The preliminary diagnostic report includes examination findings, measurement data, abnormality indications, diagnostic impressions, and recommendations for further examinations.

Images