Ultrasonic detection system and method, electronic equipment and medium

Abstract

The invention provides an ultrasonic detection system and method, electronic equipment and a medium, and the system comprises a language processing module which is used for receiving a voice instruction and carrying out the voice recognition processing, so as to obtain a text control instruction; the mechanical arm control module is used for analyzing and processing the text control instruction by utilizing an algorithm so as to obtain a mechanical arm control parameter, and adjusting the position of the mechanical arm by utilizing the control parameter; and the visual positioning module is used for acquiring an ultrasonic image and carrying out positioning segmentation processing on the ultrasonic image so as to acquire a target segmentation area and an ultrasonic detection result. According to the ultrasonic detection system, high-quality ultrasonic images can be automatically collected, manual participation is reduced, and an accurate ultrasonic detection result is obtained.

Description

Technical Field

[0001] The present application belongs to the technical field of medical image processing, and relates to an ultrasonic detection system, and in particular to an ultrasonic detection system, method, electronic equipment and medium. Background Art

[0002] In recent years, with the advancement of medical technology, ultrasound image analysis has gradually shifted from traditional manual reading to automated detection based on artificial intelligence. Extended Focused Assessment with Sonography for Trauma (eFAST) is a technology for rapid bedside assessment of patients with closed thoracic and abdominal trauma in an emergency setting. It can detect free gas and fluid in the lungs and pericardium. It is mainly used to quickly screen for effusion in the abdomen, chest and pelvis, helping doctors quickly determine whether there is internal bleeding or other emergency situations. eFAST has become an important examination method in emergency and trauma medicine because of its rapid, non-invasive and real-time performance.

[0003] Although eFAST is widely used in emergency medicine, existing technologies still face some challenges in image acquisition and analysis. Traditional eFAST examinations usually rely on experienced physicians, but due to the complexity and subjectivity of ultrasound images, differences in operating skills may lead to inconsistent image quality, which in turn affects diagnostic accuracy. Summary of the invention

[0004] In view of the above-mentioned shortcomings of the prior art, the purpose of the present application is to provide an ultrasonic detection system, method, electronic equipment and medium, which are used to solve the problem that the ultrasonic detection system in the prior art relies on manual participation and has low accuracy.

[0005] In a first aspect, the present application provides an ultrasonic detection system, which includes: a language processing module, used to receive voice instructions and perform voice recognition processing to obtain text control instructions; a robotic arm control module, used to use an algorithm to parse the text control instructions to obtain robotic arm control parameters, and use the control parameters to adjust the position of the robotic arm; a visual positioning module, used to acquire an ultrasonic image and perform positioning and segmentation processing on the ultrasonic image to obtain a target segmentation area and an ultrasonic detection result.

[0006] In this application, voice commands are used to control the robot arm control module to parse the commands to obtain the robot arm control parameters, and the control parameters are used to adjust the position of the robot arm. After the robot arm position is adjusted, an ultrasonic image is obtained, and the ultrasonic image is processed to obtain the target segmentation area and ultrasonic detection results. This ultrasonic detection system can automatically collect high-quality ultrasonic images, reduce manual participation, simplify the operation process, lower the threshold of use, improve the accuracy of ultrasonic detection results and enhance user experience.

[0007] In an implementation of the first aspect, the language processing module includes: a speech recognition unit, used to receive the voice instruction and transcribe the voice instruction into text data; and a language model unit, used to extract information from the text data to obtain the text control instruction.

[0008] In an implementation of the first aspect, the robotic arm control module includes: a first algorithm processing unit, used to process the text control instruction using an adaptive impedance control algorithm to obtain a target contact force; a second algorithm processing unit, used to process the text control instruction using a Bayesian optimization algorithm to obtain a target adjustment direction; and a position adjustment unit, used to adjust the position of the robotic arm using the target contact force and the target adjustment direction.

[0009] In an implementation of the first aspect, the first algorithm processing unit is also used to: adjust the target position and current position of the robotic arm using a stiffness coefficient and a damping coefficient to obtain the target contact force, wherein the current position is obtained using the visual positioning module.

[0010] In an implementation of the first aspect, the visual positioning module includes: an image acquisition unit, used to acquire the ultrasonic image, the ultrasonic image including a depth image and a three-channel color image; a three-dimensional positioning unit, used to perform three-dimensional positioning processing on the depth image to obtain the three-dimensional coordinates of the area to be detected; a positioning scanning unit, used to perform positioning scanning processing on the three-channel color image to obtain the positioning scanning result of the area to be detected; and an image detection unit, used to use a deep learning model to segment and detect the ultrasonic image according to the three-dimensional coordinates of the area to be detected and the positioning scanning result of the area to be detected, so as to obtain the target segmented area and the ultrasonic detection result.

[0011] In an implementation of the first aspect, the image detection unit is used to: use a first deep learning model to segment the ultrasound image according to the three-dimensional coordinates of the area to be detected and the positioning scanning results of the area to be detected to obtain the target segmented area; use a second deep learning model to extract features from the ultrasound image according to the three-dimensional coordinates of the area to be detected and the positioning scanning results of the area to be detected to obtain the ultrasound detection result.

[0012] In an implementation manner of the first aspect, the image detection unit is further used to: utilize the residual connection of the second deep learning model to perform residual mapping processing on the feature extraction process to obtain the ultrasonic detection result.

[0013] In the second aspect, the present application provides an ultrasonic detection method, which includes: receiving voice instructions and performing voice recognition processing to obtain text control instructions; using an algorithm to parse the text control instructions to obtain robot arm control parameters, and using the control parameters to adjust the position of the robot arm; obtaining an ultrasonic image and performing positioning and segmentation processing on the ultrasonic image to obtain a target segmentation area and ultrasonic detection results.

[0014] In a third aspect, the present application provides an electronic device, comprising: a memory for storing a computer program; and a processor for executing the computer program stored in the memory, so that the electronic device performs the ultrasonic detection method as described in any one of the first aspects.

[0015] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the ultrasonic detection method described in any one of the first aspects. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1A Shown is a schematic diagram of an application scenario of the ultrasonic detection method described in this application.

[0017] Figure 1B Shown are structural diagrams of the client-cloud interaction scenarios in these implementations.

[0018] Figure 2 Shown is a schematic diagram of the structure of the ultrasonic detection system described in an embodiment of the present application.

[0019] Figure 3 Shown is a schematic diagram of the structure of the ultrasonic detection system described in an embodiment of the present application.

[0020] Figure 4 Shown is a schematic diagram of the structure of the ultrasonic detection system described in an embodiment of the present application.

[0021] Figure 5 Shown is a schematic diagram of the structure of the ultrasonic detection system described in an embodiment of the present application.

[0022] Figure 6 Shown is a schematic flow chart of the ultrasonic detection method described in an embodiment of the present application.

[0023] Figure 7 Shown is a schematic diagram of the structure of an electronic device described in an embodiment of the present application.

[0024] Component number description

[0025] 1. Automated ultrasonic testing device

[0026] 11 Image acquisition equipment

[0027] 12 Local Processor

[0028] 13 Display Terminal

[0029] 2-end-cloud interactive system

[0030] 20 Terminal

[0031] 21 Cloud Server

[0032] 100 Ultrasonic testing system

[0033] 110 Language Processing Module

[0034] 120 Robotic arm control module

[0035] 121 First algorithm processing unit

[0036] 122 Second algorithm processing unit

[0037] 123 Position adjustment unit

[0038] 130 Visual Positioning Module

[0039] 131 Image acquisition unit

[0040] 132 Three-dimensional positioning unit

[0041] 133 Positioning scanning unit

[0042] 134 Image Detection Unit

[0043] 700 Electronic equipment

[0044] 710 Memory

[0045] 720 processor

[0046] 730 Display

[0047] Steps S11 to S13 DETAILED DESCRIPTION

[0048] The following describes the embodiments of the present application through specific examples, and those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. The present application can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0049] It should be noted that in the embodiments of the present application, the words "optionally" or "for example" represent examples, illustrations or descriptions. Any embodiment or design described as "optionally" or "for example" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "optionally" or "for example" is intended to present related concepts in a specific way.

[0050] In the embodiments of the present application, "at least one" refers to one or more, and "plurality" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can represent: a, b, c, ab, ac, bc or abc, where a, b, c can be single or multiple.

[0051] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present application, and thus the drawings only show components related to the present application rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed at will, and the component layout may also be more complicated.

[0052] Although extended focused assessment ultrasound (eFAST) technology has played an important role in the rapid assessment of trauma patients, it still faces the following major problems in practical applications: Experience-dependent operation mode: eFAST examinations are highly dependent on experienced physicians to operate, resulting in image quality and diagnostic results being affected by differences in operator skills. This reliance on manpower not only increases the risk of misdiagnosis, but also makes it difficult to conduct rapid and accurate examinations in emergency settings. Insufficient automation: Although the existing eFAST robot system has achieved automation to a certain extent, it lacks real-time feedback and dynamic adjustment capabilities, and cannot adjust the scanning posture and corresponding force in time during the scanning process. This limitation makes it impossible for the system to ensure the comfort of patients during the examination, especially during long-term scanning, when patients may feel uncomfortable. Dependence on manual operation: The existing eFAST robot system still relies on professionals to perform remote operations to ensure quality. This reliance requires professionals to undergo additional learning and training to adapt to the use of the robot system. This not only increases the workload of medical staff, but may also cause delays in emergency situations, affecting the timely assessment and treatment of patients.

[0053] At least to address the above-mentioned problems, an embodiment of the present application provides an ultrasonic detection system, which includes: a language processing module, used to receive voice instructions and perform voice recognition processing to obtain text control instructions; a robotic arm control module, used to use an algorithm to parse the text control instructions to obtain robotic arm control parameters, and use the control parameters to adjust the position of the robotic arm; a visual positioning module, used to acquire an ultrasonic image and perform positioning and segmentation processing on the ultrasonic image to obtain a target segmentation area and ultrasonic detection results.

[0054] In the embodiment of the present application, the robot control module is controlled by voice commands to parse the commands to obtain the robot control parameters, and the position of the robot is adjusted by the control parameters. After the robot position is adjusted, an ultrasonic image is obtained, and the ultrasonic image is processed to obtain the target segmentation area and the ultrasonic detection result. This ultrasonic detection system can automatically collect high-quality ultrasonic images, reduce manual participation, simplify the operation process, lower the threshold of use, improve the accuracy of ultrasonic detection results and enhance user experience.

[0055] Figure 1A The figure shows an application scenario of the ultrasonic detection method described in the present application. The automated ultrasonic detection device 1 can be used to implement the ultrasonic detection method provided in the embodiment of the present application, but the application scenario of the ultrasonic detection method provided in the embodiment of the present application is not limited to Figure 1A The automated ultrasonic testing device 1 shown. Figure 1AAs shown, the automated ultrasonic detection device 1 includes an image acquisition device 11, a local processor 12, and a display terminal 13. The ultrasonic detection method provided in the embodiment of the present application can be applied to the local processor 12, and the position of the image acquisition device 11 is adjusted according to the analysis result of the local processor 12 to obtain an ultrasonic image. The local processor 12 obtains the detection result according to the ultrasonic image and displays it on the display terminal 13.

[0056] in, Figure 1A The local processor 12 in the embodiment may be a local processor or a local processor cluster composed of multiple local processors or a cloud computing center, etc., which is not limited to the specifics herein. Figure 1A Only one image acquisition device 11, one local processor 12 and one display terminal 13 are shown, but it should be understood that Figure 1A The examples are only used to understand the present solution, and the specific numbers of local processors 12 and display terminals 13 should be flexibly determined based on actual conditions.

[0057] In some other implementations, the automated ultrasonic detection device 1 may not include the display terminal 13, but only include a local processor 12 with a display function and an image acquisition device 11. The ultrasonic detection method provided in the embodiment of the present application can be applied to the local processor 12. The local processor 12 with a display function may include a tablet computer, a laptop computer, a PDA, a mobile phone, a personal computer, and a voice interaction device, and may also be a monitoring device, a face recognition device, etc., which is not limited here.

[0058] In still other implementations, the ultrasonic detection method described in the present application may be applied to end-cloud interaction scenarios. Figure 1B The following is a schematic diagram of the structure of the client-cloud interaction scenario in these implementations. Figure 1B As shown, the terminal-cloud interaction system 2 includes a terminal 20 and a cloud server 21. The terminal 20 and the cloud server 21 can communicate with each other, and the communication method is not limited to wired or wireless.

[0059] Among them, the terminal 20 can be mobile or fixed, for example, the terminal 20 can be a wireless terminal or a wired terminal. The wireless terminal can refer to a device with wireless transceiver function, which can be deployed indoors, outdoors and in industrial workshops. The terminal 20 can be an ultrasonic detector, a mobile phone, a tablet computer, a laptop computer, etc., which is not limited here. The cloud server 21 can include one or more servers, or one or more processing nodes, or one or more virtual machines running on the server. The cloud server 21 can also be called a server cluster, a management platform, a data processing center, etc., which is not limited in the embodiments of the present application.

[0060] The technical solutions in the embodiments of the present application will be described in detail below in conjunction with the drawings in the embodiments of the present application.

[0061] The following embodiments of the present application provide an ultrasonic detection system, Figure 2 The structure diagram of the ultrasonic detection system described in the embodiment of the present application is shown. Figure 2 As shown, the ultrasonic detection system 100 includes a language processing module 110 , a robotic arm control module 120 and a visual positioning module 130 .

[0062] The language processing module 110 is used to receive voice instructions and perform voice recognition processing to obtain text control instructions. Optionally, the voice instructions are oral commands from clinicians, such as "scan the left thyroid lobe" or "adjust the probe angle".

[0063] In some possible implementations, the model of the language processing module 110 is trained using a multi-task and multi-language supervised dataset, and the model is trained on medical terminology using a medical language dataset, thereby improving the accuracy and efficiency of the language processing module 110 in processing medical conversations.

[0064] The robot control module 120 is used to parse the text control instruction using an algorithm to obtain robot control parameters, and use the control parameters to adjust the position of the robot. Optionally, the robot is a UR5 robot, which is a six-degree-of-freedom robot arm.

[0065] The visual positioning module 130 is used to acquire an ultrasonic image and perform positioning and segmentation processing on the ultrasonic image to obtain a target segmentation area and an ultrasonic detection result.

[0066] In some possible implementations, the language processing module 110 integrates the automatic speech recognition model Whisper API and the large language model LLM, which are used to receive the user's voice instructions and perform speech recognition processing to obtain text control instructions. The manipulator control module 120 is equipped with a force / torque sensor to achieve real-time monitoring of the applied force. It is used to parse the text control instructions using an algorithm, obtain the manipulator control parameters in combination with the current position of the manipulator obtained by the visual positioning module 130, and adjust the position of the manipulator using the control parameters. The contact force and angle of the manipulator probe are adjusted based on real-time feedback to prevent excessive or too light force and ensure stable contact between the manipulator probe and soft tissue. The visual positioning module 130 integrates the PA-Unet++ model and the ResNet50-V2 model, which are used to obtain ultrasound images and perform positioning and segmentation processing on the ultrasound images to obtain target segmentation areas and ultrasound detection results.

[0067] In the embodiment of the present application, the robot control module 120 is controlled by voice commands to parse the commands to obtain the robot control parameters, and the position of the robot is adjusted by the control parameters. After the robot position is adjusted, an ultrasonic image is obtained, and the ultrasonic image is processed to obtain the target segmentation area and the ultrasonic detection result. This ultrasonic detection system 100 can automatically collect high-quality ultrasonic images, reduce manual participation, simplify the operation process, lower the threshold of use, improve the accuracy of ultrasonic detection results and enhance user experience.

[0068] In one embodiment of the present application, the language processing module 110 includes: a speech recognition unit, used to receive the voice instruction and transcribe the voice instruction into text data; and a language model unit, used to extract information from the text data to obtain the text control instruction.

[0069] In some possible implementations, the language processing module 110 integrates the Whisper API model (Whisper Automatic Speech Recognition, automatic speech recognition model) and the large language model LLM, so that the ultrasound detection system 100 can understand the user's verbal commands and perform precise operations based on these verbal commands. For example, after receiving the voice command of "scan the left thyroid lobe" or "adjust the probe angle", Whisper converts the voice command into text data. The large language model LLM of the language model unit refers to the embedded ultrasound API and operation manual prior knowledge, understands the contextual instructions in the current medical operation, and obtains the text control instructions. For example, after receiving the text data of "perform efast ultrasound scanning", the large language model LLM extracts relevant operation suggestions from the pre-trained knowledge base and converts them into steps that can be executed by the robotic arm control module 120. The processing process of the language processing module 110 is expressed as:

[0070] C=R n (R n-1 (...R2(R1(A(U(D)))))),

[0071] Where D represents the doctor's verbal instructions, U represents the enhanced ultrasound domain knowledge, A represents the ultrasound auxiliary prompt set combined with the retrieved API and operation suggestions, and R n represents the nth interaction with the robot through dynamic execution, and C represents the task completion result after n dynamic executions.

[0072] At each iteration R nIn the present invention, the ultrasonic detection system 100 uses sensors, such as force / torque sensors that monitor contact pressure, and visual data, such as real-time feedback of ultrasonic images, to dynamically adjust the action of the robot arm. For example, when the scanned person moves during the scanning process, the language processing module 110 will update the current state in real time and recalculate the optimal scanning path, but the present application is not limited to this.

[0073] In the embodiment of the present application, the language processing module 110 integrates the Whisper API model and the LLM model, so that the ultrasound detection system 100 can efficiently understand and execute voice commands. The Whisper model can accurately transcribe complex medical terms in a noisy environment, while the LLM model is fine-tuned based on medical data sets to accurately parse instructions and execute complex operation steps, such as eFAST ultrasound scanning. The robot's task execution capability in a dynamic environment is improved, reducing dependence on human intervention.

[0074] Figure 3 The structure diagram of the ultrasonic detection system described in the embodiment of the present application is shown. Figure 3 As shown, the robot arm control module 120 includes a first algorithm processing unit 121 , a second algorithm processing unit 122 and a position adjustment unit 123 .

[0075] The first algorithm processing unit 121 is used to process the text control instruction using an adaptive impedance control algorithm to obtain a target contact force.

[0076] The second algorithm processing unit 122 is used to process the text control instruction using a Bayesian optimization algorithm to obtain a target adjustment direction.

[0077] The position adjustment unit 123 is used to adjust the position of the robot arm by using the target contact force and the target adjustment direction.

[0078] In some possible implementations, the robot control module 120 uses a UR5 robot, a six-degree-of-freedom robot arm. The UR5 robot has a repeatability of ±0.1 mm, ensuring precise positioning during ultrasound scanning, which is essential for maintaining high-quality imaging. It has a load capacity of 5 kg and can accommodate a variety of ultrasound probes and sensors without affecting their performance. The robot can automatically adjust the translation and rotation of the probe along three axes to ensure that the probe maintains stable and consistent contact with the skin of the person being tested.

[0079] In order to optimize the image quality during scanning, the second algorithm processing unit 122 dynamically adjusts the probe direction using a Bayesian optimization algorithm to obtain a target adjustment direction. The calculation formula for the target adjustment direction is:

[0080]

[0081] Among them, θ optimal represents the target adjustment direction, I(θ) is the ultrasound image, and H is the image entropy.

[0082] In one embodiment of the present application, the first algorithm processing unit 121 is also used to: use the stiffness coefficient and the damping coefficient to adjust the target position and current position of the robot arm to obtain the target contact force, wherein the current position is obtained using the visual positioning module 130.

[0083] In some possible implementations, in order to further improve performance, the first algorithm processing unit 121 uses an adaptive impedance control algorithm to obtain the target contact force as the optimal contact force between the ultrasound probe and the soft tissue. The calculation formula of the target contact force is:

[0084] F desired =K p ×(x target -x current )+D p ×(x target -x current ),

[0085] Among them, K p Denotes the stiffness coefficient, D p represents the damping coefficient, x target and x current Respectively represent the target position and current position of the probe.

[0086] In some other possible implementations, the visual positioning module 130 is used to acquire an ultrasonic image, and the current position of the robot arm is calculated based on the ultrasonic image, and then adjustments are made based on the current position and the target position to acquire the target contact force.

[0087] In the embodiment of the present application, an adaptive impedance control algorithm is introduced to dynamically adjust the contact force of the probe according to real-time feedback. At the same time, the orientation of the probe is dynamically adjusted through Bayesian optimization to maximize the image entropy, thereby further improving the quality of ultrasound image acquisition. The adaptability of the system in a dynamic environment is enhanced, ensuring the stability of high-quality image acquisition.

[0088] Figure 4 The structure diagram of the ultrasonic detection system described in the embodiment of the present application is shown. Figure 4 As shown, the visual positioning module 130 includes an image acquisition unit 131 , a three-dimensional positioning unit 132 , a positioning scanning unit 133 and an image detection unit 134 .

[0089] The image acquisition unit 131 is used to acquire the ultrasound image, which includes a depth image and a three-channel color image.

[0090] The three-dimensional positioning unit 132 is used to perform three-dimensional positioning processing on the depth image to obtain the three-dimensional coordinates of the area to be detected.

[0091] The positioning scanning unit 133 is used to perform positioning scanning processing on the three-channel color image to obtain a positioning scanning result of the area to be detected.

[0092] The image detection unit 134 is used to use a deep learning model to segment and detect the ultrasound image according to the three-dimensional coordinates of the area to be detected and the positioning scanning result of the area to be detected, so as to obtain the target segmented area and the ultrasound detection result.

[0093] In some possible implementations, the image acquisition unit 131 of the visual positioning module 130 is used to use a camera to acquire the ultrasound image, which includes a depth image and a three-channel color image, namely, an RGB image. The three-dimensional positioning unit 132 is used to perform three-dimensional positioning processing on the depth image to obtain the three-dimensional coordinates of the area to be detected. The positioning scanning unit 133 is used to perform positioning scanning processing on the three-channel color image to obtain the positioning scanning result of the area to be detected. The image detection unit 134 is used to use a deep learning model to segment and detect the ultrasound image according to the three-dimensional coordinates of the area to be detected and the positioning scanning result of the area to be detected, so as to obtain the target segmentation area and the ultrasound detection result. The deep learning model integrates the PA-Unet++ model and the ResNet50-V2 model. The PA-Unet++ model acts on target segmentation and can effectively distinguish anatomical structures such as pleura, lungs and pericardial tissues. The pyramid pooling module in the PA-Unet++ model supports multi-scale feature extraction, and can achieve high-precision segmentation even in complex ultrasound images. The calculation formula for segmentation processing is:

[0094]

[0095] Among them, S target is the target segmentation area, I ultrasound Represents the input ultrasound image. The pyramid pooling module improves the segmentation accuracy by pooling features at different resolutions and concatenating them. The pyramid pooling module ensures that the PA-Unet++ model can accurately identify the boundaries between different tissues, which is crucial for real-time detection of free gas and fluid in eFAST.

[0096] In one embodiment of the present application, the image detection unit is used to: use a first deep learning model to segment the ultrasound image according to the three-dimensional coordinates of the area to be detected and the positioning scanning results of the area to be detected to obtain the target segmented area; use a second deep learning model to extract features from the ultrasound image according to the three-dimensional coordinates of the area to be detected and the positioning scanning results of the area to be detected to obtain the ultrasound detection results.

[0097] In one embodiment of the present application, the image detection unit is further used to: utilize the residual connection of the second deep learning model to perform residual mapping processing on the feature extraction process to obtain the ultrasonic detection result.

[0098] In some possible implementations, the ResNet50-V2 model uses residual connections to solve the gradient vanishing problem in deep network training, allowing the network to learn features at a deeper level. The calculation method of residual connections is:

[0099] y=F(x;W)+x,

[0100] Where y represents the output of the residual connection, F(x;W) represents the residual mapping learned by the convolutional layer through the weight W, and x represents the input feature. The above ResNet50-V2 model can effectively capture the complex patterns in ultrasound drawings and can quickly adapt specific ultrasound features even when the training samples are limited.

[0101] In some possible implementations, Figure 5 The structure diagram of the ultrasonic detection system described in the embodiment of the present application is shown. Figure 5 As shown, the language processing module 110 integrates the automatic speech recognition model Whisper API and the large language model LLM, which are used to receive the user's voice commands and perform speech recognition processing to obtain text control commands. The robotic arm control module 120 is equipped with a force-controlled robotic arm, which uses an adaptive impedance control algorithm and a Bayesian dynamic optimization orientation algorithm to obtain the robotic arm control parameters, and uses the control parameters to adjust the position of the robotic arm. The visual positioning module 130 uses a depth camera to obtain a depth image depth image and a three-channel color RGB image for three-dimensional coordinate positioning and positioning scanning area respectively, and integrates the PA-Unet++ model and the ResNet50-V2 model to perform positioning and segmentation processing on the ultrasound image to obtain the target segmentation area and ultrasound detection results.

[0102] In the embodiment of the present application, the PA-Unet++ model and the ResNet50-V2 model are integrated into the ultrasound image analysis framework, which significantly improves the diagnostic accuracy in the eFAST examination. The PA-Unet++ model implements multi-scale feature extraction and tissue segmentation through the pyramid pooling module, and can effectively identify anatomical structures such as the pleura, lungs, and pericardium. The ResNet50-V2 model solves the gradient vanishing problem in deep networks through residual connections, and quickly adapts to ultrasound image features through transfer learning. This integrated framework supports real-time analysis of ultrasound images and accurate detection of free gas and body fluids.

[0103] In one embodiment of the present application, the ultrasonic detection method can be performed by Figure 1A The local processor 12 shown or Figure 1B It is implemented by the cloud server 21 shown. Figure 6 The flowchart of the ultrasonic detection method described in the embodiment of the present application is shown as follows: Figure 6 As shown, the ultrasonic detection method includes steps S11 to S13.

[0104] Step S11, receiving voice instructions and performing voice recognition processing to obtain text control instructions.

[0105] Step S12: parse the text control instruction using an algorithm to obtain robot arm control parameters, and use the control parameters to adjust the position of the robot arm.

[0106] Step S13, acquiring an ultrasonic image and performing positioning and segmentation processing on the ultrasonic image to obtain a target segmentation area and an ultrasonic detection result.

[0107] In some possible implementations, a user's voice command is received and voice recognition processing is performed to obtain text control instructions. The text control instructions are parsed using an algorithm, and the robot arm control parameters are obtained in combination with the current position of the robot arm obtained by the visual positioning module 130, and the position of the robot arm is adjusted using the control parameters. The contact force and angle of the robot arm probe are adjusted based on real-time feedback to prevent excessive or too light force and ensure stable contact between the robot arm probe and the soft tissue. Ultrasonic images are obtained and positioning and segmenting processing is performed on the ultrasound images to obtain target segmentation areas and ultrasound detection results.

[0108] In the embodiment of the present application, a text control instruction is obtained by using a voice command, and the text control instruction is parsed to obtain the control parameters of the robot arm, and the position of the robot arm is adjusted by using the control parameters. After the position of the robot arm is adjusted, an ultrasonic image is obtained, and the ultrasonic image is processed to obtain the target segmentation area and the ultrasonic detection result. This ultrasonic detection method can automatically collect high-quality ultrasonic images, reduce manual participation, simplify the operation process, lower the threshold of use, improve the accuracy of ultrasonic detection results and enhance user experience.

[0109] In the several embodiments provided in the present application, it should be understood that the disclosed system, device or method can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of modules / units is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or units can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules or units, which can be electrical, mechanical or other forms.

[0110] The modules / units described as separate components may or may not be physically separated, and the components displayed as modules / units may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules / units may be selected according to actual needs to achieve the purpose of the embodiments of the present application. For example, the functional modules / units in the various embodiments of the present application may be integrated into one processing module, or each module / unit may exist physically separately, or two or more modules / units may be integrated into one module / unit.

[0111] Those of ordinary skill in the art should further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0112] An embodiment of the present application also provides an electronic device. Figure 7 The structure diagram of the electronic device 700 according to the embodiment of the present application is shown. Figure 7As shown, in this embodiment, the electronic device 700 includes a memory 710 and a processor 720 .

[0113] The memory 710 is used to store computer programs; preferably, the memory 710 includes: ROM, RAM, disk, USB flash drive, memory card or CD-ROM and other media that can store program codes.

[0114] Specifically, the memory 710 may include a computer system readable medium in the form of a volatile memory, such as a random access memory (RAM) and / or a cache memory. The electronic device 700 may further include other removable / non-removable, volatile / non-volatile computer system storage media. The memory 710 may include at least one program product having a set (e.g., at least one) of program modules, which are configured to perform the functions of the various embodiments of the present application. It is understood that the memory 710 may be a volatile memory or a non-volatile memory, and may also include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM, Read Only Memory), a programmable read-only memory (PROM, Programmable Read-Only Memory), which is used as an external cache. By way of exemplary but not limiting explanation, many forms of RAM are available, such as static random access memory (SRAM, Static Random Access Memory), synchronous static random access memory (SSRAM, Synchronous Static Random Access Memory). The memory described in the embodiments of the present invention is intended to include, but is not limited to, these and any other suitable categories of memory.

[0115] The processor 720 is connected to the memory 710 and is used to execute the computer program stored in the memory 710 so that the electronic device 700 executes the ultrasonic detection method described in any embodiment of the present application.

[0116] Optionally, the processor 720 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0117] Optionally, the electronic device 700 in this embodiment may further include a display 730. The display 730 is communicatively connected with the memory 710 and the processor 720, and is used to display a graphical user interface (GUI) interaction interface related to the ultrasonic detection method described in the embodiment of the present application.

[0118] The embodiment of the present application further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the ultrasonic detection method described in any embodiment of the present application is implemented.

[0119] The terms "component", "module", "system", etc. used in this specification are used to represent computer-related entities, hardware, firmware, a combination of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program and / or a computer. By way of illustration, both applications running on a computing device and a computing device can be components. One or more components may reside in a process and / or an execution thread, and a component may be located on a computer and / or distributed between two or more computers. In addition, these components may be executed from various computer-readable media having various data structures stored thereon. Components may, for example, communicate through local and / or remote processes according to signals having one or more data packets (e.g., data from two components interacting with another component between a local system, a distributed system and / or a network, such as the Internet interacting with other systems through signals).

[0120] The descriptions of the processes or structures corresponding to the above-mentioned figures have different emphases. For parts that are not described in detail in a certain process or structure, please refer to the relevant descriptions of other processes or structures.

[0121] The above embodiments are merely illustrative of the principles and effects of the present application and are not intended to limit the present application. Anyone familiar with the technology may modify or change the above embodiments without violating the spirit and scope of the present application. Therefore, all equivalent modifications or changes made by a person of ordinary skill in the art without departing from the spirit and technical ideas disclosed in the present application shall still be covered by the claims of the present application.

Claims

1. An ultrasonic detection system, characterized in that: include: A language processing module, used for receiving voice commands and performing voice recognition processing to obtain text control commands; A robot arm control module, used to parse the text control instruction using an algorithm to obtain robot arm control parameters, and adjust the position of the robot arm using the control parameters; The visual positioning module is used to acquire an ultrasonic image and perform positioning and segmentation processing on the ultrasonic image to obtain a target segmentation area and an ultrasonic detection result.

2. The ultrasonic detection system according to claim 1, characterized in that: The language processing module comprises: a speech recognition unit, configured to receive the speech instruction and transcribe the speech instruction into text data; The language model unit is used to extract information from the text data to obtain the text control instruction.

3. The ultrasonic detection system according to claim 1, characterized in that: The robotic arm control module comprises: A first algorithm processing unit, configured to process the text control instruction using an adaptive impedance control algorithm to obtain a target contact force; A second algorithm processing unit is used to process the text control instruction using a Bayesian optimization algorithm to obtain a target adjustment direction; A position adjustment unit is used to adjust the position of the robot arm using the target contact force and the target adjustment direction.

4. The ultrasonic detection system according to claim 3, characterized in that: The first algorithm processing unit is further configured to: The target position and current position of the robot arm are adjusted using the stiffness coefficient and the damping coefficient to obtain the target contact force, wherein the current position is obtained using the visual positioning module.

5. The ultrasonic detection system according to claim 1, characterized in that: The visual positioning module includes: An image acquisition unit, configured to acquire the ultrasonic image, wherein the ultrasonic image includes a depth image and a three-channel color image; A three-dimensional positioning unit, used to perform three-dimensional positioning processing on the depth image to obtain the three-dimensional coordinates of the area to be detected; A positioning scanning unit, used for performing positioning scanning processing on the three-channel color image to obtain a positioning scanning result of the area to be detected; An image detection unit is used to use a deep learning model to segment and detect the ultrasonic image according to the three-dimensional coordinates of the area to be detected and the positioning scanning result of the area to be detected, so as to obtain the target segmented area and the ultrasonic detection result.

6. The ultrasonic detection system according to claim 5, characterized in that: The image detection unit is used for: Using a first deep learning model, segmenting the ultrasound image according to the three-dimensional coordinates of the area to be detected and the positioning scanning result of the area to be detected, to obtain the target segmented area; A second deep learning model is used to perform feature extraction on the ultrasonic image according to the three-dimensional coordinates of the area to be detected and the positioning scanning result of the area to be detected, so as to obtain the ultrasonic detection result.

7. The ultrasonic detection system according to claim 6, characterized in that: The image detection unit is also used for: The residual connection of the second deep learning model is used to perform residual mapping processing on the feature extraction process to obtain the ultrasonic detection result.

8. An ultrasonic detection method, characterized in that: include: Receive voice commands and perform voice recognition processing to obtain text control commands; Analyzing the text control instruction using an algorithm to obtain robot arm control parameters, and adjusting the position of the robot arm using the control parameters; An ultrasonic image is acquired and positioning and segmenting processing is performed on the ultrasonic image to obtain a target segmentation area and an ultrasonic detection result.

9. An electronic device, characterized in that: The electronic device comprises: Memory for storing computer programs; A processor, wherein the processor is used to execute the computer program stored in the memory so that the electronic device performs the ultrasonic detection method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the ultrasonic detection method according to any one of claims 1 to 7 is implemented.

Images