Ultrasonic image and doctor manipulation acquisition system, method and product

Abstract

The invention provides an ultrasonic image and doctor manipulation acquisition system, method and product, and relates to the technical field of ultrasonic acquisition, and the system comprises a data acquisition module which comprises an ultrasonic probe, an IMU sensor, an electromagnetic tracking sensor and an RGBD camera; the coordinate system conversion module is used for performing coordinate system conversion on the initial IMU data, the initial electromagnetic tracking data and the initial human body pose information to obtain converted IMU data, converted electromagnetic tracking data and converted human body pose information after coordinate system unification; the data synchronization module is used for performing timestamp alignment on the collected ultrasonic data and dividing a first ultrasonic data set according to the timestamp carried by each ultrasonic data; the data fusion model is used for performing data fusion on the converted IMU data and the converted electromagnetic tracking data which belong to the same first ultrasonic data set to obtain fused probe pose information and generate a second ultrasonic data set; and the data storage module is used for storing the second ultrasonic data set.

Description

Technical Field

[0001] This application relates to the technical field of ultrasonic acquisition, and particularly to an ultrasonic image and doctor's manipulation acquisition system, method and product. Background Art

[0002] Ultrasonic acquisition technology is widely used in fields such as medicine, industry, and scientific research. Its application usually requires using a handheld ultrasonic probe to acquire ultrasonic images. While performing ultrasonic acquisition, relevant information such as the pose of the ultrasonic probe is recorded to accurately locate the position of the object to be detected corresponding to the ultrasonic image.

[0003] However, relevant ultrasonic acquisition technologies often have difficulty in obtaining accurate ultrasonic information, which affects the reliability and accuracy of the finally obtained ultrasonic data. Therefore, there is an urgent need to propose an ultrasonic image and doctor's manipulation acquisition system, method and product to improve the reliability of ultrasonic data. Summary of the Invention

[0004] In view of the above problems, the embodiments of this application provide an ultrasonic image and doctor's manipulation acquisition system, method and product to overcome or at least partially solve the above problems.

[0005] In the first aspect of the embodiments of this application, an ultrasonic image and doctor's manipulation acquisition system is provided. The system includes: A data acquisition module, which includes: an ultrasonic probe, an IMU sensor, an electromagnetic tracking sensor, and an RGBD camera, where the IMU sensor and the electromagnetic tracking sensor are fixed on the ultrasonic probe; A coordinate system conversion module, configured to perform coordinate system conversion on the initial IMU data collected by the IMU sensor, the initial electromagnetic tracking data collected by the electromagnetic tracking sensor, and the initial human pose information collected by the RGBD camera to obtain the converted IMU data, the converted electromagnetic tracking data, and the converted human pose information in a unified coordinate system; A data synchronization module, configured to perform timestamp alignment on the ultrasonic data collected by the data acquisition module, and divide it into multiple first ultrasonic data sets according to the timestamps carried by each ultrasonic data; A data fusion model, configured to perform data fusion on the converted IMU data and the converted electromagnetic tracking data belonging to the same first ultrasonic data set to obtain the fused probe pose information and generate a second ultrasonic data set; A data storage module, configured to store the second ultrasonic data set in the RLDS format. Each second ultrasonic data set includes: the ultrasonic image collected by the ultrasonic probe, and the fused probe pose information and the converted human pose information corresponding to the ultrasonic image.

[0006] In the second aspect of the embodiments of the present application, a method for collecting ultrasonic images and doctor's techniques is further provided, which is applied to the ultrasonic acquisition system provided in the first aspect of the embodiments of the present application. The method includes: Collect ultrasonic data using an ultrasonic probe, an IMU sensor, an electromagnetic tracking sensor, and an RGBD camera; Perform coordinate system conversion on the initial IMU data collected by the IMU sensor, the initial electromagnetic tracking data collected by the electromagnetic tracking sensor, and the initial human body pose information collected by the RGBD camera to obtain the converted IMU data, the converted electromagnetic tracking data, and the converted human body pose information in a unified coordinate system; Align the time stamps of the collected ultrasonic data, and divide them into multiple first ultrasonic data sets according to the time stamps carried by each ultrasonic data; Fuse the converted IMU data and the converted electromagnetic tracking data belonging to the same first ultrasonic data set to obtain the fused probe pose information, and generate a second ultrasonic data set; Store the second ultrasonic data set in the RLDS format. Each second ultrasonic data set includes: the ultrasonic image collected by the ultrasonic probe, and the fused probe pose information and the converted human body pose information corresponding to the ultrasonic image.

[0007] In the third aspect of the embodiments of the present application, an electronic device is further provided, which includes a memory, a processor, and a computer program stored on the memory. The processor executes the computer program to implement the steps in the method for collecting ultrasonic images and doctor's techniques in the second aspect of the embodiments of the present application.

[0008] In the fifth aspect of the embodiments of the present application, a computer-readable storage medium is further provided, on which a computer program / instructions are stored. When the computer program / instructions are executed by a processor, the steps in the method for collecting ultrasonic images and doctor's techniques in the second aspect of the embodiments of the present application are implemented.

[0009] In the sixth aspect of the embodiments of the present application, a computer program product is further provided. When the computer program product runs on an electronic device, it causes the processor to execute the steps in the method for collecting ultrasonic images and doctor's techniques as described in the second aspect of the embodiments of the present application.

[0010] An embodiment of the present application provides an ultrasonic imaging and doctor's technique acquisition system. The system includes: a data acquisition module, which includes: an ultrasonic probe, an IMU sensor, an electromagnetic tracking sensor, and an RGBD camera. Among them, the IMU sensor and the electromagnetic tracking sensor are fixed on the ultrasonic probe; a coordinate system conversion module, which is used to perform coordinate system conversion on the initial IMU data collected by the IMU sensor, the initial electromagnetic tracking data collected by the electromagnetic tracking sensor, and the initial human body pose information collected by the RGBD camera to obtain the converted IMU data, the converted electromagnetic tracking data, and the converted human body pose information after the unified coordinate system; a data synchronization module, which is used to perform timestamp alignment on the ultrasonic data collected by the data acquisition module and divide it into multiple first ultrasonic data sets according to the timestamp carried by each ultrasonic data; a data fusion model, which is used to perform data fusion on the converted IMU data and the converted electromagnetic tracking data belonging to the same first ultrasonic data set to obtain the fused probe pose information and generate a second ultrasonic data set; a data storage module, which is used to store the second ultrasonic data set in the RLDS format. Each second ultrasonic data set includes: the ultrasonic image collected by the ultrasonic probe, and the fused probe pose information and the converted human body pose information corresponding to the ultrasonic image.

[0011] Specific beneficial effects are as follows: On the one hand, through multi-sensor integration, the embodiment of the present application fixes the IMU sensor and the electromagnetic tracking sensor on the ultrasonic probe. While collecting ultrasonic images using the ultrasonic probe, it obtains the pose information (electromagnetic tracking data and IMU data) of the corresponding ultrasonic probe, and the human body posture (human body pose information collected by the RGBD camera). By comprehensively utilizing the advantages of various sensors and making up for the deficiencies of a single sensor, it ensures the comprehensiveness of data collection, thereby improving the accuracy of the obtained ultrasonic data (for example, it can more accurately locate the position of the object to be detected corresponding to the ultrasonic image). On the other hand, the embodiment of the present application uses the coordinate system conversion module to unify the data collected by different sensors into the same coordinate system, and uses the data synchronization module to ensure the time consistency of the ultrasonic data collected by each sensor, further improving the accuracy of the obtained ultrasonic data. In addition, the embodiment of the present application also uses the data fusion module to integrate the pose information (electromagnetic tracking data and IMU data) of the ultrasonic probe collected by the IMU sensor and the electromagnetic tracking sensor to obtain more accurate pose information of the ultrasonic probe. Description of the Drawings

[0012] To more clearly illustrate the technical solutions of the embodiments of the present application, the accompanying drawings required for the description of the embodiments of the present application will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can be obtained based on these drawings.

[0013] Figure 1 is a schematic structural diagram of an ultrasonic image and doctor's manipulation acquisition system provided by an embodiment of the present application; Figure 2 is a schematic diagram of an ultrasonic acquisition process provided by an embodiment of the present application; Figure 3 is a schematic structural diagram of an ultrasonic probe provided by an embodiment of the present application; Figure 4 is a schematic diagram of a data fusion process provided by an embodiment of the present application; Figure 5 is a schematic diagram of the training process of a data fusion model provided by an embodiment of the present application; Figure 6 is a flowchart of the steps of an ultrasonic image and doctor's manipulation acquisition method provided by an embodiment of the present application; Figure 7 is a schematic diagram of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0014] The exemplary embodiments of the present application will be described in more detail below in conjunction with the accompanying drawings in the embodiments of the present application. Although the exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present application and to fully convey the scope of the present application to those skilled in the art.

[0015] Ultrasonic acquisition technology is widely used in fields such as medicine, industry, and scientific research. Its application usually requires using a handheld ultrasonic probe to acquire ultrasonic images. While performing ultrasonic acquisition, relevant information such as the pose of the ultrasonic probe is recorded for accurately positioning the position of the object to be detected corresponding to the ultrasonic image.

[0016] However, related ultrasound acquisition technologies often have difficulty in obtaining accurate ultrasound information, which affects the reliability and accuracy of the finally obtained ultrasound data. The main deficiencies are as follows: 1) Data synchronization and multi-sensor fusion: Most existing technologies rely on a single type of sensor, resulting in challenges in spatio-temporal synchronization of data and multi-sensor data fusion. 2) Reliability and accuracy of data acquisition: Whether it is an Inertial Measurement Unit (IMU), an optical tracking sensor, or an electromagnetic tracking sensor, there are problems such as data loss, drift, or interference, which affect the reliability and accuracy of data acquisition. 3) User experience and learning cost: Most existing technical solutions change the customary operation habits of ultrasound acquisition, increase the learning cost, and limit their application in actual clinical practice.

[0017] Specifically, when using an IMU sensor for ultrasound acquisition, the following problems exist: 1) Problem of position information acquisition: The IMU can only collect the attitude information of the probe, but cannot directly obtain its position information. The position information needs to be obtained by double integrating the acceleration, but the double integration process is prone to drift and error accumulation of the position information, making the position information of the IMU less reliable. 2) Change in operation habits: Doctors need to hold the end of the probe instead of the standard position, which changes the holding habits of doctors, breaks the workflow of doctors, increases the learning cost, and is not conducive to large-scale data acquisition. 3) Patient cooperation problem: During the ultrasound scanning process, the patient sometimes needs to cooperate by moving or turning the body. When collecting the pose of the probe, in fact, it is to collect the information of the probe moving relative to the human body. Therefore, the information of the patient's movement is also very important, but the related methods do not consider how to collect the patient's movement.

[0018] The optical tracking system is a method of capturing the position information and attitude information of the probe by fixing a photosensitive sphere at the end of the probe and placing it within the field of view of the optical tracking system. Although the method of using the optical tracking system to capture the pose (position and attitude) information of the ultrasound probe is effective in some cases, there are also significant deficiencies. 1) Occlusion problem: The photosensitive sphere is easily occluded. Once occluded, the optical tracking system cannot obtain the pose information of the probe, resulting in data loss. 2) Problem of force information acquisition: The optical tracking system cannot collect the force information of the probe in contact with the human body, and the force information is crucial for understanding the doctor's operation and optimizing the ultrasound image quality.

[0019] In addition, the electromagnetic tracking system obtains the position and attitude information of the probe through electromagnetic sensors, which has the advantage of being unaffected by occlusion. However, this system is prone to electromagnetic interference in a metal environment, affecting data accuracy.

[0020] In view of the above problems, an ultrasonic image and doctor's manipulation acquisition system is proposed in an embodiment of the present application to improve the accuracy of the acquired ultrasonic data. The following will, in conjunction with the accompanying drawings, elaborate on an ultrasonic image and doctor's manipulation acquisition system provided in an embodiment of the present application through some embodiments and their application scenarios.

[0021] In a first aspect of an embodiment of the present application, an ultrasonic image and doctor's manipulation acquisition system is provided. Referring to Figure 1 , Figure 1 which is a schematic structural diagram of an ultrasonic image and doctor's manipulation acquisition system provided in an embodiment of the present application. As Figure 1 shown, the system includes: A data acquisition module, which includes: an ultrasonic probe, an IMU sensor, an electromagnetic tracking sensor, and an RGBD camera. Among them, the IMU sensor and the electromagnetic tracking sensor are fixed on the ultrasonic probe.

[0022] A coordinate system conversion module, which is used to perform coordinate system conversion on the initial IMU data collected by the IMU sensor, the initial electromagnetic tracking data collected by the electromagnetic tracking sensor, and the initial human body pose information collected by the RGBD camera to obtain the converted IMU data, converted electromagnetic tracking data, and converted human body pose information in a unified coordinate system.

[0023] A data synchronization module, which is used to align the time stamps of the ultrasonic data collected by the data acquisition module and divide them into multiple first ultrasonic data sets according to the time stamps carried by each ultrasonic data.

[0024] A data fusion model, which is used to perform data fusion on the converted IMU data and the converted electromagnetic tracking data belonging to the same first ultrasonic data set to obtain the fused probe pose information and generate a second ultrasonic data set.

[0025] A data storage module, which is used to store the second ultrasonic data set in the RLDS format. Each second ultrasonic data set includes: the ultrasonic image collected by the ultrasonic probe, and the fused probe pose information and the converted human body pose information corresponding to the ultrasonic image.

[0026] Among them, the second ultrasonic data set collected by this system includes: ultrasonic images (i.e., the ultrasonic images collected by the ultrasonic probe) and doctor's manipulation (i.e., the movement trajectory information of the doctor controlling the ultrasonic probe). This movement trajectory information can be obtained by integrating the fused probe pose information and the converted human body pose information. The specific integration method is not limited in this embodiment.

[0027] In an embodiment of the present application, referring to Figure 2 ,Figure 2 shows a schematic diagram of an ultrasound acquisition process, as Figure 2 shown. In response to an ultrasound acquisition request received (for example, after the operator controls the ultrasound probe to move to a certain position and then presses the confirmation button for ultrasound acquisition), the data acquisition module controls each sensor to start data acquisition. Specifically, it controls the ultrasound probe to acquire, and after being processed by the ultrasound instrument, an ultrasound image of the corresponding position is obtained. It controls the IMU sensor to acquire, and after being processed by the IMU processing unit, IMU data at the current moment is obtained (representing the position and attitude information of the ultrasound probe at the current moment). It controls the electromagnetic tracking sensor (i.e., the receiver of the electromagnetic motion tracker) to acquire, and after being processed by the electromagnetic tracking system processing unit, electromagnetic tracking data at the current moment is obtained (representing the position and attitude information of the ultrasound probe at the current moment). It controls the RGBD camera to acquire, and after being processed by the human pose estimation module, human pose information at the current moment is obtained. Among them, an acquisition card can be used to bypass the transmission of the ultrasound image to the acquisition system to ensure the synchronous acquisition of the ultrasound image and other sensor data, so as to achieve a complete recording of the manipulation data. The RGBD camera can be used to non-contact measure human pose and motion information, which is suitable for situations where the patient needs to cooperate with movement or turning over, such as cardiac ultrasound acquisition. In this embodiment, human pose information can be extracted from the depth information image obtained by the RGBD camera through deep learning methods (such as DensePose or Mediapipe), and information such as the height and shoulder width of the human body can also be obtained from the depth information image through Mediapipe key point detection.

[0028] Referring to Figure 3 , Figure 3 shows a schematic diagram of the structure of an ultrasound probe, as Figure 3 shown, where the IMU sensor and the electromagnetic tracking sensor are fixed on the ultrasound probe (exemplarily, the fixed position is as Figure 3 shown, and it can be fixed at the end of the ultrasound probe, and the other end of the ultrasound probe is the end that is more in contact with the detection object), and during the process of the operator controlling the ultrasound probe for ultrasound acquisition, it moves along with the movement of the ultrasound probe.

[0029] In a possible implementation manner, the data acquisition module further includes: a force sensor, one end of the force sensor is connected to the ultrasound probe, and the other end is connected to the handle of the ultrasound probe to measure the interaction force data applied by the operator to the ultrasound probe through the handle of the ultrasound probe; each of the second ultrasound data sets further includes: the interaction force data corresponding to the ultrasound image.

[0030] As Figure 2As shown, the data acquisition module further includes a force sensor. When collecting ultrasonic data, it also includes controlling the force sensor to collect interaction force data, that is, the magnitude of the interaction force exerted by the operator on the ultrasonic probe at the current moment, which is equivalent to the magnitude information of the force between the ultrasonic probe and the detection object. Specifically, one end of the force sensor is fixed to the end of the ultrasonic probe through a probe clamp, and the other end of the force sensor is fixedly connected to the handle of the ultrasonic probe through a handheld clamp. Exemplarily, the specific structure can be as Figure 3 shown, the handheld housing wraps the ultrasonic probe, and a force sensor (corresponding to the force sensor in Figure 3 ) is provided between the handheld housing and the ultrasonic probe. The operator controls the ultrasonic probe by holding the handheld housing. While the force sensor transmits the force exerted by the operator to the ultrasonic probe, it detects the magnitude of the force. When used for the first time, parameters such as the weight, center of mass, and inertia of the ultrasonic probe can be identified through a load identification method. During the actual acquisition process, the data collected by the force sensor is converted into the force between the end of the probe and the human body through an algorithm.

[0031] The embodiment of the present application integrates multiple sensors, including a force sensor, which is used to obtain the force information (interaction force data) between the ultrasonic probe and the detection object, making up for the deficiencies of the optical tracking system (function modules such as IMU sensors that rely on optical information for positioning), and is beneficial to understanding the doctor's operation and optimizing the ultrasonic image quality. In addition, as Figure 3 shown, the structure of the ultrasonic probe is more in line with the holding experience of the operation habit. The entire acquisition system has little interference to the operator. The operator only needs to hold the probe in the traditional way to complete the scan, without considering the occlusion of the sensor and without changing the holding position and posture.

[0032] As Figure 2 shown, after the data acquisition module controls each sensor to collect corresponding ultrasonic data (including: the initial IMU data collected by the IMU sensor, the initial electromagnetic tracking data collected by the electromagnetic tracking sensor, and the initial human body pose information collected by the RGBD camera), the coordinate system conversion module performs coordinate system conversion on the ultrasonic data related to the pose information to obtain ultrasonic data belonging to the same coordinate system. This is because for different sensors, due to different installation positions, the coordinate systems to which the collected information belongs are different (the origin positions of the coordinate systems are different). Through the coordinate system conversion module, the collected initial data is converted into data in the same coordinate system (converted IMU data, converted electromagnetic tracking data, and converted human body pose information) to obtain more accurate ultrasonic data.

[0033] In a possible implementation, the coordinate system conversion of the initial IMU data collected by the IMU sensor, the initial electromagnetic tracking data collected by the electromagnetic tracking sensor, and the initial human body pose information collected by the RGBD camera includes: Obtain the transformation matrix between the IMU sensor, the electromagnetic tracking sensor, and the RGBD camera through calibration; According to the transformation matrix, perform coordinate system conversion on the initial IMU data, the initial electromagnetic tracking data, and the initial human body pose information.

[0034] Specifically, a target coordinate system can be determined, and then the rotation angle and translation distance (translation distance in the x direction, translation distance in the y direction, and translation distance in the z direction) of the origin of the coordinate system to which each device belongs relative to the target coordinate system can be determined through calibration. Furthermore, the data can be corrected according to the rotation angle and translation distance. Since the coordinate systems of the RGBD camera, the IMU sensor, and the electromagnetic tracking sensor are not unified, in the embodiments of the present application, the transformation matrix between the sensors is obtained through calibration, and the data collected by different devices in different coordinate systems is unified into one coordinate system for analysis and calculation, further improving the accuracy of the collected data.

[0035] In this embodiment, since multiple sensors are integrated and the data collection frequencies of different sensors are different, in response to an ultrasound collection request, the timestamps of the ultrasound data collected by each sensor controlled by the data collection module (including: the initial IMU data collected by the IMU sensor, the initial electromagnetic tracking data collected by the electromagnetic tracking sensor, the initial human body pose information collected by the RGBD camera, the ultrasound images collected by the ultrasound probe, and the interaction force data collected by the force sensor) are different. In the embodiments of the present application, the data synchronization module is used to align the timestamps of the collected ultrasound data, and the ultrasound data with the same or similar timestamps is divided into multiple first ultrasound data sets to ensure the time consistency of the data collected by each sensor.

[0036] In a possible implementation, the timestamp alignment of the ultrasound data collected by the data collection module, divided into multiple first ultrasound data sets according to the timestamp carried by each ultrasound data, includes: According to the timestamp carried by each ultrasound data, the ultrasound data with the same timestamp and the ultrasound data with the difference between timestamps less than or equal to a first preset threshold are divided into the same first ultrasound data set; The ultrasound data with the difference between timestamps greater than the first preset threshold is deleted as invalid data.

[0037] Considering that the data collection frequencies of various sensors are not unified, in order to ensure data synchronization, the embodiment of the present application introduces a data synchronization module to synchronize and filter all data, discarding the data with too long acquisition intervals and retaining the valid data within the acquisition intervals. Specifically, a first preset threshold can be set in advance. Each time ultrasonic data is collected, ultrasonic data collected by multiple sensors is obtained. The ultrasonic data with the time difference between timestamps less than or equal to the first preset threshold is divided into the same first ultrasonic data set. Exemplarily, the timestamp of the ultrasonic image A collected by the ultrasonic probe is 1.005 s, and the timestamp of the initial IMU data B collected by the IMU sensor is 1.010 s. The first preset threshold set in advance is 10 ms. Then the time difference between the timestamps of the ultrasonic image A and the initial IMU data B does not exceed the first preset threshold, and the ultrasonic image A and the initial IMU data B can be divided into the same first ultrasonic data set. In addition, for the ultrasonic data with the time difference between timestamps greater than the first preset threshold, since the acquisition time interval between the two ultrasonic data is too long, it cannot be processed as ultrasonic data at the same position. In order to improve the accuracy of the obtained data, the embodiment of the present application deletes and filters it as invalid data.

[0038] An electromagnetic motion tracker EMT (i.e., an electromagnetic tracking sensor) and an IMU sensor are used to collect the motion trajectory data of the ultrasonic probe during ultrasonic operation. By fusing the above data, the acquisition accuracy and stability are improved, and the disadvantages of a single sensor are avoided. The traditional inertial measurement unit (IMU) has high short-term accuracy, continuous output, and strong anti-interference ability. However, with the passage of time, the navigation error gradually increases. The electromagnetic sensor (EMT) can provide high-precision navigation information, but it is greatly affected by the environment and it is difficult to achieve continuous output. Therefore, the embodiment of the present application combines the IMU and the EMT to achieve complementary advantages. Specifically, after the embodiment of the present application aligns the collected ultrasonic data by using the data synchronization module, one or more first ultrasonic data sets are obtained. Each first ultrasonic data set contains the ultrasonic data obtained by performing an ultrasonic acquisition on a specific position. Since the data collected by the IMU sensor and the electromagnetic tracking sensor are both used to characterize the pose information of the ultrasonic probe, the embodiment of the present application proposes to use a data fusion model to fuse the transformed IMU data and the transformed electromagnetic tracking data belonging to the same first ultrasonic data set to obtain more accurate pose information of the ultrasonic probe (i.e., the fused probe pose information). Then, the obtained fused probe pose information is used to replace the transformed IMU data and the transformed electromagnetic tracking data in the first ultrasonic data set for data fusion to generate the corresponding second ultrasonic data set.

[0039] In a possible implementation, the IMU sensor collects IMU data at a first frequency, and the electromagnetic tracking sensor collects electromagnetic tracking data at a second frequency; the data fusion model includes: a KRR module, a first KF module, a second KF module, a buffer module, and an ANN module; the data fusion of the converted IMU data and the converted electromagnetic tracking data belonging to the same first ultrasound data set includes: Step S101: Input the converted electromagnetic tracking data of the previous t cycles into the KRR module to obtain interpolated electromagnetic tracking data, where the interpolated electromagnetic tracking data is the electromagnetic tracking data with the same timestamp as the converted IMU data.

[0040] Specifically, referring to Figure 4 , Figure 4 shows a schematic diagram of a data fusion process. As shown in Figure 4 , first select a first ultrasound data set, and input the converted IMU data and the converted electromagnetic tracking data in the first ultrasound data set into the data fusion model. Since the IMU sensor collects IMU data at a first frequency and the electromagnetic tracking sensor collects electromagnetic tracking data at a second frequency, the timestamps of the converted IMU data and the converted electromagnetic tracking data in the obtained first ultrasound data set cannot be exactly the same. For example, the converted IMU data f IMU (n) is the IMU data collected at time n (actually representing the pose information of the ultrasound probe at time n), and the converted electromagnetic tracking data f EMT (n’) is the electromagnetic tracking data collected at time n' (actually representing the pose information of the ultrasound probe at time n').

[0041] To synchronize the electromagnetic tracking data with the converted IMU data f IMU (n) in this application embodiment, the KRR module is used to interpolate the interpolated electromagnetic tracking data within the sampling interval of the electromagnetic tracking sensor, that is, the electromagnetic tracking data that the electromagnetic tracking sensor should collect at time n f EMT ’(n) . The KRR module uses the Kernel Ridge Regression (KRR) algorithm to obtain the interpolated electromagnetic tracking data by fitting the converted electromagnetic tracking data collected in the nearest t cycles (t cycles when the electromagnetic tracking sensor collects electromagnetic tracking data at the second frequency) input into the module fEMT ’(n) Among them, the value of t can be set based on actual application requirements and is not limited in this embodiment.

[0042] Step S102: Determine the difference between the interpolated electromagnetic tracking data and the converted IMU data as the first error. Specifically, the converted IMU data f IMU (n) is the IMU data collected at time n, and the interpolated electromagnetic tracking data f EMT ’(n) is the predicted electromagnetic tracking data collected at time n. As Figure 4 shown, determine the difference between the two as the first error △ f’(n) .

[0043] Step S103: Filter the first error through the first KF module to obtain the filtered error. Specifically, the KF module is a Kalman filter, which is used to filter the calculated first error to remove the noise in it and obtain a more accurate filtered error δ f’(n) .

[0044] Step S104: Input the filtered error into the buffer module to generate an error sequence, where the error sequence represents a sequence composed of the filtered errors in the previous m cycles in chronological order. Specifically, the buffer module stores historical information and obtains the filtered errors in the previous m cycles to generate an error sequence δ f’(n)…… δ f’(n - m). Among them, the value of m can be set based on actual application requirements and is not limited in this embodiment.

[0045] Step S105: Input the error sequence into the ANN model for prediction to obtain an estimated error; the ANN model is used to correct the estimated error caused by filtering using the interpolated electromagnetic tracking data. The estimated error represents the error generated by the predicted first KF module during filtering. Specifically, use the generated error sequence as the input of the ANN model, and this ANN model is used to predict the error (i.e., the estimated error) δ generated by the KF module during filtering f’’(n) .

[0046] Step S106: Calculate the fused probe pose information based on the converted IMU data, the filtered error, and the estimated error.

[0047] Specifically, the converted IMU data is corrected using the filtered error and the estimated error according to the following calculation formula to obtain the fused probe pose information: ; Where is the fused probe pose information, is the converted IMU data, is the filtered error, is the estimated error. Traditional inertial measurement units (IMUs) have high short-term accuracy, continuous output, and strong anti-interference ability. However, as time accumulates, the navigation error gradually increases. Electromagnetic tracking sensors (EMTs) can provide high-precision navigation information, but they are greatly affected by the environment and it is difficult to achieve continuous output. Therefore, in the embodiments of the present application, through the above steps S101-S106, the IMU and EMT are combined based on a data fusion algorithm of machine learning to collect and fuse sensor data at different frequencies to achieve complementary advantages.

[0048] In a possible implementation manner, the data fusion model is trained according to the following steps: Step S201, obtaining a training sample data pair, where the training sample data pair includes: sample IMU data and sample electromagnetic tracking data. Specifically, referring to Figure 5 , Figure 5 shows a schematic diagram of the training process of a data fusion model. As shown in Figure 5 , first, a training sample data pair is selected. The sample IMU data in this training sample data pair (such as Figure 5 shown as ) is the data collected by the IMU sensor for IMU data at the first frequency, and the sample electromagnetic tracking data is the data collected by the electromagnetic tracking sensor for electromagnetic tracking data at the second frequency. Due to different collection frequencies, the timestamps between the two cannot be exactly the same (the difference between the two timestamps can be within the first preset threshold range).

[0049] Step S202, inputting the historical sample electromagnetic tracking data of the previous t-1 cycles before the sample electromagnetic tracking data and the sample electromagnetic tracking data into the KRR module to obtain interpolated sample electromagnetic tracking data. The interpolated sample electromagnetic tracking data (such as Figure 5 shown as ) is the predicted electromagnetic tracking data with the same timestamp as the sample IMU data.

[0050] Step S203: Determine the difference between the interpolated sample electromagnetic tracking data and the sample IMU data as the first sample error. Specifically, the sample IMU data (such as Figure 5 shown in ) is the IMU data collected at time n, and the interpolated sample electromagnetic tracking data (such as Figure 5 shown in ) is the predicted electromagnetic tracking data collected at time n. Determine the difference between the two as the first sample error (such as Figure 5 shown in ).

[0051] Step S204: Filter the first sample error through the first KF module to obtain the first filtered sample error (such as Figure 5 shown in ).

[0052] Step S205: Input the first filtered sample error into the buffer module to generate an error sample sequence, where the error sample sequence represents a sequence composed of the first filtered sample errors of the previous m cycles in chronological order. Specifically, store historical information through the buffer module, obtain the first filtered sample errors of the previous m cycles, and generate an error sample sequence δ f’(n)…… δ f’(n - m). where the value of m can be set according to actual application requirements and is not limited in this embodiment.

[0053] Step S206: Calculate the true error value based on the sample IMU data and the sample electromagnetic tracking data without KRR interpolation.

[0054] In a possible implementation manner, in step S206, calculating the true error value based on the sample IMU data and the sample electromagnetic tracking data without KRR interpolation includes: Step S2061: Determine the difference between the sample IMU data and the sample electromagnetic tracking data as the second sample error (such as Figure 5 shown in ).

[0055] Step S2062: Filter the second sample error through the second KF module to obtain the second filtered sample error (such as Figure 5 shown in ).

[0056] Step S2063: Determine the difference between the second filtered sample error and the first filtered sample error as the true error value (such as Figure 5 shown in ).

[0057] Step S207: Determine the true error value as the training label, determine the error sample sequence as the training data, input them into the ANN module, and obtain the estimated sample error.

[0058] Step S208: Calculate the loss function value based on the estimated sample error and the true error value, and update the model parameters according to the loss function value. Specifically, update the parameters of the KRR module, ANN model, first KF module, and second KF module in the data fusion model by calculating the loss.

[0059] Step S209: Repeat the above steps until the preset training times are reached or the loss function converges, end the training, and obtain the trained data fusion model.

[0060] In the embodiment of the present application, a data storage module is further included, which is used to store the second ultrasound data set in the RLDS format. Each second ultrasound data set includes: the ultrasound image collected by the ultrasound probe, and the fused probe pose information and the transformed human pose information corresponding to the ultrasound image. In actual application, the corresponding second ultrasound data set is obtained from the data storage module and displayed, that is, not only the collected ultrasound image is displayed, but also the probe pose information, human pose information, interaction force magnitude, and other information corresponding to the ultrasound image are displayed. The technical solution proposed in the embodiment of the present application optimizes the data acquisition process by combining multiple sensors (IMU, electromagnetic tracking, RGBD camera, etc.), and improves the synchronization and accuracy of the collected ultrasound data (the position and pose of the probe, human pose, ultrasound image, contact force between the probe and the human body, three-dimensional point cloud information of the scene). In addition, after obtaining multiple second ultrasound data sets, the obtained data can be used to train an intelligent agent to perform ultrasound acquisition actions and complete ultrasound scanning, providing strong technical support for the realization of an automated ultrasound robot in the later stage.

[0061] In the embodiments of the present application, through multi-sensor integration, the IMU sensor and the electromagnetic tracking sensor are fixed to the ultrasonic probe. While collecting ultrasonic images using the ultrasonic probe, the pose information of the corresponding ultrasonic probe (electromagnetic tracking data and IMU data) and the human body pose (human body pose information collected by the RGBD camera) are obtained. By comprehensively utilizing the advantages of various sensors to make up for the deficiencies of a single sensor, the comprehensiveness of data collection is ensured, thereby improving the accuracy of the obtained ultrasonic data (for example, the position of the object to be detected corresponding to the ultrasonic image can be more accurately located). In the embodiments of the present application, by using the coordinate system conversion module, the data collected by different sensors are unified into the same coordinate system, and by using the data synchronization module, the time consistency of the ultrasonic data collected by each sensor is ensured, further improving the accuracy of the obtained ultrasonic data. In addition, the embodiments of the present application also use the data fusion module to integrate the pose information of the ultrasonic probe (electromagnetic tracking data and IMU data) collected by the IMU sensor and the electromagnetic tracking sensor to obtain more accurate pose information of the ultrasonic probe. The ultrasonic acquisition system proposed in the present application can adapt to different types of operation tasks and environmental requirements, such as medical surgery, industrial assembly, etc., and has high versatility and scalability.

[0062] In the second aspect of the embodiments of the present application, there is also provided a method for collecting ultrasonic images and doctor's techniques, which is applied to the ultrasonic image and doctor's technique collection system described in the first aspect. Referring to Figure 6 , Figure 6 shows a flowchart of the steps of a method for ultrasonic images and doctor's techniques. As shown in Figure 6 , the method includes: Step S301, collecting ultrasonic data using an ultrasonic probe, an IMU sensor, an electromagnetic tracking sensor, and an RGBD camera; Step S302, performing coordinate system conversion on the initial IMU data collected by the IMU sensor, the initial electromagnetic tracking data collected by the electromagnetic tracking sensor, and the initial human body pose information collected by the RGBD camera to obtain the converted IMU data, the converted electromagnetic tracking data, and the converted human body pose information after being unified into the same coordinate system; Step S303, aligning the timestamps of the collected ultrasonic data, and dividing them into multiple first ultrasonic data sets according to the timestamps carried by each ultrasonic data; Step S304, performing data fusion on the converted IMU data and the converted electromagnetic tracking data belonging to the same first ultrasonic data set to obtain the fused probe pose information and generate a second ultrasonic data set; Step S305, store the second ultrasound data set in the RLDS format. Each second ultrasound data set includes: the ultrasound image collected by the ultrasound probe, and the fused probe pose information and the transformed human body pose information corresponding to the ultrasound image.

[0063] Among them, the second ultrasound data set collected by this method includes: ultrasound images (i.e., the ultrasound images collected by the ultrasound probe) and doctor's maneuvers (i.e., the movement trajectory information of the doctor controlling the ultrasound probe). This movement trajectory information can be obtained by integrating the fused probe pose information and the transformed human body pose information. The specific integration method is not limited in this embodiment.

[0064] In a possible implementation manner, the IMU sensor collects IMU data at a first frequency, and the electromagnetic tracking sensor collects electromagnetic tracking data at a second frequency; the data fusion of the transformed IMU data and the transformed electromagnetic tracking data belonging to the same first ultrasound data set includes: Input the transformed electromagnetic tracking data of the previous t cycles into the KRR module to obtain interpolated electromagnetic tracking data, where the interpolated electromagnetic tracking data is the electromagnetic tracking data with the same timestamp as the transformed IMU data; Determine the difference between the interpolated electromagnetic tracking data and the transformed IMU data as the first error; Filter the first error through the first KF module to obtain the filtered error; Input the filtered error into the buffer module to generate an error sequence, where the error sequence represents a sequence composed of the filtered errors of the previous m cycles in chronological order; Input the error sequence into the ANN model for prediction to obtain an estimated error; the estimated error represents the error generated by filtering through the predicted first KF module; Calculate the fused probe pose information according to the transformed IMU data, the filtered error, and the estimated error.

[0065] In a possible implementation manner, the KRR module, the first KF module, the buffer module, and the ANN module are trained according to the following steps: Obtain training sample data pairs, where the training sample data pairs include: sample IMU data and sample electromagnetic tracking data; Input the historical sample electromagnetic tracking data of the previous t - 1 cycles before the said sample electromagnetic tracking data and the said sample electromagnetic tracking data into the KRR module to obtain interpolated sample electromagnetic tracking data, where the interpolated sample electromagnetic tracking data is the electromagnetic tracking data with the same timestamp as that carried by the said sample IMU data; Determine the difference between the interpolated sample electromagnetic tracking data and the said sample IMU data as the first sample error; Filter the first sample error through the first KF module to obtain the first filtered sample error; Input the first filtered sample error into the buffer module to generate an error sample sequence, where the error sample sequence represents a sequence composed of the first filtered sample errors of the previous m cycles in chronological order; Calculate the true error value according to the said sample IMU data and the sample electromagnetic tracking data without KRR interpolation; Determine the true error value as the training label, determine the error sample sequence as the training data, input them into the ANN module to obtain the estimated sample error; Calculate the loss function value according to the estimated sample error and the true error value, and update the model parameters according to the loss function value; Repeat the above steps until the preset training times are reached or the loss function converges, and end the training.

[0066] In a possible implementation manner, the calculating the true error value according to the said sample IMU data and the sample electromagnetic tracking data without KRR interpolation includes: Determine the difference between the said sample IMU data and the sample electromagnetic tracking data as the second sample error; Filter the second sample error through the second KF module to obtain the second filtered sample error; Determine the difference between the second filtered sample error and the first filtered sample error as the true error value.

[0067] In a possible implementation manner, the method further includes: measuring the interaction force data applied by the operator to the ultrasonic probe through the handle of the ultrasonic probe by using a force sensor; each said second ultrasonic data set further includes: the interaction force data corresponding to the ultrasonic image.

[0068] In a possible implementation manner, the time - stamping alignment of the collected ultrasonic data and dividing them into multiple first ultrasonic data sets according to the timestamp carried by each ultrasonic data includes: According to the timestamps carried by each piece of the ultrasonic data, the ultrasonic data with the same timestamps, and the ultrasonic data with the difference between timestamps less than or equal to the first preset threshold are divided into the same first ultrasonic data set; The ultrasonic data with the difference between timestamps greater than the first preset threshold is deleted as invalid data.

[0069] In a possible implementation manner, the coordinate system conversion of the initial IMU data collected by the IMU sensor, the initial electromagnetic tracking data collected by the electromagnetic tracking sensor, and the initial human body pose information collected by the RGBD camera includes: Obtain the transformation matrix between the IMU sensor, the electromagnetic tracking sensor, and the RGBD camera through calibration; According to the transformation matrix, perform coordinate system conversion on the initial IMU data, the initial electromagnetic tracking data, and the initial human body pose information.

[0070] The embodiment of the present application also provides an electronic device. Refer to Figure 7 , Figure 7 is a schematic diagram of the electronic device proposed by the embodiment of the present application. As Figure 7 shown, the electronic device 100 includes: a memory 110 and a processor 120. The memory 110 and the processor 120 are communicatively connected through a bus. A computer program is stored in the memory 110, and the computer program can run on the processor 120, thereby implementing the steps in the ultrasonic image and doctor's technique acquisition method disclosed in the embodiment of the present application.

[0071] The embodiment of the present application also provides a computer-readable storage medium, on which a computer program / instructions are stored. When the computer program / instructions are executed by a processor, the steps in the ultrasonic image and doctor's technique acquisition method disclosed in the embodiment of the present application are implemented.

[0072] The embodiment of the present application also provides a computer program product. When the computer program product runs on an electronic device, it causes the processor to execute the steps in the ultrasonic image and doctor's technique acquisition method disclosed in the embodiment of the present application.

[0073] Each embodiment in this specification is described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The same or similar parts among the embodiments can be referred to each other.

[0074] Embodiments of the present application are described with reference to the flowcharts and / or block diagrams of methods, apparatuses, electronic devices, and computer program products according to embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing terminal devices generate a device for implementing the functions specified in one process Figure 1 one process or multiple processes and / or blocks Figure 1 or multiple blocks.

[0075] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device that implements the functions specified in one process Figure 1 one process or multiple processes and / or blocks Figure 1 or multiple blocks.

[0076] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device, such that a series of operation steps are executed on the computer or other programmable terminal device to generate a computer-implemented process, so that the instructions executed on the computer or other programmable terminal device provide steps for implementing the functions specified in one process Figure 1 one process or multiple processes and / or blocks Figure 1 or multiple blocks.

[0077] Although the preferred embodiments of the embodiments of the present application have been described, those skilled in the art can make additional changes and modifications once they learn the basic creative concepts. Therefore, the appended claims are intended to be construed as including the preferred embodiments and all changes and modifications falling within the scope of the embodiments of the present application.

[0078] Finally, it should also be noted that in this text, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or terminal device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or terminal device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or terminal device comprising said element.

[0079] The above has introduced in detail an ultrasonic imaging and doctor's technique acquisition system, method and product provided by the present application. Specific examples are used in this text to elaborate on the principle and implementation manner of the present application. The description of the above embodiments is only used to help understand the method and its core idea of the present application; at the same time, for those of ordinary skill in the art, according to the idea of the present application, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present application.

Claims

1. An ultrasonic imaging and doctor's technique acquisition system, characterized in that, The system includes: A data acquisition module, which includes: an ultrasonic probe, an IMU sensor, an electromagnetic tracking sensor, and an RGBD camera. Among them, the IMU sensor and the electromagnetic tracking sensor are fixed on the ultrasonic probe; A coordinate system conversion module, which is used to perform coordinate system conversion on the initial IMU data collected by the IMU sensor, the initial electromagnetic tracking data collected by the electromagnetic tracking sensor, and the initial human body pose information collected by the RGBD camera, so as to obtain the converted IMU data, the converted electromagnetic tracking data, and the converted human body pose information after the unified coordinate system; A data synchronization module, which is used to align the timestamps of the ultrasonic data collected by the data acquisition module, and divide them into multiple first ultrasonic data sets according to the timestamps carried by each ultrasonic data; A data fusion model, which is used to perform data fusion on the converted IMU data and the converted electromagnetic tracking data belonging to the same first ultrasonic data set to obtain the fused probe pose information and generate a second ultrasonic data set; A data storage module, which is used to store the second ultrasonic data set in the RLDS format. Each second ultrasonic data set includes: the ultrasonic image collected by the ultrasonic probe, and the fused probe pose information and the converted human body pose information corresponding to the ultrasonic image.

2. The ultrasonic imaging and doctor's manipulation acquisition system according to claim 1, wherein, The IMU sensor collects IMU data at a first frequency, and the electromagnetic tracking sensor collects electromagnetic tracking data at a second frequency; the data fusion model includes: a KRR module, a first KF module, a buffer module, and an ANN module; the data fusion of the converted IMU data and the converted electromagnetic tracking data belonging to the same first ultrasonic data set includes: Input the converted electromagnetic tracking data of the previous t cycles into the KRR module to obtain interpolated electromagnetic tracking data, which is the electromagnetic tracking data with the same timestamp as the converted IMU data; Determine the difference between the interpolated electromagnetic tracking data and the converted IMU data as the first error; Filter the first error through the first KF module to obtain the first filtered error; Input the first filtered error into the buffer module to generate an error sequence, which represents the sequence composed of the first filtered errors of the previous m cycles in chronological order; Input the error sequence into the ANN model for prediction to obtain an estimated error, and the ANN model is used to correct the estimated error caused by using the interpolated electromagnetic tracking data for filtering; Calculate the fused probe pose information according to the converted IMU data, the filtered error, and the estimated error.

3. The ultrasonic imaging and physician's manipulation acquisition system according to claim 2, wherein The data fusion model is trained according to the following steps: Obtain training sample data pairs, which include: sample IMU data and sample electromagnetic tracking data; Input the historical sample electromagnetic tracking data of the previous t-1 cycles before the sample electromagnetic tracking data and the sample electromagnetic tracking data into the KRR module to obtain interpolated sample electromagnetic tracking data, where the interpolated sample electromagnetic tracking data is the electromagnetic tracking data interpolated to have the same timestamp as the sample IMU data carried; Determine the difference between the interpolated sample electromagnetic tracking data and the sample IMU data as the first sample error; Filter the first sample error through the first KF module to obtain the first filtered sample error; Input the first filtered sample error into the buffer module to generate an error sample sequence, where the error sample sequence represents a sequence composed of the first filtered sample errors of the previous m cycles in chronological order; Calculate the true error value according to the sample IMU data and the sample electromagnetic tracking data without KRR interpolation; Determine the true error value as the training label, determine the error sample sequence as the training data, input them into the ANN module to obtain the estimated sample error; Calculate the loss function value according to the estimated sample error and the true error value, and update the model parameters according to the loss function value; Repeat the above steps until the preset training times are reached or the loss function converges, end the training, and obtain the trained data fusion model.

4. The ultrasonic imaging and doctor's manipulation acquisition system according to claim 3, wherein The calculating the true error value according to the sample IMU data and the sample electromagnetic tracking data without KRR interpolation includes: Determine the difference between the sample IMU data and the sample electromagnetic tracking data as the second sample error; Filter the second sample error through the second KF module to obtain the second filtered sample error; Determine the difference between the second filtered sample error and the first filtered sample error as the true error value.

5. The ultrasonic imaging and doctor's manipulation acquisition system according to claim 1, characterized in that, The data acquisition module further includes: a force sensor, one end of the force sensor is connected to the ultrasonic probe, and the other end is connected to the handle of the ultrasonic probe to measure the interaction force data applied by the operator to the ultrasonic probe through the handle of the ultrasonic probe; each second ultrasonic data set further includes: the interaction force data corresponding to the ultrasonic image.

6. The ultrasonic imaging and doctor's manipulation acquisition system according to claim 1, characterized in that, The time stamp alignment of the ultrasonic data collected by the data acquisition module and dividing them into multiple first ultrasonic data sets according to the time stamp carried by each ultrasonic data includes: According to the time stamp carried by each ultrasonic data, divide the ultrasonic data with the same time stamp and the ultrasonic data with the time stamp difference less than or equal to the first preset threshold into the same first ultrasonic data set; Delete the ultrasonic data with the time stamp difference greater than the first preset threshold as invalid data.

7. The ultrasonic imaging and doctor's manipulation acquisition system according to claim 1, characterized in that The coordinate system conversion of the initial IMU data collected by the IMU sensor, the initial electromagnetic tracking data collected by the electromagnetic tracking sensor, and the initial human body pose information collected by the RGBD camera includes: Obtain the transformation matrix between the IMU sensor, the electromagnetic tracking sensor, and the RGBD camera through calibration; According to the transformation matrix, perform coordinate system conversion on the initial IMU data, the initial electromagnetic tracking data, and the initial human body pose information.

8. An ultrasonic imaging and doctor's technique acquisition method, characterized in that, Applied to the acquisition system according to any one of claims 1-7, the method includes: Use an ultrasound probe, an IMU sensor, an electromagnetic tracking sensor, and an RGBD camera to collect ultrasound data; Perform coordinate system conversion on the initial IMU data collected by the IMU sensor, the initial electromagnetic tracking data collected by the electromagnetic tracking sensor, and the initial human body pose information collected by the RGBD camera to obtain the converted IMU data, the converted electromagnetic tracking data, and the converted human body pose information in a unified coordinate system; Perform timestamp alignment on the collected ultrasound data, and divide it into multiple first ultrasound data sets according to the timestamp carried by each ultrasound data; Perform data fusion on the converted IMU data and the converted electromagnetic tracking data belonging to the same first ultrasound data set to obtain the fused probe pose information, and generate a second ultrasound data set; Store the second ultrasound data set in the RLDS format, and each second ultrasound data set includes: the ultrasound image collected by the ultrasound probe, and the fused probe pose information and the converted human body pose information corresponding to the ultrasound image.

9. An electronic device, characterized in that, Includes a memory, a processor, and a computer program stored on the memory, and the processor executes the computer program to implement the ultrasound image and doctor's manipulation acquisition method according to claim 8.

10. A computer-readable storage medium, characterized in that, A computer program is stored thereon, and when the computer program is executed by a processor, it implements the ultrasound image and doctor's manipulation acquisition method according to claim 8.

Images