Ultrasonic probe control method, device, terminal equipment and storage medium

Abstract

The present application is applicable to the field of ultrasonic scanning technology, and provides an ultrasonic probe control method, device, terminal device and storage medium, wherein the method comprises: obtaining the real-time posture control instruction of the ultrasonic probe according to the ultrasonic image of the target object collected in real time by the ultrasonic probe, and gradually navigating the ultrasonic probe to the standard scanning section of the target object; controlling the ultrasonic probe to collect ultrasonic images of the target object, and obtaining the ultrasonic image of the standard scanning section. The present application can avoid the problems of uneven imaging quality, time-consuming and labor-intensive, and low flexibility brought by traditional methods.

Description

Technical Field

[0001] The present application belongs to the field of ultrasonic scanning technology, and in particular, relates to an ultrasonic probe control method, device, terminal equipment and storage medium. Background Art

[0002] Ultrasound examination is widely used in diagnostic and interventional applications in many medical fields. In conventional ultrasound examinations, clinical sonographers usually hold an ultrasound probe to scan the patient and generate reports for subsequent diagnosis or treatment of the disease. However, current clinical ultrasound examinations require professional sonographers to manually manipulate the ultrasound probe to move on the patient's skin surface based on anatomical knowledge and understanding of ultrasound images to find the standard scanning section of the target organ. Therefore, the conventional manual ultrasound scanning process is time-consuming and labor-intensive, has low flexibility, and the imaging quality is difficult to guarantee. Summary of the invention

[0003] The embodiments of the present application provide an ultrasound probe control method, apparatus, terminal device, and storage medium, which can solve the problems that the conventional manual ultrasound scanning process is time-consuming and labor-intensive, has low flexibility, and is difficult to ensure imaging quality.

[0004] In a first aspect, an embodiment of the present application provides an ultrasound probe control method, comprising:

[0005] According to the ultrasound image of the target object acquired in real time by the ultrasound probe, a real-time posture control instruction of the ultrasound probe is obtained, and the ultrasound probe is gradually navigated to a standard scanning section of the target object;

[0006] The ultrasonic probe is controlled to acquire ultrasonic images of the target object to obtain ultrasonic images of the standard scanning section.

[0007] Optionally, according to the ultrasound image of the target object acquired in real time by the ultrasound probe, a real-time posture control instruction of the ultrasound probe is obtained, and the ultrasound probe is gradually navigated to a standard scanning section of the target object, including:

[0008] Acquire an ultrasonic image obtained by acquiring an ultrasonic image of a target object by an ultrasonic probe at time n;

[0009] According to the ultrasonic image collected by the ultrasonic probe at time n, a posture control instruction of the ultrasonic probe at time n+1 is obtained;

[0010] Controlling the ultrasonic probe to collect an ultrasonic image of the target object at time n+1 according to the posture control instruction at time n+1, so as to obtain an ultrasonic image at time n+1;

[0011] The ultrasonic image at time n+1 is used as the ultrasonic image collected by the ultrasonic probe at time n, and the step of obtaining the posture control instruction of the ultrasonic probe at time n+1 according to the ultrasonic image collected by the ultrasonic probe at time n is returned to execute until the posture of the ultrasonic probe no longer changes.

[0012] Optionally, obtaining a posture control instruction of the ultrasonic probe at time n+1 according to the ultrasonic image collected by the ultrasonic probe at time n includes:

[0013] The ultrasonic image collected by the ultrasonic probe at time n is input into the ultrasonic probe control model to obtain the position and posture control instruction of the ultrasonic probe at time n+1.

[0014] Optionally, the training process of the ultrasound probe control model includes:

[0015] Acquire a standard posture corresponding to a standard scanning section of the target object and ultrasound image sample data at time t, where the ultrasound image sample data at time t includes the posture of the ultrasound probe for ultrasound image acquisition of the target object at time t and the acquired ultrasound image;

[0016] The ultrasonic image at time t is input into the ultrasonic probe control model to obtain the position and posture control instruction of the ultrasonic probe at time t+1, and the position and posture of the ultrasonic probe for ultrasonic image acquisition of the target object at time t+1 and the acquired ultrasonic image are used as the ultrasonic image sample data at time t+1;

[0017] Calculate the reward value of the ultrasonic probe control model according to the ultrasonic image sample data at time t, the ultrasonic image sample data at time t+1, and the standard posture;

[0018] Update the parameters of the ultrasonic probe control model, use the ultrasonic image at time t+1 as the ultrasonic image at time t, and return to execute the step of inputting the ultrasonic image at time t into the ultrasonic probe control model to obtain the position and posture control instruction of the ultrasonic probe at time t+1, until the position and posture of the ultrasonic probe no longer changes;

[0019] The ultrasound probe control model that performs stably on the training data is used as the trained model.

[0020] Optionally, the reward value of the ultrasonic probe control model is calculated according to the ultrasonic image sample data at time t, the ultrasonic image sample data at time t+1, and the standard posture, including:

[0021] Calculate the position difference and angle difference between the position and posture of the ultrasonic probe at time t and the standard position and posture, and the position difference and angle difference between the position and posture of the ultrasonic probe at time t+1 and the standard position and posture;

[0022] The improvement between the position difference at time t+1 and the position difference at time t is taken as the probe position reward, and the improvement between the angle difference at time t+1 and the angle difference at time t is taken as the probe orientation reward;

[0023] Calculating the ultrasound confidence of the ultrasound image at time t and the ultrasound confidence of the ultrasound image at time t+1;

[0024] The improvement between the ultrasound confidence at time t+1 and the ultrasound confidence at time t is taken as the image quality reward value;

[0025] The probe position reward value, the probe orientation reward value, and the image quality reward value are added together to obtain the reward value.

[0026] Optionally, after obtaining the real-time posture control instruction of the ultrasonic probe, the ultrasonic probe control method further includes:

[0027] Acquire body surface tracking data of the target object;

[0028] The real-time posture control instructions are adjusted according to the body surface tracking data to make the ultrasound probe contact with the body surface of the target object.

[0029] Optionally, after adjusting the real-time posture control instruction according to the body surface tracking data so that the ultrasound probe contacts the body surface of the target object, the ultrasound probe control method further includes:

[0030] A constant contact force is applied between the ultrasound probe and the target object's body surface.

[0031] In a second aspect, an embodiment of the present application provides an ultrasound probe control device, comprising:

[0032] An image processing and posture prediction module is used to obtain a real-time posture control instruction of the ultrasound probe based on an ultrasound image of the target object collected in real time by the ultrasound probe, and gradually navigate the ultrasound probe to a standard scanning section of the target object;

[0033] The probe acquisition standard image module is used to control the ultrasound probe to acquire ultrasound images of the target object to obtain the ultrasound image of the standard scanning section.

[0034] Optionally, the image processing and pose prediction module includes:

[0035] An image acquisition unit, used to acquire an ultrasonic image obtained by the ultrasonic probe acquiring an ultrasonic image of the target object at time n;

[0036] A posture acquisition unit, used to obtain a posture control instruction of the ultrasonic probe at time n+1 according to the ultrasonic image collected by the ultrasonic probe at time n;

[0037] A probe control unit, used to control the ultrasonic probe to collect an ultrasonic image of the target object at time n+1 according to the posture control instruction at time n+1, so as to obtain an ultrasonic image at time n+1;

[0038] The posture cycle unit is used to use the ultrasonic image at time n+1 as the ultrasonic image collected by the ultrasonic probe at time n, and return to execute the step of obtaining the posture control instruction of the ultrasonic probe at time n+1 according to the ultrasonic image collected by the ultrasonic probe at time n, until the posture of the ultrasonic probe no longer changes.

[0039] Optionally, the posture acquisition unit is used to input the ultrasonic image collected by the ultrasonic probe at time n into the ultrasonic probe control model to obtain the posture control instruction of the ultrasonic probe at time n+1.

[0040] Optionally, the ultrasound probe control device further includes:

[0041] A body surface tracking module, used to obtain body surface tracking data of a target object;

[0042] The posture adjustment module is used to adjust the real-time posture control instructions according to the body surface tracking data so that the ultrasound probe contacts the body surface of the target object.

[0043] Optionally, the ultrasound probe control device further includes:

[0044] The contact force applying module is used to apply a constant contact force between the ultrasound probe and the body surface of the target object.

[0045] In a third aspect, an embodiment of the present application provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the ultrasound probe control method as described above when executing the computer program.

[0046] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the ultrasound probe control method as described above is implemented.

[0047] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0048] In an embodiment of the present application, by obtaining real-time position and posture control instructions of the ultrasound probe based on the ultrasound image of the target object acquired in real time by the ultrasound probe, the ultrasound probe is gradually navigated to the standard scanning section of the target object and the ultrasound probe is controlled to acquire the ultrasound image of the target object, thereby obtaining the ultrasound image of the standard scanning section, thereby avoiding the problems of uneven imaging quality, time-consuming and labor-intensive, and low flexibility brought about by traditional methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the specific implementation methods of the present application or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0050] Figure 1 A schematic diagram of the structure of a robot automatic ultrasonic scanning system provided in an embodiment of the present application;

[0051] Figure 2 A schematic diagram of a flow chart of an ultrasonic probe control method provided in an embodiment of the present application;

[0052] Figure 3 A flowchart of a specific implementation method of step S10 provided in an embodiment of the present application;

[0053] Figure 4 A schematic diagram of a flow chart of a training process of an ultrasound probe control model provided in an embodiment of the present application;

[0054] Figure 5 A flowchart of a specific implementation method of step S43 provided in an embodiment of the present application;

[0055] Figure 6 An overall flow chart of the ultrasonic probe control method provided in the embodiment of the present application;

[0056] Figure 7 A schematic diagram of the structure of an ultrasound probe control device provided in an embodiment of the present application;

[0057] Figure 8 A schematic diagram of the structure of a terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0058] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0059] It should be noted that the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0060] Currently, the conventional manual ultrasound scanning process is time-consuming and labor-intensive, has low flexibility, and has difficulty in ensuring imaging quality.

[0061] In response to the above problems, an embodiment of the present application provides an ultrasound probe control method, which obtains real-time posture control instructions of the ultrasound probe based on the ultrasound image of the target object acquired in real time by the ultrasound probe, gradually navigates the ultrasound probe to the standard scanning section of the target object and controls the ultrasound probe to acquire ultrasound images of the target object, thereby obtaining an ultrasound image of the standard scanning section. This method can avoid the problems of uneven imaging quality, time-consuming and labor-intensive, and low flexibility brought about by traditional methods.

[0062] The ultrasonic probe control method provided in the present application is exemplarily described below in conjunction with specific embodiments.

[0063] Embodiment 1

[0064] The ultrasonic probe control method provided in the embodiment of the present application can be implemented as follows: Figure 1 The method is performed by the robot automatic ultrasonic scanning system shown in the figure, and the following embodiments take the method performed by the robot automatic ultrasonic scanning system as an example. Figure 1 As shown, as an example, the robotic automatic ultrasound scanning system may include a robotic arm 101, an ultrasound probe 102, an ultrasound machine 103 and a data processing and control computer 104, wherein the ultrasound probe 102 is rigidly connected to the actuator at the end of the robotic arm 101, the ultrasound machine 103 is connected to the ultrasound probe 102, and the data processing and control computer 104 is connected to the ultrasound machine 103.

[0065] like Figure 2 As shown, an embodiment of the present application provides an ultrasound probe control method, the method comprising the following steps:

[0066] S10. According to the ultrasound image of the target object acquired in real time by the ultrasound probe, a real-time posture control instruction of the ultrasound probe is obtained, and the ultrasound probe is gradually navigated to a standard scanning section of the target object.

[0067] In some embodiments of the present application, during the examination, the target object lies flat on the examination bed, and the mechanical arm is installed near the examination bed to control the movement of the ultrasound probe. The data processing and control computer is located near the examination bed, and it can send the ultrasound probe posture control instruction to the mechanical arm, so that the mechanical arm controls the ultrasound probe to move along the surface of the target object, and gradually navigates the ultrasound probe to the standard scanning section of the target object.

[0068] S20, controlling the ultrasonic probe to collect ultrasonic images of the target object to obtain ultrasonic images of standard scanning sections.

[0069] In some embodiments of the present application, after the ultrasound probe gradually navigates to the standard scanning section of the target object, the data processing and control computer can control the ultrasound probe to collect ultrasound images of the target object and generate ultrasound images on the ultrasound machine. The ultrasound image at this time is the ultrasound image corresponding to the standard scanning section of the target object, and the ultrasound image corresponding to the standard scanning section is the ultrasound image with the highest imaging quality of the target object.

[0070] The ultrasound probe control method provided in the embodiment of the present application obtains the real-time posture control instruction of the ultrasound probe based on the ultrasound image of the target object acquired in real time by the ultrasound probe, gradually navigates the ultrasound probe to the standard scanning section of the target object and controls the ultrasound probe to acquire the ultrasound image of the target object according to the target posture, thereby obtaining the ultrasound image of the standard scanning section, which can avoid the problems of uneven imaging quality, time-consuming and labor-intensive, and low flexibility brought about by traditional methods.

[0071] The specific implementation of the above step S10 is exemplarily described below with reference to the accompanying drawings.

[0072] Embodiment 2

[0073] This embodiment is a further explanation of the first embodiment. For details of the same or similar parts as the first embodiment, please refer to the relevant description of the first embodiment, which will not be repeated here.

[0074] In some embodiments of the present application, Figure 3 As shown, the above step S10, according to the ultrasound image of the target object collected in real time by the ultrasound probe, obtains the real-time posture control instruction of the ultrasound probe, and gradually navigates the ultrasound probe to the standard scanning section of the target object, includes the following steps:

[0075] S11, obtaining an ultrasonic image obtained by acquiring an ultrasonic image of a target object by an ultrasonic probe at time n.

[0076] In some embodiments of the present application, the nth moment may be the nth moment after the robot automatic ultrasonic scanning system starts to collect ultrasonic images of the target object, or may be the nth moment and the n-1 moments before the nth moment. If it is the nth moment and the n-1 moments before the nth moment, the collected ultrasonic images are an image sequence. After the ultrasonic probe collects ultrasonic images of the target object at the nth moment, the ultrasonic machine generates the ultrasonic image, and the ultrasonic machine transmits the ultrasonic image to the data processing and control computer in real time.

[0077] S12. Obtaining a position control instruction of the ultrasonic probe at time n+1 according to the ultrasonic image collected by the ultrasonic probe at time n.

[0078] In some embodiments of the present application, time n+1 is the next time after time n, and the data processing and control computer stores an ultrasound image analysis algorithm and a robotic arm control algorithm, which can calculate the position and posture of the ultrasound probe at time n+1 based on the ultrasound image at time n, and send control instructions for the position and posture to the robotic arm.

[0079] S13, controlling the ultrasonic probe to collect ultrasonic images of the target object at time n+1 according to the posture control instruction at time n+1, to obtain the ultrasonic image at time n+1.

[0080] S14. Use the ultrasonic image at time n+1 as the ultrasonic image collected by the ultrasonic probe at time n, and return to the step of obtaining the posture control instruction of the ultrasonic probe at time n+1 according to the ultrasonic image collected by the ultrasonic probe at time n, until the posture of the ultrasonic probe no longer changes.

[0081] In some embodiments of the present application, the ultrasound image at time n+1 is used as the ultrasound image collected by the ultrasound probe at time n, and the process returns to step S12 to enter a loop until the position of the ultrasound probe no longer changes. It is then considered that the ultrasound probe has navigated to the standard scanning section of the target object, and navigation is terminated.

[0082] The ultrasound probe control method provided in the embodiment of the present application guides the robotic automatic ultrasound scanning system to autonomously control the ultrasound probe posture in real time through automatic analysis and understanding of ultrasound images based on artificial intelligence, so as to find the standard scanning section of the target object, thereby avoiding the problems of uneven imaging quality, time-consuming and labor-intensive, and low flexibility brought about by traditional methods.

[0083] In some embodiments of the present application, the above step S12, obtaining the position and posture control instruction of the ultrasonic probe at time n+1 according to the ultrasonic image collected by the ultrasonic probe at time n, includes the following steps:

[0084] The ultrasonic image collected by the ultrasonic probe at time n is input into the ultrasonic probe control model to obtain the position and posture control instruction of the ultrasonic probe at time n+1.

[0085] In some embodiments of the present application, the above-mentioned ultrasound probe control model is constructed based on a deep Q network and stored in a data processing and control computer. After the ultrasound image collected by the ultrasound probe at time n is input into the ultrasound probe control model, the Q values ​​of 12 actions corresponding to the 6-DOF posture of the ultrasound probe can be obtained, and the action with the largest Q value is output as the posture control instruction of the ultrasound probe at time n+1.

[0086] The following is an exemplary description of the training process of the ultrasound probe control model with reference to the accompanying drawings.

[0087] In some embodiments of the present application, in order to train the parameters of the deep Q network, a deep reinforcement learning agent can be constructed, whose state is defined as the 6-DOF position and ultrasound image of the ultrasound probe, and whose action is defined as 12 actions corresponding to the 6-DOF position of the ultrasound probe (translating a unit distance d in the positive and negative directions of the x / y / z axis of the probe coordinate system, respectively). step , rotate along the positive and negative directions of the x / y / z axis by a unit angle θ step ).like Figure 4 As shown in FIG. 1 , the training process of the ultrasound probe control model includes the following steps:

[0088] S41. Obtain a standard posture corresponding to a standard scanning section of the target object and ultrasound image sample data at time t. The ultrasound image sample data at time t includes the posture of the ultrasound probe for acquiring ultrasound images of the target object at time t and the acquired ultrasound image.

[0089] In some embodiments of the present application, the standard posture corresponding to the standard scanning section of the target object can be marked by an expert physician, and the ultrasound image sample data at time t includes the posture of the ultrasound probe for acquiring an ultrasound image of the target object at time t, and the ultrasound image obtained by the ultrasound probe for acquiring the ultrasound image according to the posture at time t.

[0090] S42. Input the ultrasound image at time t into the ultrasound probe control model to obtain the position and posture control instruction of the ultrasound probe at time t+1, and use the position and posture of the ultrasound probe for ultrasound image acquisition of the target object at time t+1 and the acquired ultrasound image as the ultrasound image sample data at time t+1.

[0091] In some embodiments of the present application, the ultrasound image sample data at time t+1 includes the posture of the ultrasound probe for acquiring an ultrasound image of the target object at time t+1, and the ultrasound image obtained by the ultrasound probe for acquiring the ultrasound image according to the posture at time t+1.

[0092] S43. Calculate the reward value of the ultrasonic probe control model according to the ultrasonic image sample data at time t, the ultrasonic image sample data at time t+1, and the standard posture.

[0093] S44. Update the parameters of the ultrasound probe control model, use the ultrasound image at time t+1 as the ultrasound image at time t, and return to execute the step of inputting the ultrasound image at time t into the ultrasound probe control model to obtain the posture control instruction of the ultrasound probe at time t+1, until the posture of the ultrasound probe no longer changes.

[0094] S45. The ultrasound probe control model that performs stably on the training data is used as the trained model.

[0095] In some embodiments of the present application, the ultrasound probe control model that performs stably on the training data may be a model when the error between the final posture of the ultrasound probe and the standard posture is less than a preset value.

[0096] The specific implementation of the above step S43 is exemplarily described below with reference to the accompanying drawings.

[0097] In some embodiments of the present application, Figure 5 As shown, the above step S43, according to the ultrasonic image sample data at time t, the ultrasonic image sample data at time t+1 and the standard posture, calculates the reward value of the ultrasonic probe control model and updates the parameters of the ultrasonic probe control model, including the following steps:

[0098] S431, calculating the position difference and angle difference between the position and posture of the ultrasonic probe at time t and the standard position and posture, and the position difference and angle difference between the position and posture of the ultrasonic probe at time t+1 and the standard position and posture.

[0099] In some embodiments of the present application, it is assumed that the position and posture of the ultrasound probe at time t is expressed as (p t ,q t ), the pose corresponding to the standard scanning section is expressed as (p g ,q g ), then the position difference d between the position of the ultrasound probe at time t and the position corresponding to the standard scanning section t and the angle difference θ t The calculation formulas are:

[0100] d t =||p t -p g ||2,θ t =2 arccos(| t ,q g >|)

[0101] S432. The improvement between the position difference at time t+1 and the position difference at time t is taken as the probe position reward value, and the improvement between the angle difference at time t+1 and the angle difference at time t is taken as the probe orientation reward value.

[0102] In some embodiments of the present application, the probe position reward value Δd t and the probe heading reward Δθ t The calculation formulas are:

[0103]

[0104] S433: Calculate the ultrasound confidence of the ultrasound image at time t and the ultrasound confidence of the ultrasound image at time t+1.​

[0105] In some embodiments of the present application, the ultrasound confidence level is used to measure the clarity of the ultrasound image. The ultrasound confidence level c at time t is t The calculation formula is:

[0106]

[0107] Among them, C t is the ultrasound confidence map calculated from the ultrasound image at time t. The value of each image (i, j) represents the imaging quality of the corresponding pixel (i, j) in the ultrasound image, and S is the region of interest in the ultrasound image.

[0108] S434: The improvement between the ultrasound confidence level at time t+1 and the ultrasound confidence level at time t is taken as the image quality reward value.

[0109] In some embodiments of the present application, the image quality reward value Δc t The calculation formula is: Δc t =c t+1 -c t .

[0110] S435 , adding the probe position reward value, the probe orientation reward value, and the image quality reward value to obtain a reward value.

[0111] In some embodiments of the present application, the calculation formula of the reward value r is: r = Δd t +Δθ t +Δc t .

[0112] The ultrasound probe control method provided in the embodiment of the present application can train the parameters of the deep Q network through the deep Q-learning algorithm by modeling the state, action and reward value, so that the deep neural network can learn the image understanding and organ anatomical structure knowledge in the clinical ultrasound scanning process, thereby autonomously guiding the ultrasound probe to navigate to the standard scanning section of the target object.

[0113] In some embodiments of the present application, after obtaining the real-time posture control instruction of the ultrasound probe, the ultrasound probe control method further includes the following steps:

[0114] Obtain the target object's body surface tracking data.

[0115] In some embodiments of the present application, Figure 1 As shown, the robot automatic ultrasound scanning system may further include an RGB-D camera 105, which is installed above the examination bed. The above-mentioned acquisition of the body surface tracking data of the target object may be acquisition of the body surface tracking data of the target object by using the RGB-D camera.

[0116] The real-time posture control instructions are adjusted according to the body surface tracking data to make the ultrasound probe contact with the body surface of the target object.

[0117] In some embodiments of the present application, an RGB-D image processing algorithm is also stored in the data processing and control computer, and the above-mentioned real-time posture control instruction adjusted according to the body surface tracking data may be an instruction for the data processing and control computer to adjust the real-time posture control instruction according to the body surface tracking data.

[0118] In some embodiments of the present application, after adjusting the real-time posture control instruction according to the body surface tracking data so that the ultrasound probe contacts the body surface of the target object, the ultrasound probe control method further includes:

[0119] A constant contact force is applied between the ultrasound probe and the target object's body surface.

[0120] In some embodiments of the present application, by applying a constant contact force between the ultrasound probe and the body surface of the target object, good contact and ultrasonic coupling between the ultrasound probe and the body surface of the target object can be ensured.

[0121] The ultrasound probe control method provided in the embodiment of the present application uses an RGB-D camera to observe the surface of the target object to obtain surface tracking data, and adjusts real-time posture control instructions to track the surface of the target object. This can better maintain the contact between the ultrasound probe and the surface of the target object and improve the quality of ultrasound images.

[0122] like Figure 6 As shown, the overall process of the ultrasound probe control method provided by the embodiment of the present application is as follows: during inspection, the robotic arm controls the ultrasound probe to move along the surface of the target object, the ultrasound probe collects ultrasound images of the target object, the ultrasound machine generates ultrasound images and inputs the ultrasound probe control model in the data processing and control computer, the data processing and control computer outputs the real-time posture control instructions of the ultrasound probe to the robotic arm according to the ultrasound images, the RGB-D camera obtains the surface tracking data of the target object, the data processing and control computer adjusts the real-time posture control instructions according to the surface tracking data, so that the ultrasound probe contacts the surface of the target object, and applies a constant contact force between the ultrasound probe and the surface of the target object. Repeat the above steps to gradually navigate the ultrasound probe to the standard scanning section of the target object.

[0123] The ultrasonic probe control device provided in the present application is exemplarily described below with reference to the accompanying drawings.

[0124] Embodiment 3

[0125] Corresponding to the ultrasonic probe control method described in the above embodiment, Figure 7As shown, an embodiment of the present application provides an ultrasound probe control device, and the ultrasound probe control device 700 includes:

[0126] The image processing and posture prediction module 701 is used to obtain the real-time posture control instruction of the ultrasound probe according to the ultrasound image of the target object collected in real time by the ultrasound probe, and gradually navigate the ultrasound probe to the standard scanning section of the target object;

[0127] The probe acquisition standard image module 702 is used to control the ultrasound probe to acquire ultrasound images of the target object to obtain the ultrasound image of the standard scanning section.

[0128] Optionally, the image processing and posture prediction module 701 includes:

[0129] An image acquisition unit, used to acquire an ultrasonic image obtained by the ultrasonic probe acquiring an ultrasonic image of the target object at time n;

[0130] A posture acquisition unit, used to obtain a posture control instruction of the ultrasonic probe at time n+1 according to the ultrasonic image collected by the ultrasonic probe at time n;

[0131] A probe control unit, used to control the ultrasonic probe to collect an ultrasonic image of the target object at time n+1 according to the posture control instruction at time n+1, so as to obtain an ultrasonic image at time n+1;

[0132] The posture cycle unit is used to use the ultrasonic image at time n+1 as the ultrasonic image collected by the ultrasonic probe at time n, and return to execute the step of obtaining the posture control instruction of the ultrasonic probe at time n+1 according to the ultrasonic image collected by the ultrasonic probe at time n, until the posture of the ultrasonic probe no longer changes.

[0133] Optionally, the posture acquisition unit is used to input the ultrasonic image collected by the ultrasonic probe at time n into the ultrasonic probe control model to obtain the posture control instruction of the ultrasonic probe at time n+1.

[0134] Optionally, the ultrasound probe control device 700 further includes:

[0135] A body surface tracking module, used to obtain body surface tracking data of a target object;

[0136] The posture adjustment module is used to adjust the real-time posture control instructions according to the body surface tracking data so that the ultrasound probe contacts the body surface of the target object.

[0137] Optionally, the ultrasound probe control device 700 further includes:

[0138] The contact force applying module is used to apply a constant contact force between the ultrasound probe and the body surface of the target object.

[0139] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0140] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0141] The present application embodiment also provides a terminal device 800, such as Figure 8 As shown, it includes a memory 801, a processor 802, and a computer program 803 stored in the memory 801 and executable on the processor 802. When the processor 802 executes the computer program 803, the ultrasound probe control method described above is implemented.

[0142] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the ultrasound probe control method as described above is implemented.

[0143] It should be noted that a computer program includes computer program code, which may be in source code form, object code form, executable file or some intermediate form, etc. Computer-readable storage media may include: any entity or device capable of carrying computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc.

[0144] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0145] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for controlling an ultrasonic probe, characterized in that: include: According to the ultrasonic image of the target object collected in real time by the ultrasonic probe, a real-time posture control instruction of the ultrasonic probe is obtained, and the ultrasonic probe is gradually navigated to the standard scanning section of the target object, including: inputting the ultrasonic image collected by the ultrasonic probe into an ultrasonic probe control model, and the training process of the ultrasonic probe control model includes: obtaining a standard posture corresponding to the standard scanning section of the target object, and ultrasonic image sample data at time t, wherein the ultrasonic image sample data at time t includes the posture of the ultrasonic probe for collecting ultrasonic images of the target object at time t and the collected ultrasonic image; The ultrasonic image at time t is input into the ultrasonic probe control model to obtain the position and posture control instruction of the ultrasonic probe at time t+1, and the position and posture of the ultrasonic probe for ultrasonic image acquisition of the target object at time t+1 and the acquired ultrasonic image are used as the ultrasonic image sample data at time t+1; According to the ultrasonic image sample data at time t, the ultrasonic image sample data at time t+1 and the standard posture, the reward value of the ultrasonic probe control model is calculated, including: calculating the position difference and angle difference between the posture of the ultrasonic probe at time t and the standard posture, and the position difference and angle difference between the posture of the ultrasonic probe at time t+1 and the standard posture; taking the improvement between the position difference at time t+1 and the position difference at time t as the probe position reward value, and taking the improvement between the angle difference at time t+1 and the angle difference at time t as the probe orientation reward value; calculating the ultrasonic confidence of the ultrasonic image at time t and the ultrasonic confidence of the ultrasonic image at time t+1; taking the improvement between the ultrasonic confidence at time t+1 and the ultrasonic confidence at time t as the image quality reward value; adding the probe position reward value, the probe orientation reward value and the image quality reward value as the reward value; updating the parameters of the ultrasonic probe control model, taking the ultrasonic image at time t+1 as the ultrasonic image at time t, and returning to the step of inputting the ultrasonic image at time t into the ultrasonic probe control model to obtain the position and posture control instruction of the ultrasonic probe at time t+1, until the position and posture of the ultrasonic probe no longer changes; The ultrasound probe control model that is stable on the training data is used as a trained model; The ultrasonic probe is controlled to acquire an ultrasonic image of the target object to obtain an ultrasonic image of the standard scanning section.

2. The ultrasonic probe control method according to claim 1, characterized in that: The method of acquiring a real-time posture control instruction of the ultrasound probe based on an ultrasound image of the target object acquired in real time by the ultrasound probe, and gradually navigating the ultrasound probe to a standard scanning section of the target object includes: Acquire an ultrasonic image obtained by acquiring an ultrasonic image of the target object by the ultrasonic probe at time n; Obtaining a position and posture control instruction of the ultrasonic probe at time n+1 according to the ultrasonic image collected by the ultrasonic probe at time n; Controlling the ultrasonic probe to collect an ultrasonic image of the target object at time n+1 according to the posture control instruction at time n+1, so as to obtain an ultrasonic image at time n+1; The ultrasonic image at time n+1 is used as the ultrasonic image collected by the ultrasonic probe at time n, and the step of obtaining the posture control instruction of the ultrasonic probe at time n+1 based on the ultrasonic image collected by the ultrasonic probe at time n is returned to execute until the posture of the ultrasonic probe no longer changes.

3. The ultrasonic probe control method according to claim 2, characterized in that: The step of obtaining a position and posture control instruction of the ultrasonic probe at time n+1 according to the ultrasonic image collected by the ultrasonic probe at time n includes: The ultrasonic image collected by the ultrasonic probe at time n is input into the ultrasonic probe control model to obtain the position and posture control instruction of the ultrasonic probe at time n+1.

4. The ultrasonic probe control method according to claim 1, characterized in that: After acquiring the real-time posture control instruction of the ultrasonic probe, the ultrasonic probe control method further includes: Acquiring body surface tracking data of the target object; The real-time posture control instruction is adjusted according to the body surface tracking data so that the ultrasound probe contacts the body surface of the target object.

5. The ultrasonic probe control method according to claim 4, characterized in that: After adjusting the real-time posture control instruction according to the body surface tracking data so that the ultrasound probe contacts the body surface of the target object, the ultrasound probe control method further includes: A constant contact force is applied between the ultrasound probe and the body surface of the target object.

6. An ultrasonic probe control device, characterized in that: include: An image processing and posture prediction module is used to obtain a real-time posture control instruction of the ultrasound probe based on an ultrasound image of the target object collected in real time by the ultrasound probe, and gradually navigate the ultrasound probe to a standard scanning section of the target object; The image processing and posture prediction module includes: a posture acquisition unit, which is used to input the ultrasound image collected by the ultrasound probe into the ultrasound probe control model. The training process of the ultrasound probe control model includes: Acquire a standard posture corresponding to a standard scanning section of the target object and ultrasound image sample data at time t, where the ultrasound image sample data at time t includes the posture of the ultrasound probe for ultrasound image acquisition of the target object at time t and the acquired ultrasound image; The ultrasonic image at time t is input into the ultrasonic probe control model to obtain the position and posture control instruction of the ultrasonic probe at time t+1, and the position and posture of the ultrasonic probe for ultrasonic image acquisition of the target object at time t+1 and the acquired ultrasonic image are used as the ultrasonic image sample data at time t+1; According to the ultrasonic image sample data at time t, the ultrasonic image sample data at time t+1 and the standard posture, the reward value of the ultrasonic probe control model is calculated, including: calculating the position difference and angle difference between the posture of the ultrasonic probe at time t and the standard posture, and the position difference and angle difference between the posture of the ultrasonic probe at time t+1 and the standard posture; taking the improvement between the position difference at time t+1 and the position difference at time t as the probe position reward value, and taking the improvement between the angle difference at time t+1 and the angle difference at time t as the probe orientation reward value; calculating the ultrasonic confidence of the ultrasonic image at time t and the ultrasonic confidence of the ultrasonic image at time t+1; taking the improvement between the ultrasonic confidence at time t+1 and the ultrasonic confidence at time t as the image quality reward value; adding the probe position reward value, the probe orientation reward value and the image quality reward value as the reward value; updating the parameters of the ultrasonic probe control model, taking the ultrasonic image at time t+1 as the ultrasonic image at time t, and returning to the step of inputting the ultrasonic image at time t into the ultrasonic probe control model to obtain the position and posture control instruction of the ultrasonic probe at time t+1, until the position and posture of the ultrasonic probe no longer changes; The ultrasound probe control model that is stable on the training data is used as a trained model; The probe acquisition standard image module is used to control the ultrasound probe to acquire ultrasound images of the target object to obtain the ultrasound image of the standard scanning section.

7. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the ultrasound probe control method according to any one of claims 1 to 5 is implemented.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the ultrasound probe control method according to any one of claims 1 to 5 is implemented.

Images