Method for generating operation of medical assistance robot, and medical assistance robot system

Abstract

The present invention provides: a method for generating an operation of a medical assistance robot that is capable of generating, in accordance with physical differences among individuals and the quality of an acquired image, a movement route of a robot arm for autonomously holding an ultrasonic probe; and a medical assistance robot system. Provided is a method for generating an operation of a medical assistance robot, said method comprising: a step (S220) for acquiring input data for a trained operation generation model, said input data including arm position information that indicates the position of a robot arm, force sensor information that indicates the force with which an ultrasonic probe makes contact with a subject, and an image feature amount of the ultrasonic image of the subject that is obtained by a scanning operation of the ultrasonic probe; and a step (S230) for using the acquired input data as input data for the trained operation generation model and causing the trained operation generation model to output next step data, said next step data including the arm position information, the force sensor information, and the image feature amount of the next step.

Description

Method for generating motion of medical support robot and medical support robot system

[0001] The present invention relates to a medical support robot system.

[0002] Developed countries are facing a serious shortage of obstetricians and gynecologists. The average age of obstetricians and gynecologists is also rising, meaning that there are very few young doctors willing to become obstetricians and gynecologists. This situation could lead to diagnostic errors, making it urgent to improve prenatal care. Meanwhile, research and development in the field of robotics has made remarkable progress, and robotic devices are being used in the medical field. For example, Patent Document 1 proposes a technology for obtaining images of consistent quality by having a robotic arm hold an ultrasound probe and have the robotic device perform ultrasound scans.

[0003] Japanese Patent Application Laid-Open No. 2022-52345

[0004] However, there are individual differences in the shape and characteristics of each patient's body, and experts determine how to move the ultrasound probe by taking these individual differences into consideration. The present invention aims to provide a motion generation method for a medical support robot and a medical support robot system that can autonomously generate a movement path for a robot arm that holds an ultrasound probe, in accordance with individual differences in the body and the quality of the acquired images.

[0005] In order to solve the above-mentioned problem, one aspect of the present invention is a method for generating movements of a medical support robot, the method including: a step of acquiring input data for a learned movement generation model, the input data including arm position information indicating a position of a robot arm of the medical support robot, force sensor information indicating a contact force of an ultrasound probe operated by the medical support robot on a subject, and image features of an ultrasound image of the subject obtained by a scanning operation of the ultrasound probe; and a step of using the acquired input data as input data for the learned movement generation model, and the learned movement generation model outputting next step data, the next step data including arm position information, force sensor information, and image features of the next step.

[0006] Another aspect of the present invention is the above method, further comprising the step of outputting the arm position information for the next step output by the learned motion generation model to the medical support robot.

[0007] Another aspect of the present invention is the above method, in which the image features are output by the trained image feature extraction model using an ultrasound image of the subject as input data to the trained image feature extraction model.

[0008] Another aspect of the present invention is the above-mentioned method, wherein the learned motion generation model is a model learned using training data including motion parameters of the robot arm, shape information of the subject, and an ultrasound image of the subject obtained by a scanning operation of the ultrasound probe performed based on the motion parameters and the shape information of the subject.

[0009] Another aspect of the present invention is the above-mentioned method, wherein the learned motion generation model is a Multiple Time-scale Recurrent Neural Network (MTRNN).

[0010] Another aspect of the present invention is the above-mentioned method, wherein the trained motion generation model is a model trained using an error function in which shape information of the subject is mapped to a latent space and is weighted when the robot arm is correctly scanning the subject with the ultrasound probe and ignored when the robot arm is not correctly scanning the subject. Another aspect of the present invention is the above-mentioned method, wherein the trained image feature extraction model is a convolutional autoencoder. Another aspect of the present invention is a computer device comprising a processor and a memory, which executes the above-mentioned method. Another aspect of the present invention is a computer program for causing a computer device comprising a processor and a memory to execute the above-mentioned method.

[0011] FIG. 1 is a diagram showing an example of the configuration of a medical support robot system according to an embodiment of the present invention. FIG. 2 is a diagram showing an outline of a DNN model used in the medical support robot system according to an embodiment of the present invention. FIG. 3 is a diagram showing an example of parameters used to construct CAE. ｓ Node, C ｆ 13 is a diagram showing the time constants used in nodes and input / output (IO) nodes and the number of nodes. FIG. 14 is a diagram showing a phantom constructed in a test of a medical support robot system according to an embodiment of the present invention. FIG. 15 is a diagram explaining a plurality of phantoms constructed in a test of a medical support robot system according to an embodiment of the present invention. FIG. 16 is a diagram showing the number of dots erased by the scanning operation of an ultrasound probe in a medical support robot system according to an embodiment of the present invention. FIG. 17 is a diagram showing an overview of a scanning operation generated using a model according to an embodiment of the present invention, an original image, and a CAE-reconstructed ultrasound image. FIG. 18 is a diagram showing the state of a robot arm when an incorrect posture of the robot arm is generated in a test of a medical support robot system according to an embodiment of the present invention. FIG. 19 is a diagram showing an example of functional blocks of a computer device. FIG. 19 is a diagram showing an example of a process for training a DNN model of a medical support robot system according to an embodiment of the present invention. FIG. 19 is a diagram showing an example of details of the process in step S110 of FIG. 11. FIG. 19 is a diagram showing an example of a process for verifying a trained DNN model of a medical support robot system according to an embodiment of the present invention. FIG. 19 is a diagram showing an example of details of the process in step S220 of FIG. 13.

[0012] Embodiments of the present invention will be described in detail below with reference to the drawings. A medical support robot system according to this embodiment is a medical support robot system that performs fetal ultrasound examinations and enables autonomous scanning with an ultrasound probe while adapting to the shape of the patient's (subject's) abdomen. The system uses a deep neural network (DNN). The use of DNNs for generating the motion of medical support robot systems has attracted increasing attention in robotics research. By using a DNN, the medical support robot system of this embodiment is able to perform ultrasound examinations by mimicking the actions of an experienced doctor, taking into account individual differences in the patient's body. The medical support robot system of this embodiment is controlled using a DNN model including a multiple time-scale recurrent neural network (MTRNN). The DNN model performs appropriate scanning operations based on the characteristics of ultrasound images obtained when scanning the abdomen with an ultrasound probe, as well as information on the position and contact force of the ultrasound probe.

[0013] (Configuration of Medical Support Robot System) FIG. 1 is a diagram showing an example of the configuration of a medical support robot system according to this embodiment. The medical support robot system 1 shown in FIG. 1 is a system actually constructed by the inventors of the present application. The medical support robot system 1 according to this embodiment includes a collaborative robot arm 10 (UR5e, Universal Robot, Denmark) and a passively actuated end effector 20 that holds an ultrasound probe 30 (Vscan Air, GE Healthcare, USA). The collaborative robot arm 10 is configured to be movable around a patient's abdomen 5, which is the subject of an ultrasound examination ( FIG. 1 is a diagram for explanatory purposes, and the patient's abdomen 5 is substituted here with a phantom that resembles an abdomen). Ultrasound image data acquired by contacting the ultrasound probe 30 with the patient's abdomen 5 is transmitted to a computer device 60.

[0014] The collaborative robot arm 10 is connected to the end effector 20 via a 6-axis force / torque sensor (Axia80, ATI Industrial Automation, USA) (not shown) to measure the contact force at the end effector 20 .

[0015] The computer device 60 is connected to each of the robot arm 10 and the end effector 20 of the medical support robot system 1 and can exchange data with these components. For example, the computer device 60 can send control signals for controlling each component. The computer device 60 can also receive data acquired or generated by each component from each component. The computer device 60 can be realized with a hardware configuration similar to that of a general computer device. The computer device 60 can include, for example, a processor, random access memory (RAM), read-only memory (ROM), an internal hard disk drive, removable memory such as an external hard disk drive, CD, DVD, USB memory, memory stick, or SD card, an input / output user interface (display, keyboard, mouse, touch panel, speaker, microphone, LED (light emitting diode), etc.), and a wired / wireless communication interface capable of communicating with each component of the medical support robot system 1 and other computer devices.

[0016] The computer device 60 may have a processor that reads a computer program for causing the computer device 60 to control the robot device, the computer program having been stored in advance in a hard disk drive, ROM, or removable memory, into a memory such as RAM, and then executes the program while reading necessary data from the hard disk drive, ROM, or removable memory as needed. This operation of the computer device 60 may realize various processes in the medical support robot system 1 described in this embodiment. Furthermore, various data used in each process described in this embodiment is stored in a storage device or storage medium such as a hard disk drive, RAM, or removable memory, and is read and used by the processor as needed during processing. Furthermore, the computer program for realizing various processes in the medical support robot system 1 according to this embodiment may be stored in part or in whole in a computer-readable medium or downloaded via a wired or wireless communication network. The configuration shown in FIG. 1 is merely an example, and the configuration of the medical support robot system 1 according to this embodiment is not limited thereto.

[0017] (Learning Model) Fig. 2 is a diagram showing an outline of the DNN model used in this embodiment. The DNN model in this embodiment is composed of a convolutional autoencoder (CAE) and an MTRNN. In this embodiment, abdominal shape information indicating the shape of the patient's abdomen is mapped to a latent space.

[0018] (A) Convolutional Autoencoder (CAE) The DNN model (hereinafter sometimes simply referred to as "model") according to this embodiment uses CAE to compress images into low-dimensional features so that the MTRNN can learn the data in a well-balanced way. The MTRNN can obtain the output reconstruction data y by minimizing the error E as follows: Ｉｍｇ (t) is the input data x Ｉｍｇ (t) was trained to be identical to (t).

[0019]

[0020] Figure 3 shows the parameters used to construct the CAE. Image features yImgFeature(t) were extracted using 20 nodes from the hidden layer. Convolution was performed with a kernel size of 6 × 6, a sliding width of 2, and zero padding of 1. The module was trained for 1000 epochs.

[0021] (B) MTRNN We used MTRNN as the main module to integrate the position and contact force of the ultrasound probe, recognize the shape of the abdomen, and generate motion. MTRNN is a recurrent neural network that predicts the next step from current and past inputs. Its unique features are multiple nodes with different time constants and a slow context. ｓ Node and Fast Context C ｆ 4 shows the C ｓ Node, C ｆ 1 shows the time constants and number of nodes used in the nodes and input / output (IO) nodes. ｓ Since the node has a large time constant, it learns continuous information and behaves as a latent space. Therefore, the abdominal shape information of each patient is the initial state of the node (C ｓ (0)). Also, the normalized abdomen height and width are passed to C, as shown below. ｓ (0) was entered.

[0022] The forward calculation is described as follows: First, the internal value U of neuron i at step t is ｉ is calculated as follows:

[0023]

[0024] where N is the number of neurons, τ i is the time constant of neuron i, w iｊ is the weight value from neuron j to neuron i, x ｊ (t) is the input value from neuron j to neuron i. The output value of neuron i is calculated as follows:

[0025]

[0026] Using this output value, the module can generate the robot's movement by controlling the robot's (robot arm's) movement according to the robot arm's position. ｉ Using the value of (t) as the next input value, x ｉ (t+1) is expressed as follows:

[0027]

[0028] Here, T(t) represents input data i, which is either training data (teacher data) used in training the model or real-time data used in evaluation testing. ｉ (t) is the previous step y ｊ The feedback rate is adjusted by multiplying the output value of (t-1) and the data T(t) by a feedback rate α (0≦α≦1). When the feedback rate is high, the model can stably predict the next step, while when the feedback rate is low, the model can easily adapt to real-time situations. In this embodiment, the feedback rate is set to 0.9 during model training and 0.28 during evaluation.

[0029] To achieve backward calculation during training, a backward propagation through time algorithm was used to minimize the training error. The inventors formulated an error function by considering the number of white pixels P(t) in an ultrasound image. That is, P(t) is high when the ultrasound probe 30 is in contact with the abdomen 5 and is low when the probe 30 is away from the abdomen 5. The error function allows the robot (robot arm) to recognize deviations from the correct position and return to the correct position. The model can be trained to focus on the error function E when the probe 30 can efficiently scan the abdomen 5 and to ignore the error function E when the probe 30 is away from the abdomen 5.

[0030] The weights are updated as follows:

[0031] Here, η (=0.001) is the learning rate, and n (=5000) is the number of epochs.

[0032] (Tests) Tests conducted by the inventors on the medical support robot system according to this embodiment will be described below.

[0033] (A) Hardware Design and Data Sampling In this test, the medical support robot system illustrated in Figure 1 was used.

[0034] (B) Distribution of Objects and Data Used. To collect training data and validate the model, tissue-mimicking ultrasound phantoms were prepared. The inventors mixed agar and gelatin to construct a phantom capable of reproducing the abdominal shape and acoustic characteristics for ultrasound imaging. Figure 5 shows the constructed phantom. The agar concentration in water was 1.17%, and the gelatin concentration was 3.30%. Phantom 5 had an approximate spherical truncation shape with a hemisphere partially cut away by a plane (right diagram in Figure 5). As shown in Figure 6, the inventors created phantoms 5 with several different diameters and heights to obtain training and validation data. Note that the units of "diameter" and "height" in Figure 6 are millimeters (mm). Over each phantom used for training, the ultrasound probe 30 held by the robot arm 10 was moved back and forth along the direction shown in the right diagram in Figure 5. When acquiring the training data, the robot arm 10 was manually manipulated so that the ultrasound probe 30 was approximately perpendicular to the phantom surface. During the scan, the robot arm position (6 axes), two force / torque sensor data (6 axes x 2), and ultrasound images (128 width x 128 height x 1 channel) were recorded as training data. The sampling frequency of the robot arm position and ultrasound images was 1 Hz, and the sampling frequency of the force / torque sensor was 15 Hz. The scan was repeated five times for each of the five training phantoms, resulting in a total of 25 data sets. The time step size ranged from 61 to 97 steps.

[0035] (C) Evaluation Test As shown in FIG. 5, in order to evaluate the performance of the ultrasound scanning model of this embodiment using the evaluation criteria, seven target dot marks were painted at 2 cm (centimeter) intervals along the center line of the phantom 5. The painted dots were wiped away and erased when the robot scanned each dot with the ultrasound probe 30. Using the trained model, ultrasound scans were performed three times on four phantoms that were not used for model training. Then, the number of dots erased by the robot wiping (scanning) with the ultrasound probe 30 was evaluated. Furthermore, two models different from the trained model were used to compare and verify the trained model. These two models are a learning model (a) that does not use abdominal shape information (C in Equation (2)), ｓ (0) = [0,0]) and a learning model (b) that does not use an error function that includes the number of white pixels (P(t) = 1 in equation (6)).

[0036] (Results and Discussion) FIG. 7 shows the number of dots erased by the scanning motion of the ultrasound probe. The numbers in FIG. 7 are the "number of erased dots / 7 (number of applied dots)." FIG. 8 shows an overview of the scanning motion generated using the model of this embodiment, as well as the original image and the CAE-reconstructed ultrasound image. Various abdominal surfaces were successfully scanned along the motion trajectory generated by the model of this embodiment. Although the position of the ultrasound probe 30 sometimes deviated from the ideal trajectory during scanning, the model was able to correct the position of the ultrasound probe 30 after a while. The overall percentage of erased dots (i.e., correctly scanned dots) was 91.7%. In contrast, both of the above-mentioned two learning models (a) and (b) were unable to scan along the appropriate trajectory and generated an incorrect posture for the robot arm 10. In that case, the robot arm 10 was located at the edge of the phantom 5 (FIG. 9). This demonstrates that the shape information of the abdomen 5 and the error function contribute to correctly correcting the position when a deviation from the corrected trajectory is detected.

[0037] As demonstrated in this test, the medical support robot system according to this embodiment can adapt the ultrasound probe 30 to the abdominal surface even if the exact shape of the patient's abdomen is not known in advance. Furthermore, to automate the acquisition of fetal ultrasound images, the medical support robot system according to this embodiment provides a DNN model for generating appropriate ultrasound scanning operations for the robotic device. The DNN model can predict the appropriate scanning trajectory by mapping the height and width of the abdomen to a latent space. Furthermore, the medical support robot system according to this embodiment can detect deviations of the robot arm 10 from the appropriate trajectory and return the robot arm 10 to the correct position. This can be achieved by using an error function that is weighted when the robot arm 10 is on the correct trajectory and ignored when it is away from the abdomen.

[0038] (Functional Blocks) Figure 10 is a diagram showing an example of functional blocks of the computer device 60. The computer device 60 includes a teacher data generation unit 610, an image feature extraction model learning unit 620, a motion generation model learning unit 630, an input data acquisition unit 640, a next step data output unit 650, and an arm position information transmission unit 660. The teacher data generation unit 610, the image feature extraction model learning unit 620, and the motion generation model learning unit 630 are components related to learning the DNN model. The input data acquisition unit 640, the next step data output unit 650, and the arm position information transmission unit 660 are components related to motion generation of the DNN model robot device.

[0039] The training data generation unit 610 generates training data for the DNN model to learn. The training data generation unit 610 sets operation parameters for the training medical support robot system 1 (robot arm 10). The training data generation unit 610 also acquires abdominal shape information of the patient's abdomen 5. The operation parameters and abdominal shape information can be input by a user of the computer device 60. The training medical support robot system 1 (robot arm 10) performs a scan operation of the abdomen 5 based on the operation parameters and abdominal shape information to acquire an ultrasound image. Training data is generated using the operation parameters, abdominal shape information, and ultrasound image. The image feature extraction model learning unit 620 learns (trains) an image feature extraction model (CAE). The motion generation model learning unit 630 learns (trains) a motion generation model (MTRNN) using the training data generated by the training data generation unit 610.

[0040] The input data acquisition unit 640 acquires arm position information of the robot arm 10 and force sensor information measured by the force / torque sensor. The input data acquisition unit 640 also acquires an ultrasound image of the abdomen 5 by the scanning operation of the ultrasound probe 30 held by the end effector 20. The ultrasound image is used as input data for a trained image feature extraction model (CAE), and image features are output by the trained image feature extraction model.

[0041] The next step data output unit 650 outputs next step data using a trained motion generation model (MTRNN) with the arm position information, force sensor information, and image feature amounts acquired by the input data acquisition unit 640 as input data. The next step data includes the arm position information, force sensor information, and image feature amounts for the next step.

[0042] The arm position information transmitting unit 660 transmits to the end effector 20 arm position information indicating the position of the robot arm 10 in the next step, which is included in the next step data output by the next step data output unit 650 .

[0043] 11 to 14 are flow diagrams showing an example of processing executed by the computer device 60 of the medical support robot system according to this embodiment. Figures 11 and 12 are flow diagrams relating to model training (learning), and Figures 13 and 14 are flow diagrams relating to verification.

[0044] 11 is a diagram showing an example of a process for training a DNN model. First, the training data generating unit 610 generates training data for training the DNN model (step S110). (Details of this step will be described later.)

[0045] Next, the image feature extraction model learning unit 620 learns (trains) the image feature extraction model (step S120). Next, the action generation model learning unit 630 learns (trains) the action generation model using the teacher data generated in step S110 (step S130).

[0046] Figure 12 is a diagram showing an example of the details of the processing in step S110 of Figure 11. Operation parameters of the medical support robot system 1 (robot arm 10) for learning, which is used to train the model, are set (step S1110). Next, abdominal shape information of the patient's abdomen 5 is acquired (step S1120). A scan operation of the abdomen 5 is performed based on the operation parameters set in step S1110 and the abdominal shape information of the abdomen 5 acquired in step S1120 (step S1130). An ultrasound image of the abdomen 5 is acquired by the scan operation of step S1130 (step S1140). Teacher data is generated based on the data set or acquired in steps S1110 to S1140 (step S1150).

[0047] (Verification Flow) Figure 13 is a diagram showing an example of processing for verifying a trained DNN model. First, the ultrasound probe 30 is moved to an initial position on the surface of the abdomen 5 (step S210). Next, the input data acquisition unit 640 acquires input data for the motion generation model (MTRNN) (step S220). (Details of this step will be described later.)

[0048] Next, the next step data output unit uses the input data (arm position information, force sensor information, and image feature amounts) acquired in step S220 as input data and outputs next step data using the trained motion generation model (MTRNN) (step S230). The next step data includes the arm position information, force sensor information, and image feature amounts for the next step. The arm position information transmission unit 660 transmits arm position information indicating the position of the robot arm 10 for the next step, which is included in the next step data output in step S230, to the end effector 20 (step S240). Note that the position of the robot arm 10 can also be referred to as the position of the ultrasound probe 30 (this also applies throughout this specification). The processes of steps S220 to S240 are repeated until the final step is reached (step S250).

[0049] Figure 14 is a diagram showing an example of detailed processing in step S220 of Figure 13. Arm position information of the robot arm 10 is acquired (step S2210). Note that the arm position information can be acquired by a known method. Force sensor information indicating the contact force of the ultrasonic probe 30 on the surface of the abdomen 5, measured by the force / torque sensor, is acquired (step S2220). An ultrasound image of the abdomen 5 is acquired by the scanning operation of the ultrasonic probe 30 held by the end effector 20 (step S2230). Using the ultrasound image acquired in step S2230 as input data, image features are output by a trained image feature extraction model (CAE) (step S2240).

[0050] Although one embodiment of the present invention has been described above, it goes without saying that the present invention is not limited to the above-described embodiment and may be embodied in various different forms within the scope of the technical concept thereof.

[0051] The scope of the present invention is not limited to the exemplary embodiments shown and described, but includes all embodiments that have equivalent effects to the object of the present invention. Furthermore, the scope of the present invention is not limited to the combination of the features of the invention defined by each claim, but can be defined by any desired combination of specific features among all the disclosed features. Note that the following configurations also fall within the technical scope of the present invention.

[0052] (Invention 1) A method for generating movements of a medical support robot, comprising: a step of acquiring input data for a learned movement generation model, the input data including arm position information indicating a position of the robot arm of the medical support robot, force sensor information indicating a contact force of an ultrasound probe operated by the medical support robot on a subject, and image features of an ultrasound image of the subject obtained by a scanning operation of the ultrasound probe, and a step of the learned movement generation model outputting next step data using the acquired input data as input data for the learned movement generation model, the next step data including arm position information of the next step, force sensor information, and image features. (Invention 2) The method according to Invention 1, further comprising the step of outputting the arm position information of the next step output by the learned movement generation model to the medical support robot.

[0053] (Invention 3) The method according to Invention 1 or Invention 2, wherein the image feature is output by a trained image feature extraction model using an ultrasound image of the subject as input data to the trained image feature extraction model.

[0054] (Invention 4) The method according to any one of Inventions 1 to 3, wherein the learned motion generation model is a model trained using training data including motion parameters of the robot arm, shape information of the subject, and an ultrasound image of the subject obtained by a scanning motion of the ultrasound probe executed based on the motion parameters and the shape information of the subject. (Invention 5) The method according to any one of Inventions 1 to 4, wherein the learned motion generation model is a Multiple Time-scale Recurrent Neural Network (MTRNN).

[0055] (Invention 6) The method according to Invention 5, wherein the trained motion generation model is a model in which shape information of the object is mapped to a latent space and trained using an error function that is weighted when the robot arm is correctly scanning the object with the ultrasound probe and is ignored when the robot arm is not correctly scanning the object. (Invention 7) The method according to any one of Inventions 1 to 6, wherein the trained image feature extraction model is a convolutional autoencoder.

[0056] (Invention 8) A computer device comprising a processor and a memory, the computer device executing the methods described in Inventions 1 to 7. (Invention 9) A computer program for causing a computer device comprising a processor and a memory to execute the methods described in Inventions 1 to 7.

[0057] DESCRIPTION OF SYMBOLS 1... Medical support robot system 5... Phantom simulating a patient's abdomen 10... Robot arm 20... End effector 30... Ultrasound probe 60... Computer device 62... PID controller 610... Teacher data generation unit 620... Image feature extraction model learning unit 630... Action generation model learning unit 640... Input data acquisition unit 650... Next step data output unit 660... Arm position information transmission unit

Claims

1. A method for generating movements of a medical support robot, comprising: a step of acquiring input data for a learned movement generation model, the input data including arm position information indicating a position of the robot arm of the medical support robot, force sensor information indicating a contact force of an ultrasound probe operated by the medical support robot on a subject, and image features of an ultrasound image of the subject obtained by a scanning operation of the ultrasound probe; and a step of the learned movement generation model outputting next step data using the acquired input data as input data for the learned movement generation model, the next step data including arm position information, force sensor information, and image features of the next step.

2. The method according to claim 1, further comprising a step of outputting the arm position information of the next step output by the learned motion generation model to the medical support robot.

3. The method according to claim 1, wherein the image features are output by a trained image feature extraction model using an ultrasound image of the subject as input data to the trained image feature extraction model.

4. The method according to claim 1, wherein the learned motion generation model is a model trained using training data including motion parameters of the robot arm, shape information of the subject, and an ultrasound image of the subject obtained by a scanning operation of the ultrasound probe performed based on the motion parameters and the shape information of the subject.

5. The method of claim 1, wherein the learned action generation model is a Multiple Time-scale Recurrent Neural Network (MTRNN).

6. The method of claim 5, wherein the learned motion generation model is a model trained using an error function in which shape information of the object is mapped into a latent space and is weighted when the robot arm is correctly scanning the object with the ultrasound probe and is ignored when the robot arm is not correctly scanning the object.

7. The method of claim 1, wherein the trained image feature extraction model is a convolutional autoencoder.

8. A computer device comprising a processor and a memory, the computer device being adapted to carry out the method according to claims 1 to 7.

9. A computer program product for causing a computer device, comprising a processor and a memory, to carry out the method according to claims 1 to 7.

Images