Method for guiding a robot arm, guiding system

Abstract

Disclosed is a method for real-time guidance of a robot arm (6) for the treatment of surface data of a body, comprising the steps of: ▪ generating a model of a body (MOD_P) comprising a mesh (M1, M2) of points; ▪ planning a treatment path (TRAJ1) on the surface of the body model (MOD_S) by means of a computer; sensor parameters ▪ activating at least one emission from an emitter and / or one acquisition by a sensor of an operator device (8), said operator device (8) being arranged at the distal end of the robotic arm (6), the activation being carried out when the orientation of the sensor or emitter axis is coincident with a predefined straight line passing through the target point (PCIBLE), said target point (PCIBLE) being referenced on the generated body model (MOD_P).

Description

[0001] METHOD FOR GUIDING A ROBOT ARM, GUIDING SYSTEM

[0002] The field of the invention relates to methods for constructing a three-dimensional image of the human body in order to dynamically control a device located near the body. More specifically, the field of the invention relates to methods for real-time dynamic control requiring rapid computation of the three-dimensional surface. The invention therefore deals with methods for generating a surface from algorithms combining parameterized models and machine learning using an artificial intelligence technique.

[0003] Several techniques currently exist for reconstructing three-dimensional surfaces in space. However, these are not suitable for the real-time guidance of a medical device intended to perform treatment near a patient's skin surface. Indeed, reconstructing the surface of a body requires image processing that demands significant computing resources and often numerous acquisition systems. Furthermore, this complexity most often necessitates partial processing or an approximation requiring human assistance.

[0004] Currently, this limitation prevents the implementation of complex spatial guidance for a device. Typically, guiding a robotic arm in a controlled environment based on a calculated surface that may undergo deformation or displacement remains difficult. However, there is a need for robotic arms that are responsive and automatically guided according to specific real-time analyses. To date, existing solutions require additional operator intervention or lengthy processing times.

[0005] Therefore, there is a need to define a solution that can address this problem.

[0006] The present invention aims to resolve the aforementioned drawbacks.

[0007] According to one aspect, the invention relates to a real-time guidance method for a robotic arm for processing surface data of a body, comprising the steps of: ■ Generation of a model of a body comprising a mesh of points;

[0008] ■ Planning a treatment trajectory on the surface of the body model using a computer; sensor parameters

[0009] ■ Calculation of a guidance trajectory for the distal end of the robot arm;

[0010] ■ Generation of a kinematic of the robot arm such that a point of the head arranged at the distal end of the robot arm is servo-controlled on the guidance trajectory;

[0011] ■ Activation of the robot arm's kinematics to follow the guidance trajectory, said trajectory being calculated in real time from the generated body model;

[0012] ■Receiving at least one position of a target point in space;

[0013] ■ Activation of at least one emission from a transmitter and / or an acquisition from a sensor of an operator device, said operator device being arranged at the distal end of the robotic arm, the activation being carried out when the orientation of the axis of the sensor or transmitter coincides with a predefined line passing through the target point, said target point being referenced on the generated body model.

[0014] One advantage is the use of a guidance method that takes into account a body model reproducing a patient's body. This reduces computational load and enables real-time control of medical equipment. The reconstruction of a real world is optimized by using a meshed body model calculated from a patient's body. Another advantage is the ability to account for a patient's potential movements or anticipate possible collisions by using a body model that is easy to process. The body model used comprises a set of points or a mesh, which can be easier to use for real-time guidance than a system based on acquiring direct images of a patient to anticipate the robot's movements. In one aspect, the invention relates to a real-time guidance method for a robotic arm for processing surface data of a body, comprising the following steps:

[0015] ■ Generation of a 3D model of all or part of a surface of a human body comprising a point mesh;

[0016] ■ Planning a processing trajectory on the surface of the body model using a computer;

[0017] ■ Calculation of a guidance trajectory for the distal end of the robot arm;

[0018] ■ Generation of a kinematic of the robot arm such that a point of the head arranged at the distal end of the robot arm is servo-controlled on the guidance trajectory;

[0019] ■ Activation of the robot arm's kinematics to follow the guidance trajectory, said guidance trajectory being calculated in real time from the 3D surface model of the human body;

[0020] ■the generation of a new 3D model of the human body surface at regular intervals ensuring the calculation of new kinematics for the robotic arm to automatically control its movements in real time, a new calculated surface of the 3D model leading to the calculation of a new processing trajectory and a new guidance trajectory,

[0021] ■ Receiving at least one position of a target point in space;

[0022] ■ Activation of at least one emission from a transmitter and / or an acquisition from a sensor of an operator device, said operator device being arranged at the distal end of the robotic arm, the activation being carried out when the orientation of the axis of the sensor or transmitter coincides with a predefined line passing through the target point, said target point being referenced on the generated body model.

[0023] In one embodiment, the body model is generated from a patient body image acquisition system, the system being arranged at a point in space. An advantage is that acquiring a partial view of the human body allows for the generation of a complete human body using an algorithm based on the exploitation of a neural network. In another embodiment, the processing trajectory is generated within a processing area of ​​a surface of the body model.

[0024] One advantage is defining a guidance trajectory that is generated relative to the surface generated by the input body model. Guidance is therefore optimized due to the simplified use of a meshed body model.

[0025] According to one embodiment, the treatment path is automatically generated from the definition of a treatment area defined on a surface of the body model.

[0026] One advantage is automating a task for an operator who only needs to select a region. The region is defined on the generated body model. Thus, its definition is simplified and the trajectory can be easily generated.

[0027] According to one embodiment, the guidance trajectory is calculated from a predefined distance and orientation setpoint allowing the generation of a relative position of a distal end of the robot arm with respect to the treatment trajectory.

[0028] One advantage is to obtain simplified guidance through a servo system that takes into account a minimum number of parameters.

[0029] According to one embodiment, the robot arm comprises at least 3 branches articulated in pairs, one with respect to the other by a pivot joint, the set of pivot joints comprising a planned kinematics for a given guidance trajectory and a generated body model.

[0030] One advantage is to separate the generation of the robot arm kinematics, which can be done within a computer, from the robotic arm based on a position of a trajectory guidance point generated from an operating console.

[0031] In one embodiment, the guidance process involves generating a new body model at regular intervals so that the robotic arm can be controlled in real time based on the generation of a new robotic arm trajectory. A new calculated surface of the body model results in a modification of the treatment trajectory. The process also includes calculating new robotic arm kinematics based on the definition of the new trajectory. One advantage is achieving good real-time guidance performance. Using a body model simplifies guidance calculations. The robotic arm can be controlled quickly, thus limiting collisions or sudden patient movements.

[0032] In one embodiment, the guidance process includes a step of calibrating the position of the robotic arm from a fixed position at which images are acquired to generate a body model. An advantage of calibration is knowing the position of the body model relative to the position of the end of the robotic arm.

[0033] According to one embodiment, the processing trajectory is used to establish a strategy for firing a laser beam from a laser emitter, said laser emitter being fixed to the operator device.

[0034] In one embodiment, the body model includes a homogeneous mesh so as to create a new mesh defined by an arrangement of circles, at least one circle defining the position of a target point in space. An advantage is that it simplifies the activation of an operator device at the head of the robotic arm based on points that can be easily located on a mesh.

[0035] In one embodiment, the new mesh is generated based on a firing power instruction and / or an instruction relating to an overlap rate between two shots. One advantage is having a mesh adapted to the specific use case of the robotic arm.

[0036] In one embodiment, the sampling of the number of shots fired is a function of the new mesh, the configured firing power, and the overlap rate defined between two co-located circles. One advantage is providing an operator with a user-friendly control console.

[0037] According to one embodiment, the processing trajectory is used to establish a firing strategy for an ultrasonic signal from an ultrasonic transmitter, said ultrasonic transmitter being fixed to the operator device.

[0038] In one embodiment, a set of images is acquired from an optical device mounted on the operating device, each image being indexed to at least one node or cell of the body model's mesh based on the position of the target point on said body model and the viewing angle of the optical device. An advantage is the ability to reconstruct an envelope, such as skin, on the parameterized or parametric body model corresponding to a patient's body.

[0039] According to one embodiment, at least one pair of images is acquired simultaneously from an optical device comprising two sensors joined together and arranged on different axes of the operator device, each image being indexed to at least one node or mesh of the body model mesh as a function of the position of the target point on said body model and a viewing angle of the optical device, said images being processed so as to reconstruct a local three-dimensional image.

[0040] According to one embodiment, the local three-dimensional image is superimposed in a 3D representation onto a generated body model.

[0041] According to one embodiment, a first image acquired by a first optical device of the operator device is compared to a set of real-time images, the comparison operation including, where appropriate, the generation of an indicator of interest, said indicator of interest enabling the generation of a digital control command for the robot arm.

[0042] In one embodiment, the guidance instruction triggers an initial movement of the robotic arm to the target position, from which the first image is extracted. A second movement of the operator device then activates a second optical device, which captures a second near-field image of an area containing the target point. One advantage is the generation of an image database that can be used for diagnostic purposes, qualitative studies of patients and diseases, or to create a patient image history containing skin information at various scales.

[0043] In one embodiment, the comparison step is a classification step of the acquired image using a neural network trained on a set of images. One advantage is that it optimizes the detection of singularities on a patient's skin.

[0044] In another aspect, the invention relates to a robotic arm guidance system comprising:

[0045] ■ a computer to generate a plurality of body models at different times;

[0046] ■ a robotic arm controlled by a computer in a control unit calculating the kinematics of the robotic arm from a trajectory calculated from points defined on a mesh of an acquired body model;

[0047] ■ an operator device arranged at the end of the robot arm comprising at least one transmitter or sensor, the activation of at least one transmitter or sensor being achieved when the orientation of the axis of the sensor or transmitter is directed along a straight line passing through the target point, said target point being referenced on the generated body model.

[0048] According to one aspect, the invention relates to a robotic arm guidance system comprising a device for acquiring an image of a patient's body to produce a parameterized body model and a robotic arm equipped with an operator device for implementing the method of the invention.

[0049] The system of the invention comprises, according to different embodiments, the various means described in the different implementations of the invention. Furthermore, according to the different embodiments, it is configured according to the different realizations of the process of the invention.

[0050] According to one aspect, the invention relates to a guidance system for a robotic arm comprising:

[0051] ■ a robotic arm controlled by a computer in a control unit calculating the kinematics of the robotic arm from a trajectory calculated from points defined on a mesh of a 3D model of an acquired human body surface;

[0052] ■ a computer to generate a plurality of body models at different times, the generation of a new 3D model of the surface of the human body at regular times ensuring the calculation of a new kinematic of the robotic arm to control its movements automatically in real time, a new calculated surface of the 3D model leading to the calculation of a new trajectory;

[0053] ■ an operator device arranged at the end of the robot arm comprising at least one transmitter or sensor, the activation of at least one transmitter or sensor being achieved when the orientation of the axis of the sensor or transmitter is directed along a straight line passing through the target point, said target point (being referenced on the 3D model of a generated human body surface).

[0054] According to another aspect, the invention relates to a method for generating a three-dimensional working surface of a human body comprising the steps of:

[0055] ■ Receiving input data corresponding to geometric data;

[0056] ■ Generation of an initial point cloud from the input data, each point being associated with a three-dimensional spatial coordinate;

[0057] ■ Determination of a plurality of attributes at each point of the first point cloud, said attributes comprising at least one shape descriptor, at least one shape descriptor being calculated from a subset of points in the vicinity of the point considered, at least one attribute of a point comprising the data describing a normal vector at that point of the first point cloud,

[0058] ■Association of said attributes to each point to generate a second enriched point cloud;

[0059] ■ Calculation of a set of geometric parameters comprising at least one rotation at at least one point of the second point cloud and at least one shape coefficient by means of a regression, said regression being carried out from a series of matrix operations performed according to different layers of a neural network trained from a plurality of partial views of parameterized parametric models with different parameterization configurations;

[0060] ■ Determination of a parameterized model to generate a body model of a body including a first mesh.

[0061] In one example, the input data includes data describing images. Geometric data, for instance, can be derived from data describing 2D images. In one case, the input data is therefore images.

[0062] One advantage of this method is that it allows for real-time control of medical equipment guidance in close proximity to a patient's body, using a minimalist acquisition system such as a pair of cameras. The guidance performance is achieved primarily through a high-performance neural network learning process.

[0063] In one embodiment, the process further includes processing the first mesh to generate a second homogeneous mesh. An advantage of using a homogeneous mesh is that it facilitates the control of the sensor(s) or transmitter(s) at the head of the robotic arm. Indeed, processing a point on a homogeneous mesh simplifies the operation of the device at the head of the robotic arm.

[0064] In one embodiment, a shape descriptor is calculated at at least one given point of the first point cloud from the adjacency matrix of the first graph using a spectral partitioning method for the first point cloud. One advantage is that, in the neighborhood of a point, the shape descriptor is weighted according to a distance criterion to the point where it is applied. Another advantage is that it allows the local surface topology to be taken into account in the neighborhood of a point.

[0065] In one embodiment, at least one local shape descriptor and / or one global shape descriptor are used, these descriptors being of the type: "Wave Kernel Signature," "Heat Kernel Signature," or "Gaussian Curvature." One advantage is the ability to define shape descriptors from a spectral analysis of the region surrounding a point. Another advantage is the simplified use of standardized equations or operators whose numerical processing can be optimized.

[0066] In one embodiment, the calculation of a normal vector at a given point in the first point cloud is performed by considering a surface generated locally around each point, based on the points in its neighborhood. An advantage is the ability to parameterize a desired resolution or granularity for the analysis of the local surface topology of the surface of the body being processed.

[0067] According to one embodiment, the surface generated locally at a given point in the first point cloud is calculated:

[0068] ■ Either, starting from the underlying surface defined by the points of the first point cloud; ■ Either, from an approximation of a local surface directly obtained from a method of the k nearest neighbors of the point considered.

[0069] In one embodiment, at least two shape descriptors are associated with each point in the first point cloud. One advantage is that this enriches the model description. Another benefit is improved learning during neural network training. Furthermore, another advantage is that it enhances the performance of the neural network's regression stage.

[0070] According to one embodiment, the first point cloud is generated from the obtained data:

[0071] ■By acquiring a plurality of images from at least two optics, each position of a point in the first point cloud being deduced from a stereoscopic measurement of at least two acquired images and / or;

[0072] ■ By acquiring an image containing a distorted pattern, said pattern being projected by a projector and having a known geometry and / or;

[0073] ■ By acquiring ultrasound signals emitted and reflected off the surface of a patient's skin and / or;

[0074] ■ By acquiring a laser emitted onto the surface of a patient's skin.

[0075] One advantage is the ability to implement the invention's process using various acquisition techniques. Another benefit is obtaining a point cloud that can be used by the invention's process, particularly for guiding medical devices. The different acquisition techniques can be combined or used interchangeably. Image acquisition systems, such as cameras, remain simple to implement, configure, and deploy.

[0076] According to one embodiment, the parameterized body models are obtained from a parametric model of the human body in which a set of joints are predefined, said modeling parameters:

[0077] at least one angle of rotation for at least one joint and; ■ at least one form factor.

[0078] One advantage is the ability to define simple parameters for a human body that take into account the different degrees of freedom of each limb. Another advantage is the ability to achieve real-time control of a guided device based on a patient's possible movements.

[0079] According to one embodiment, these parameters also model:

[0080] ■ at least one global translation of the model in space, and / or;

[0081] ■ at least one global rotation of the model in space.

[0082] One advantage is that it allows for consideration of a patient's movements and calibration of the patient's body acquisition system according to the arrangement of the different equipment.

[0083] In one embodiment, a parameterized body model is obtained from a given parameterization definition. This parameterization includes the definition of a set of modeling parameters that define a specific representation of the parameterized body model. This parameterized body model can then be represented by a third point cloud, a graph, or a mesh. One advantage is the ability to generate a directly usable point cloud from the obtained parameterized bodies. Another advantage is the ability to obtain a large number of bodies in different poses, thus achieving good generalization performance for the neural network.

[0084] In one embodiment, a plurality of partial views is generated. Each partial view can, for example, be obtained from a subset of points in the third point cloud. One advantage is the generation of numerous views likely to correspond to actual image captures. The neural network is therefore better trained, and the performance of the regression step is improved, when trained on a large number of partial views.

[0085] According to one embodiment, each subset of points in the third point cloud is obtained by selecting points seen from a given viewpoint of space, the set of partial views being generated for a plurality of viewpoints of space.

[0086] In one embodiment, the regression includes: ■Convolutions or neural network layers comprising a plurality of matrix multiplications with weighting coefficients obtained from a learning process;

[0087] ■ non-linear operations.

[0088] In one embodiment, the regression includes the following inputs:

[0089] ■ points from the point cloud and the value of their attributes and / or;

[0090] ■ the adjacency matrix of the graph corresponding to a scatter plot.

[0091] According to one embodiment, the method includes a preliminary step of training the neural network, said preliminary step being carried out by a deep learning method comprising:

[0092] ■ a training exercise performed using a plurality of partial views of parameterized parametric models with different parameterization configurations,

[0093] ■ the production of matrices including weighting coefficients for each layer of the trained network.

[0094] According to another aspect, the invention relates to a system comprising an input data acquisition interface, a computer and a memory for processing input data acquired by means of the method to generate a parameterized body model.

[0095] According to one embodiment, the invention relates to a system comprising a means for acquiring distance data from a set of points on the surface of a body located in space, a guided robotic arm comprising at least one degree of freedom, said robotic arm further comprising a sensor or transmitter whose position is controlled by a guidance trajectory obtained from a processing trajectory defined on the surface of a partial view of a body model, said body model being generated from the method of the invention.

[0096] The system of the invention can be configured according to the various parameters stated to describe the method of the invention. Furthermore, the system of the invention comprises, depending on the different implementations, the various means described in the different implementations of the method of the invention. Other features and advantages of the invention will become apparent from the detailed description that follows, with reference to the accompanying figures, which illustrate:

[0097] Fig. 1: the main steps of an embodiment of the process of the invention;

[0098] Fig. 2: an embodiment of a system for spatially controlling the movements of a robotic arm near a surface of a body to be treated; Fig. 3: a representation of an embodiment of a system of the invention comprising an operating console, a positioning system and a robotic arm controllable from the method of the invention; Fig. 4: an example of a robotic arm of the invention comprising six axes guided to follow a planned trajectory;

[0099] Fig. 5: an example of a robotic arm of the invention comprising six axes illustrating a reference frame of the head of the robotic arm;

[0100] Fig. 6A: a first example of an operator device arranged at the head of a robotic arm of the invention intended for example for a dermatological application; Fig. 6B: a second example of an operator device arranged at the head of a robotic arm of the invention;

[0101] Fig. 7: an example of a first mesh of points of a surface to be treated according to the process of the invention;

[0102] Fig. 8: a first example of a second mesh obtained from an embodiment of the invention,

[0103] Fig. 9: a second example of a second mesh obtained from an embodiment of the invention.

[0104] The invention relates to a method for acquiring images of a body's surface and reconstructing a three-dimensional surface of said body. The invention also relates to an image acquisition and processing system. The invention further relates to a method and system for activating a medical device using a modeled body. The invention also relates to the guidance, for example, of a robotic object for the diagnosis or treatment of a patient's skin.

[0105] In the following description, a point acquisition system 5 for the body of a patient 1 is preferably positioned at a fixed point in space. For example, this point acquisition system can be fixed to the ceiling, the wall, or any other point in space intended to remain fixed during the execution of the method of the invention. This acquisition system 5 is referenced in a frame denoted Ro (O, xo, yo, zo). It can also be referenced in a frame Ri (Oi, xi, yi, zi) associated with the acquisition system 5. Based on a position calibration between the "world" frame and the acquisition system 5, the acquisition system 5 can be referenced in the frame Ro by means of a transformation Ri -> Ro.

[0106] In the following description, a robotic arm 6 is positioned within the same space covered by the acquisition field of the acquisition device 5. This could be a treatment room in a hospital or a room in a doctor's office. The robotic arm 6 can be positioned on a base, such as a removable cabinet 25, which can be fixed in a given position during the treatment performed using the method of the invention. This robotic arm 6 is referenced in a coordinate system denoted Ro (O, xo, yo, zo). The position of the robotic arm 6 in space can be located within the coordinate system R2 (O2, X2, y2, Z2), in which the movements of the robotic arm 6 can be easily described. A change of reference frame R2 -> Ro allows the movements of the robot arm 6 to be expressed in the reference frame of the acquisition system Ro 5. This change of reference frame can prove particularly effective when performing a collision detection function between the robot arm 5 and the body of a patient 1.

[0107] According to one embodiment, the acquisition system 5 can be fixed to the reference frame R2. In this example, the acquisition system can be attached to the base supporting the robot arm 6. This configuration can be achieved, for example, using a support rod (not shown) to raise the acquisition system 5. One advantage of this configuration is that it simplifies the installation of the system of the invention. Another advantage is that it provides a pre-calibration between the positioning of the acquisition system 5 and the robot arm 6.

[0108] In another embodiment, the acquisition system 5 can be fixed to a base on which an operating console 20 is installed. This configuration provides a self-contained acquisition device comprising both hardware and software components. A support rod can also be used to fix the acquisition system 5 or any other mechanical support device. In the following description, a head 8 of the robot arm 6, also called the operator device 8, is arranged at the distal end of the robot arm 6. The head 8 can be linked to a frame denoted R3 (O3, X3, y3, Z3) shown in Figure 5. The position of the head 8 of the robot arm 6 can be easily identified in the frame R2 (O2, X2, y2, Z2) with knowledge of the kinematics animating the branches of the robot arm 6. One advantage is to reference the images acquired by the operator device 8, for example, in the frame Ro (O, xo, yo, zo).

[0109] This description applies to any type of robotic object, including a robotic arm. The robotic arm of the invention may have a plurality of axes of freedom, notably 2 and 9 movable axes. In the following description, we will consider the application of a 6-axis robotic arm to describe one embodiment. However, the invention is not limited to this example.

[0110] Depending on the specific configuration, the robotic arm can be fixed to a horizontal base, with the first arm extending vertically, or it can be fixed to a vertical base, with the first arm extending horizontally. Various configurations of the invention are compatible.

[0111] In the following description, a "body model" refers to a three-dimensional model of a body. This body model can be represented by a point cloud. The point cloud can be generated from a given model of a parametric body obtained with a given parameterization.

[0112] Furthermore, this body model can be used to generate different partial views of the body, each corresponding to a subset of points from a point cloud modeling the entire body. To this end, a viewpoint can be defined to extract a partial view from the body model. Such a body model can be obtained from a configuration of a given parameterization of a parametric model of the human body.

[0113] In the following description, a partial view of a "body model" is defined as a three-dimensional model of a body from a given viewpoint from which a surface of the previous body model is observed.

[0114] In the following description, we will refer to a "parametric human body model," MOD_S, as a model that can be parameterized to generate transformations of certain parts of the body or views based on a given parameter configuration. A previously defined body model can thus be generated from a parametric human body model with a given parameterization; this is then called a parameterized body model, MOD_P.

[0115] The parametric body model can be used in a first application during the process of the invention to perform the regression step in order to determine a set of parameters for generating a body model to guide medical equipment, for example. It can also be used in a second application during deep learning to generate weight matrices that will be used in the regression step.

[0116] As an example, a MOD_S parametric body model can be obtained from the parametric model with the following parameter inputs:

[0117] ■ Global translation of the model in space: Tx, Ty, Tz, i.e. 3 parameters;

[0118] ■ Global rotation of the model in space: Rx, Ry, Rz, i.e. 3 parameters;

[0119] ■Rotation angles of each joint, for example using a parametric model with 24 joints each having 3 rotation parameters: Rxi, Ryi, Rzi defining a total of 72 parameters;

[0120] ■ Shape coefficients, for example using a parametric model with 20 shape coefficients. These influence the shape of the human body.

[0121] In this example, 88 parameters can be determined to generate a human body model.

[0122] Other examples can be implemented with the method or system of the invention.

[0123] Point acquisition

[0124] In one embodiment, the process includes a step of acquiring ACQ points. This acquisition can be carried out, for example, using an image acquisition device. In one embodiment, an optical system 5 in Figure 3 comprises two sensors 51, 52 for acquiring IM-1, IM2 images. These could be, for example, cameras such as 3D cameras comprising at least two color or infrared cameras. In various embodiments, color or infrared cameras can be used. Pairs of identical cameras are preferred. A preliminary calibration step can be performed to configure the colors of each camera and match them for the same points in space.

[0125] Another calibration method, called "stereoscopic calibration," can be performed between the two cameras to create a 3D reconstruction of a subject. This calibration involves calculating a transformation between two images acquired by each camera.

[0126] As an example, this transformation might involve keypoint detection and the calculation of descriptors obtained through a scale-invariant feature transform (SIFT). A correlation step to match keypoints can be implemented, for example, using an estimation method such as RANSAC (RANdom Sample Consensus). Next, an isometric transformation is calculated between the detected keypoints. Finally, the transformation triangulates the image points, thus constructing a depth map.

[0127] According to one method, 2D images acquired of the same object from two viewpoints whose arrangement is known can be combined using known techniques to obtain a so-called depth or distance map.

[0128] The method of the invention allows for the processing of 2D images, which are then combined to obtain a point cloud Gi. These points constitute a first graph G1. For example, the points of the first graph G1 obtained are undirected and unconnected. In its simplest form, the graph G1 is an unordered list of points whose coordinates are referenced in space, for example, in R0, R1, R2, or R3. The points are then defined independently of each other. In its simplest form, the graph G1 is therefore a point cloud.

[0129] Images IM1 and IM2 can be acquired simultaneously. Advantageously, the optics are directed along parallel axes and are co-located. One benefit is the ability to process the images using a stereoscopic technique to generate a depth map. The depth map allows spatial coordinates (x, y, z) to be assigned to a plurality of points.

[0130] According to one embodiment, the colors of the images are used to generate differences in tone and therefore variations in distance.

[0131] The depth map can be obtained by encoding the distances between different points, the resolution of which can be configured. The grayscale levels can vary within a range of values. By choosing an encoding, for example, of 16 bits, the grayscale encoding of the depth map allows for high accuracy in determining the distance between the different points.

[0132] The method of the invention enables the continuous acquisition of a stream of images by the optics 51, 52 in order to process them in real time. The notations IM-i, IM2 refer to both the images acquired and the stream of images acquired by each of the cameras 51, 52.

[0133] Images can be processed to extract an acquisition perimeter that limits the area to be processed. To achieve this, a template can be used to avoid including points outside a region of interest defined by the body's surface.

[0134] Acquiring two images, IM1 and IM2, and covering the entire surface to be processed allows for the generation of a depth map of the whole body. The depth map includes a set of points and their relative distances to each other.

[0135] In another example, no surface is processed at this stage of the process. The distances between points are generated independently of any shape recognition, such as the outline of a body.

[0136] As an example, the number of points generated per unit area is configurable according to the desired point cloud resolution. To this end, the acquisition can be configured to acquire more or fewer images in order to increase the resolution.

[0137] According to one embodiment, a positioning system SYS_POS defines a reference frame and a coordinate system to locate each point within a single reference frame. This positioning system can advantageously be the acquisition device 5. Stereoscopic calibration enables the transformation between Ro and Ri, with R1 being the reference frame associated with the acquisition system 5. Subsequently, the method of the invention calibrates the robotic object 6 with respect to the acquisition system 5, thus performing image transformations in the reference frames R2->RI->RO. When a reference frame R3 is associated with the head 8 of the robotic arm 6, a transformation from R3 to R2 is known because the articulation kinematics of each arm are known by a computer and memory of the system controlling the robotic arm 6.

[0138] Other acquisition techniques

[0139] In one embodiment, a projection device is used to project a pattern onto the body's surface. The acquired images can then be processed to analyze the deformation of the projected pattern. This technique provides additional data and improves 3D reconstruction. In particular, detecting deformation in a projected pattern improves the accuracy of distance calculations used to generate the depth map. Furthermore, the projector reduces noise during point cloud construction to generate the initial G1 graph.

[0140] Consequently, stereoscopic reconstruction of a depth map can be achieved with pairs of cameras, possibly used with at least one pattern projector. The projected light can be in the visible or non-visible range, such as the infrared range. As an example, the acquisition device 5 could include a pair of 3D color cameras and at least one infrared projector.

[0141] According to another embodiment, the cameras used can be infrared cameras or cameras having a range of frequencies in the near-infrared.

[0142] Other techniques for constructing a depth map can be used, such as a laser transceiver. The reflection of the laser beam generates data relating to the geometry of the reflected surface. Another technique can rely on an ultrasonic transceiver. An example of a system that could be implemented is a LiDAR-type system.

[0143] According to one embodiment, the process of the invention makes it possible to retrieve a point cloud from the acquired images and to transfer it for example to a graphics processor such as a GPU, GPU meaning in Anglo-Saxon terminology "Graphics Processing Unit".

[0144] Generation of a first Graph G1

[0145] The method of the invention includes a step for generating a set of points associated with spatial coordinates referenced, for example, in a Ro frame of space. These coordinates can be obtained from a prior calibration of a SYS_POS positioning system. The calibration can be performed using a specific pattern acquired by the optics of the SYS_POS positioning system at the start of the method of the invention. In one embodiment, the SYS_POS positioning system and the SYS_OPT optical system may be a single system. In another embodiment, they may correspond to two distinct systems.

[0146] Thus, a depth map obtained by a SYS_OPT acquisition system can be used to obtain a set of points in space associated with coordinates in a Ro frame of space.

[0147] The process of the invention includes a data reception step REC_DATAi, the DATAi data corresponding to points in space referenced in a frame Ro of space.

[0148] As an example, the method of the invention may include a step for segmenting the data associated with a predefined surface. The surface may be, for example, the surface of a human body. This segmentation may be performed automatically by analyzing data such as data from acquired 2D images and / or the resulting depth map. For example, the method of the invention may include analyzing the distance differences between certain points or by performing colorimetric processing. The depth map may be used to filter the point cloud and retain only the points of interest, i.e., those of the human body.

[0149] In another example, a surface contour can be acquired from an operating console such as the PC 20 shown in Figure 3. According to this example, the surface contour of a body can be obtained using a selection tool such as a stylus or mouse and an image displayed on the console 20 screen by an operator. In other examples, a selection can be made by touch control on a screen or by voice command. One use case is to exclude, for example, points belonging to the table 22 on which patient 1 is lying.

[0150] The method of the invention comprises the generation of a first graph GEN_Gi. The first graph Gi can be obtained from a matrix representation of the set of points generated from the surface of a patient's body 1. Each point is denoted vi and can be represented by a vector including, in particular, its coordinates in a coordinate system Ro. In one embodiment, at this stage the first graph Gi is unordered in the acquisition frame. The notation "graph" is used here even in the simplest embodiment where the set of points corresponds to a list of points having coordinates, that is, a simple point cloud. In other embodiments, the graph may be connected or unconnected, directed or undirected, etc.

[0151] Filtering

[0152] According to one embodiment, the method of the invention includes a step of filtering the resulting graph; the filtering may be temporal and / or spatial. This filtering step reduces the noise generated during the generation of the first graph G1 following the detection step.

[0153] Determining the attributes

[0154] The method of the invention includes a step of determining at least one attribute DET_ATi for each point vi. The attributes correspond to data associated with the point vi; these attributes can be of different types: Cartesian data, shape descriptors, identifiers, vectors, etc.

[0155] In one embodiment, an attribute comprises data describing a normal vector n at the point vi of the first graph Gi. The normal vector n can be determined in various ways. The advantage of calculating the normal vector as an attribute of a point in the point cloud is to produce information about the convexity of the local surface.

[0156] In one embodiment, the normal vector ni at a given point vi of the first graph Gi is calculated by considering a surface generated locally around each point vi from the points ViLk in its neighborhood. To this end, the spatial coordinates of each point are compared to the coordinates of the point under consideration. In one example, the acquired distance or depth data are used to identify the points in the vicinity of the point under consideration.

[0157] According to a first example, the surface considered to calculate the normal vector is calculated from the underlying surface defined by the points of the first graph Gi.

[0158] In a second example, the surface used to calculate the normal vector is calculated using a GPU-implemented method. In another example, the normal vector is calculated from an approximation of a local surface obtained directly using a k-nearest neighbors method. This triangulation method, which connects a list of 3D points, can nevertheless be more computationally intensive than the GPU-implemented method.

[0159] One possible use of calculating the Normal vector at a point in the graph is to increase the description of a point in order to enrich the point cloud.

[0160] Shape descriptor

[0161] According to one embodiment, an attribute includes at least one shape descriptor, denoted Dfi, at each point vi of the first graph Gi. A shape descriptor is a local descriptive signature of the space at a point.

[0162] As an example, a shape descriptor Dfi is calculated from a subset of points in the vicinity of the point Vi under consideration. The process therefore includes a step of generating a reconstructed or extracted point graph from a body model, from the perspective of each image acquisition device capable of processing this data. For example, when an acquisition device is positioned at the distal end of a robotic arm, a partial view of a body model can be generated from the acquisition device's position to perform a specific processing task.

[0163] Shape descriptors can also be derived at each point of a 3D body model obtained from the point cloud. Since they exhibit symmetry and isotropy properties, the deformed descriptors remain invariant under isometric transformations, such as rotations of the body model. This is useful for transposing shape descriptors regardless of the viewing angle. An advantage is that the shape descriptors can be calculated directly on the points of the first graph Gi. The invention thus optimizes the calculation of these shape descriptors, which are not applied to surface portions but directly to predefined points of a graph. The descriptor calculations can be performed on the adjacency matrix of a connected graph obtained from the graph Gi. A connected graph implies that each point is connected to all the others.The adjacency matrix allows us to obtain the points that have direct connections to a given point. Furthermore, the degree of proximity of one point to another can be deduced from the adjacency matrix. Depending on the shape descriptors, a neighborhood of a point can also be deduced from the adjacency matrix.

[0164] Alternatively, the method of the invention allows a distance threshold to be applied so that a point is connected only with its N nearest neighbors.

[0165] In one embodiment, the shape descriptors are chosen to be isometric and invariant. One advantage is obtaining shape descriptors that remain invariant when a body movement is performed with an isometric transformation.

[0166] In the first example, the shape descriptor is of the type: "Wave Kernel Signature", denoted WKS. In the second example, the shape descriptor is of the type: "Heat Kernel Signature", denoted HKS. In the third example, the shape descriptor is of the type: "Gaussian Curvature".

[0167] According to different embodiments, a plurality of shape descriptors Dfi is calculated at each point considered of the first graph Gi and is associated at each of these last points.

[0168] Some shape descriptors belong to the group of spectral shape analysis methods, such as the HKS or WKS descriptors. The HKS descriptor, in particular, is based on a solution to the heat equation and is based on the Laplace-Beltrami operator.

[0169] One of the advantages of these shape descriptors is their invariance under certain transformations, particularly isometric transformations. Furthermore, this shape descriptor can be calculated from the adjacency matrix of the graph Gi. An advantage is that it allows us to consider the relative distance of a given point from all the points in its neighborhood.

[0170] According to one example, the method of the invention can be used by considering thresholds applied to the adjacency matrix so as to consider only a subset of points for which the distance is greater than a given threshold.

[0171] The WKS descriptor is based on a solution to the Schrödinger equation. One advantage of the WKS descriptor is its ability to provide spectral separation of the model obtained by the operator.

[0172] Shape descriptors can be chosen to model a local or global three-dimensional shape. When these descriptors encode a local model, a radius of validity can be introduced. However, a global model will benefit from all the geometric invariances of the operator used.

[0173] The "Global Point Signature" shape descriptor, abbreviated GPS, can also be used. This descriptor is based on a Laplace-Beltrami type operator. One advantage of this operator is its ability to obtain a global model.

[0174] The shape descriptor "Gaussian Curvature", denoted GC, can also be used. This descriptor offers the possibility of considering a parameterized surface and, conversely, of considering points that locally define a surface.

[0175] The "Spectral Graph Wavelet Signature" descriptor, denoted SGWS, can also be used to define a local multi-resolution descriptor.

[0176] One advantage of shape descriptors based on spectral shape analysis is their ability to be mathematically processed to assemble connected or neighboring surfaces, segment areas and perform shape recognition algorithms.

[0177] One advantage is to enrich the description of a point and therefore improve the performance of neural network regression.

[0178] Another advantage of shape descriptors is that they generate an identity at a point in the first graph Gi. When a patient's body moves or changes over time, the body surface is considered a deformable object whose transformations are non-isometric. Shape descriptors allow the generation of eigencharts at each point, which can be recovered using the method of the invention after deformation.

[0179] Attribute association

[0180] Thus, each point vi of the first graph Gi can be associated with coordinates {xi, y\, z\}, a normal vector n, and a set of shape descriptors {Dfn , DFi2, DFÎ3, DfiN}. This step is denoted ASSO_ATi in Figure 1.

[0181] The point vi can then be constructed as a finite-dimensional vector where each dimension corresponds to a given attribute. A second graph G2 can then be generated. This second graph G2 corresponds to a list of vectors enriched with their ATi attributes.

[0182] Regression

[0183] A computational step enables the three-dimensional reconstruction of the acquired and enhanced human body using an approximation by a parametric human body model. This step is labeled REG in Figure 1. The objective of the regression step is to estimate given parameters to configure the parametric body model with these parameters. Thus, from a partial view acquired by the camera, a complete parametric body model can be estimated.

[0184] To this end, the method of the invention then includes a step of calculating a set of geometric parameters comprising shape and rotation parameters ROTi.

[0185] Rotations can be calculated at a plurality of points in the second graph G2. In one embodiment, rotations are calculated for each point in the second graph G2. In another embodiment, each rotation can be associated with a joint. A joint can be associated with a bounded region of the parameterized body model MOD_S, and a given joint can therefore be associated with a parameterized body model MOD_P. A given joint can thus be associated, as a result of the regression operation, with a set of points in the second graph G2.

[0186] Shape parameters can be defined by a set of point positions of a parameterized body model in a given coordinate system.

[0187] According to one embodiment, the input parameters are the set of points in the point cloud G1 or G2 equipped with their attributes calculated in Euclidean space R 3Alternatively, the input parameters can be the adjacency matrix corresponding to the graph Gi of each point in the point cloud. These inputs allow for analysis within the spectral domain of the input and do not necessarily require consideration of attributes.

[0188] The shape parameters and rotations of a set of joints correspond to the output parameters of the neural network. These parameters will be used to configure the MOD_S parametric body model to generate a MOD_P parameterized body model.

[0189] According to one embodiment, the regression step comprises a series of mathematical operations such as matrix operations between the input matrix and a weight matrix previously trained by a deep learning process.

[0190] Each of these operations is followed by a non-linear function. These are activation or transfer functions that produce the output of a network node. For example, non-linear functions such as ReLU, Sigmoid, or Tanh can be implemented.

[0191] According to one example, the step includes a regression performed from a series of layer convolutions of a neural network trained from a plurality of parameterized models of predefined fields MPPi}ie[1 ;N]

[0192] These can be convolutions or layers of the neural network. Indeed, the network may include convolutions in its initial layers, or fully connected layers at the end of the model. In the latter case, these are neurons connected to all the neurons in the preceding layer and to all those in the following layer.

[0193] The convolutional layers may involve scanning an input matrix, producing a series of matrix calculations. The other layers of the neural network typically include matrix calculations based on the size of the input matrix. In all cases, the step involves matrix calculations.

[0194] As an example, each convolution involves matrix multiplication between an input matrix and a weight matrix, along with the application of an additional bias. Layer processing within the neural network involves a series of matrix multiplications followed by a nonlinear function to produce the output of a layer. The sequence of these operations defines the depth of the neural network.

[0195] In one implementation example, the neural network is a multilayer perceptron, known by the acronym MLP and in English as "multi-layer perceptron." In another example, the neural network can be a network equivalent to the MLP.

[0196] In one example, the neural network is configured so that the result remains unchanged under permutation of the points in the input list. This configuration is made possible by the use of aggregation functions such as the function known as "MAX POOLING" or "AVG POOLING".

[0197] In one embodiment, the neural network structure allows for the extraction of attributes from the input at different scales. The neighborhood of each point in the point cloud at different scales is successively considered to extract a reduced-dimensional encoding of the input.

[0198] This encoding is then passed through a series of fully connected layers to perform a regression on all parameters of the parametric human body model.

[0199] Successively, sampling and / or grouping steps of the neighborhood of the sampled points and then grouping of these points are repeated and encoded by convolution layers to create maps of the parameters obtained.

[0200] The method of the invention advantageously allows the creation of a function that preserves the symmetry of all input points of the network with that of the network's output. Thus, the application of the neural network to a disconnected, undirected graph can be ensured by implementing this symmetry-preserving function. This function ensures the independence of this step from the ordering of the input points.

[0201] This function also allows the implementation of autoencoders within the regression processing. All parameterized models can be derived from a generic model. One advantage of such a model is its ability to generate multiple poses of a human body based on joint parameters and shape parameters. A particular feature of such a body model is that it can be modeled by a set of joints, each with at least one rotation and one position. A joint rotation can be defined by an angle. The joint position can be associated with at least one position of at least one point on the graph, or with several points on the same graph. In another example, the joint position is obtained from an operation combining different positions on the graph, for instance, to generate a new position for a new point.This new point could be, for example, the centroid of a subset of points in the graph. Shape parameters can be used to define global shape parameters of the body.

[0202] The set of all given configurations of a set of joints in position and rotation allows us to define a given pose of the body model. Thus, it is possible to generate a plurality of poses of the body model.

[0203] According to one embodiment, the parametric modeling of the human body is a mesh network of points. Each parameterized joint generates a configuration associating a subset of points from the mesh network of points.

[0204] According to this last example, 24 joints can be defined from 72 rotation parameters, each joint being associated with 3 rotations. This parametric model can also include 3 global rotations. A subset of joints can be extracted to implement an embodiment of the invention, such as the wrist, elbow, etc.

[0205] Each joint can also include a set of shape parameters such as wrist width, wrist length, etc.

[0206] In one example, a principal component analysis (PCA) function extracts general parameters whose configuration allows for action on a set of parameters across multiple joints. For example, an individual's height, ventral zone thickness, body mass index, pelvic size, etc., can be configured to preset a set of local parameters related to multiple joints.

[0207] The parameterization of a body can include a plurality of vectors defining a shape and an angle. For example, this could be the ankle, the waist, the arm, etc. A general configuration allows obtaining a given silhouette or template of the body, which can then be parameterized.

[0208] Other parametric body models can be used with the method of the invention.

[0209] Neural network learning

[0210] The neural network trained by machine learning, better known by the English name "machine learning".

[0211] Such a machine learning algorithm might include a step of retrieving a database of high-definition human body scans. Then, it would involve steps of deforming a body template from the parametric model by applying isometric transformations. These transformations could include translations, rotations, and combinations thereof applied to different facets of the template to conform to the scans from the retrieved data. This step might also include executing a cost function to be optimized, for example, by gradient descent.

[0212] The process includes a step of aggregating the transformations performed on each scan. The principal components of the transformations are then deduced by a PCA-type method.

[0213] Applying this algorithm allows us to extract the main parameters of shape and position of a human body.

[0214] To generate a set of partial VP views of a set of parameterized body models obtained from a plurality of parameterizations of a parametric body model, a ray tracing algorithm can be applied. The algorithm can consider a large number of different viewpoints to generate a multitude of partial VP views from a given parameterized body model. One advantage is that it avoids a calibration step during the execution of the invention's method and brings the images of a patient's body closer to the real world. Multiplying the training with a large number of partial views eliminates body processing steps such as translations or rotations of the acquired data representing the patient's body.

[0215] One advantage of this solution is the ability to generate a large number of body models by varying parameters with different partial views of a body model. These partial views (VP) correspond to a subset of the point cloud defining the entire body. Another benefit is the ability to train the neural network with a multitude of configurations that can correspond to the acquisition of a real body when using the method of the invention. The viewpoints correspond to capture points from a camera, for example. To create the most comprehensive possible library of data representing body models, viewpoints are multiplied around the surface of each body generated from the parametric model. Thus, a multitude of body surfaces represented from different viewpoints are stored in memory.The set of data generated to train a neural network in order to perform the regression step is a set of partial views VP of a plurality of parameterized field models MOD_P.

[0216] One advantage is training the regression weights with the shape descriptors associated with each point of a body model as it might be perceived in reality. Shape descriptors can be obtained for each generated partial view (VP) of the body model and for each point of a given body model. The shape descriptors obtained through training allow the definition of weight matrices, also known as matrices containing weighting coefficients for each layer of the neural network. During real-time regression calculations of a patient's body model, the regression step is advantageously performed using the weight matrices defined during the training phase.

[0217] The parameterized MOD_P body models thus generated allow the neural network used to perform the REG regression step to be trained. The real-time regression then allows the extraction of parameters characterizing the body under consideration. This step makes it possible to obtain a body model whose surface can be easily extracted and to consider a complete parameterized body model, rather than just a part as seen by the camera.

[0218] One advantage of the invention is its ability to reconstruct a three-dimensional surface in real time through inference. This solution relies on the use of models trained using a deep learning technique. A key benefit is the elimination of optimization methods, such as iterative methods using a gradient descent algorithm, which require longer computation times and impose integration limitations for real-time control of the robotic arm. A further advantage of the invention is that optimization is performed during the training phase, rather than during the processing of data acquired while guiding the robotic arm.

[0219] Deep learning inference includes matrix multiplication operations with pre-trained weights.

[0220] In one embodiment, the layers of the neural network have a resolution that depends on the layer in question. The scale can become increasingly larger as one moves down through the network layers. An example of a neural network such as an MLP network can be used.

[0221] First mesh, Mi

[0222] The process therefore includes a step of extracting a meshed surface. The mesh of the resulting surface is a first mesh, denoted Mi. In one embodiment, the first mesh Mi is a mesh in which triangles are defined by points of the considered surface and their neighboring points. Figure 7 shows an example of a mesh Mi comprising a plurality of triangles formed by triplets of points n colocalized within the considered surface.

[0223] According to another embodiment, another mesh pattern can be considered in other embodiments of the invention: square quadrilaterals, pentagons and generally any type of polygon whose vertices are points of the mesh.

[0224] The Mi mesh is created after the regression operation. Before the regression step, the point cloud is considered in such a way that each point is independent of the others; the points are not linked to each other.

[0225] Second mesh, M2

[0226] According to one embodiment, the method of the invention includes a step of defining a second mesh M2 from the first mesh Mi. The second mesh M2 can advantageously be defined so as to generate semi-homogeneous triangles, for example, isosceles triangles. In this example, the triangles can be transformed according to a given algorithm, for example, using symmetry properties of the generated mesh.

[0227] Figure 8 shows an example of the realization of a second mesh M2 obtained from a transformation. The mesh M2 comprises a set of points including a first set of points n\ which remains invariant and a second set of points ni' generated by creating an angle bisector of each considered triangle to create two new isosceles triangles.

[0228] Other techniques can be used to homogenize a mesh in order to regularize it. According to another technique, equilateral triangles are generated, as shown in Figure 9. One advantage is the generation of a completely homogeneous mesh, which notably facilitates the guidance of the operator device 8.

[0229] The method of the invention makes it possible to apply transformations to the points of the mesh so that their attributes are also transformed according to the Cartesian positions of each new point generated.

[0230] As an example, the desired mesh resolution M1 or M2 is obtained based on the settings of the sensor(s) or emitter(s) at the end of the robotic arm. The resolution of the graph G1 can be configured to a variable size and is independent of the mesh resolution M1 or M2. For example, circles 36, 37, and 38 represent the desired targeting points for the laser power to be delivered to a skin surface of patient 1. One advantage of this solution is that it generates a dermatological resolution adapted to the treatment to be applied.

[0231] Thus, Figure 8 illustrates 36 circles of different sizes obtained from a mesh containing isosceles triangles of varying dimensions. A coefficient must therefore be applied at the time of firing to adjust the power applied to the signal. According to another example, the second mesh M2 can be homogenized through a series of geometric adjustments to obtain triangles of equivalent dimensions.

[0232] Figure 9 represents a targeting point 37 or 38 of a laser power to be delivered onto a skin surface. One advantage of this mesh configuration is the ability to obtain the same firing pattern from one triangle to another. The triangles 38 are tangent to each other and allow control of an overlap rate parameter.

[0233] According to one embodiment, a new mesh M3 is defined by circles 36, 37, 38 generated from the second mesh M2.

[0234] Real-time control

[0235] According to one embodiment, the acquisition device 5 performs in real time an ACQ acquisition of points of all or part of the body of a patient 1 in order to calculate a parameterized body model MOD_P from the process of the invention which is adapted to control the movements of a robotic arm 6. The guidance of a robotic arm 6 is carried out from a treatment trajectory TRAJ1, noted 30 on the figure 4, established within a defined area 9 of the surface thus modeled and recalculated.

[0236] The TRAJ2 trajectory of the robotic arm 6 can be automatically generated from the TRAJ1 treatment trajectory based on an orientation and a relative distance between the head 8 of the robotic arm 6 and the skin surface, which can be defined, for example, using a user interface (UI). The orientation is set to 90° by default, meaning that the principal axis of a sensor or transmitter on the head 8 of the robotic arm 6 coincides with the normal (N) to the target point on the body surface, specifically the parameterized body model MOD_P, which was previously generated. The distance corresponds to the distance between a point on the sensor or transmitter of the head 8 of the robotic arm 6 and a target point, denoted as PCIBLE, on the generated modeled body surface at the patient's position.

[0237] One advantage of real-time calculation of the modeled surface of a patient's body is the ability to anticipate, for example, patient's movements and to account for movements resulting from their breathing. The advantage of the invention's method is its ability to generate a new modeled surface in less than 30 milliseconds. These response times make it possible to control the movements of a robotic arm with sufficient speed to anticipate collisions or sudden movements and move away as quickly as possible. The camera acquisition settings can be configured, for example, from 10 to 90 frames per second.

[0238] The real-time control of the robotic arm 6 enables the activation of at least one transmission from a transmitter and / or one acquisition from a sensor of an operator device 8. Activation occurs when the orientation of the sensor or transmitter axis is perpendicular to the line passing through the target point PCIBLE. The target point PCIBLE is referenced on the generated parameterized body model. This model is generated at the position and orientation of the operator device 8. The target point PCIBLE can be obtained before the guidance of the robotic arm 6, for example, during the planning of a laser firing strategy or an ultrasonic firing. Alternatively, the target point PCIBLE is obtained during guidance, for example, when a comparison of acquired images triggers a change in guidance to activate another detector.

[0239] In all cases, the target point PCIBLE is defined according to a guidance strategy and its position can be corrected when the surface of the body under consideration undergoes a transformation requiring an adaptation of the guidance strategy.

[0240] According to one embodiment, the ACQ acquisition performed by the device 5 can also be coupled with an image acquisition performed by an optic arranged at the end of the robot arm 6. This can be the case when the movements of the robot arm 6 mask the area of ​​interest of the patient's body 1 to be acquired by the point acquisition device 5 or when movements of small amplitudes are controlled by the distal optic of the robot arm 6.

[0241] Thus, two control modes can be implemented and possibly combined to generate a new modeled surface during the execution of a process.

[0242] Robotic arm

[0243] The robot arm 6 is guided by a motion controller GEN_TRAJ, which takes into account various constraints to control the robot arm. Specifically, this controller considers a planned, configured trajectory and a user-defined area to be processed. This area is defined on the body surface reconstructed according to the invention's method. Furthermore, the CTRL controller can take into account a command from a collision detection module to adapt the robot arm's movement in real time. Finally, the CTRL controller considers data from the robot head 6 via a CNTR_8 module, for example, when the detection of a unique image requires specific robot manipulation.

[0244] In one embodiment, the robot arm 6 comprises a set of arms 61, 62, 63, 64, 65 articulated by means of pivot joints 612, 623, 634, 645, 656. In this case, the robot arm 6 comprises 5 arms articulated by means of pivot joints. In another variant, it comprises 6 articulated arms, and in other variants, it may comprise 7, 8, or 9 articulated arms. Indeed, in this embodiment, each joint is capable of performing at least one rotation between two adjacent arms.

[0245] In one embodiment, each arm of the robotic arm is equipped with a force sensor. One advantage is the prevention of sudden collisions. When contact is established between an arm of the robotic arm and an object, the robotic arm's movements can be automatically stopped or its guidance modified so that it changes its trajectory.

[0246] According to one embodiment, a calibration step of the robot arm 6 is performed prior to the method of the invention. The calibration may include the relative positioning of the robot arm with respect to the reference frame Ro as well as its initial position.

[0247] Operator device and optics

[0248] According to one embodiment, the robot arm 6 includes an operator device 8 advantageously arranged at the end of the robot arm 6. According to the embodiments, the optics 67 can be arranged so as to be offset with respect to the lower robot head 8 or it can be arranged at the end of this head as in the case of Figure 6A.

[0249] One advantage of the configuration shown in Figure 3 is that it allows the robotic arm head to be equipped, for example, with a laser or ultrasound device for localized treatments, and the image acquisition optics to be relocated. Another advantage of the configuration shown in Figure 6A is that it allows a robotic arm to be configured for dermatological imaging applications. However, all configurations are possible, as well as combinations of the previous ones.

[0250] As an example, the operator device 8 includes at least one sensor and / or at least one transmitter. The operator device 8 is also called the "head" of the robot arm 6.

[0251] In the example shown in Figure 5, the robotic arm 6 is equipped with one or more optical sensors 67 to acquire local images of the skin surface. One advantage of acquiring local images at the head of the robotic arm 6 is the possibility of reconstructing a "colorized" body surface, that is, one in which the acquired images are integrated into the parameterized body model MOD_P generated by the process of the invention. To this end, the optical device located at the end of the robotic arm 67 or {83; 84} may preferably comprise two optics so as to reconstruct a depth map associated with the images acquired simultaneously by the two optics. The images can then be indexed onto the parameterized body surface.

[0252] Other issuance / acquisition devices

[0253] Furthermore, the operator device 8 may include an ultrasonic, radio frequency, or laser transmitter, or any other signal source capable of generating an incident beam. In one embodiment, the operator device s may include an array of focused transmitters and means for controlling the direction and power emitted.

[0254] When an ultrasound transmitter is arranged at the end of the operating device 8, an ultrasound receiver can be associated to measure the reflected signal in order to construct an image of the dermis and / or an image of the skin thickness of a patient, or even an image of vessels or an organ.

[0255] Figure 6 shows another embodiment in which the operating device 8 includes a receiver head 80 positioned at the end of said operating device. The receiver head 80 has two local optics 83, 84 for taking images of the body surface and a removable bidirectional millimeter or microscopic optic 86 in a longitudinal direction. One advantage is the ability to generate images with millimeter resolution suitable for dermatological applications, for example, for diagnostic applications. Robotic arm guidance

[0256] Initially, the generation of the movement of the robot arm 6 is defined by means of a trajectory configured by the user using a component labeled PATH in Figure 2. The user can select an area of ​​interest on the surface of the body of a patient 1, such as an area: "an arm", "a leg" or a portion of this area or even the whole body.

[0257] An area can be selected using a user interface. The area can be outlined using a stylus or mouse, or by selecting one or more predefined areas. As another example, the selection can be made using a touch command directly on a display or via voice command.

[0258] The selection can be performed on a parameterized body model MOD_P generated by the process of the invention 10. The module for selecting an area is labeled ROI in Figure 2. The parameterized body model MOD_P can be displayed on a screen of the user interface Ul. When an area is selected, a portion of the surface can be extracted, on which the trajectory of the operator device 8 will be defined. For example, a trajectory 30 can be automatically defined from the dimensions of a selected area. For instance, depending on the operation to be performed with the operator device 8, such as image acquisition, laser processing, and / or processing from an ultrasonic source, the automatic trajectory generation function can be implemented in various ways.When capturing images over a predefined area, the goal is to cover the entire selected zone. Therefore, an optimized trajectory can be calculated based on the lens's resolution and field of view.

[0259] When a user defines a trajectory on the selected area, the trajectory is acquired by the PLAN_TRAJ component which will optimize or calculate the stroke parameters of the robot arm to cover the selected surface.

[0260] The method of the invention enables real-time guidance of the robotic arm 6. To this end, a GEN_TRAJ motion generation module for the robotic arm 6 is implemented. In order to calculate the trajectory in real time based, for example, on patient movements, the capture of a specific image, or the control of an end-of-travel limit switch in case of contact with an object, the module allows for the real-time recalculation of the robotic arm 6's travel.

[0261] The guidance of robot arm 6 is configured, for example, using a Cartesian coordinate system referenced in the Ro frame. The orientation of the end of robot arm 6 can be configured by controlling the orientation of the normal N shown in Figure 3 or 5. The normal N is defined by a 90° angle with the line of sight on the body surface. One advantage of this parameterization is that each point in the enriched mesh already includes an attribute corresponding to the normal of a point in the model. Thus, the orientation of the robot arm head can be guided by aligning the robot's orientation with the normal N of the body surface.

[0262] According to one embodiment, the distance between the tip of the head 8 of the robotic arm 6 and a sighting point 35 on the modeled surface of a patient's body 1 is controlled to remain substantially constant, for example at a distance of 5 cm. Thus, the guidance of the robotic arm 6 is controlled according to an orientation and a distance to the skin of patient 1.

[0263] A collision detection function DET_COLLI can then be implemented to provide a control command to the motion generation module GEN_TRAJ of the robot arm 6. Furthermore, this module takes into account the initial trajectory established by the PLAN_TRAJ component. Finally, the movement of the lower robot can also be controlled according to a CNTR_8 control function of the head 8 of the robot arm 6. When one or more optical devices are mounted at the head of the robot arm 6, servo control of the robot arm may be necessary to activate a second optical device.

[0264] In one embodiment, the optical system 5 is configured to control each movement of the body 1 by means of a control loop. This control allows for the comparison of differences between images acquired at a given time with new images acquired consecutively. The purpose of such control is to modify the guidance of the robot arm 5 by adapting its initial stroke to a new trajectory. To this end, a collision detection function DET_COLLI can be implemented to adapt the movement of the robot arm 6's stroke. The generation of the robot arm 6's movement is determined by the function GEN_TRAJ. In one embodiment, the head of the robot arm 6 is removable so that it can be used for different applications.A head with one or more optical devices for dermatological applications can, for example, be replaced by a head with a laser for skin surface treatment applications, such as hair removal, tattoo removal, or Ultherapy for lifting applications.

[0265] Depending on the intended application, a given optical configuration may be preferred.

[0266] According to one example, an arrangement of optics having different focal points is implemented at the end of the head of the robot arm 6. These optics can be articulated or moved independently or jointly depending on the embodiments.

[0267] As an example, at least one optic is a confocal microscopy system. As another example, the focal plane can be shifted to produce images at different depths. A local three-dimensional image of the skin surface can be reconstructed. This application can be useful for reconstructing a volume, such as that of a mole on the skin's surface.

[0268] In the case of using confocal optics, illumination can be obtained from a light source, such as a laser.

[0269] In another example, at least one optical system is configured to acquire a series of cross-sectional images to implement a tomographic technique. Depending on the configuration, this technique may be OCT, also known as optical coherence tomography. These techniques often involve coupling a sensor with an emitter, such as a laser emitter.

[0270] Other possible applications involve treatments using ultrasound emissions. For this purpose, at least one ultrasound emitter can be positioned at the head of the robotic arm. This emitter, or these emitters, can be combined with a sensor to measure an echo signal or a change in the shape of the emitted wave. For example, this could be an array of emitters arranged to generate a beam focused to a single point. The power levels can be configured to burn tissue or to measure the elasticity of a skin lesion by creating acoustic pressure at the skin surface. In one embodiment, the emitter is configured to treat a subcutaneous area. An image point can be defined with reference to the PCIBLE target point, this image point being located on the surface of the body model.The image point is, for example, located a few millimeters, or even a few centimeters below the skin surface along the normal to the surface of the model passing through the target point PCIBLE.

[0271] Transmitter guidance can be achieved using the PCIBLE target point and by constructing an image point. Alternatively, transmitter guidance can be achieved directly by defining a point in space to be reached and defining the image point, this image point being located under the patient's skin. When this point is not defined relative to the PCIBLE target point, the PCIBLE target point is used to acquire an image or to plan a modified robot trajectory.

[0272] The head of the robotic arm can also be equipped with a sensor to probe a mechanical property of the skin, such as its elasticity, firmness, hydration or roughness.

[0273] In another example, a fluorescence sensor is mounted at the distal end of robotic arm 6. Such a sensor allows for the tracking of biological actives, for example, in cosmetic applications. In yet another example, a spectroscopic technique can be used to perform molecular characterization.

[0274] Dermatological application

[0275] In one embodiment, the head of the robotic arm 6, also called the "operator device" 8, is equipped with an optical system for dermatological applications. In this case, a dual optical device can be arranged. A first optical device 83, 84 comprises cameras that perform a so-called "macroscopic" scan due to the movement of the robotic arm 6. These cameras capture images of interest of the skin surface. In one example, a light source illuminates the captured area. The emitted light can be in the visible or infrared spectrum, or even a fluorescent source, for example. In another example, an LED system with a polarizer is configured to emit polarized light. In the case of polarized light, the LED light source is covered with a polarizing filter. In this example, a camera arranged at the end of the robotic arm head is equipped with a polarizing filter.The filters of the light source and the camera are, in one example, configured orthogonally.

[0276] When a singularity is detected, the robotic arm 6 is commanded to alter its trajectory. The head is positioned at a point within the detected singularity, for example, the center or centroid of the region forming the singularity. Guidance is then stopped, and the initial trajectory is temporarily suspended.

[0277] A controller activates a second optical device 85, enabling millimeter or microscopic image capture with dermatological resolution. To obtain good resolution, the optical device 85 can be controlled by a movement 86 along an axis perpendicular to the plane of one of the optics 83, 84, or 85.

[0278] Figure 4 represents a portion of a surface 9 of a patient's body 1 on which a trajectory 30 is shown. The trajectory 30 can be obtained from a scanning strategy of the robotic arm 6. In one example, the generated trajectory 30 is obtained using a planning module PLAN_TRAJ. In the example in Figure 4, three dermatological singularities 91 are shown on a portion 9 of a patient's skin. These could be, for example, a scar, a melanoma, a mole, or any other skin surface singularity. When the optical field 87 of the first optical device 83, 84 intercepts a singularity 91, the acquired images are dynamically analyzed to control the robotic arm 6.

[0279] The second optical device thus allows for the acquisition of an image of a close area of ​​a patient's skin surface. For example, the second optical device is a dermatoscope. This second optical device may include a glass plate that comes into contact with the skin. In one embodiment, the second optical device may include a polarized light emitter to obtain image information deep within the epidermis.

[0280] To this end, the last arm 66 of the robotic arm 6 or the operator device 81 can be equipped with a force feedback sensor to slow down the movement of the second optical device 85, or even stop and stabilize it. When the second optical device 85 is in contact with the skin of a patient 1, a scan can be triggered. In another example, each axis of the robot is equipped with a force sensor. The analysis of the force values ​​recorded by the sensors is used to control the pressure exerted on the surface. Thus, thanks to its torque sensors in each axis, the robot can control the contact pressure between a tool and the work surface, in this case, the skin of a patient. This can also be used to perform a "glide" of the tool on a work surface.

[0281] In one embodiment, a scanning device 87 is activated to move hairs away from the lens 85. Such a scanning device 87 can be automatically activated during the stabilization of the lens of the second optical device 85. One advantage is to avoid capturing images of hairs located near a singularity that is to be imaged. The scanning device 87, in one embodiment, comprises a rotating rod that activates flexible brushes near the lens 85. One advantage is that it rotates the hair so that it does not remain between the lens 85 and the skin. In one embodiment, a plurality of scanning devices 87 are arranged around the lens 85. Figure 6B shows two of them.

[0282] Once the second optics has captured one or more images with a higher resolution of the singularity, the second device 85 can retract to return to its initial position. The robotic arm can then continue its movement and resume the planned guidance trajectory.

[0283] According to one example of implementation, guidance is carried out at a distance of between 1 cm and 20 cm from the surface of the body of a patient 1. According to a preferred mode, guidance is carried out at a distance of between 2 cm and 8 cm from the surface of the skin.

[0284] In one embodiment, at a given distance from the robot's surface, and given the optical characteristics, the field of view of the optics can be determined. These characteristics allow the coverage area of ​​the images acquired over the complex surface of the patient's body to be determined. One advantage is the ability to precisely match the images with the three-dimensional surface of the patient's body. In addition to the possible indexing of the acquired images onto the generated parameterized body model MOD_P, the images can be arranged in a given position on said model. Thus, a three-dimensional representation of a body can be archived with images accessible to a user capable of representing skin singularities. The association of the acquired images with the generated parameterized body model can be performed using the Mi or M2 mesh.When the mesh includes, for example, triangles, an image can be subdivided so as to match said triangles with the surface of the image.

[0285] In one embodiment, the invention includes a step of referencing captured images with a set of mesh nodes. In one example, each triangle in the mesh includes an identifier. The images can then be saved with data associating said image with a set of identifiers.

[0286] In one embodiment, the images acquired by the optics 83, 84, and 85 are recorded in a memory, for example, the memory of the motion controller of the robotic arm 6. Alternatively, the operator device 8 comprises an electronic board equipped with at least one computer and a memory. The acquired images can be initially recorded locally before being transmitted to a remote electronic device.

[0287] According to one embodiment, each photograph taken simultaneously by optics 83 and 84 is associated with the other. The acquired images can, for example, be time-stamped and associated with a point in space in the frame of reference Ro. They can also be associated with a point located relative to a reference frame R3 linked to the operator device of the robotic arm 6, or referenced with respect to the trajectory 30.

[0288] When an image is acquired with the second optical device 85, the latter can be associated with the images acquired by the first optical device 83, 84 and in particular with the same position of the trajectory 30 at which the operator device 8 activated a movement of the second optical device 85.

[0289] One advantage of the invention is the ability to index each acquired image to an image on the parametric model generated during the REG regression step. Thus, the method allows for the localization of an image portion on the skin and its indexing to a generated model. Another benefit is the preservation of a precise record of the photographed area relative to the body's geometry. Therefore, when a patient changes position or returns for a subsequent visit, the recorded image indexing allows for the direct generation of a new pointing to the already indexed area of ​​interest based on a new image acquisition.

[0290] One benefit is to create a bank of images of different resolutions of different portions of the surface of the body under consideration.

[0291] To ensure accurate response from the robot arm 6 during real-time image acquisition by the operator device, the acquired images can be compared to a database of characteristic images. As another example, a trained neural network can classify the acquired images and generate a command based on the classification results. For instance, detecting certain singularities does not require activating the second optical device 85, while detecting other singularities requires capturing images with a higher resolution.

[0292] According to another implementation example, the robotic arm 6 executes a first planned trajectory 30 with image coverage acquired by the first device 83, 84. The robotic arm 6 then performs a second image acquisition at the points where a singularity has been detected. When a first trajectory 30 is executed to guide the robotic arm 6 over a portion selected by an operator and a set of images acquired by the first optical device is obtained, the robotic arm 6 initiates a second trajectory 31. This trajectory can be automatically generated from the singular points identified during the execution of the first trajectory 30. In order to collect a second series of images at identified points, the second device 85 is automatically activated at characteristic points of the second trajectory 31.

[0293] Laser application

[0294] In another application, the robotic arm 6 includes a laser emitter at its distal end. The laser emitter can be, for example, mounted on the distal end of the operating device 8. The latter can be used for hair removal, the treatment of certain skin conditions, or tattoo removal. Furthermore, laser therapy or any medical or cosmetic dermatological treatment using a laser can be performed with a robotic arm equipped with a laser source. For this purpose, the configuration of the laser source can be adapted according to the application, based on parameters such as power, wavelength, pulsed or continuous mode, etc.

[0295] In the case of laser source control, an M2 mesh can be advantageously used to optimize the laser firing configuration, particularly for controlling firing power. Indeed, a homogeneous mesh, comprising, for example, equilateral triangles, allows for the definition of a mesh of circles circumscribed about or inscribed within these triangles. One advantage of a mesh of circles circumscribed about the triangles is that it defines overlap between the circles and therefore between the shots. This configuration allows for the definition of target firing zones adjacent to a given target zone. An advantage of constructing tangent circles directly from a homogeneous mesh is the ability to generate a target zone for a laser shot while managing a homogeneous overlap parameter between two consecutive shots on neighboring targets.

[0296] The invention allows the coverage rate to be controlled in order to compensate for the inhomogeneity of the energy quantity of a laser beam around the main firing axis. Thus, it becomes possible to deposit a substantially homogeneous amount of energy over a given area with a homogeneous mesh.

[0297] In the case of multiple laser pulses required to treat a specific skin area, the guidance system of the robotic arm 6 can be configured to stop at a set of points and deliver successive pulses according to a predetermined trajectory plan. This trajectory plan (PLAN_TRAJ) is defined via a user interface (UE). The trajectory of the robotic arm 6 can be adjusted in real time to account for factors such as laser source configuration, patient movement, or arm collision detection.

[0298] The system of the invention includes a means for defining laser firing parameters, namely a firing power and / or a target diameter on the skin surface, as well as a coverage rate. The laser emitter and the guidance path can be configured to achieve these objectives. In one embodiment, the resolution of the homogeneous mesh can be adapted according to a given firing configuration. Typically, when a laser firing area is defined with a maximum target radius on the skin surface, the mesh is configured to define circles whose dimensions correspond to the firing constraint.

[0299] In each application, during real-time guidance, the robotic arm 6, which follows a given trajectory, is required to evaluate the trajectory in real time. This evaluation is performed using a computer and the implementation of the method of the invention.

[0300] When the body is stationary, the recalculated images of the body surface used to position the robotic arm 6 involve only rotations of a previously calculated surface; that is, the transformation is isometric and corresponds to one or more rotations of the robotic arm depending on the portion of its trajectory. The generated position of the body surface is calculated for a specific position of the robotic arm's head.

[0301] When the body moves, the recalculated images of the body surface used to position the robotic arm 6 involve a non-isometric transformation; the surface is recalculated using the method of the invention. In addition to accounting for any rotations, the method of the invention allows for regression on the shape descriptors and regeneration of the body surface from the perspective of the position of the robotic arm's head.

[0302] According to one embodiment, particularly in the context of an arrangement of a laser or ultrasonic emitter on the head 8 of the robotic arm 6, when a region of interest is smaller than a given surface area threshold, a mesh optimization step for the surface of interest can be implemented. This optimization can be performed so that the transformation in this region is considered isometric. In one example, the method of the invention makes it possible to determine key points on the considered surface of the body in order to delimit a region of interest that has undergone an isometric shape transformation.

[0303] One advantage of the invention is that it can be adapted to dermatological, surgical, cardiological, etc. applications.

Claims

DEMANDS 1. A real-time guidance method for a robotic arm (6) for processing surface data of a body, comprising the steps of: ■ Generation of a 3D model (MOD_P) of all or part of a surface of a human body comprising a mesh (Mi, M2) of points; ■ Planning a processing trajectory (TRAJ1) on the surface of the body model (MOD_S) using a computer; ■ Calculation of a guidance trajectory (TRAJ2) of the distal end of the robot arm (6); ■ Generation of a kinematic (PILOT_ROBOT) of the robot arm (6) such that a point of the head arranged at the distal end of the robot arm (6) is servoed to the guidance trajectory (TRAJ2); ■Activation of the kinematics of the robot arm (6) to travel the guidance trajectory (TRAJ2), said guidance trajectory (TRAJ2) being calculated in real time from the 3D model (MOD_P) of the surface of the human body; ■ the generation of a new 3D model (MOD_P) of the surface of the human body at regular times ensuring the calculation of a new kinematic of the robotic arm (6) to control its movements automatically in real time, a new calculated surface of the 3D model (MOD_P) leading to a calculation of a new processing trajectory (TRAJ-T) and a calculation of a new guidance trajectory (TRAJ2'), ■ Receiving at least one position of a target point (PCIBLE) in space; ■Activation of at least one emission from a transmitter and / or an acquisition from a sensor of an operator device (8), said operator device (8) being arranged at the distal end of the robotic arm (6), the activation being carried out when the orientation of the axis of the sensor or transmitter coincides with a predefined line passing through the target point (PCIBLE), said target point (PCIBLE) being referenced on the generated body model (MOD_P).

2. Guidance method according to claim 1, characterized in that the processing trajectory (TRAJi) is generated within a processing zone of a surface of the body model (MOD_S).

3. Guidance method according to claim 2, characterized in that the processing trajectory (TRAJi) is automatically generated from the definition of a processing zone (34, 35) defined on a surface of the body model (MOD_S).

4. Guidance method according to claim 1, characterized in that the guidance trajectory (TRAJ2) is calculated from a predefined distance and orientation setpoint allowing to generate a relative position of a distal end of the robot arm (6) with respect to the treatment trajectory (TRAJ1, TRAJ-T).

5. Guidance method according to claim 1, characterized in that the robot arm (6) comprises at least 3 branches (61, 62, 63, 64, 65, 81) articulated in pairs, one with respect to the other by a pivot joint (612, 623, 634, 645, 656), the set of pivot joints comprising a planned kinematics for a given guidance trajectory (TRAJ2) and a generated body model (MOD_P).

6. Guidance method according to claim 1, characterized in that it comprises the generation of a new body model (MOD_S) at regular times so that the robot arm (6) can be controlled in real time from the generation of a new trajectory (TRAJ2') of the robot arm (6), a new calculated surface of the body model resulting in a modification of the processing trajectory (TRAJ-T), the method further comprising a calculation of a new kinematic of the robot arm (6) controlled on the definition of the new trajectory (TRAJ2').

7. A guiding method according to any one of claims 1 to 6, characterized in that it comprises a position calibration step of the robotic arm (6) from a fixed position at which images are acquired allowing the generation of a body model (MOD_S).

8. Guidance method according to claim 1, characterized in that the processing trajectory (TRAJi, TRAJ-T) is used to establish a firing strategy of a laser beam from a laser emitter, said laser emitter being fixed to the operator device (8).

9. Guiding method according to claim 8, characterized in that the body model comprises a homogeneous mesh (M2) so as to create a new mesh (M3) defined by an arrangement of circles (38), at least one circle defining the position of a target point (PCIBLE) in space.

10. Guidance method according to claim 9, characterized in that the new mesh (M3) is generated as a function of a firing power setpoint and / or a setpoint relating to an overlap rate between two shots. 1 1. Guidance method according to claim 10, characterized in that the sampling of the number of shots made is a function of the new mesh (M3), the configured firing power and the overlap rate defined between two colocalized circles.

12. Guidance method according to claim 1, characterized in that the processing trajectory (TRAJ1, TRAJ-T) is used to establish a firing strategy of an ultrasonic signal from an ultrasonic transmitter, said ultrasonic transmitter being fixed to the operator device (8).

13. Guidance method according to claim 1, characterized in that a set of images is produced from an optical device (83, 84, 86) arranged on the operator device (8), each image (IM1, IM2) being indexed to at least one node or mesh of the mesh (Mi, M2, M3) of the body model (MOD_P) as a function of the position of the target point (PCIBLE) on said body model (MOD_P) and a viewing angle of the optical device (83, 84, 86).

14. Guidance method according to claim 1 3, characterized in that at least one pair of images is acquired simultaneously from an optical device (83, 84) comprising two sensors joined and arranged on different axes of the operator device (8), each image (IM-i , IM2) being indexed to at least one node or mesh of the body model mesh (MOD_P) as a function of the position of the target point (PCIBLE) on said body model (MOD_P) and a viewing angle of the optical device (83, 84, 86), said images being processed so as to reconstruct a local three-dimensional image.

15. Guiding method according to claim 14, characterized in that the local three-dimensional image is superimposed in a 3D representation on a generated body model.

16. A guidance method according to any one of claims 1 to 15, characterized in that a first image acquired by a first optical device (83, 84) of the operator device is compared to a set of images in real time, the comparison operation comprising, where appropriate, the generation of an indicator of interest, said indicator of interest allowing the generation of a digital control command for piloting the robot arm (6).

17. Guidance method according to claim 16, characterized in that the guidance command causes a first piloting of the robot arm (6) to the target position from which the first image is extracted and a second piloting of the operator device (8) to activate a second optical device (85), the activation of the second optical device (85) allowing a second near-field image to be taken of an area containing the target point (PCIBLE).

18. A guidance method according to any one of claims 1 5 to 16, characterized in that the comparison step is a classification step of said image acquired from a neural network trained on a set of images.

19. A robotic arm guidance system (6) comprising: ■ a robotic arm (6) controlled by a computer of a control unit calculating a kinematic of the robotic arm (6) from a trajectory calculated from points defined on a mesh of a 3D model of an acquired human body surface (Mi, M2); ■a computer to generate a plurality of body models (MOD_P) at different times, the generation of a new 3D model (MOD_P) of the surface of the human body at regular times ensuring the calculation of a new kinematic of the robotic arm (6) to control its movements automatically in real time, a new calculated surface of the 3D model (MOD_P) leading to a calculation of a new trajectory (TRAJ1', TRAJ2'); ■ an operator device (8) arranged at the end of the robot arm (6) comprising at least one transmitter or sensor, the activation of at least one transmitter or sensor being carried out when the orientation of the axis of the sensor or transmitter is directed along a straight line passing through the target point (PCIBLE), said target point (PCIBLE) being referenced on the 3D model of a generated human body surface (MOD_P).

20. A robotic arm guidance system (6) characterized in that it comprises an image acquisition device (5) for a patient's body to produce a parameterized body model (MOD_P) and a robotic arm (6) equipped with an operator device (8) for implementing the method of any one of claims 1 to 18.

Images