Method and guidance system for guiding a robot arm

Abstract

A method for guiding a robot arm (6) for processing body surface data in real time. The method comprises: generating a body model (MOD_P) comprising a grid of points (M1, M2); planning a treatment trajectory (TRAJ1) on the surface of the body model (MOD_S) by means of a computing device; triggering at least one emission of a transmitter and / or acquisition of a sensor of an operating device (8) arranged at the distal end of the robot arm (6), the triggering being performed when the orientation of the axis of the sensor or transmitter merges with a predefined straight line through a target point (PTAT), the target point (PTAT) TARGET ) is used as a reference on the generated body model (MOD_P).

Description

Technical Field

[0001] The field of the invention relates to the field of methods that make it possible to construct a three-dimensional image of the human body in order to dynamically control devices close to the body. The field of the invention more specifically relates to methods for achieving real-time dynamic control that requires rapid three-dimensional surface calculations. The invention therefore relates to methods for generating surfaces from algorithms that combine parameterized models and algorithms that learn through artificial intelligence techniques. Background Art

[0002] There are currently different techniques for reconstructing three-dimensional surfaces in space. However, the latter are not suitable for real-time guidance of medical devices intended to perform treatments close to the surface of a patient's skin. Indeed, reconstructing the body surface requires image processing, which requires significant computing resources and often a large number of signal acquisition systems. Moreover, this complexity often results in local processing or approximations that rely on manual assistance.

[0003] Until now, this limitation has made it impossible to carry out complex guidance of the device in space. Generally, it remains difficult to achieve a robotic arm that can be guided in space by calculation of the surface to be treated and can undergo deformations and displacements. Despite this, there is still a need to manipulate a reactive robotic arm that is automatically guided by a function that performs specific analyses in real time. Until now, existing solutions have required an additional operator for manipulation or long processing times.

[0004] Therefore, a solution that can address this problem needs to be identified. Summary of the invention

[0005] The object of the present invention is to solve the above-mentioned disadvantages.

[0006] According to one aspect, the invention relates to a method for real-time guidance of a robotic arm for processing data of a surface of a body, comprising the following steps:

[0007] generating a model of the body comprising a mesh formed of points;

[0008] The treatment trajectory is planned by computer on the surface of the body model; sensor parameters

[0009] calculating a guidance trajectory of a distal end of the robotic arm;

[0010] Generate the dynamics of the robot arm so that a point set at the distal head of the robot arm is constrained on the guide trajectory;

[0011] triggering the dynamics of the robotic arm to cover the guidance trajectory, which is calculated in real time from the generated body model;

[0012] receiving at least one position of a target point in space;

[0013] The acquisition signal of the sensor of at least one transmitting and / or actuating device of the triggering transmitter is arranged at the distal end of the mechanized arm, and the triggering is performed when the direction of the axis of the sensor or the transmitter merges with a predetermined straight line passing through a target point, which is referenced on the generated body model.

[0014] An advantage is that a guidance method is used that takes into account a body model that reproduces the patient's body. Computation is reduced and real-time control of medical devices is enabled. The reconstruction of the real world is optimized by using a meshed body model calculated from the patient's body. An advantage is that potential patient movements or prediction of potential collisions are taken into account by using an easily available body model. The body model used comprises a set of points or a grid that can be more easily utilized for real-time guidance, rather than systems based on acquiring direct images of the patient, to predict the robot's movements.

[0015] According to one aspect, the invention relates to a method for real-time guidance of a robotic arm for processing data of a surface of a body, comprising the following steps:

[0016] generating a 3D model of all or part of a human body surface including points forming a mesh;

[0017] The treatment trajectory is planned on the surface of the body model by computer;

[0018] calculating a guidance trajectory of a distal end of the robotic arm;

[0019] generating the dynamics of the robot arm so that a point of a head disposed at a distal end of the robot arm is constrained on a guide trajectory;

[0020] Triggering the dynamics of the robotic arm to cover the guidance trajectory, which is calculated in real time from a 3D model of the human body surface;

[0021] The generation of a 3D model of the human body surface at a fixed moment ensures the calculation of a new kinematics of the robot arm to automatically manage its displacement in real time, the new surface of the calculated 3D model brings about the calculation of a new treatment trajectory and a new guidance trajectory;

[0022] receiving at least one position of a target point in space;

[0023] The acquisition signal of the sensor of at least one transmitting and / or actuating device of the triggering transmitter is arranged at the distal end of the mechanized arm, and the triggering is performed when the direction of the axis of the sensor or the transmitter merges with a predetermined straight line passing through a target point, which is referenced on the generated body model.

[0024] According to an embodiment, the body model is generated from a system for acquiring images of the patient's body, the system being arranged at a point in space. One advantage is that the acquisition of partial views of the human body makes it possible to generate the entire human body from an algorithm based on the use of a neural network.

[0025] According to an embodiment, a treatment trajectory is generated within a region for treating the surface of the body model.

[0026] One advantage is that the guidance trajectory is defined to be generated relative to the surface of the body model generated as input. Hence, the guidance is optimized due to the utilization of a simplified meshed body model.

[0027] According to an embodiment, the treatment trajectory is automatically generated from the definition of a treatment area defined on the surface of the body model.

[0028] One advantage is that the operator's task is automated and the operator only needs to select a region. The region is defined on the generated body model. Therefore, its definition is simplified and trajectories can be easily generated.

[0029] According to an embodiment, the guidance trajectory is calculated based on predetermined distance and direction instructions, making it possible to generate a relative position of the distal end of the robot arm with respect to the treatment trajectory.

[0030] One advantage is that a simplified guidance is obtained by a control system taking into account a minimum of parameters.

[0031] According to one embodiment, the robotic arm comprises at least three branches articulated in pairs, one branch corresponding to the other branches via a pivot connection, the pivot connection group comprising planned dynamics, the planned dynamics being used for a given guidance trajectory and a generated body model.

[0032] One advantage is the generation of the kinematics of the separate robotic arm, which can be executed in the computer of the robotic arm as a function of the position of the guide point of the trajectory generated from the operator console.

[0033] According to an embodiment, the guidance method comprises generating a new body model at a fixed time so that the robot arm can be directed in real time by the new trajectory generated from the robot arm, the new surface calculated from the body model brings about the modification of the treatment trajectory, the method further comprises: calculating new dynamics of the robot arm, the new dynamics being based on the definition of the new trajectory. An advantage is that a good real-time guidance performance is obtained. Taking the body model into account simplifies the calculation of the guidance. The robot arm can be quickly controlled and thus makes it possible to limit the occurrence of situations of collision or sudden movement of the patient.

[0034] According to an embodiment, the guidance method comprises a step of calibrating the position of the robot arm from a fixed position where the images are acquired, making it possible to generate a body model. An advantage of this calibration is that the position of the body model corresponding to the position of the robot arm end is known.

[0035] According to one embodiment, the treatment trajectory is used to establish a firing strategy for a laser beam from a laser emitter that is fixed to the effector device.

[0036] According to an embodiment, the body model comprises a uniform network so as to create a new grid defined by an arrangement of circles, at least one circle defining the position of a target point in space. An advantage is that the triggering at the robot head is simplified as a function of a point that can be easily precisely located on the grid.

[0037] According to an embodiment, the new network is generated as a function of an indication of the transmission power and / or an indication of the amount of overlap between two transmissions.An advantage is that a grid adapted to the use case of the mechanized arm can be obtained.

[0038] According to an embodiment, the sampling of the number of transmissions performed is a function of the new network, the set transmission power and the amount of overlap defined between the two co-located circles. An advantage is that an easy to use console is provided for the operator.

[0039] According to one embodiment, the treatment trajectory is used as a strategy for establishing the transmission of ultrasound signals from an ultrasound transmitter that is fixed to the effector device.

[0040] According to one embodiment, a set of images is generated from an optical device disposed on an actuator, each image being indexed by at least one node or mesh of a mesh of a body model that is a function of the location of a target point on the body model and the viewing angle of the optical device. One advantage is that an outer layer such as skin is reconstructed on a parameterized or parametric model corresponding to the patient's body.

[0041] According to one embodiment, at least one pair of images is obtained simultaneously from an optical device comprising two sensors integrated and arranged on different axes of an actuator, each image being indexed by at least one node or mesh of a grid of a body model, the grid being a function of the position of a target point on the body model and the line of sight angle of the optical device, the images being processed to reconstruct a three-dimensional local image.

[0042] According to an embodiment, the three-dimensional partial image is superimposed on the generated 3D representation of the body model.

[0043] According to one embodiment, a first image acquired by a first optical device of the actuator is compared in real time with a set of images, the comparison operation comprising: if appropriate, generating an indicator of interest making it possible to generate digital commands to manage the robot arm.

[0044] According to one embodiment, the guidance command brings about a first management of the robot arm at the target position from which the first image is extracted and a second management of the actuator to trigger the second optical device, the triggering of the second optical device making it possible to generate a second image in the near field of the area including the target point. An advantage is that an image library is generated, which can be used in particular for diagnostic purposes, qualitative studies of patients and diseases, or alternatively a patient image history including skin information covering different scales.

[0045] According to an embodiment, the step of comparing is a step of classifying images obtained from a neural network trained on a set of images.An advantage is that the step of detecting singular points on the patient's skin is optimized.

[0046] According to another aspect, the present invention relates to a system for guiding a robotic arm, comprising:

[0047] a computer for generating multiple body models at different times;

[0048] a robotic arm controlled by a computer of a control unit, the computer of the control unit calculating the kinematics of the robotic arm from trajectories calculated from points defined on a mesh of the obtained body model;

[0049] An actuator arranged at the end of the robot arm includes: at least one transmitter or sensor, which is triggered when the direction of the axis of the sensor or transmitter points along a straight line passing through a target point, and the target point is used as a reference on the generated body model.

[0050] According to one aspect, the invention relates to a system for guiding a robotic arm, comprising a device for acquiring an image of a patient's body in order to generate a parametric body model, and comprising a robotic arm equipped with actuators for implementing the method of the invention.

[0051] According to different optional embodiments, the system of the present invention includes the devices described in different embodiments. Further, according to different schemes, it is configured according to different embodiments of the method of the present invention.

[0052] According to one aspect, the present invention relates to a system for guiding a robotic arm, comprising:

[0053] a robotic arm controlled by a computer of a control unit, the computer of the control unit calculating the kinematics of the robotic arm from trajectories calculated from points defined on a grid of the obtained 3D model of the human body surface;

[0054] The actuator arranged at the end of the robot arm includes: at least one transmitter or sensor, which is triggered when the direction of the axis of the sensor or transmitter points along a straight line passing through a target point, and the target point is used as a reference on the generated 3D model of the human body surface.

[0055] According to another aspect, the present invention relates to a method for generating a three-dimensional working surface of a human body, comprising the following steps:

[0056] receiving input data corresponding to geometric data;

[0057] generating a first point cloud based on the input data, each point being associated with a three-dimensional space coordinate;

[0058] determining a plurality of attributes for each point of the first point cloud, the attributes comprising at least one shape descriptor, at least one attribute of a point comprising data describing a normal vector of the point in the point cloud;

[0059] Associating the attribute with each point to generate a second enriched point cloud;

[0060] calculating a set of geometric parameters by regression, the geometric parameters comprising at least one rotation of at least one point of the second point cloud or of at least one joint of the human body model, and at least one shape coefficient, the regression being performed according to a series of matrix operations performed according to different layers of a neural network trained from a plurality of local views of the parametric model parameterized with different parameter configurations;

[0061] A parameterized model is determined to generate a body model including a first mesh.

[0062] According to an example, the input data comprises data describing an image. For example, the geometric data may be derived from data describing a 2D image. According to one case, the input data is therefore an image.

[0063] One advantage of the method is that it enables real-time control of guided medical devices close to the patient's body and with a minimal image acquisition system comprising, for example, a pair of cameras. Due to the learning method of a high-performance neural network, a remarkable guiding performance is obtained.

[0064] According to one embodiment, the method further comprises processing the first grid to generate a second uniform grid. One advantage of using a uniform grid is that it facilitates control commands of one or more sensors or one or more transmitters at the head of the manipulator. In fact, processing the points of the uniform grid makes it possible to facilitate the use of the equipment at the head of the manipulator.

[0065] According to an embodiment, a shape descriptor is calculated at at least one given point of the first point cloud from the adjacency matrix of the first graph by means of a method for spectral segmentation of the first point cloud. One advantage is that the neighborhood of the point of the shape descriptor weighted according to a distance criterion at the point in question is taken into account. Another advantage is that it is possible to take into account the topology of the local surface in the neighborhood of the point.

[0066] According to an embodiment, at least one local shape descriptor and / or global shape descriptor is of the following type: "fluctuation kernel feature" or "thermal kernel feature" or "Gaussian curvature". An advantage is that the shape descriptor is defined based on a spectral analysis of an area in the neighborhood of a certain point. An advantage is that the use of formulas or normalization operators is simplified, which normalization operators can optimize digital processing.

[0067] According to one embodiment, the calculation of the normal vector of a given point of the first point cloud is performed by considering the locally generated faces around each considered point from the points in its neighborhood. An advantage is that the required resolution and granularity for analyzing the topology of the faces of the surface of the body to be treated is parameterized.

[0068] According to one embodiment, the locally generated surface at the considered point of the first point cloud is calculated in the following way:

[0069] According to a potential surface defined by points of the first point cloud;

[0070] Alternatively, an approximation of the local surface is obtained directly from the method of the k nearest neighbors of the point under consideration.

[0071] According to an embodiment, at least two shape descriptors are associated with each point of the first point cloud. An advantage is that the description of the model is richer. A benefit is that better learning is obtained during the training of the neural network. Further, an advantage is that the performance of the regression step of the neural network is improved.

[0072] According to one embodiment, the first point cloud is generated by data obtained by:

[0073] By acquiring a plurality of images from at least two optical devices, each position of a point of the first point cloud is derived from stereo measurements of at least two acquired images; and / or

[0074] By acquiring an image comprising a deformed pattern projected by a projection operator comprising a known geometry; and / or

[0075] Acquisition of ultrasonic signals transmitted or reflected by the surface of the patient's skin; and / or

[0076] The collection of laser light delivered across the surface of the patient's skin.

[0077] One advantage is that the method of the invention is implemented with different acquisition technologies. One benefit is that a point cloud is obtained that can be utilized by the method of the invention, in particular for the guidance of medical devices. Different acquisition technologies can be combined or replaced with each other. The image acquisition system, such as a camera, remains nothing more than an acquisition system that is easy to implement, easy to configure and easy to deploy.

[0078] According to an embodiment, the parameterized body model is obtained from a parameterized model of a human body in which a set of joints is predefined, the parameterized model comprising:

[0079] at least one rotation angle for at least one joint; and

[0080] At least one form factor.

[0081] One advantage is that a simple parameterization of the human body is defined, taking into account the different degrees of freedom of each limb of the body. One advantage is that control of the device is guided by a function of the possible movements of the patient.

[0082] According to one embodiment, the parameterized model further comprises:

[0083] At least one global translation of the model in space, and / or

[0084] At least one global rotation of the model in space.

[0085] One advantage is that the position of the patient can be taken into account and the acquisition system can be calibrated to the patient's body according to the arrangement of different kinds of equipment.

[0086] According to an embodiment, the parameterized body model is obtained from a given parameterization process, the given parameterization process comprising a definition of a set of model parameters such that a given rendering of the parameterized body model can be defined. The parameterized body model can then be represented by a third point cloud, graph or mesh. One advantage is that a directly usable point cloud can be generated from the obtained parameterized body model. Another advantage is that a large number of body models are obtained according to different postures so that the generalization of the neural network obtains good performance.

[0087] According to an embodiment, multiple partial views are generated. Each partial view can be obtained, for example, from a subset of points of the third point cloud. One advantage is that many views that can correspond to real image capture are generated. Therefore, with the learning of a large number of partial views, the neural network is better trained and the performance of the regression step is better.

[0088] According to an embodiment, each subset of points of the third point cloud is obtained by a selection of points seen from a given viewpoint of the space, the set of local views being generated for a plurality of viewpoints of the space.

[0089] According to one embodiment, the regression includes:

[0090] A neural network convolution or layer comprising a plurality of matrix products including weighting coefficients obtained from a machine learning method;

[0091] Non-linear operation.

[0092] According to one embodiment, the regression includes the following as input:

[0093] Points of a point cloud and their attribute values, and / or;

[0094] Adjacency matrix of the graph corresponding to the point cloud.

[0095] According to one embodiment, the method comprises a preliminary step of learning the neural network, the preliminary step being performed by a deep learning method, the deep learning method comprising:

[0096] Performing training from multiple partial views of a parametric model parameterized with different parameterization configurations,

[0097] Generate a matrix that includes the weight coefficients for each layer of the trained network.

[0098] According to another aspect, the invention relates to a system comprising an interface for acquiring input data, computing means and a memory for processing the input data acquired by the method to generate a parameterized human body model.

[0099] According to one embodiment, the present invention relates to a system comprising: a device for acquiring distance data of a set of points on the surface of a body located in space; a guiding robot arm having at least one degree of freedom, the robot arm also comprising a sensor or a transmitter, the position of which is defined by a guiding trajectory obtained from a treatment trajectory defined at the surface of a local view of a body model, wherein the body model is generated by the method of the present invention.

[0100] The system of the present invention can be configured according to the different parameters listed to describe the method of the present invention. In addition, according to different embodiments, the system of the present invention includes different devices described in different embodiments of the method of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0101] Other features and advantages of the present invention will become apparent from reading the following detailed description and from referring to the accompanying drawings, in which:

[0102] Figure 1 The main steps of an embodiment of the method of the present invention are shown;

[0103] Figure 2 An embodiment of a system for spatially controlling the movement of a robotic arm near a body surface to be treated is shown;

[0104] Figure 3 An embodiment of the system of the present invention is shown, the system comprising an operating console, a positioning system and a robotic arm manageable by the method of the present invention;

[0105] Figure 4 An example of a robotic arm of the present invention is shown, comprising six axes guided to follow a planned trajectory;

[0106] Figure 5 An example of a robot arm of the present invention is shown, comprising six axes illustrating reference points of the head of the robot arm;

[0107] Fig. 6A A first example of a manipulation device arranged at the head of a robot arm of the invention is shown, which is intended for use, for example, in dermatological applications;

[0108] Figure 6B shows an example of a trajectory generated via the system of the present invention;

[0109] Figure 7 An example of a first grid of points of a surface to be treated according to the method of the invention is shown;

[0110] Figure 8 shows a first example of a second grid obtained via an embodiment of the present invention; and

[0111] Fig. 9 A second example of a second grid obtained via an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0112] The invention relates to a method for acquiring images of a surface of a body and reconstructing the three-dimensional surface of the body. The invention also relates to an image acquisition and processing system. The invention also relates to a method and a system for actuating a medical device according to a modeled body model. The invention relates, for example, to the guidance of a robotic object for the diagnosis or treatment of a patient's skin covering.

[0113] In the remainder of the description, the system for acquiring 5 points of the patient's 1 body is preferably arranged at a fixed spatial point. As an example, the point acquisition system can be fixed to a ceiling, a wall or alternatively to any other spatial point intended to remain fixed during the performance of the method of the invention. The acquisition system 5 is referenced in a reference point marked R0 (O, x0, y0, z0). It can also be referenced in a reference point R1 (O1, x1, y1, z1) associated with the acquisition system 5. According to the position calibration between the "world" reference system and the acquisition system 5, the acquisition system 5 can be referenced in the reference point R0 by means of a transformation R1->R0.

[0114] In the remainder of the description, the robotic arm 6 is arranged in the same space covered by the acquisition area of ​​the acquisition device 5. This space can be a treatment room of a hospital, or instead a room corresponding to a doctor's surgical operation. The robotic arm 6 can be positioned on a base, such as a portable item 25 of furniture, and can be fixed in a given position due to the method of the invention during the performance of the treatment. The robotic arm 6 is referenced to a reference point marked R0 (O, x0, y0, z0). The position of the robotic arm 6 in space can be precisely located in a reference point R2 (O2, x2, y2, z2), in which the movement of the robotic arm 6 can be easily represented. The change of the reference point R2→R0 makes it possible to represent the movement of the robotic arm 6 in the reference point R0 of the system of the acquisition device 5. This change of the reference point can prove to be particularly effective in the implementation of the collision detection function between the robotic arm 5 and the body of the patient 1.

[0115] According to an exemplary embodiment, the acquisition system 5 can be integrated with the reference point R2. In this exemplary case, the acquisition system can be fixed to a base supporting the robotic arm 6. This configuration is obtained, for example, by a holding rod (not shown), so that the acquisition system 5 can be raised. The advantage of this configuration is that the installation of the system of the invention is facilitated. Another advantage is the available pre-calibration between the positioning of the acquisition system 5 and the robotic arm 6.

[0116] According to another exemplary embodiment, the acquisition system 5 can be fixed to a base on which the operating console 20 is mounted. This situation makes it possible to obtain an autonomous acquisition device including a hardware part and a software part. It is also possible to use a holding rod to fix the acquisition system 5 or any other mechanical holding device.

[0117] In the remainder of the description, the head 8 of the robot arm 6 (also referred to as the manipulator device 8) is arranged at the distal end of the robot arm 6. The head 8 can be linked to Figure 53 (03, x3, y3, z3) in the reference point indicated in FIG. 1 . The position of the head 8 of the robot 6 can be easily identified in the reference point R2 (02, x2, y2, z2) using the knowledge of the kinematic animation of the branches of the robot 6. It is of interest to refer to an image acquired by the manipulator device 8, for example, in the reference point R0 (0, x0, y0, z0).

[0118] This description is applicable to any type of robotized object, in particular a robotic arm. The robotic arm of the present invention may include multiple degrees of freedom, in particular 2 and 9 degrees of freedom. In the remainder of the description, we will consider the case of applying a robotic arm with 6 axes to describe the embodiment. However, the present invention is not limited to this example.

[0119] According to different alternatives, the robot arm can be fixed to a horizontal base, wherein the first arm extends vertically, or the robot arm can be fixed to a vertical base, wherein the first arm extends horizontally. Different configurations are compatible with the present invention.

[0120] In the remainder of the specification, a "body model" refers to a three-dimensional model of a body. The body model may be represented by a point cloud. The point cloud may be generated from a given modeling of a parametric body model obtained using a given parameterization.

[0121] Furthermore, the body model may be used in order to generate different partial views of the body, which correspond to subsets of the points of the point cloud modeling the entire body. To this end, viewpoints may be defined to extract partial views of the body model. Such a body model may be obtained from a given parameterized configuration of a parametric model of a human body.

[0122] In the remainder of this specification, a partial view of a “body model” represents a three-dimensional model of the body according to a given viewing angle from which the surface of the previous body model is observed.

[0123] In the remainder of the description, a "parameterized body model", MOD_S, specifies a model that can be parameterized in order to generate transformations of certain parts or views of the body according to a configuration of given parameters. Thus, a previously defined body model can be generated from a parameterized body model with given parameterization, which is called a parameterized body model, MOD_P.

[0124] The parametric body model can be used according to the first use of the method of the invention to perform the regression step in order to determine a set of parameters enabling the generation of a body model for guiding, for example, medical equipment. The weight matrix to be used precisely during the regression step can be generated according to the second use of deep learning.

[0125] According to an example, the parametric body model MOD_S can be obtained from the parametric model using the following parameters as input:

[0126] Global transformation of the spatial model: Tx, Ty, Tz, i.e. 3 parameters;

[0127] The global rotation of the model in space: Rx, Ry, Rz, i.e. 3 parameters;

[0128] For example, using a parametric model consisting of 24 joints, each joint has 3 rotation parameters: Rxi, Ryi, Rzi, a total of 72 parameters are defined;

[0129] Shape coefficients, for example, use a parametric model comprising 20 shape coefficients. The latter influence the shape of the human body.

[0130] In this example, 88 parameters may be determined to generate the human body model.

[0131] Other examples can be implemented using the method or system of the present invention.

[0132] Point acquisition

[0133] According to an embodiment, the method comprises the step of acquiring the point ACQ. The acquisition can be done, for example, with the aid of an image acquisition device. In an exemplary embodiment, Figure 3 The optical system 5 in comprises two sensors 51, 52 for acquiring images IM1, IM2. They can be, for example, cameras including at least two color or infrared cameras, such as 3D cameras. According to different exemplary embodiments, color or infrared cameras can be used. Preferably, a camera pair consisting of identical cameras will be selected. A previous calibration step can be performed to configure the color of each camera and make them correspond to the same spatial point.

[0134] Another inter-camera calibration, known as "stereo calibration", can be performed between two cameras to produce a 3D reconstruction of an object. Calibration aims to compute the transformation between the two images acquired by each camera in the camera pair.

[0135] According to one example, the transformation may include the detection of key points and the calculation of descriptors obtained by a scale-invariant visual feature transform abbreviated as SIF. For example, the relevant steps aimed at making the key points correspond can be implemented using an estimation method such as the method called "RANSAC" that specifies "random sample consistency". Next, the calculation of isometric transformations between the detected key points is performed. Finally, the transformation makes it possible to triangulate the points of the image and thus construct a depth map.

[0136] According to a first approach, 2D images of the same object acquired from two viewpoints whose arrangement is known may be combined according to known techniques to obtain a so-called depth or distance map.

[0137] The method of the invention enables processing of 2D images, which are then combined to obtain a cloud of points G1. The points will constitute a first graph G1. According to the example, the points of the first graph G1 obtained are non-oriented and non-connected. In its simplest form, the graph G1 is an unordered list of points whose coordinates are referenced in space (for example in R0, R1, R2 or R3). Each point is then defined independently of the other points. In its simplest form, the graph G1 is therefore a cloud of points.

[0138] The images IM1, IM2 may be acquired simultaneously. Advantageously, the optics are oriented along parallel axes and are colocalised. It is of interest to be able to process the images by stereo techniques to generate a depth map. The depth map makes it possible to address spatial coordinates x, y, z at multiple points.

[0139] According to an embodiment, the color of the image is used to generate hue differences, and thus distance variations.

[0140] The depth map can be obtained by encoding the distances obtained between the different points, the resolution of which can be configured. The grayscale can vary according to the values ​​of the first range. By choosing the encoding, for example on 16 bits, the encoding in the grayscale of the depth map makes it possible to obtain a high precision of the distances separating the different points.

[0141] The method of the invention makes it possible to continuously acquire a stream of images through the optics 51,52 in order to process the latter in real time. The notations IM1,IM2 designate not only the acquired images but also the stream of images acquired by each of the cameras 51,52.

[0142] The image may be processed in order to extract an acquisition perimeter that limits the zone to be treated. To this end, a template may be used to avoid considering points outside the zone of interest defined by the surface of the body.

[0143] The acquisition of two images IM1 , IM2 and the coverage of the entire surface to be treated makes it possible to generate a depth map of the entire body. A depth map comprises a set of points and distances relative to each other.

[0144] According to another example, no surface is processed in this stage of the method. The distances between points are generated independently of the shape recognition (eg the outer layer of the body).

[0145] According to an example, the number of points generated per surface unit may be configured according to the desired resolution of the point cloud. To this end, the acquisition may be configured to acquire more or fewer images in order to increase the resolution.

[0146] According to an embodiment, the positioning system SYS_POS enables the definition of a reference system and a coordinate system to accurately locate each point in a single reference system. The positioning system may advantageously be the acquisition device 5.

[0147] Stereo calibration makes it possible to obtain a transformation between R0 and R1, where R1 is a reference point associated with the acquisition system 5. Next, the method of the invention makes it possible to calibrate the automated target 6 relative to the acquisition system 5, thereby performing a transformation of the images in the reference system R2->R1->R0. When the reference system R3 is associated with the head 8 of the robotic arm 6, the transformation R3->R2 is known, since the kinematics of the joints of each arm are known by the computing means and the memory of the system managing the robotic arm 6.

[0148] Other acquisition techniques

[0149] According to an embodiment, a projection device is used to project a pattern onto the surface of a body. The acquired images can then be processed in order to analyze the deformation of the projected pattern. This technique makes it possible to obtain additional data and to improve the 3D reconstruction. In particular, the detection of the deformation of the projected pattern makes it possible to improve the accuracy of the 3D reconstruction. In particular, the detection of the deformation of the projected pattern makes it possible to improve the accuracy of the evaluation of the calculated distance to generate a depth map. In addition, the projector makes it possible to reduce noise during the creation of the point cloud to generate the first graph G1.

[0150] Thus, a stereoscopic reconstruction through a depth map can be obtained using a camera pair optionally used with at least one pattern projector. The projected light can be in the visible range or not in the visible range, such as the infrared range. According to one example, the acquisition device 5 may include a pair of color 3D cameras and at least one infrared projector.

[0151] According to another embodiment, the camera used may be an infrared camera or have a near infrared frequency range.

[0152] Other techniques for constructing a depth map may be used, such as a laser transceiver device. The reflection of the laser beam makes it possible to generate data relative to the geometry of the reflecting surface. Another technique may be by means of an ultrasonic transceiver. An example of a system that can be implemented is a LIDAR type system.

[0153] According to an embodiment, the method of the invention makes it possible to recover a point cloud derived from an acquired image and to transfer it, for example, to a graphics processor, such as a GPU (Graphics Processing Unit).

[0154] Generation of the first graph G1

[0155] The method of the invention comprises a step aimed at generating a set of points associated with spatial coordinates referenced, for example, in a reference point R0 of space. These coordinates may be obtained from a previous calibration of the positioning system SYS_POS. This calibration may be performed at the beginning of the method of the invention from a specific pattern acquired by the optics of the positioning system SYS_POS. According to an embodiment, the positioning system SYS_POS and the optical system SYS_OPT may be the same system. According to another embodiment, they may correspond to two different systems.

[0156] Therefore, the depth map obtained by the acquisition system SYS_OPT can be used to obtain a set of spatial points associated with coordinates in the reference point R0 of the space.

[0157] The method of the invention comprises a step of receiving data REC_DATA1 corresponding to a spatial point referenced in a reference point R0 of space.

[0158] According to an example, the method of the present invention may include a step of segmenting data associated with a predefined surface. The surface may be, for example, the surface of a human body. The segmentation may be automatically performed by analysis of data (e.g., data derived from a captured 2D image and / or a depth map obtained). For example, the method of the present invention may include analysis of differences in distances between specific points, or alternatively by performing a colorimetric process. The depth map may be used to filter the point cloud and retain only points of interest, that is, those of the human body.

[0159] According to another example, it is possible to Figure 3 According to this latter example, the contour of the surface of the body can be obtained by a selection tool such as a probe or a mouse and an image represented on the screen of the console 20 by the operator. According to other examples, the selection can be made by a tactile command on the screen or a verbal command. The point of interest is to eliminate, for example, points belonging to the table 22 on which the patient 1 rests.

[0160] The method of the present invention comprises generating a first map GEN_G1. The first map G1 can be obtained from a matrix representation of a set of points generated from the body surface of the patient 1. Each point is in v i , and can be represented by a vector notably including its coordinates in the reference point R0. According to an embodiment, at this stage, the first graph G1 is unordered in the acquisition frame of reference. Even in the simplest embodiment, the symbol "graph" is used, or the group of points corresponds to a series of points with coordinates, that is, a simple point cloud. In other embodiments, the graph can be connected or non-connected, directed or non-directed, etc.

[0161] Filtering

[0162] According to an embodiment, the method of the invention comprises a step of filtering the pattern obtained, the filtering being temporal and / or spatial. The filtering step makes it possible to reduce the noise generated during the generation of the first pattern G1 after the detection step.

[0163] Attribute determination

[0164] The method of the present invention comprises for each point v i Determine at least one attribute DET_AT i These properties correspond to the steps of i These attributes can be of different natures: Cartesian data, shape descriptors, identifiers, vectors, etc.

[0165] According to one embodiment, the attributes include a description of a point v in the first graph G1. i The data of the normal vector ni at the point. The normal vector ni can be determined in different ways. The benefit of calculating the normal vector as an attribute of a point of the point cloud is that it can yield information about the convexity of the local surface.

[0166] According to one embodiment, by considering the points v in its vicinity iLk At each considered point v i The surface generated locally around the given point v in the first graph G1 is calculated i The normal vector ni at . To this end, the spatial coordinates of each point are compared with the coordinates of the point under consideration. According to one example, the acquired distance or depth data are used to identify points in the vicinity of the point under consideration.

[0167] According to a first example, the surface considered for calculating the normal vectors is calculated from the underlying surface defined by the points of the first graph G1 .

[0168] According to a second example, the surface considered for calculating the normal vector is calculated according to a method implemented on the GPU. According to another example, the normal vector is calculated according to an approximation of the local surface obtained directly by the method of the k nearest neighbors of the point considered. However, this triangulation method of connecting a list of 3D points may be more expensive than the method implemented on the GPU.

[0169] Computing normal vectors at points in the graph can increase the description of the points so as to enrich the point cloud.

[0170] Shape Descriptor

[0171] According to one embodiment, the attribute comprises at each point v of the first graph G1 i At least one shape descriptor at i . A shape descriptor is a local descriptive signature of the space at a point.

[0172] According to an example, according to the point v considered i A subset of nearby points is used to compute the shape descriptor Df i The method thus comprises the step of generating a graph of points reconstructed or extracted from the body model according to the viewing angle of each image acquisition device capable of utilizing these data. As an example, when the acquisition device is arranged at the distal end of a robotic arm, a partial view of the body model can be generated according to the position of the acquisition device to perform a specific treatment.

[0173] A shape descriptor can also be inferred at each point of the 3D body model obtained from the point cloud. Insofar as they have symmetry and isotropic properties, shape descriptors remain unchanged during isometric transformations such as rotations of the body model. This facilitates transposing the shape descriptor regardless of the viewpoint considered.

[0174] The advantage is that the calculation of the shape descriptors is performed directly on the points of the first graph G1. Therefore, the invention makes it possible to optimize the calculation of these shape descriptors that are not applied to surface parts but directly to predefined points of the graph. The calculation of the descriptors can be performed on the adjacency matrix of a connected graph obtained from the graph G1. A connected graph means that each point is connected to all other points. The adjacency matrix makes it possible to obtain points that are directly connected to a given point. In addition, the proximity of a point relative to another point can be inferred from the adjacency matrix. Based on the shape descriptors, the nearby points of a point can be inferred from the adjacency matrix.

[0175] Optionally, the method of the invention makes it possible to apply a distance threshold in order to connect a point only with its N closest neighbours.

[0176] According to an embodiment, the shape descriptor is chosen to be isometric and invariant. It is of interest to obtain a shape descriptor that remains invariant when the body is moved using an isometric transformation.

[0177] According to a first example, the shape descriptor is of the "wave kernel feature" type, annotated WKS. According to a second example, the shape descriptor is of the "heat kernel feature" type, annotated HKS. According to a third example, the shape descriptor is of the "Gaussian curvature" type.

[0178] According to various embodiments, a plurality of shape descriptors Df are calculated at each considered point of the first graph G1. i , and multiple shape descriptors Df i Associated with each of the following points.

[0179] Certain shape descriptors are shape descriptors belonging to a group of spectral shape analysis methods, such as the HKS or WKS descriptors. The HKS descriptor is based in particular on the solution of the heat equation, which is based on the Laplace-Polamie operator.

[0180] One of the advantages of these shape descriptors is their invariance in certain transformations, in particular isometric transformations. Furthermore, the shape descriptors can be calculated from the adjacency matrix of the graph G1. This has the advantage of making it possible to consider the relative distances of the considered point to the set of points in its vicinity.

[0181] According to one example, the method of the invention can be used while taking into account a threshold applied to the adjacency matrix, so as to consider only a subset of points whose distance is greater than a given threshold.

[0182] The WKS descriptor itself is based on the solution of the Schrödinger equation.One benefit of the WKS descriptor is its ability to model spectral separation obtained by the operator.

[0183] Shape descriptors can be chosen such that they model local or global 3D shapes. When these descriptors encode local modeling, a radius of validity can be introduced. However, global modeling will benefit from the sum of the geometric invariances of the operators used.

[0184] It is also possible to use a shape descriptor called "global point feature", or GPS. This descriptor is based on an operator of the Laplace-Beltrami type. The benefit of this operator is its ability to achieve global modeling.

[0185] It is also possible to use the shape descriptor "Gaussian curvature", GC. This descriptor offers the possibility of considering a parameterized surface and thus conversely the points that locally define the surface.

[0186] The descriptor "spectral wavelet feature" (denoted as SGWS) can also be used to define multi-resolution local descriptors.

[0187] The benefit of shape descriptors based on spectral shape analysis is that they can be mathematically processed to aggregate connected or adjacent surfaces, segment regions, and perform shape recognition algorithms.

[0188] The advantage is that it enriches the description of the point, thereby improving the regression performance of the neural network.

[0189] Another advantage of the shape descriptor is that it generates identifications at points of the first graph G1. When the patient's body moves or changes over time, the surface of the body is considered a deformable object that deforms unequally. The shape descriptor enables the generation of specific features at each point, which can be found again after deformation, thanks to the method of the present invention.

[0190] Attribute association

[0191] Therefore, each point v of the first graph G1 i can be compared with the coordinates {x i ,yi , z i}, normal vector ni, a set of shape descriptors {Df i1 , Df i2 , Df i3 ,…,Df iN}. This step is in Figure 1 It is marked as ASSO_AT i .

[0192] Point v i Then it can be constructed as a finite dimensional vector, where each dimension corresponds to a given attribute. Then a second graph G2 can be generated. This second graph G2 corresponds to using its attribute AT i Extensive list of vectors.

[0193] return

[0194] The computational step enables the three-dimensional reconstruction of the acquired and enriched human body by means of an approximation of a parametric human body model. Figure 1 denoted as REG in . The goal of the regression step is to estimate given parameters in order to configure a parameterizable human body model with these parameters. Thus, from the partial views acquired by the camera, a complete parameterized human body model can be estimated.

[0195] To this end, the method of the present invention further comprises calculating the shape and rotation ROT i Parameters are a set of geometric parameter steps.

[0196] The rotation may be calculated at a plurality of points of the second graph G2. According to an embodiment, the rotation is calculated for each point of the second graph G2. According to an embodiment, each rotation may be associated with a joint motion. The joint motion may be associated with a defined area of ​​the parameterized body model MOD_S, and a given joint motion may thus be associated with the parameterized body model MOD_P. A given joint motion may thus be associated with a set of points of the second graph G2 at the end of the regression operation.

[0197] The shape parameters may be defined by a set of positions of points of the parameterized body model in a given reference point.

[0198] According to one embodiment, the input parameter is a set of points of a point cloud G1 or G2 provided with their attributes calculated in the Euclidean space R3 .

[0199] According to an alternative, the input parameter can be the adjacency matrix corresponding to the graph G1 for each point of the point cloud. These inputs enable studies to be conducted in the spectral domain of the inputs and do not necessarily require taking into account the attributes.

[0200] A set of shape parameters and rotations of the joints corresponds to the output parameters of the neural network. These parameters will make it possible to configure the parametric body model MOD_S to generate a parameterized body model MOD_P.

[0201] According to one embodiment, the regression step includes a series of mathematical operations, such as matrix operations between an input matrix and a weight matrix pre-trained by a deep learning method.

[0202] Each of its operations is followed by a nonlinear function. These are trigger functions or transfer functions that make it possible to produce the output of a node of the network. As an example, nonlinear functions such as ReLU, S-shaped or hyperbolic tangent can be performed.

[0203] According to an example, this step comprises performing a regression according to a series of convolutions of a layer of a neural network trained from a plurality of predefined parameterized body models MPPi}i∈[1,N].

[0204] They can be convolutions or layers of a neural network. In fact, a network can include convolutions in its first layer, or a layer of fully connected neurons at the end of the model. In the latter case, they are neurons that are connected to all neurons of the previous layer and to all neurons of the following layer.

[0205] A convolutional layer may include a scan of the input matrix resulting in a series of matrix calculations. Other layers of a neural network typically include matrix calculations on the size of the input matrix. In all cases, this step includes matrix calculations.

[0206] According to an example, each convolution comprises a matrix product between an input matrix and a weight matrix and takes into account an additional bias.

[0207] The application of processing performed by successive layers within a neural network consists of applying a series of matrix multiplications followed by a non-linear function to produce the output of the layer. The succession of these operations defines the depth of the neural network.

[0208] According to an exemplary embodiment, the neural network is a multilayer perceptron known by the acronym MLP. According to an example, the neural network may be a network equivalent to an MLP.

[0209] According to one example, the neural network is configured so that the result does not vary with the permutation of the points of the input list. This configuration is made possible by using a pooling function, such as a function called "MAX_POOLING" or "AVG_POOLING".

[0210] According to an embodiment, the structure of the neural network enables the extraction of attributes of the input at different scales. The nearby points of each point of the point cloud at different scales are successively considered to extract the encoding in the space of reduced input dimensionality.

[0211] This encoding is then passed through a series of fully connected layers to perform regression on a set of parameters of a parameterized human body model.

[0212] The successive steps of sampling and / or grouping nearby points of the sampled points together and then grouping these points together again are repeated and encoded through convolutional layers to create a map of the obtained parameters.

[0213] The method of the invention advantageously makes it possible to create a function that preserves the symmetry of the set of points at the input of the network and the set of points at the output of the network. Thus, the application of the neural network to non-connected, non-directed graphs can be ensured by implementing this symmetry preserving function. This function ensures the independence of this step with respect to the ordering of the input points.

[0214] This function also enables the implementation of autoencoders within the process of regression.

[0215] A collection of parameterized models can be obtained from a generic model. The benefit of such a model is the ability to generate multiple poses of a human body from the parameterization of the connections and the shape parameters. The particularity of such a body model is that it can be modeled by a collection of connections, each connection comprising at least one rotation and a position. The rotation of a connection can be defined by an angle. The position of a connection can be associated with at least one point of a figure or at least one position of several points of the same figure. According to another example, the position of a connection is obtained from an operation of combining different positions of a figure, for example to generate a new position of a new point. The new point can be, for example, the center of gravity of a subset of the points of the figure. The shape parameters can be parameterized to define global shape parameters of the body.

[0216] The set of configuration data of the set of joints in position and rotation makes it possible to define a given pose of the body model. Thus, multiple poses of the body model can be generated.

[0217] According to an embodiment, the parametric modeling of the human body is a meshed network of points. Each parametric joint generates a configuration of a subset of points of the meshed network of associated points.

[0218] According to this latter example, 24 articulations can be defined from 72 rotation parameters, each articulation being associated with 3 rotations. The parametric model can also include 3 overall rotations. A subset of articulations can be extracted to implement embodiments of the present invention, such as a wrist, elbow, etc.

[0219] Each joint may also include a set of shape parameters, such as the width of the wrist, the length of the wrist, etc.

[0220] According to an exemplary embodiment, the PCA (Principal Component Analysis) function makes it possible to extract general parameters, the configuration of which makes it possible to act on a set of parameters for multiple joints. As an example, the size of the individual, the thickness of the ventral region, the expansion, the size of the pelvis, etc. can be configured as a set of local parameters pre-conditioned to multiple joints.

[0221] The parameterization of the body may include multiple vectors defining shapes and angles. As an example, it may be an ankle, a waist, an arm, etc. The general configuration makes it possible to obtain a contour or a given template of a body that can be parameterized.

[0222] Other parameterizable body models may be used with the method of the present invention.

[0223] Learning Neural Networks

[0224] Neural networks are trained through automatic learning, better known as "machine learning".

[0225] Such a machine learning algorithm may include the step of recovering a database of high-resolution human scans. Then, the step of deforming a body template of a parametric model by applying an isometric transformation. The transformation may include translations and rotations and combinations thereof applied to different faces of the template to conform to the scan derived from the recovered data. This step may include executing a cost function to optimize, for example, by descent of the gradient.

[0226] The method includes a step of pooling the transformations performed on each scan. The principal components of the transformations are then derived by an ACP-type method.

[0227] Applying this algorithm makes it possible to extract the main shape and position parameters of the human body.

[0228] In order to generate a set of partial views VP of a parameterized body model obtained from a plurality of parameterizations of the parameterized body model, a ray tracing algorithm can be applied. This algorithm is able to take into account a large number of different viewing angles in order to generate a large number of partial views VP from a given parameterized body model. The advantage is that a calibration step during the execution of the method of the invention is avoided and it is closer to the real world when images of the patient's body are to be acquired. The multiplication of learning with a large number of partial views makes it possible to avoid steps of manipulating the body, such as translation or rotation of the data representing the acquisition of the patient's body.

[0229] One benefit of this solution is the generation of a large number of body models by varying the parameters using different partial views of the body model. These latter partial views VP correspond to subsets of the point cloud defining the whole body. One advantage is the training of the neural network using a large number of configurations that can correspond to the acquisition of a real body during use of the method of the invention. The viewpoint corresponds, for example, to the capture point of a camera. In order to constitute the most complete possible database representing the body models, this viewpoint is multiplied around the surface of each body generated from the parameter model. Thus, a large number of body surfaces represented according to different viewpoints are saved in a memory. The data set generated for the training of the neural network using viewpoints in order to perform the regression step is a set of partial views VP of a plurality of parameterized body models MOD_P.

[0230] The advantage is that the weights of the regression are trained with shape descriptors associated with each point of the body model so that it can be perceived in reality. A shape descriptor can be obtained for each partial body model view VP generated and for each point of a given body model. The shape descriptors obtained as a result of the learning enable the definition of a weight matrix, otherwise known as a matrix comprising weight coefficients for each layer of the neural network. In the process of performing the regression in real time during the calculation of the patient's body model, the regression step will advantageously be performed with the weight matrix defined by the learning step.

[0231] The parameterized body model MOD_P thus generated makes it possible to train a neural network for performing the regression step REG. The regression performed in real time then makes it possible to extract the parameters characterizing the body under consideration.

[0232] This step enables obtaining a body model from which surfaces can be easily extracted and a complete parameterized body model is considered instead of only the part seen from the camera.

[0233] The advantage of the method of the invention is to reconstruct the three-dimensional surface in real time by inference. The solution is based on the use of a model trained thanks to deep learning techniques. It is interesting that this can be done by gradient algorithms without optimization methods (such as iterative methods), which require long calculation times and integration restrictions on the real-time control of the robot arm. The benefit of the method of the invention is that the optimization is achieved during learning rather than during the processing of data acquired during the guidance of the robot arm.

[0234] Deep learning inference consists of matrix multiplication operations with previously trained weights.

[0235] According to an embodiment, the layers of the neural network include a resolution that depends on the layer under consideration. As reasoning descends into the layers of the network, the scale can become larger and larger. Examples of neural networks such as MLP networks can be used.

[0236] First grid M1

[0237] The method thus comprises a step of extraction of a mesh surface. The mesh of the surface obtained is a first mesh M1. According to an embodiment, the first mesh M1 is a mesh in which triangles are defined by points of the surface under consideration and points in its vicinity. Figure 7 An example of a mesh M1 comprising a plurality of triangles formed by triplets of points co-located within the surface under consideration is represented.

[0238] According to another embodiment, another grid pattern may be considered in other embodiments of the invention: quadrilaterals, squares, pentagons and generally any type of polygon whose vertices are the points of the grid.

[0239] The mesh M1 is created after the regression operation. Before the regression step, the point cloud is considered such that each point is independent from each other and the points are not linked together.

[0240] Second grid M2

[0241] According to an embodiment, the method of the present invention comprises a step of defining a second mesh M2 from a first mesh M1. The second mesh M2 can advantageously be defined so as to generate semi-uniform triangles, such as isosceles triangles. In this example, the triangles can be transformed according to a given algorithm, for example using the symmetry properties of the generated mesh.

[0242] Figure 8 An exemplary embodiment of a second mesh M2 obtained from the transformation is represented. The mesh M2 comprises a set of points comprising a first set of points ni which remain unchanged and a second set of points ni' generated from the creation of a bisector of each considered triangle to create two new isosceles triangles.

[0243] Other techniques can be used to homogenize the mesh in order to regularize the mesh. According to another technique, equilateral triangles are generated, such as Fig. 9 The advantage is that a completely uniform grid is generated, which notably makes it possible to facilitate the guidance of the operating device 8 .

[0244] The method of the invention makes it possible to apply a transformation to the points of this grid so that their properties are also transformed as a function of the Cartesian position of each new point generated.

[0245] According to an example, the desired resolution for the grid M1 or M2 is obtained as a function of the parameterization of one or more sensors or one or more transmitters at the end of the robotic arm. The resolution of the graph G1 can be configured to have a variable size and be independent of the resolution of the grid M1 or M2. As an example, the circles 36, 37, 38 represent the desired aiming points of the power to be delivered from the laser to the surface of the skin of the patient 1. The benefit of this solution is to generate a dermatological resolution suitable for the treatment to be applied.

[0246] therefore, Figure 8 Circles 36 of different sizes are shown, obtained by a mesh with isosceles triangles of different sizes. Therefore, a coefficient must be applied at the moment of laser emission to adapt the power applied to the signal. According to another example, the second mesh M2 can be made uniform by a series of geometrical treatments in order to obtain triangles of equal size.

[0247] Fig. 9 Represents the aiming point 37 or 38 of the power delivered from the laser onto the skin surface. The benefit of this grid configuration is that the same shooting configuration can be obtained from one triangle to another. The triangles 38 are tangent to each other and allow the amount of overlap parameters to be controlled.

[0248] According to an embodiment, the new mesh M3 is defined by circles 36 , 37 , 38 generated from the second mesh M2 .

[0249] Real-time control

[0250] According to an embodiment, the acquisition device 5 performs in real time the acquisition ACQ of points of all or part of the body of the patient 1 in order to calculate a parameterized body model MOD_P according to the method suitable for controlling the movement of the robot 6 according to the invention. According to the treatment trajectory TRAJ1 established in the defined area 9 of the surface thus modeled and recalculated (in Figure 4 30) is used to guide the robot arm 6.

[0251] The trajectory TRAJ2 of the robot arm 6 can be automatically generated from the treatment trajectory TRAJ1 according to the orientation and the relative distance between the head 8 of the robot arm 6 and the skin surface, for example defined with the aid of a user interface UI. The orientation is determined by default to 90°, that is, the main axis of the sensor or transmitter of the head 8 of the robot arm 6 merges with the normal N of the target point of the surface of the body under consideration (in particular, a pre-generated parameterized body model MOD_P). The distance corresponds to the distance between the point of the sensor or transmitter of the head 8 of the robot arm 6 and the target point P on the surface of the modeled body generated at the position of the patient. TARGET distance.

[0252] A benefit of real-time calculation of the modeled surface of the body of the patient 1 is to predict, for example, the movement of the patient 1 to take into account, for example, movement caused by the patient's 1 breathing.

[0253] The advantage of the method of the invention is that a new modeling surface can be generated in less than 30ms. These response times significantly make it possible to constrain the movement of the robot 6 at sufficient speed to anticipate collisions or sudden movements so as to become clear as quickly as possible. The configuration of the camera's acquisition can be, for example, 10 to 90fps.

[0254] The real-time control of the robot 6 makes it possible to trigger at least one firing of the transmitter and / or the acquisition of the sensor of the operating device 8. Then, when the orientation of the axis of the sensor or transmitter is perpendicular to the axis passing through the target point P TARGET When the target point P TARGET is referenced to the generated parameterized body model. The latter is generated at the position and orientation of the operating device 8. The target point P TARGET It can be obtained before the guidance of the robot arm 6, for example, during the planning of the laser emission or ultrasound emission strategy. According to another case, the target point P is obtained during the guidance TARGET , for example, when a comparison of the images achieved triggers a change in guidance to trigger another detector.

[0255] In all cases, the target point P is defined according to the guidance strategy TARGET , and when the considered body surface undergoes a transformation that applies the adaptation of the guidance strategy, the target point P can be corrected TARGET location.

[0256] According to an embodiment, the acquisition ACQ performed by the device 5 can also be coupled to the acquisition of images performed by an optical device arranged at the end of the robotic arm 6. This can be the case when the movement of the robotic arm 6 masks the area of ​​interest of the patient's 1 body so that the acquisition device 5 cannot acquire, or when small movements are controlled by the distal optical device of the robotic arm 6.

[0257] Thus, both layout modes may be executed and possibly combined to generate a new modeling surface during execution of the process.

[0258] Robotic Arm

[0259] The robot 6 is guided by a mobile controller GEN_TRAJ which takes into account the different constraints used to control the robot. It is worth noting that this controller takes into account the planning of the configuration trajectory and the definition of the area to be processed by the user. According to the method of the invention, the surface is defined on the surface of the reconstructed body model. In addition, the controller CTRL can take into account the instructions from the collision detection module to adjust the movement of the robot in real time. Finally, the controller CTRL takes into account the data from the head of the robot 6, for example when the detection of a single image imposes a specific manipulation on the robot.

[0260] According to an embodiment, the robot 6 comprises a set of branches 61, 62, 63, 64, 65 articulated by pivot joints 612, 623, 634, 645, 656. In this case, the robot 6 comprises 5 branches articulated from a pivot link. According to an alternative, the robot 6 comprises 6 articulated branches, and according to other alternatives, the robot 6 may comprise 7, 8 or 9 articulated branches. In fact, in this embodiment, each joint is capable of at least one rotation between two adjacent branches.

[0261] According to one embodiment, each branch of the robot arm comprises a force sensor. The advantage is that sudden collisions are prevented. When contact is established between a branch of the robot arm 6 and an object, the movement of the robot arm can be automatically stopped or the guidance of the robot arm can be modified so that the movement of the robot arm changes its trajectory.

[0262] According to an embodiment, the method of the invention is preceded by a step of calibration of the robot arm 6. The calibration may comprise the relative positioning of the robot arm with respect to a reference point R0 and its initial position.

[0263] Operating equipment and optics

[0264] According to an embodiment, the robot arm 6 comprises a manipulation device 8, which is advantageously arranged at the end of the robot arm 6. According to an embodiment, the optical device 67 can be arranged off-axis with respect to the head 8 of the robot arm, or it can be arranged at the end of the head, such as Fig. 6A situation.

[0265] Figure 3 One benefit of the configuration is to provide the head of the robotic arm with, for example, a laser device or a device that generates ultrasound for localized treatment and to displace the image acquisition optics. Fig. 6A One benefit of the configuration is to configure the robotic arm 6 for dermatological image capture applications. However, all configurations and combinations of the foregoing are contemplated.

[0266] According to the example, the operating device 8 comprises at least one sensor and / or at least one transmitter. The operating device 8 is also referred to as the "head" of the robot arm 6.

[0267] exist Figure 5 In the example of , the robot arm 6 is provided with one or more optical sensors 67 for acquiring local images of the skin surface. The advantage of capturing local images at the head of the robot arm 6 is that a "colored" body surface can be reconstructed, that is, wherein the captured image is integrated in the parameterized body model MOD_P generated by the method of the present invention. To this end, the optical device or {83; 84} located at the end of the robot arm 67 may preferably include two optical devices in order to reconstruct a depth map associated with the image acquired simultaneously by the two optical devices. The image can then be indexed on the surface of the parameterized body.

[0268] Other transmission / acquisition equipment

[0269] Furthermore, the operating device 8 may comprise an emitter of ultrasound, radiofrequency or laser type, or any other signal source capable of generating an incident light beam. According to an embodiment, the operating device 8 may comprise an array of focused emitters and means for controlling the direction and the emission power.

[0270] When an ultrasound transmitter is arranged at the end of the operating device 8, an ultrasound receiver may be associated to measure the reflected signal in order to construct an image of the dermis and / or an image of the thickness of the patient's skin, or even an image of blood vessels or organs.

[0271] Fig. 6 shows another embodiment in which the operating device 8 comprises a receiving head 80 positioned at the end of the operating device. The receiving head 80 comprises two local optics 83, 84 to take images of the surface of the body, and millimeter or microscopic optics movable in two directions along the longitudinal direction 86. It is of interest to generate images with millimeter resolution including resolution for skin applications (e.g. for diagnostic applications).

[0272] Robotic arm guidance

[0273] Initially, the movement of the robot arm 6 is generated by the user by means of Figure 2 The user may select a region of interest on the surface of the body of the patient 1, such as an "arm", a "leg" or a portion of this region or an area instead of the entire body.

[0274] The selection of the area can be done with the help of a user interface UI. The area can be marked with the help of a stylus or with the help of a mouse or alternatively by selecting one or more predefined areas of the user's choice. According to another example, the selection can be done with the help of a tactile command directly on the display or with the help of a voice command.

[0275] The selection can be made on the parameterized body model MOD_P generated by the method 10 of the present invention. Figure 2 The module ROI enabling the selection of a zone is marked in . The parameterized body model MOD_P can be represented on the display of the user interface UI. When a zone is selected, an extraction of a surface portion can be performed on which the trajectory of the operating device 8 will be defined. According to the example, the trajectory 30 can be automatically defined according to the size of the selected zone. For example, as a function of the operation performed with the operating device 8 (such as image capture, laser treatment and / or treatment from an ultrasound source), the automatic trajectory generation function can be implemented in different ways. When it comes to image capture on a predefined zone, the goal is to cover the entirety of the selected zone. Therefore, an optimized trajectory can be calculated according to the resolution of the optics and its field of view.

[0276] When the user defines the trajectory himself on the selected area, the trajectory is obtained by the component PLAN_TRAJ, which will optimize or calculate the travel parameters of the robot to cover the selected surface.

[0277] The method of the invention enables the guidance of the robot arm 6 to be performed in real time. To this end, a module for generating the GEN_TRAJ movements of the robot arm 6 is implemented. This module enables the path of the robot arm 6 to be recalculated in real time in order to calculate the trajectory in real time, for example according to the movement of the patient, the capture of a single image or alternatively the control of the end of travel in case of contact with an object.

[0278] The guidance of the robot arm 6 is configured, for example, as a Cartesian coordinate system referenced from a reference point R0. Figure 3 or Figure 5 The orientation of the end of the robot arm 6 is configured by the orientation of the normal N represented in . The normal N is defined by an angle of 90° with the aiming point of the surface of the body. The benefit of this parameterization is that each point in the enriched mesh already includes the attribute corresponding to the normal of the point of the model. Thus, the guidance of the orientation of the head of the robot arm can be performed by aligning the orientation of the robot arm on the normal N of the surface of the body.

[0279] According to an embodiment, the distance between the end of the head 8 of the robot arm 6 and the aiming point 35 of the modeled surface of the body of the patient 1 is defined to remain substantially constant, for example, to maintain a distance of 5 cm. Therefore, the guidance of the robot arm 6 is defined along the direction and distance to the skin of the patient 1.

[0280] The collision detection function DET_COLLI can then be implemented in order to deliver control instructions to the movement generation module GEN_TRAJ of the robot 6. In addition, this module takes into account the initial trajectory established by the component PLAN_TRAJ. Finally, the movement of the robot 6 can also be managed according to the control function CNTR_8 of the head 8 of the robot 6. When one or more optical devices are mounted on the head of the robot 6, control of the robot may be required to trigger the second optical device.

[0281] According to an embodiment, the optical system 5 is configured to control each movement of the body 1 by means of a control loop. This control makes it possible to compare the difference between an image acquired at one instant and a new image acquired successively. The benefit of this control is to modify the guidance of the robot 5 by adapting its initial travel to the new trajectory. To this end, a collision detection function DET_COLLI can be implemented to adapt the movement of the travel of the robot 6. The generation of the movement of the robot 6 is determined by the function GEN_TRAJ.

[0282] According to an embodiment, the head of the robot arm 6 is removable so that the latter can be used for different applications. A head comprising one or more optical devices for dermatological applications can be replaced, for example, by a head comprising a laser for applications such as hair removal, tattoo removal or UItherapy for lifting applications for treating the skin surface.

[0283] Depending on the target application, a given optical configuration may be preferably selected.

[0284] According to an example, the arrangement of optics with different focal points is implemented at the end of the head of the robot 6. According to an embodiment, these optics can be articulated or displaced independently or jointly.

[0285] According to an example, at least one optical device is a confocal microscopy system. According to an example, the focal plane can be shifted to produce images at different depth levels. A local three-dimensional image of the skin surface can be reconstructed. The application can be used to reconstruct a volume (such as the volume of a beauty spot on the skin surface).

[0286] Where confocal optics are used, illumination may be obtained from a light source such as a laser.

[0287] According to another example, at least one optical device is configured to generate a series of cross-sectional images to implement a tomographic technique. Depending on the given configuration, the technique may be of the OCT (Optical Coherence Tomography) type. These techniques generally involve coupling a sensor with an emitter such as a laser emitter.

[0288] Other possible applications involve treatment from ultrasound emission. To this end, at least one ultrasound emitter can be arranged at the level of the head of the robotic arm. This emitter or these emitters can be associated with a sensor to measure the modification of the echo signal or the emission waveform. According to an example, it can be a matrix of emitters organized to generate a beam focused at one point. The power can be configured to burn the tissue by creating an acoustic pressure on the surface of the skin or to measure the elasticity of a singularity on the skin covering.

[0289] According to one embodiment, the transmitter is configured to treat the subcutaneous area. The image point can be referenced to the target point P TARGET The point image is located on the surface of the body model. TARGET , the point image is located, for example, a few millimeters, or even a few centimeters, from the skin surface.

[0290] From the target point P TARGET Alternatively, the guidance of the emitter can be performed directly from the definition of a spatial point that reaches and defines a point image, which is located under the patient's skin. When this point is relative to the target point P TARGET When not limited, the target point P TARGET Planning of modified trajectories for acquiring images or dumping robots.

[0291] The head of the robotic arm may also be provided with sensors making it possible to detect the mechanical properties of the skin, such as its elasticity, its hardness, its hydration level or its roughness.

[0292] According to another example, a fluorescence sensor is arranged at the distal end of the robotic arm 6. Such a sensor allows monitoring of bioactive ingredients, for example for cosmetic applications. According to another example, spectroscopic techniques can be used to perform molecular characterization.

[0293] Applications in dermatology

[0294] According to one embodiment, the head of the robot 6 (also called "operating device" 8) is equipped with an optical system for dermatological applications. In this case, a dual optical device can be arranged. The first optical device 83, 84 includes a camera that performs a so-called "macro" scan due to the movement performed by the robot 6. These cameras perform the capture of images of interest on the skin surface. According to an example, the light source makes it possible to illuminate the captured area. The transmitted light can be, for example, visible light or infrared light, or alternatively a fluorescent light source. According to an example, an LED system including a polarizer is configured to transmit polarized light. In the case of polarized light, the LED-type light source is covered with a polarization filter. In this example, the camera arranged at the end of the head of the robot is equipped with a polarization filter. According to an example, the filters of the light source and the camera are configured orthogonally.

[0295] When a singularity is detected, the robot 6 is controlled to modify its trajectory. Positioning of the head is performed on the detected singularity, for example on the center or center of gravity of the area where the singularity is formed. Guidance is thus stopped and the initial trajectory is temporarily suspended.

[0296] The controller triggers the second optical device 85, enabling the capture of millimeter or microscopic images with dermatological resolution. To obtain good resolution, the optical device 85 can be controlled according to a movement 86 along an axis perpendicular to the plane of one of the optics 83, 84 or 85.

[0297] Figure 5 represents a portion of a surface 9 of the body of the patient 1 on which a trajectory 30 is represented. The trajectory 30 may be obtained from the scanning strategy of the robot 6. According to an example, the generated trajectory 30 is obtained due to the planning module PLAN_TRAJ. Figure 4 In the example of FIG. 1 , three dermatological singularities 91 are represented on a portion 9 of the skin of the patient 1. These singularities may be, for example, scars, melanomas, cosmetic spots, or any other skin surface singularities. When the light field 87 of the first optical device 83, 84 intercepts the singularity 91, the acquired image is dynamically analyzed to ensure control of the robot arm 6.

[0298] The second optical device thus makes it possible to take an image of a close-up area of ​​the surface of the skin of the patient 1. According to an example, the second optical device 85 is a dermatoscope. The second optical device 85 may comprise a glass plate in contact with the skin. According to an exemplary embodiment, the second optical device may comprise an emitter of polarized light in order to obtain information of a depth image of the epidermis.

[0299] To this end, the last branch 66 of the robot arm 6 or the operating device 81 can be equipped with a force feedback sensor to brake the travel of the second optical device 85, or even stop it and stabilize it. When the second optical device 85 comes into contact with the skin of the patient 1, the emission of, for example, a laser can be triggered.

[0300] According to another example, each axis of the robotic arm is equipped with a force sensor. The analysis of the force value read on the sensor is used to control the pressure applied to the surface. Thus, thanks to these torque sensors on each axis, the robotic arm makes it possible to control the contact pressure between the tool and the working surface (in this case the patient's skin). This can also be used to perform a "sliding" of the tool on the working surface.

[0301] According to an embodiment, the scanning device 87 is triggered so as to move the hair away from the optical device 85. Such a scanning device 87 can be automatically implemented during the stabilization of the optics of the second optical device 85. It is useful to avoid image capture of hairs located near a singular point where image capture is desired. According to an embodiment, the scanning device 87 comprises a rotating rod, which triggers a flexible clamp near the optical device 85. The advantage is that the hair is rotated so that it is no longer retained between the optical device 85 and the skin. According to an exemplary embodiment, a plurality of scanning devices 87 are arranged around the optical device 85. Figure 6B Indicates two of them.

[0302] When the second optics has captured one or more images with a more precise singularity resolution, the second device 85 may perform a retracting movement to return to its initial position. The robot arm may then continue its travel and return to the already planned guidance trajectory.

[0303] According to an exemplary embodiment, the guiding is performed at a distance between 1 cm and 20 cm from the surface of the body of the patient 1. According to a preferred embodiment, the guiding is performed at a distance between 2 cm and 8 cm from the surface of the skin.

[0304] According to an embodiment, at a given distance from the surface of the robot, the characteristics of the optics are known and the field of view of the optics can be obtained. These characteristics make it possible to know the part of the image acquired covering the complex surface of the patient's body. The advantage is that the image can be made to correspond exactly to the three-dimensional surface of the patient's body. In addition, with the possible indexing of the images acquired on the generated parametric body model MOD_P, the images can be arranged in given positions of the model. Thus, the three-dimensional representation of the body can be archived with user-accessible images capable of representing skin singularities. The association of the images acquired using the generated parametric body model can be performed from the mesh M1 or M2. When the mesh comprises, for example, triangles, the image can be subdivided so that the triangles correspond to the surface of the image.

[0305] According to an embodiment, the invention comprises the step of referencing the captured image using a set of nodes of a mesh. According to an exemplary case, each triangle of the mesh comprises an identifier. The image can then be saved together with data associating the image with a set of identifiers.

[0306] According to an embodiment, the images acquired by the optical devices 83, 84 and 85 are saved in a memory, for example, in the memory of the motion controller of the robot arm 6. According to an alternative, the operating device 8 comprises an electronic board provided with at least one computing device and a memory. The acquired images can first be saved locally before being sent to a remote electronic device the next time.

[0307] According to an exemplary embodiment, each photograph taken simultaneously by the optical devices 83 and 84 is associated with each other. The acquired images can, for example, be time-stamped and associated with a spatial point in the reference point R0. They can also be associated with a point precisely located relative to a frame of reference points R3 associated with the manipulator of the robot arm 6, or alternatively referenced relative to the trajectory 30.

[0308] When acquiring an image with the second optical device 85 , the second optical device 85 can be associated with the image acquired by the first optical device 83 , 84 and in particular at the same position of the trajectory 30 as the position where the operating device 8 has triggered the movement of the second optical device 85 .

[0309] The advantage of the invention is the ability to index each acquired image with an image on the parametric model resulting from the regression step REG. The method thus makes it possible to locate a portion of the image on the skin and index it on the resulting model. The advantage is that a precise track of the photographed area with respect to the body geometry is saved. Thus, when the patient changes position or when the patient returns for another subsequent visit, the indexing of the saved photographs makes it possible to directly generate a new drawing of the area of ​​interest that has been indexed according to a new image acquisition.

[0310] Of interest is a set of images of different resolutions constituting different parts of the considered surface of the body.

[0311] In order to obtain a good response of the robot 6 arm during the real-time acquisition of the images captured by the operating device, the acquired images can be compared with a set of characteristic images. According to another example, the trained neural network makes it possible to classify the acquired images to classify them and generate commands according to the result of the classification. As an example, the detection of some singularities does not require the triggering of the second optical device 85, while the detection of other singularities requires the capture of images with a more accurate resolution.

[0312] According to another exemplary embodiment, the robot 6 executes a first planned trajectory 30 with an overlap of images acquired by the first device 83, 84. The robot 6 then secondly performs a second capture of images at the point where the singularity has been detected. When the first trajectory 30 is executed to guide the robot 6 on the portion selected by the operator and when the set of images acquired by the first optical device is achieved, the robot 6 executes a second trajectory 31. This trajectory can be automatically generated from the singularity identified during the course of the first trajectory 30. In order to collect a second series of images at the identified point, the second device 85 is automatically triggered at the characteristic point of the second trajectory 31.

[0313] Application of laser

[0314] According to another case of application, the robot arm 6 includes a laser transmitter at its distal end. The laser transmitter can be arranged, for example, on the distal end of the operating device 8. The operating device 8 can be used for hair removal, treatment of certain skin diseases or tattoo removal. In addition, a robot arm including a laser source can be used to implement UItherapy treatment or any medical or aesthetic dermatology treatment that relies on lasers. To this end, the configuration of the laser source can be adapted according to the use as a function of parameters such as power, wavelength, pulse or continuous state.

[0315] In the case of managing a laser source, the grid M2 can be advantageously used to optimize the configuration of the laser shots, in particular to manage the shot power. In practice, a uniform grid comprising, for example, an equilateral triangle makes it possible to define a grid of circles circumscribed on the triangle or inscribed in the triangle. The advantage of the grid of circles circumscribed on the triangle is the definition of the overlap between the circles and, therefore, the overlap between the shots. This configuration makes it possible to define a shot target zone adjacent to the target zone under consideration. The benefit of constructing a tangent circle directly derived from a uniform grid is the generation of a target laser shot zone while making it possible to manage the parameters of the uniform overlap amount between two consecutive shots on adjacent targets.

[0316] The invention makes it possible to manage the amount of overlap to compensate for the non-uniformity of the energy of the laser beam around the main irradiation axis. Thus, it is possible to deposit a substantially uniform amount of energy on a given area with a uniform type of grid.

[0317] In the case of performing multiple laser shots to achieve the treatment goal of a skin area, the guidance of the robot arm 6 can be configured to become immobile at a set of points to perform consecutive shots according to a given trajectory plan. The trajectory plan PLAN_TRAJ is determined with the aid of the user interface UI. The trajectory of the robot arm 6 can be adapted in real time to take into account, for example, data associated with the configuration of the laser source, the movement of the patient, or alternatively to take into account the detection of a collision of the arm.

[0318] The system of the invention comprises means for defining the parameters of the laser irradiation, namely the irradiation power and / or the diameter reaching the skin surface and the amount of overlap. The laser emitter and the guide trajectory can be configured to achieve these objectives.

[0319] According to an embodiment, the resolution of the uniform grid can be adjusted according to a given illumination configuration. Typically, when the surface illuminated by the laser is defined to have a maximum radius obtained on the surface of the skin, the grid is configured to define a circle whose size corresponds to the illumination constraints.

[0320] In each application, during real-time guidance, the robot 6 covering a given trajectory is made to evaluate the trajectory in real time. This evaluation is performed by means of computing means and the implementation of the method of the invention.

[0321] When the body is not moving, the image of the surface of the body that is recalculated for positioning the robot 6 involves only rotations of the surface that have already been calculated, that is, the transformation is equiangular and corresponds to one or more rotations of the robot according to the portion of the trajectory covered. The resulting position of the surface of the body is generated for the position of the head of the robot.

[0322] When the body moves, recalculating the image of the surface of the body in order to position the robot 6 involves non-isometric transformations, and the method according to the invention recalculates the surface. In addition to taking into account potential rotations, the method of the invention makes it possible to perform regression on the shape descriptors and regenerate the surface of the body at the viewpoint of the position of the head of the robot.

[0323] According to an embodiment, in particular in the case of an arrangement of emitters on the head 8 of a robotic arm 6 of the laser or ultrasound type, when the region of interest is smaller than a given surface threshold, a step of optimizing the mesh of the surface of interest can be implemented. This optimization can be performed so as to consider the transformations in this region as isometric. According to an exemplary embodiment, the method of the invention makes it possible to determine key points on the considered surface of the body so as to delimit the region of interest that has been subjected to isometric shape transformations.

[0324] An advantage of the present invention is that it is adaptable to applications in dermatology, surgery, cardiology, etc.

Claims

1. A method for guiding a robotic arm (6) in real time to process data on a body surface, comprising: Generating a 3D model (MOD_P) of all or part of a body surface, wherein the 3D model (MOD_P) comprises a grid (M1, M2) of points; the 3D model (MOD_P) is generated from images acquired from a first image acquisition system (5); planning a treatment trajectory (TRAJ1) on the surface of a body model (MOD_S) by means of a computing device; calculating a guide trajectory (TRAJ2) of a distal end of a robotic arm (6), the robotic arm (6) being movable relative to the first image acquisition system (5); Generate the kinematics (PILOT_ROBOT) of the robot (6) so that the point of the head arranged at the distal end of the robot (6) is controlled and constrained on the guide trajectory (TRAJ2); triggering the movement of the robot arm (6) to cover the guide trajectory (TRAJ2), wherein the guide trajectory (TRAJ2) is calculated in real time based on the 3D model (MOD_P) of the body surface; periodically generating a new 3D model of the body surface (MOD_P) ensuring the calculation of new kinematics of the robot arm (6) to automatically manage its displacements in real time, wherein the newly calculated surface of the 3D model (MOD_P) leads to the calculation of a new treatment trajectory (TRAJ1') and the calculation of a new guidance trajectory (TRAJ2'); The target point in the receiving space (P TARGET ) at least one position; The sensor of the operating device (8) arranged at the far end of the robot arm (6) triggers the target point (P TARGET ) of the region, when the orientation of the axis of the sensor is in a direction passing through the target point (P TARGET ) is merged with the predetermined straight line, the trigger is executed, and the target point (P TARGET ) is referenced on the generated 3D model (MOD_P).

2. The guiding method according to claim 1, characterized in that: The treatment trajectory (TRAJ1) is generated within a treatment area on the surface of the body model (MOD_S).

3. The guiding method according to claim 2, characterized in that: The treatment trajectory (TRAJ1) is automatically generated according to a treatment area (34, 35) defined on the surface of the body model (MOD_S).

4. The guiding method according to claim 1, characterized in that: The guidance trajectory (TRAJ2) is calculated according to predetermined distance and orientation instructions, which enable the generation of the relative position of the distal end of the robot arm (6) relative to the treatment trajectory (TRAJ1) and the new treatment trajectory (TRAJ1').

5. The guiding method according to claim 1, characterized in that: The robot arm (6) comprises at least three branches (61, 62, 63, 64, 65, 81), which are articulated relative to each other in pairs by pivot links (612, 623, 634, 645, 656), and the pivot link group comprises kinematics planned for a given guide trajectory (TRAJ2) and a generated 3D model (MOD_P).

6. The booting method according to claim 1, further comprising: A new body model (MOD_S) is generated periodically so that the robotic arm (6) can be controlled in real time based on the generation of a new guidance trajectory (TRAJ2') of the robotic arm (6), the newly calculated surface of the body model causing the modification of the new treatment trajectory (TRAJ1'), the guidance method also comprising calculating new kinematics of the robotic arm (6) according to the definition of the new guidance trajectory (TRAJ2').

7. The guiding method according to any one of claims 1 to 6, further comprising: The position of the robot arm (6) is calibrated from a fixed position at which images are acquired so that a body model (MOD_S) can be generated.

8. The guiding method according to claim 1, characterized in that: The treatment trajectory (TRAJ1) and the new treatment trajectory (TRAJ1') are used to establish an emission strategy of a laser beam from a laser emitter fixed to the operating device (8).

9. The guiding method according to claim 8, characterized in that: The body model comprises a uniform grid (M2) so as to create a new grid (M3) defined by an arrangement of circles (38), wherein at least one circle defines a target point (P TARGET )’s location.

10. The guiding method according to claim 9, characterized in that: The new grid (M3) is generated according to the transmission power instruction and / or the instruction relative to the overlap amount between two transmissions.

11. The guiding method according to claim 10, characterized in that: The sampling of the number of transmissions performed is a function of the new grid (M3), the configured transmission power and the amount of overlap defined between two co-circles.

12. The guiding method according to claim 1, characterized in that: The treatment trajectory (TRAJ1) and the new treatment trajectory (TRAJ1') are used to establish a transmission strategy of ultrasound signals from an ultrasound transmitter, which is fixed to the operating device (8).

13. The guiding method according to claim 1, characterized in that: A set of images is generated from an optical device (83, 84, 86) arranged on the operating device (8), each image (IM1, IM2) using at least one node or mesh of the meshes (M1, M2) and the new mesh (M3) of the 3D model (MOD_P) as a target point (P) on the 3D model (MOD_P) TARGET ) is indexed as a function of the position of the optical device (83, 84, 86) and the viewing angle of the optical device (83, 84, 86).

14. The guiding method according to claim 13, characterized in that: At least one pair of images is simultaneously acquired from an optical device (83, 84), wherein the optical device (83, 84) comprises two sensors integrated together and arranged on different axes of the operating device (8), each image (IM1, IM2) using at least one node or mesh of the mesh of the 3D model (MOD_P) as a target point (P) on the 3D model (MOD_P) TARGET ) and a function of the viewing angle of the optical device (83, 84, 86), wherein the image is processed in such a way as to reconstruct a three-dimensional local image.

15. The guiding method according to claim 14, characterized in that: The three-dimensional partial image is superimposed on the generated body model in a 3D representation.

16. The guiding method according to claim 15, characterized in that: A first image captured by the first optical means (83, 84) of the operating device is compared in real time with a set of images, the comparison operation comprising, if appropriate, generating a validity indicator enabling the generation of a digital control for managing the robot arm (6).

17. The guiding method according to claim 16, characterized in that: The guidance instruction leads to a first control of the robot arm (6) at a target position for extracting the first image, and leads to a second control of the operating device (8) to trigger a second optical device (85), the triggering of the second optical device (85) making it possible to obtain a first image at a target position including the target point (P TARGET ) produces a second image in the near field of the area.

18. The guiding method according to claim 16, characterized in that: The comparing operation includes classifying the acquired image by a neural network, wherein the neural network is trained from a set of images.

19. A system for guiding a robotic arm (6), comprising: a robotic arm (6) controlled by a computing device of a control unit, wherein a trajectory is calculated from points defined on a grid (M1, M2) obtained from a surface 3D model of the body, said computing device calculating the kinematics of said robotic arm (6) from said trajectory; Calculation means for generating a plurality of 3D models (MOD_P) at different moments from the images acquired by the first image acquisition system (5), and for periodically generating a new 3D model (MOD_P) of the body surface to ensure the calculation of new kinematics of the robot arm (6) to automatically manage its displacement in real time, the newly calculated surface of the 3D model (MOD_P) leading to the calculation of a new treatment trajectory (TRAJ1') and a new guidance trajectory (TRAJ2'); The operating device (8) is arranged at the end of the robot arm (6) and includes at least one sensor to obtain a target point (P TARGET ), the target point is referenced by the body model (P TARGET ), when the sensor axis is oriented along a line passing through the target point (P TARGET ) is pointed to by a straight line, the triggering of at least one sensor is performed, wherein the target point (P TARGET ) is referenced on the generated 3D model of the body surface (MOD_P).

20. A system for guiding a robot arm (6), characterized in that The system comprises means for acquiring images of a patient's body to generate a 3D model (MOD_P) and a robot arm (6) provided with operating means (8) for implementing the method according to any one of claims 1 to 18.

Images