Method and system for guiding a robot arm

Abstract

A method for real-time guidance of a robotic arm (6) for processing data on a surface of a body, comprising the steps of generating a body model (MOD_P) comprising a mesh of points (M1, M2), planning a treatment trajectory (TRAJ1) on the surface of the body model (MOD_S) using computer and sensor parameters, and activating at least one transmission of a transmitter and / or acquisition of a sensor of an operator device (8), said operator device (8) being located at the distal end of the robotic arm (6), the activation being performed when the orientation of the axis of the sensor or transmitter is directed to a target point (P TARGET ) and the target point (P TARGET ) describes a method including an activation step based on the generated body model (MOD_P).

Description

[Technical field]

[0001] The field of the invention relates to that of methods making it possible to build a three-dimensional image of the human body in order to dynamically control devices close to the body. The field of the invention relates more particularly to methods making it possible to perform dynamic control in real time, which requires high speed of calculation of the three-dimensional surface. Thus, the invention relates to a method for generating surfaces from an algorithm combining a parameterized model and learning by artificial intelligence techniques. [Background technology]

[0002] Currently, various techniques exist to reconstruct three-dimensional surfaces in space. However, they are not suitable for the real-time guidance of medical devices intended to perform procedures close to the surface of the patient's skin. Indeed, the reconstruction of the body's surface requires image processing that demands significant computational resources and often very numerous acquisition systems. Moreover, this complexity demands partial processing or approximations that in most cases require human assistance.

[0003] Today, this limitation does not make it possible to perform complex guidance of devices in space. In general, the guidance of a robotized arm in space governed by the calculation of the treatment object's surface, which may undergo deformations or displacements, remains difficult to implement. Nevertheless, there is a need to operate a robotized arm that is highly responsive and automatically guided according to specific analyses performed in real time. So far, existing solutions require additional operator operations or still long processing times.

[0004] Thus, there is a need to define a solution that can meet this challenge. Summary of the Invention [Problem to be solved by the invention]

[0005] The present invention aims to overcome the aforementioned drawbacks. [Means for solving the problem]

[0006] According to one aspect, the present invention provides a method for producing a method for manufacturing a semiconductor device comprising: generating a model of the body comprising a mesh of points; planning a treatment trajectory on a surface of a body model using computer and sensor parameters; calculating a guided trajectory for a distal end of the robot arm; generating a kinematic for the robot arm such that a point of a head disposed at a distal end of the robot arm is controlled on a guided trajectory; activating the kinematics of the robot arm to cover a guided trajectory, the trajectory being calculated in real time from a generated body model; receiving at least one location of a target point in space; activating at least one transmission of a transmitter and / or acquisition of a sensor of an operator device, the operator device being disposed at a distal end of the robotic arm, the activation being performed when an orientation of an axis of the sensor or transmitter coincides with a predetermined line passing through a target point, the target point being referenced on the generated body model; The present invention relates to a method for guiding a robotic arm in real time for processing of data on a surface of a body, comprising:

[0007] An advantage consists in using a guidance method that considers a body model that reproduces the patient's body. The computations are reduced and the medical device can be controlled in real time. The reconstruction of the real world is optimized by using a meshed body model calculated from the patient's body. An advantage consists in considering the possible movements of the patient or anticipating possible collisions by using a body model that is easy to use. The body model used comprises a set of points or a mesh that may be easier to use for real time guidance, rather than a system based on acquiring direct images of the patient to anticipate the robot's movements.

[0008] According to one aspect, the present invention provides a method for producing a method for manufacturing a semiconductor device comprising: generating a 3D model of all or a portion of a surface of a human body comprising a mesh of points; planning a treatment trajectory on a surface of a body model using a computer; calculating a guided trajectory for a distal end of the robot arm; generating kinematics of the robot arm such that a point of a head disposed at a distal end of the robot arm is controlled on a guided trajectory; activating the kinematics of the robot arm to cover a guided trajectory, the guided trajectory being calculated in real time from a 3D model of the surface of the human body; The generation of a new 3D model of the surface of the human body at regular intervals ensures the calculation of new kinematics of the robot arm for automatically managing the displacements of the robot arm in real time, the calculated new surface of the 3D model leads to the calculation of new treatment trajectories and the calculation of new guidance trajectories, receiving at least one location of a target point in space; activating at least one transmission of the transmitter and / or acquisition of the sensor of the operator device, the operator device being disposed at a distal end of the robotic arm, the activation being performed when an orientation of an axis of the sensor or transmitter coincides with a predetermined line passing through a target point, the target point being referenced on the generated body model; The present invention relates to a method for guiding a robotic arm in real time for processing of data on a surface of a body, comprising:

[0009] According to one embodiment, the body model is generated from a system for acquiring images of a patient's body, said system being positioned at a point in space. Advantageously, the acquisition of partial views of the body allows the generation of the entire body from an algorithm based on the use of neural networks.

[0010] According to one embodiment, a treatment trajectory is generated within a zone for treating a surface of a body model.

[0011] An advantage is that the generated guidance trajectory is defined with respect to a generated surface of a body model used as input, thus optimizing the guidance by simplified use of the meshed body model.

[0012] According to one embodiment, the treatment trajectory is generated automatically from a definition of a treatment zone defined on the surface of the body model.

[0013] The advantage is that it automates the task of the operator who only has to select the region, which is defined on the generated body model, thus simplifying its definition and allowing the trajectory to be easily generated.

[0014] According to one embodiment, the guided trajectory is calculated from predefined distance and orientation commands that allow generating a relative position of the distal end of the robot arm with respect to the treatment trajectory.

[0015] The advantage is to obtain a simplified guidance by the control system that takes into account the minimum of the parameters.

[0016] According to one embodiment, the robot arm includes at least three branches articulated two by two with each other via pivot links, the set of pivot links including the planned kinematics for a given guided trajectory and the generated body model.

[0017] The advantage lies in decoupling the generation of the kinematics of the robot arm which can be performed in the computer of the robotized arm according to the positions of the guidance points of the trajectory generated from the operator console.

[0018] According to one embodiment, the method of navigation includes the generation of a new body model at regular intervals so that the robot arm can be commanded in real time from the generation of a new trajectory of the robot arm, the new surfaces calculated from the body model lead to a modification of the treatment trajectory, and the method further includes the calculation of new kinematics of the robot arm controlled on the defined area of ​​the new trajectory. The advantage is to obtain a good navigation performance in real time. The consideration of the body model simplifies the navigation calculation. The control of the robot arm can be quicker, thus making it possible to suppress cases of collisions or sudden movements of the patient.

[0019] According to one embodiment, the method of guidance comprises a step of calibration of the position of the robot arm from a fixed position from which the images are acquired making it possible to generate a body model. The advantage of the calibration is to know the position of the body model relative to the position of the end of the robot arm.

[0020] According to one embodiment, the treatment trajectory is used to establish a direction of emission of a laser beam from a transmitter laser, said transmitter laser being fixed to an operator device.

[0021] According to one embodiment, the body model includes a homogenous mesh to form a new mesh defined by an arrangement of circles, at least one of which defines the location of a target point in space. An advantage is that it simplifies the activation of an operator device at the head of the robot arm according to points that can be easily located on the mesh.

[0022] According to one embodiment, the new mesh is generated as a function of the ejection power command and / or the command regarding the amount of overlap between two ejections. An advantage is to make available an adapted mesh in case of use of a robotized arm.

[0023] According to one embodiment, the sampling of the number of shots performed depends on the new mesh, the shot power set and the amount of overlap defined between the two co-localized circles. An advantage is to provide the operator with an easy to use operation console.

[0024] According to one embodiment, the treatment trajectory is used to establish a firing strategy of an ultrasound signal from an ultrasound transmitter, the ultrasound transmitter being fixed to an operator device.

[0025] According to one embodiment, a set of images is created from an optical device arranged on an operator device, each image indexing at least one node or mesh of a mesh of the body model according to a position of a target point on said body model and a viewing angle of the optical device. An advantage is in reconstructing an outer layer, such as skin, on a parameterized or parametric body model corresponding to a patient's body.

[0026] According to one embodiment, at least a pair of images are simultaneously acquired from an optical device comprising two sensors which are integral and positioned on different axes of an operator device, each image indexing at least one node or mesh of a mesh of the body model depending on the position of a target point of said body model and the viewing angle of the optical device, said images being processed to reconstruct a three-dimensional local image.

[0027] According to one embodiment, the three-dimensional local image is superimposed on the generated body model in a 3D representation.

[0028] According to one embodiment, a first image acquired by a first optical device of the operator device is compared in real time with the set of images, the comparison operation including, where appropriate, the generation of an indicator of interest, said indicator of interest making it possible to generate digital instructions for managing the robotic arm.

[0029] According to one embodiment, the guidance command leads to a first management of the robot arm at the target position from which the first image is extracted and a second management of the operator device for activating the second optical device, the activation of the second optical device allowing to create a second image in an area in the vicinity of the zone containing the target point. An advantage is in particular in generating a set of images that can be used for diagnostic purposes, i.e. for a qualitative examination of the patient and the disease or alternatively for generating a history of images for the patient containing information about the skin coverage at different scales.

[0030] According to one embodiment, the comparison step is a step of classification of said images obtained from a neural network trained on a set of images. An advantage lies in optimizing a step of detection of singular points on the patient's skin covering.

[0031] According to another aspect, the present invention provides a method for producing a method for manufacturing a pharmaceutical composition comprising: A computer for generating a plurality of body models at different times; a robot arm controlled by a control unit computer that calculates the kinematics of the robot arm from a trajectory calculated from points defined on the mesh of the acquired body model; an operator device arranged at the end of the robot arm, the operator device including at least one transmitter or sensor, the activation of the at least one transmitter or sensor being performed when the axis of the sensor or transmitter is oriented along a line passing through a target point, the target point being referenced on the generated body model; The present invention relates to a system for guiding a robotic arm, comprising:

[0032] According to one aspect, the invention relates to a system for guiding a robotic arm, comprising a device for acquiring images of a patient's body to create a parameterized body model and a robotic arm equipped with an operator device for the implementation of the method of the invention.

[0033] The system of the present invention, according to various alternative embodiments, includes the various means described in the various embodiments of the present invention. Moreover, according to various alternatives, it is configured according to the various embodiments of the method of the present invention.

[0034] According to one aspect, the present invention provides a method for producing a method for manufacturing a semiconductor device comprising: a robotic arm controlled by a control unit computer that calculates the kinematics of the robotic arm from trajectories calculated from points defined on a mesh of an acquired 3D model of the surface of the human body; A computer for generating a plurality of body models at different time points, where the generation of a new 3D model of the surface of the human body at each time point ensures the calculation of a new kinematic of the robot arm for automatically managing the displacement of the robot arm in real time, and the new calculated surface of the 3D model leads to the calculation of a new trajectory; an operator device arranged at the end of the robot arm, the operator device including at least one transmitter or sensor, the activation of the at least one transmitter or sensor being performed when the orientation of the axis of the sensor or of the transmitter is directed along a line passing through a target point, the target point being referenced on a generated 3D model of the surface of the human body; The present invention relates to a system for guiding a robotic arm, comprising:

[0035] According to another aspect, the present invention provides a method for producing a method for manufacturing a pharmaceutical composition comprising: receiving input data corresponding to the geometric data; generating a first cloud of points from the input data, each point being associated with a three-dimensional spatial coordinate; determining a plurality of attributes at each point of the first cloud of points, said attributes including at least one shape descriptor, at least one attribute of the point including data describing a normal vector at that point of the first cloud of points; associating the attributes with each point to generate a second enriched point cloud; calculating a set of geometric parameters comprising at least one rotation and at least one shape coefficient for at least one point of the second point cloud or at at least one joint of the human body model by means of regression, said regression being performed from a series of matrix operations performed according to different layers of a neural network trained from multiple partial views of the parametric model parameterized with different parameterization configurations; determining a parameterized model for generating a body model of the body including the first mesh; The present invention relates to a method for generating a three-dimensional working surface of a human body, comprising:

[0036] According to one example, the input data comprises data describing an image. The geometric data may for example be inferred from data describing a 2D image. Thus, according to one case, the input data is an image.

[0037] An advantage of the method is that it allows real-time control of the guidance of a medical device near the patient's body with a minimal acquisition system, e.g., a pair of cameras. The guidance performance is obtained in particular by a sophisticated neural network learning method.

[0038] According to an embodiment, the method further comprises processing the first mesh to generate a second homogeneous mesh. The use of a homogeneous mesh has the advantage of facilitating the command of one or more sensors or one or more transmitters in the head of the robot arm. In fact, the processing of the points of the homogeneous mesh makes it possible to facilitate the use of devices in the head of the robot arm.

[0039] According to one embodiment, a shape descriptor is calculated at at least one given point of the first point cloud from the adjacency matrix of the first graph using a method for spectral partitioning of the first point cloud. An advantage is that it takes into account the shape descriptor in the neighborhood of the point, weighted according to a distance criterion at the point to which it is applied. Another advantage is that it makes it possible to take into account the local surface topology in the neighborhood of the point.

[0040] According to an embodiment, at least one local and / or global shape descriptor is of the "Wave Kernel Signature" or "Heat Kernel Signature" or "Gaussian Curvature" type. An advantage lies in defining the shape descriptor from a spectral analysis of an area in the vicinity of a point. An advantage lies in the simplification of the use of formulas or regularized operators for which the digital processing can be optimized.

[0041] According to one embodiment, the calculation of the normal vector at a given point of the first point cloud is calculated by considering a surface locally generated around each point of interest from its neighboring points.An advantage is to parameterize the desired resolution or granularity for analyzing the local surface topology of the surface of the body to be treated.

[0042] According to one embodiment, the surface generated locally at the points of interest of the first point cloud is from an underlying surface defined by the points of the first cloud of points; or from a local surface approximation obtained directly from the k-nearest neighbor method of the point of interest It is calculated as either

[0043] According to an embodiment, at least two shape descriptors are associated with each point of the first point cloud. An advantage consists in enriching the description of the model. An advantage consists in obtaining better learning during training of the neural network. Furthermore, an advantage consists in improving the performance of the regression step of the neural network.

[0044] According to one embodiment, the first cloud of points is: Acquiring a plurality of images from at least two optical elements, where each position of a point of the first point cloud is inferred from a stereo measurement of the at least two acquired images; and / or Acquiring an image including a deformed pattern, the pattern being projected by a projector and including a known geometry; and / or Acquiring ultrasound signals transmitted to and reflected from the surface of the patient's skin; and / or Obtaining a laser transmitted to the surface of the patient's skin It is generated from data obtained by

[0045] An advantage lies in implementing the method of the invention with various acquisition techniques. An advantage lies in obtaining a point cloud that can be utilized by the method of the invention, in particular for guiding medical devices. Various acquisition techniques may be combined or substituted for each other. Nevertheless, an image acquisition system such as a camera remains an acquisition system that is simple to implement, easy to set up and easily deployable.

[0046] According to one embodiment, the parameterized body model is obtained from a parametric model of the human body in which a set of joints is predefined, said parameters being: At least one rotation angle for at least one joint; At least one shape factor and Model the following.

[0047] An advantage is to define a simple parameterization of the human body that takes into account the various degrees of freedom of each limb of the body. An advantage is to perform real-time control of the device that is guided according to the possible movements of the patient.

[0048] According to one embodiment, the parameters are: At least one global translation of the model in space, and / or At least one global rotation of the model in space Further modeling.

[0049] An advantage is that it allows for the patient's displacement to be taken into account and for the acquisition system to be calibrated on the patient's body for different equipment placements.

[0050] According to one embodiment, the parameterized body model is obtained from a given parameterization definition, the given parameterization including the definition of a set of modeling parameters making it possible to define a given representation of the parametric body model. Said parameterized body model may then be represented by a third point cloud, a graph or a mesh. An advantage consists in generating a directly usable point cloud from the obtained parameterized body. Another advantage consists in obtaining a large number of bodies according to different poses in order to obtain a good performance of generalization of the neural network.

[0051] According to an embodiment, a number of partial views are generated, each of which may for example be obtained from a subset of points of the third point cloud. The advantage is to generate a large number of views that can correspond to real-world imaging. Thus, learning on a large number of partial views allows the neural network to be better trained and the regression step to perform better.

[0052] According to one embodiment, each subset of points of the third point cloud is obtained by selection of points visible from a given viewpoint in the space, and a set of partial views is generated for multiple viewpoints in the space.

[0053] According to one embodiment, the regression comprises: Neural network convolutions or layers that involve multiple multiplications of matrices with weighting coefficients obtained from the learning method, Nonlinear Operations Includes.

[0054] According to one embodiment, the regression takes as input: the points of the point cloud and the values ​​of their attributes, and / or Adjacency matrix of the graph corresponding to the point cloud Includes.

[0055] According to one embodiment, the method comprises a preliminary step of training the neural network, said preliminary step comprising: Training performed from multiple partial views of a parametric model parameterized in different parameterization configurations, Create a matrix containing the weighting coefficients for each layer of the trained network This is performed by deep learning methods including

[0056] According to another aspect, the invention relates to a system including an interface for acquiring input data, a computer and a memory for processing the input data acquired using a method for generating a parameterized body model.

[0057] According to one embodiment, the invention relates to a system comprising means for acquiring distance data of a set of points of a surface of a body arranged in space, a guided robotic arm having at least one degree of freedom, said robotic arm further comprising a sensor or transmitter whose position is controlled by a guided trajectory obtained from a treatment trajectory defined on the surface of a partial view of a body model, said body model being generated from the method of the invention.

[0058] The system of the present invention may be configured according to the various parameters recited for describing the method of the present invention. Furthermore, the system of the present invention, according to various embodiments, includes the various means described in the various embodiments of the method of the present invention.

[0059] Other features and advantages of the present invention will become apparent from the following detailed description, taken in conjunction with the accompanying drawings, in which: [Brief description of the drawings]

[0060] [Figure 1] 1 shows the main steps of one embodiment of the method of the present invention. [Diagram 2] 1 illustrates one embodiment of a system for spatially controlling the movement of a robotic arm near a body surface to be treated. [Diagram 3]1 is a representation of one embodiment of a system of the present invention including an operation console, a positioning system and a robotic arm, manageable from the method of the present invention. [Figure 4] An example of a robotic arm of the present invention having six axes that can be guided to travel a planned trajectory. [Diagram 5] FIG. 2 is an example of a robot arm of the present invention having six axes indicating the reference points of the head of the robot arm. [Figure 6A] This is a first example of an operator device to be placed on the head of a robotic arm of the present invention, intended for example for dermatology applications. [Figure 6B] 1 is an example of a trajectory generated by the system of the present invention. [Figure 7] 2 is an example of a first mesh of points of a surface of an object to be treated according to the method of the invention. [Figure 8] 1 is a first example of a second mesh obtained from an embodiment of the present invention. [Figure 9] 11 is a second example of a second mesh obtained from an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0061] The present invention relates to a method for acquiring images of a surface of a body and for reconstructing a three-dimensional surface of said body. The present invention also relates to an image acquisition and processing system. The present invention also relates to a method and a system for activating a medical device from a modeled body model. The present invention relates to the guidance of e.g. a robotic object for the diagnosis or treatment of a skin covering of a patient.

[0062] In the remainder of the description, a system for acquiring five points on the body of the patient 1 is preferentially placed at a fixed point in space. By way of example, this point acquisition system may be fixed to the ceiling, to a wall or alternatively to any other point in space intended to remain fixed during the execution of the method of the invention. This acquisition system 5 is referenced to a reference point denoted R0(O,x0,y0,z0). It may also be referenced to a reference point R1(O1,x1,y1,z1) associated with the acquisition system 5. From the positional calibration between the "world" frame of reference and the acquisition system 5, the acquisition system 5 may be referenced to the reference point R0 with the transformation R1→R0.

[0063] In the remainder of the description, a robot arm 6 is placed in the same space covered by the acquisition field of the acquisition device 5. This may be a hospital treatment room or alternatively a room corresponding to a doctor's surgical procedure. The robot arm 6 may be positioned on a pedestal, such as a portable installation 25, and may be capable of being fixed in a given position during the procedure carried out by the method of the invention. This robot arm 6 is referenced to a reference point represented by R0(O,x0,y0,z0). The position of the robot arm 6 in space may be specified by a reference point R2(O2,x2,y2,z2) by which the movements of the robot arm 6 may be easily represented. The change of the reference point R2→R0 makes it possible to represent the movements of the robot arm 6 in the reference point of the system R0 of the acquisition 5. This change of reference point may prove to be particularly efficient during the realization of a collision detection function between the robot arm 5 and the body of the patient 1.

[0064] According to an exemplary embodiment, the acquisition system 5 may be integral with the reference point R2. In this exemplary case, the acquisition system may be fixed to a base that supports the robot arm 6. This configuration may for example be obtained by a holding rod (not shown) that allows the acquisition system 5 to be elevated. The advantage of this configuration is that it makes it easier to mount the system of the invention. Another advantage is that it makes it possible to make use of a pre-calibration during the positioning of the acquisition system 5 and the robot arm 6.

[0065] According to another exemplary embodiment, the acquisition system 5 may be fixed to a base on which the operation console 20 is mounted. This scenario makes it possible to have an autonomous acquisition device, including a hardware part and a software part. To fix the acquisition system 5, a holding rod or any other mechanical holding device may be employed.

[0066] The head 8 of the robot arm 6, also referred to as operator device 8 in the remainder of the description, is placed at the distal end of the robot arm 6. The head 8 may be related to a reference point represented by R3 (O3, x3, y3, z3) shown in FIG. 5. The position of the head 8 of the robot arm 6 can be easily identified at the reference point R2 (O2, x2, y2, z2) if the kinematics of moving the branches of the robot arm 6 are known. The advantage is to reference the images acquired by the operator device 8 to, for example, a reference point R0 (O, x0, y0, z0).

[0067] The present description applies to any type of robotized object, in particular to a robotized arm. A robotized arm of the invention may have multiple axes of freedom, in particular between 2 and 9 degrees of freedom. In the remainder of the description, an embodiment will be described considering the application of a robotized arm with 6 axes. However, the invention is not limited to this example.

[0068] According to various alternatives, the robotic arm may be fixed to a horizontal base and the first arm may extend vertically, or the robotic arm may be fixed to a vertical base and the first arm may extend horizontally. Various configurations are compatible with the present invention.

[0069] In the remainder of the description, "body model" refers to a three-dimensional model of a body. This body model may be represented by a point cloud. The point cloud may be generated from a given modeling of a parametric body model obtained by a given parameterization.

[0070] Furthermore, this body model may be used to generate various partial views of this body, corresponding to a subset of points of the point cloud that models the whole body. For this purpose, viewpoints may be defined to extract partial views of the body model. Such a body model may be obtained from a given parameterization configuration of a parametric model of the human body.

[0071] In the remainder of the description, a partial view of a "body model" refers to a three-dimensional model of a body according to a given viewpoint from which the surface of said body model is observed.

[0072] In the remainder of the description, the "parametric human body model" MOD_S refers to a model that can be parameterized to generate a transformation of a particular part of this body or a view according to a given parameter configuration. Thus, the body model defined above may be generated from a parametric human body model with a given parameterization, which is known as the parameterized body model MOD_P.

[0073] The parametric body model may be used according to a first use during the method of the invention to carry out a regression step to determine a set of parameters making it possible to generate a body model for the guidance of a medical device, which may be used according to a second use during deep learning to generate a weight matrix to be used with precision during the regression step.

[0074] According to one example, the parametric body model MOD_S has the following parameters: The global translation of the model in space, i.e., the three parameters Tx, Ty, and Tz, The global rotation of the model in space, i.e., the three parameters Rx, Ry, Rz, For example, a parametric model containing 24 joints, each with three rotation parameters, Rxi, Ryi, Rzi, and Ryi, is used to define the rotation angle of each joint, for a total of 72 parameters. For example, shape factors using a parametric model including 20 shape factors The latter may be obtained from a parametric model with inputs that affect the shape of the human body.

[0075] In this example, 88 parameters may be determined to generate the human body model.

[0076] Other examples may be implemented with the methods or systems of the present invention.

[0077] Getting points According to an embodiment, the method comprises a step ACQ of acquisition of points. This acquisition may be done, for example, by means of an image acquisition device. In an exemplary embodiment, the optical system 5 of FIG. 3 comprises two sensors 51, 52 for acquiring images IM1, IM2. These may be cameras, for example a 3D camera comprising at least two color or infrared cameras. According to various exemplary embodiments, color or infrared cameras may be used. Preferentially, a pair of identical cameras is selected. A pre-calibration step may be performed to set the color of each camera and to associate them for the same point in space.

[0078] To create a 3D reconstruction of the object, another inter-camera calibration, known as "stereo calibration", may be performed between the two cameras, which aims to calculate the transformation between the two images acquired by each of the cameras.

[0079] According to one example, this transformation may include the detection of interest points and the calculation of a descriptor obtained by the scale invariant visual feature transform, better known by the abbreviation SIF. A correlation step may be implemented with the aim of associating the interest points with each other, together with an estimation method, for example the method known as "RANSAC" for "RANdom SAmple Consensus". Then, an isometric transformation calculation between the detected interest points is performed. Finally, the transformation makes it possible to triangulate the image points and thus to build a depth map.

[0080] According to a first method, 2D images of the same object acquired from two viewpoints of known geometry may be combined according to known techniques to obtain a so-called depth or distance map.

[0081] The method of the invention then makes it possible to process the 2D images that are combined to obtain a cloud of points G1, which constitute a first graph G1. According to one example, the points of the obtained first graph G1 are undirected and unconnected. In its simplest form, the graph G1 is an unordered list of points whose coordinates are relative to a space, for example R0, R1, R2 or R3, where each point is defined independently of the others. In its simplest form, the graph G1 is thus a cloud of points.

[0082] The images IM1, IM2 may be acquired simultaneously. Advantageously, the optical elements are oriented along parallel axes and are co-localized. An advantage is that the images can be processed by stereo techniques to generate a depth map. The depth map allows the position of the spatial coordinates x, y, z at multiple points to be specified.

[0083] According to one embodiment, the color of the image is used to generate the color tone difference and thus the distance variation.

[0084] The depth map may be obtained by encoding the obtained distances between the different points, the resolution of which may be set. The greyscale may vary according to a first range of values. By choosing, for example, an encoding in 16 bits, the encoding of the depth map into greyscale makes it possible to obtain with high precision the distances separating the different points.

[0085] The method of the invention allows for the continuous acquisition of image streams by optical elements 51, 52 in order to process the image streams in real time. The designations IM1, IM2 refer not only to the acquired images but also to the image streams acquired by each of the cameras 51, 52.

[0086] The images may be processed to extract an acquisition boundary that limits the zone of interest for treatment. To do this, a template may be used to avoid considering points outside the zone of interest defined by the surface of the body.

[0087] The acquisition of two images IM1, IM2 of the entire surface of the treatment object makes it possible to generate a depth map of the entire body, which comprises a set of points and their distances relative to each other.

[0088] According to another example, at this stage of the method, there is no surface to be treated: the distances between the points are generated independently of any shape recognition, such as the outer layer of the body.

[0089] According to one example, the number of points generated per surface unit can be set depending on the desired resolution of the point cloud: for this purpose, the acquisition may be configured to acquire more or fewer images to increase the resolution.

[0090] According to one embodiment, the positioning system SYS_POS makes it possible to define a reference frame and a coordinate system for locating each point in a single reference frame. This positioning system may advantageously be the acquisition device 5.

[0091] Stereo calibration makes it possible to obtain the transformation between R0 and R1, the reference point R1 being associated to the acquisition system 5. The method of the invention then makes it possible to calibrate the robotized object 6 with respect to the acquisition system 5 and thus to carry out a transformation of the images in the reference frame R2 → R1 → R0. If the reference frame R3 is associated to the head 8 of the robotized arm 6, the transformation R3 → R2 is known due to the fact that the kinematics of the joints of each arm are known by the computer and the memory of the system managing the robot arm 6.

[0092] Other Acquisition Techniques According to an embodiment, a projection device is used to project a pattern onto the surface of the body. The acquired images may then be processed to analyze the deformation of the projected pattern. This technique allows obtaining additional data and improving the 3D reconstruction. In particular, the detection of the deformation of the projected pattern allows improving the accuracy of the 3D reconstruction. In particular, the detection of the deformation of the projected pattern allows improving the accuracy of the evaluation of the distances calculated to generate the depth map. Furthermore, the projector allows reducing noise during the construction of the point cloud to generate the first graph G1.

[0093] As a result, a stereoscopic reconstruction of the depth map may be obtained by a pair of cameras, optionally used together with at least one pattern projector. The projected light may be in the visible range or elsewhere, for example in the infrared range. According to one example, the acquisition device 5 may include a pair of color 3D cameras and at least one infrared projector.

[0094] According to another embodiment, the camera used may be an infrared camera or one having a frequency range in the near infrared.

[0095] Other techniques for constructing a depth map may be used, such as laser transceiver devices. The reflection of a laser beam makes it possible to generate data on the geometry of the reflecting surface. Another technique may be to use an ultrasonic transceiver. One example of a system that may be implemented is a LIDAR type system.

[0096] According to one embodiment, the method of the invention makes it possible to recover a point cloud derived from the acquired images and transfer it to a graphics processor, for example a GPU (Graphic Processing Unit).

[0097] Creating the first graph G1 The method of the invention comprises a step aimed at generating a set of points associated with spatial coordinates, for example with respect to a spatial reference point R0. These coordinates may be obtained from a prior calibration of the positioning system SYS_POS. The calibration may be performed from a specific pattern acquired by the optical elements of the positioning system SYS_POS at the start of the method of the invention. According to one embodiment, the positioning system SYS_POS and the optical system SYS_OPT may be the same system. According to another embodiment, they may correspond to two separate systems.

[0098] Thus, the depth map obtained by the acquisition system SYS_OPT may be used to obtain a set of spatial points associated with coordinates at the spatial reference point R0.

[0099] The method of the invention comprises a step REC_DATA1 of receiving data, namely data DATA1 corresponding to a point in space referenced to a reference point R0 in space.

[0100] According to one example, the method of the invention may comprise a step aimed at segmenting data associated with a predefined surface, which may for example be the surface of a human body. This segmentation may be performed automatically by analysis of data, such as data derived from the acquired 2D images and / or the obtained depth map. For example, the method of the invention may comprise an analysis of the difference in distances between certain points, or alternatively performing a colorimetry process. The depth map may be used to filter the point cloud and keep only points of interest, i.e. points of the human body.

[0101] According to another example, the surface contour may be obtained from an operating console, such as the PC 20 shown in Fig. 3. According to this example, the contour of the body surface may be obtained by an operator using a selection tool, such as a stylet or a mouse, and an image displayed on the screen of the console 20. According to another example, the selection may be made by tactile or voice commands on the screen. The advantage consists in ignoring points that belong, for example, to the table 22 on which the patient 1 rests.

[0102] The method of the invention includes the generation of a first graph GEN_G1. The first graph G1 may be obtained from a matrix representation of a set of generated points of the surface of the body of the patient 1. Each point is represented by a matrix of v i and in particular by a vector containing its coordinates at the reference point R0. According to one embodiment, at this stage the first graph G1 is unordered in the acquisition frame of reference. The notation "graph" is used herein even in the simplest embodiment or the set of points corresponds to a list of points with coordinates, i.e. a simple cloud of points. In other embodiments the graph may be connected or not, oriented or not, etc.

[0103] filtering According to one embodiment, the method of the invention comprises a step of filtering the graph obtained, which may be a temporal and / or spatial filtering, which step makes it possible to reduce the noise generated during the generation of the first graph G1, continuing up to the detection step.

[0104] Determining Attributes The method of the present invention calculates the i Step DET_AT to determine at least one attribute for i The attributes include the points v i These attributes may be of different nature, such as Cartesian data, shape descriptors, identifiers, vectors, etc.

[0105] According to one embodiment, the attribute is i Normal vector n in i Contains data describing the normal vector n i may be determined in a variety of ways. The advantage of calculating the normal vectors as attributes of the points of the point cloud is that it yields information about the convexity of the local surface.

[0106] According to one embodiment, for a given point v of the first graph G1, i Normal vector n in i is the vector for each point v i A point v in the neighborhood of iLk From point v i is calculated by considering a surface generated locally around the point. For this purpose, the spatial coordinates of each point are compared with the coordinates of the point of interest. According to one example, the acquired distance or depth data is used to identify points in the vicinity of the point of interest.

[0107] According to a first example, the surface to which the normal vectors are calculated is calculated from an underlying surface defined by the points of a first graph G1.

[0108] According to a second example, the surface of interest for which the normal vectors are to be calculated is calculated from a method implemented on a GPU. According to another example, the normal vectors are calculated from a local surface approximation obtained directly from a method of k nearest neighbors of the point of interest. However, this method of triangulation of a list of 3D points can be more expensive than a method implemented on a GPU.

[0109] A possible use of computing normal vectors at the points of a graph is to augment the description of the points to enrich the point cloud.

[0110] Shape Descriptor According to one embodiment, the attribute is: i The shape descriptor is a local descriptive feature of the space at a point.

[0111] According to one example, the shape descriptor Dfi is i The point of view is calculated from a subset of points in the vicinity of the body model. Thus, the method includes generating a graph of reconstructed or extracted points from the viewpoint of each image acquisition device that can utilize these data and from the body model. As an example, if the acquisition device is located at the distal end of a robotic arm, a partial view of the body model may be generated from the position of the acquisition device to perform a particular procedure.

[0112] A shape descriptor may be estimated at each point of a 3D body model obtained from the point cloud. The shape descriptor remains invariant during isometric transformations such as rotations of the body model, as long as it is symmetric and isotropic. This has the advantage that the shape descriptor is permuted regardless of the viewing angle considered.

[0113] An advantage consists in performing the calculation of shape descriptors directly on the points of the first graph G1. Thus, the invention makes it possible to optimize the calculation of these shape descriptors, which are not applied to parts of the surface, but are applied directly to predefined points of the graph. The calculation of the descriptors may be performed on an adjacency matrix of a connected graph obtained from the graph G1. A connected graph implies that each point is connected to all other points. The adjacency matrix makes it possible to obtain the points that have a direct connection with a given point. Furthermore, the proximity of a point with respect to another point may be inferred from the adjacency matrix. The proximity of a point may be inferred from the adjacency matrix according to the shape descriptor.

[0114] Alternatively, the method of the present invention allows for the application of a distance threshold so that a point is only connected to its N nearest neighbors.

[0115] According to one embodiment, the shape descriptors are chosen to be isometric and invariant. The advantage is to obtain shape descriptors that remain invariant when body movements involving isometric transformations occur.

[0116] According to a first example, the shape descriptor is of the type "wave kernel feature" denoted by WKS. According to a second example, the shape descriptor is of the type "thermal kernel feature" denoted by HKS. According to a third example, the shape descriptor is of the type "Gaussian curvature".

[0117] According to various embodiments, a number of shape descriptors Dfi are calculated at each point of the object of the first graph G1 and associated with each of said points.

[0118] Particular shape descriptors are those that belong to the group of spectral shape analysis methods, such as the HKS or WKS descriptors. The HKS descriptor is in particular based on the solution of the heat equation, which is based on the Laplace-Beltrami operator.

[0119] One advantage of these shape descriptors is in particular their invariance under certain transformations, in particular isometric transformations. Moreover, they may be computed from the adjacency matrix of the graph G1. An advantage is that they make it possible to take into account the relative distance of a point of interest with a set of its neighbours.

[0120] According to one example, the method of the present invention may be used considering a threshold applied to the adjacency matrix, so as to consider only the subset of points whose distance is greater than a given threshold.

[0121] The WKS descriptor is itself based on the solution of the Schrödinger equation. The advantage of the WKS descriptor is that it allows a spectral separation of the modelling obtained by the operators.

[0122] Shape descriptors may be chosen to model local or global 3D shapes. If these descriptors encode local modeling, an effective radius may be introduced. However, global modeling will benefit from the completeness of the geometric invariance of the operators used.

[0123] A shape descriptor "Global Point Signature", denoted GPS, may be used. This descriptor is based on an operator of the Laplace-Beltrami type. The advantage of this operator is that it makes it possible to obtain a global modeling.

[0124] A shape descriptor "Gaussian curvature", denoted GC, may be used, which offers the possibility to consider parameterized surfaces and thus, inversely, the points that locally define the surface.

[0125] A descriptor, denoted SGWS, "Spectral Graph Wavelet Signature", may be used to define a multi-resolution local descriptor.

[0126] The advantage of shape descriptors based on spectral shape analysis is that they can be mathematically processed to assemble connected or adjacent surfaces, partition zones, and implement shape recognition algorithms.

[0127] The advantage is that it expands the description of the points and thus improves the regression performance of the neural network.

[0128] Another advantage of the shape descriptor is that it generates identification information at the points of the first graph G1. If the patient's body moves or changes over time, the body surface is considered as a deformable object whose transformations are non-isometric. The shape descriptor makes it possible to generate specific features at each point that can be found again by the method of the invention after deformation.

[0129] Attribute Association Therefore, each point vi of the first graph G1 has coordinates {x i ,y i ,z i}, normal vector n i , a set of shape descriptors {Df i1 ,D.F. i2 ,D.F. i3 ,…,Df iN This step may be associated with ASSO_AT in FIG. i It is expressed as:

[0130] point v i may then be constructed as a finite dimensional vector with each dimension corresponding to a given attribute. A second graph G2 may then be generated. This second graph G2 may be constructed as a finite dimensional vector with each dimension corresponding to a given attribute AT i corresponds to the list of vectors augmented by

[0131] Regression A computational step that allows a three-dimensional reconstruction of the human body to be obtained and augmented with an approximation by a parametric human body model. This step is denoted by REG in Fig. 1. The goal of the regression step is to estimate given parameters and to set these parameters in a parameterizable body model. Thus, from partial views acquired by the cameras, a complete parameterized body model may be estimated.

[0132] For this purpose, the method of the invention then performs a shape and rotation ROT i The method includes calculating a set of geometric parameters including the parameter.

[0133] Rotations may be calculated at a number of points of the second graph G2. According to one embodiment, rotations are calculated for each point of the second graph G2. According to one embodiment, each rotation may be associated with a joint. A joint may be associated with a bounded region of the parametric body model MOD_S, and thus a given joint may be associated with a parameterized body model MOD_P. Thus, a given joint may be associated with a set of points of the second graph G2 at the end of the regression operation.

[0134] The shape parameters may be defined by a set of point locations of a parameterized body model at a given reference point.

[0135] According to one embodiment, the input parameters are in the Euclidean space R 3 is the set of points in point cloud G1 or G2 with attributes computed at

[0136] According to an alternative, the input parameters may be the adjacency matrix corresponding to the graph G1 for each point of the point cloud. These inputs allow an analysis in the spectral domain of the inputs and do not necessarily require taking into account attributes.

[0137] The shape parameters and rotations of the set of joints correspond to the output parameters of the neural network. These parameters will make it possible to configure the parametric body model MOD_S to generate the parameterized body model MOD_P.

[0138] According to one embodiment, the regression step involves a series of mathematical operations such as matrix operations between an input matrix and a weight matrix pre-trained by a deep learning method.

[0139] Each of its operations results in a nonlinear function. These are activation functions or transfer functions that make it possible to generate the outputs of the nodes of the network. By way of example, nonlinear functions such as ReLU, Sigmoid or Tanh may be performed.

[0140] According to one example, this step includes:

[0141] It involves regression performed from a series of convolutions in a layer of a neural network trained from

[0142] These can be convolutions or layers of a neural network. In fact, a network may contain convolutions in its first layer, or a fully connected layer of neurons at the end of the model. In this case, they are neurons connected to all the neurons of the preceding layer and connected to all the neurons of the following layer.

[0143] A convolutional layer may involve scanning an input matrix resulting in a series of matrix calculations. Other layers of neural networks generally involve matrix calculations on the size of the input matrix. In all cases, the steps involve matrix calculations.

[0144] According to one example, each convolution involves a matrix multiplication between an input matrix and a weight matrix and consideration of an additional bias.

[0145] The application of processing by subsequent layers in a neural network involves the application of a series of matrix multiplications that result in a nonlinear function to generate the layer's output. This series of operations defines the depth of the neural network.

[0146] According to an exemplary embodiment, the neural network is a multi-layer perceptron, known by the abbreviation MLP. According to one example, the neural network may be a network corresponding to MLP.

[0147] According to one example, a neural network is constructed in such a way that the results are invariant to the permutation of the points in the input list, which is made possible by the use of pooling functions such as those known under the names "MAX POOLING" or "AVG POOLING".

[0148] According to one embodiment, the structure of the neural network allows extracting attributes of the input at different scales: the neighborhood of each point of the point cloud at different scales is subsequently considered to extract an encoding in a reduced space of the input dimensionality.

[0149] This encoding is then passed to a series of fully connected layers to perform regression on a set of parameters of a parametric human body model.

[0150] Subsequently, the steps of sampling and / or grouping together neighbourhoods of sampled points and then grouping these points together are repeated and encoded by a convolutional layer to produce a map of the resulting parameters.

[0151] The method of the invention advantageously makes it possible to generate a function that preserves the symmetry of the set of points at the input of the network with the set of points at the output of the network. The application of neural networks to disconnected and undirected graphs may thus be ensured by the implementation of this symmetry-preserving function, which ensures the independence of this step with respect to the ordering of the input points.

[0152] This function also allows the implementation of autoencoders in the regression process.

[0153] The set of parameterized models may be obtained from a generic model. The advantage of such a model is that it is possible to generate several poses of the human body from the parameterization of the joints and the shape parameters. The particularity of such a body model is that it can be modeled by a set of joints, each having at least one rotation and position. The rotation of the joints may be defined by an angle. The position of the joints may be associated with at least one point of the graph or at least one position of several points of the same graph. According to another example, the position of the joints is obtained from an operation of combining different positions of the graph to generate a new position of, for example, a new point. This new point may be, for example, the centroid of a subset of the points of the graph. The shape parameters may be parameterized to define the global shape parameters of the body.

[0154] A set of configuration data for a set of joints in position and rotation makes it possible to define a given pose of the body model. Thus, multiple poses of the body model can be generated.

[0155] According to one embodiment, the parametric model of the human body is a meshed network of points, where each joint that is parameterized generates a configuration that associates a subset of the points of the meshed network of points.

[0156] According to this example, 24 joints may be defined from 72 rotation parameters, each of which is associated with three rotations. The parametric model may include three global rotations. A subset of the joints, such as the wrist, elbow, etc., may be extracted to implement an embodiment of the present invention.

[0157] Each joint may include a set of shape parameters such as wrist width, wrist length, etc.

[0158] According to an exemplary embodiment, a PCA (Principal Component Analysis) function allows to extract general parameters of the configuration that can be acted on a set of parameters of multiple joints. As an example, individual size, abdominal thickness, obesity level, pelvic size, etc. may be set to pre-adjust a set of local parameters related to multiple joints.

[0159] The parameterization of the body may include multiple vectors that define the shape and angles. As an example, this may be the ankles, waist, arms, etc. The general configuration makes it possible to obtain a body contour or a given template that can be parameterized.

[0160] Other parameterizable body modeling may be used with the methods of the present invention.

[0161] Neural Network Training Neural networks are trained by automated learning, better known as "machine learning."

[0162] Such a machine learning algorithm may include a step of reconstruction of a database of high-resolution human body scans, followed by a step of deformation of a parametric model body template by application of an isometric transformation, which may include translations and rotations and combinations thereof applied to different faces of the template to fit the scan derived from the reconstructed data. This step may include the execution of a cost function for optimization, for example by gradient descent.

[0163] The method involves pooling the transformations made for each scan. The principal components of the transformations are then estimated by an ACP type method.

[0164] The application of this algorithm makes it possible to extract the main shape and position parameters of the human body.

[0165] A ray tracing algorithm may be applied to generate a set of partial views VP of a set of parameterized body models resulting from multiple parameterizations of a parametric body model. The algorithm can take into account a large number of different viewpoints in order to generate a large number of partial views VP from a given parameterized body model. An advantage is that it avoids a calibration step during the execution of the method of the invention and is closer to the real world when the images of the patient's body are acquired. The combination of learning and a large number of partial views makes it possible to avoid steps of manipulating the body, such as translating or rotating the acquired data representing the patient's body.

[0166] The advantage of this solution is to generate a number of body models with parameters that vary with different partial views of the body model. These partial views VP correspond to subsets of the point cloud that defines the entire body. The advantage is to train the neural network with a number of configurations that can correspond to the case of acquisition of a real body during the use of the method of the invention. The viewpoints correspond for example to the shooting positions of the camera. In order to constitute the most complete possible library of data representing the body model, the viewpoints are multiplied around each body surface generated from the parametric model. Thus, a number of body surfaces represented according to different viewpoints are stored in the memory. The data set generated for training the neural network for the purpose of performing the regression step is a set of partial views VP of a number of parameterized body models MOD_P.

[0167] An advantage consists in training the regression weights with shape descriptors associated to each point of the body model as it can be perceived in reality. A shape descriptor may be obtained for each partial body model view VP generated and for each point of a given body model. The shape descriptors obtained by learning make it possible to define, for each layer of the neural network, a weight matrix, otherwise called a matrix containing weighting coefficients. In the process of regression, which is carried out in real time during the calculation of the patient's body model, the regression step is advantageously carried out with a weight matrix defined by the learning step.

[0168] The parameterized body model MOD_P thus generated thereby makes it possible to train a neural network used to carry out a regression step REG, the regression then being carried out in real time making it possible to extract parameters characterizing the body of the subject.

[0169] This step obtains a body model whose surface can be easily extracted, and makes it possible to take into account the complete parameterized body model and not just the part visible to the camera.

[0170] The advantage of the method of the invention is that it reconstructs the three-dimensional surface in real time by inference. The solution is based on the use of models trained by deep learning techniques. The advantage is that it is possible to dispense with optimization methods, such as iterative methods with gradient algorithms, which require longer calculation times and integration limitations for real-time control of the robotized arm. The advantage of the method of the invention is that it realizes the optimization during learning and not during the processing of data obtained during the guidance of the robotized arm.

[0171] Deep learning inference involves matrix multiplication operations with previously trained weights.

[0172] According to one embodiment, the layers of the neural network have a resolution that depends on the layer of interest. The scale may increase as one descends into the layers of the network. An example of a neural network may be used, such as an MLP network.

[0173] First mesh M1 Thus, the method comprises a step of extraction of a mesh-like surface. The resulting surface mesh is a first mesh, denoted M1. According to one embodiment, the first mesh M1 is a mesh in which the points of the object's surface together with their neighbouring points define triangles. Figure 7 shows the extraction of a triad of points n colocalized in the object's surface. i 4 shows an example of a mesh M1 including a number of triangles formed by

[0174] According to alternative embodiments, other mesh patterns, i.e. quadrilaterals, squares, pentagons and generally any type of polygon whose vertices are mesh points, may be considered in other embodiments of the invention.

[0175] The mesh M1 is generated after the regression operation: Before the regression step, the point cloud is considered such that each point is independent of each other and the points are not related together.

[0176] Second mesh M2 According to one embodiment, the method of the invention comprises a step of defining a second mesh M2 from a first mesh M1. The second mesh M2 may advantageously be defined so as to generate semi-homogeneous triangles, for example isosceles triangles. In this example, the triangles may be transformed according to a given algorithm, for example using the symmetry of the mesh to be generated.

[0177] 8 represents an exemplary embodiment of a second mesh M2 obtained from the transformation. The mesh M2 is a mesh obtained by transforming the first set of points n i and a second set of points n generated from constructing the bisectors of each of the triangles to create two new isosceles triangles. i' includes the set of points including and.

[0178] Other techniques may be employed to homogenize the mesh with the aim of regularizing the mesh. According to another technique, equilateral triangles are generated as represented in Fig. 9. The advantage is that it generates a perfectly homogenous mesh, which makes it possible in particular to facilitate the guidance of the operator device 8.

[0179] The method of the invention makes it possible to apply a transformation to the points of the mesh so that each new point generated has its attributes transformed according to its Cartesian position.

[0180] According to one example, the desired resolution of the mesh M1 or M2 is obtained depending on the parameterization of one or more sensors or one or more transmitters at the end of the robotized arm. The resolution of the graph G1 is set as being of variable size and may be independent of the resolution of the mesh M1 or M2. By way of example, the circles 36, 37, 38 represent the desired aim points of the power to be delivered by the laser on the surface of the skin of the patient 1. The advantage of this solution is that it generates a dermatological resolution that is suitable for the treatment to be applied.

[0181] Thus, Fig. 8 shows circles 36 of different sizes obtained by a mesh with isosceles triangles of different dimensions. A coefficient must therefore be applied at the moment of emission to adapt the power applied to the signal. According to another example, in order to obtain triangles of comparable dimensions, the second mesh M2 may be homogenized by a series of geometrical operations.

[0182] Figure 9 shows the aim points 37 or 38 of the power to be delivered by the laser on the surface of the skin. The advantage of this mesh configuration is that it is possible to obtain the same injection settings from triangle to triangle. The triangles 38 are tangent to each other, allowing the overlap parameter to be controlled.

[0183] According to one embodiment, a new mesh M3 is defined by circles 36, 37, 38 generated from the second mesh M2.

[0184] Real-time Control According to one embodiment, the acquisition device 5 performs in real time an acquisition ACQ of all or part of the body points of the patient 1 in order to calculate a parameterized body model MOD_P from the method of the invention adapted to govern the movements of a robot arm 6. The guidance of the robot arm 6 is performed from a treatment trajectory TRAJ1, represented by 30 in Fig. 4, which is set within a defined zone 9 of the surface so modeled and recalculated.

[0185] The trajectory TRAJ2 of the robot arm 6 may be generated automatically from the treatment trajectory TRAJ1 depending on the orientation and the relative distance between the head 8 of the robot arm 6 and the surface of the skin to be defined, for example by means of a user interface UI. The orientation is determined by default to 90°, i.e. the main axis of the sensor or transmitter of the head 8 of the robot arm 6 coincides with the normal N to a target point on the surface of the subject's body, in particular of a pre-generated parameterized body model MOD_P. The distance is determined by the distance between the sensor or transmitter point of the head 8 of the robot arm 6 and the target point P on the surface of the modeled body generated at the patient's position. TARGET The distance corresponds to the distance to the aim point expressed as

[0186] An advantage of a real-time calculation of the modelled surface of the patient's 1 body is to anticipate movements of the patient 1, for example to take into account movements caused by the patient's breathing.

[0187] An advantage of the method of the invention is that it is possible to generate new modeled surfaces in less than 30 ms. These response times make it possible in particular to control the movements of the robot arm 6 fast enough to anticipate collisions or sudden movements in order to leave the scene as quickly as possible. The camera acquisition settings can be for example 10 to 90 fps.

[0188] The real-time control of the robot arm 6 makes it possible to activate at least one transmission of the transmitter and / or acquisition of the sensor of the operator device 8. The activation then occurs when the axis of the sensor or transmitter is oriented to the target point P TARGET This is executed when the target point P is perpendicular to the line that passes through it. TARGET is referenced on the parameterized body model to be generated. It is generated at the position and orientation of the operator device 8. The target point P TARGET may be obtained before the guidance of the robot arm 6, for example during the planning of the laser or ultrasonic firing strategy. TARGET is obtained when activating a different detector, for example a comparison of images realized upon stimulation activates a change in stimulation.

[0189] In all cases, the target point P TARGET is defined according to a guidance strategy and its position may be modified when the surface of the body object undergoes transformations that require adaptation of the guidance strategy.

[0190] According to one embodiment, the acquisition ACQ made by the device 5 may be combined with the acquisition of images made by an optical element arranged at the end of the robot arm 6. This may be the case when the movements of the robot arm 6 cover a zone of interest of the body of the patient 1 that is to be acquired by the acquisition device points 5, or when small amplitude movements are controlled by an optical element at the tip of the robot arm 6.

[0191] Thus, two modes of control may be implemented, possibly combined, to generate new modeled surfaces during the course of a procedure.

[0192] Robot Arm The robot arm 6 is guided by a motion controller GEN_TRAJ, which takes into account the various constraints for controlling the robot arm. In particular, this controller takes into account the planning of the set trajectory and the definition of the zone to be treated by the user, whose surface is defined on the surface of the reconstructed body according to the method of the invention. Furthermore, the controller CTRL may take into account the commands coming from the collision detection module for adapting the progress of the robot arm in real time. Finally, the controller CTRL takes into account the data coming from the head of the robot 6 via the module CNTR_8, for example when the detection of a single image requires a specific manipulation of the robot.

[0193] According to one embodiment, the robot arm 6 comprises a set of branches 61, 62, 63, 64, 65 articulated by means of pivot joints 612, 623, 634, 645, 656. The robot arm 6 comprises five branches, in this case articulated from the pivot links. According to an alternative, it may comprise six articulated branches, and according to other alternatives, seven, eight or nine articulated branches. Indeed, in this embodiment, each joint is capable of performing at least one rotation between two adjacent branches.

[0194] According to an embodiment, each branch of the robot arm includes a force sensor. The advantage is that it prevents sudden collisions. When contact is established between a branch of the robot arm 6 and an object, the movement of the robot arm may be automatically stopped or its guidance may be modified to change its trajectory.

[0195] According to one embodiment, a step of calibration of the robot arm 6 is carried out prior to the method of the invention. The calibration may include the relative positioning of the robot arm with respect to a reference point R0 and its initial position.

[0196] Operator Devices and Optical Elements According to one embodiment, the robot arm 6 comprises an operator device 8, which is advantageously arranged at the end of the robot arm 6. According to said embodiment, the optical element 67 may be arranged off-axis with respect to the head 8 of the robot arm or may be arranged at the end of this head, as is the case in Fig. 6A.

[0197] The advantage of the configuration of Fig. 3 is to equip the head of the robot arm with, for example, a laser device or a device for generating ultrasound for localized treatment, replacing the image acquisition optics. The advantage of the configuration of Fig. 6A is to configure the robot arm 6 for dermatological imaging applications. However, any configuration may be envisaged, as well as combinations of the aforementioned cases.

[0198] According to one example, the operator device 8 includes at least one sensor and / or at least one transmitter. The operator device 8 is also referred to as the “head” of the robot arm 6.

[0199] In the example of Fig. 5, the robotized arm 6 comprises one or more optical sensors 67 for acquiring local images of the surface of the skin. The advantage of acquiring local images at the head of the robot arm 6 is that it is possible to reconstruct a "colored" body surface, i.e. the acquired images are integrated with the parameterized body model MOD_P generated by the method of the invention. For this purpose, the optical device 67 or {83, 84} arranged at the end of the robot arm may preferably comprise two optical elements so as to reconstruct a depth map associated with the images acquired simultaneously by the two optical elements. These images may then be indexed on the parameterized body surface.

[0200] Other transmission / acquisition devices Furthermore, the operator device 8 may include an ultrasonic, radio frequency or laser type transmitter or any other signal source that can be used to generate an incident beam. According to one embodiment, the operator device 8 may include an array of focused transmitters and means for controlling the direction and transmission power.

[0201] An ultrasound transmitter may be arranged at the end of the operator device 8 and an ultrasound receiver may be associated with the measurement of the reflected signal so as to construct an image of the dermis and / or an image of the patient's skin thickness, or even an image of blood vessels or organs.

[0202] Fig. 6 represents another embodiment in which the operator device 8 comprises a receiving head 80 positioned at the end of said operator device. The receiving head 80 comprises two local optical elements 83, 84 for taking images of the surface of the body and a millimetre or microscopic optical element movable in two directions along a longitudinal direction 86. The advantage is to generate images with millimetre resolution, including resolution for dermatological applications, e.g. diagnostic applications.

[0203] Robot arm guidance First, the generation of the movement of the robot arm 6 is defined by means of a trajectory set by the user using the component represented by PATH in Fig. 2. The user can select the zone of interest on the surface of the body of the patient 1, such as the zone "arm", "leg" or a part of this zone or alternatively the whole body.

[0204] The selection of the zones may be performed using a user interface UI. The zones may be highlighted using a stylet or using a mouse, or alternatively by selection of one or more predefined zones selected by the user. According to another example, the selection may be performed using direct tactile commands on the display or using voice commands.

[0205] The selection may be made on a parameterized body model MOD_P generated by the method 10 of the invention. The module allowing the selection of the zone is represented in FIG. 2 by ROI. The parameterized body model MOD_P may be displayed on the display of the user interface UI. Once the zone is selected, an extraction of a portion of the surface on which the trajectory of the operator device 8 is to be defined may be performed. According to an example, the trajectory 30 may be automatically defined from the dimensions of the selected zone. Depending on the operations to be performed with the operator device 8, for example imaging, laser treatment and / or treatment from an ultrasound source, the automatic trajectory generation function may be implemented in different ways. If this requires performing imaging in a predefined zone, the objective is to cover the entirety of the selected zone. Thus, depending on the resolution of the optical elements and their range, an optimized trajectory may be calculated.

[0206] If the user himself defines a trajectory over a selected zone, the trajectory is obtained by the component PLAN_TRAJ, which will optimize or calculate the progression parameters of the robot arm to cover the selected surface.

[0207] The method of the invention makes it possible to carry out a real-time guidance of the robot arm 6. For this purpose, a module GEN_TRAJ is implemented for generating the movements of the robot arm 6. Said module makes it possible to recalculate in real time the path of the robot arm 6, in order to calculate a trajectory in real time, for example depending on the patient's movements, the taking of a single image or alternatively to control the end of the progression upon contact with an object.

[0208] The guidance of the robot arm 6 is set, for example, from a Cartesian coordinate system referenced to a reference point R0. The orientation of the end of the robot arm 6 may be set by controlling the orientation of the normal N shown in FIG. 3 or FIG. 5. The normal N is defined at an angle of 90° with the aim point of the body surface. The advantage of this parameterization is that each point of the expanded mesh already contains an attribute corresponding to the normal of the model point. Thus, the guidance of the orientation of the head of the robot arm may be performed by orienting the robot to the normal N of the body surface.

[0209] According to one embodiment, the distance between the end of the head 8 of the robot arm 6 and the aim point 35 on the modelled surface of the body of the patient 1 is controlled to remain substantially constant, for example at a distance of 5 cm. Thus, the guidance of the robot arm 6 is controlled according to its orientation and distance with respect to the skin of the patient 1.

[0210] A collision detection function DET_COLLI may then be implemented to supply control instructions to a movement generation module GEN_TRAJ of the robot arm 6. Furthermore, this module takes into account the initial trajectory established by the component PLAN_TRAJ. Finally, the movement of the robot arm may be managed according to a control function CNTR_8 of the head 8 of the robot arm 6. If one or more optical devices are mounted on the head of the robot arm 6, control of the robot arm may be necessary to activate a second optical element.

[0211] According to an embodiment, the optical system 5 is configured to control each movement of the body 1 by means of a control loop. This control makes it possible to compare the difference between simultaneously acquired images with new images acquired in succession. The advantage of such a control is the correction of the guidance of the robot arm 5 by adapting its initial progression to the new trajectory. For this purpose, a collision detection function DET_COLLI may be implemented to adapt the progression movement of the robot arm 6. The generation of the movement of the robot arm 6 is determined by the function GEN_TRAJ.

[0212] According to one embodiment, the head of the robot arm 6 is removable so that it can be used for different applications: a head containing one or more optical devices for dermatology applications may be replaced by a head containing a laser for applications treating the surface of the skin, such as hair removal, tattoo removal, or Ultherapy for lifting applications.

[0213] Depending on the target application, a given optical configuration may be preferentially selected.

[0214] According to one example, an arrangement of optical elements with different focal points is implemented at the end of the head of the robot arm 6. These optical elements may be articulated or displaced independently or jointly depending on the embodiment.

[0215] According to one example, the at least one optical element is a confocal microscope system. According to one example, the focal plane may be displaced to create images at different depth levels. A local three-dimensional image of the surface of the skin may be reconstructed. This application may find advantage for reconstructing volumes such as moles on the surface of the skin.

[0216] When confocal optics are used, illumination may be obtained from a light source such as a laser.

[0217] According to another example, the at least one optical element is configured to create a series of cross-sectional images for implementing a tomography technique. According to a given configuration, this technique may be of the OCT (Optical Coherence Tomography) type. These techniques often require coupling a sensor with a transmitter, such as a laser transmitter.

[0218] Other possible applications concern treatments from ultrasound transmissions. For this purpose, at least one ultrasound transmitter may be placed at the level of the head of the robot arm. This or these transmitters may be associated with a sensor for measuring the echo signal or the modification of the transmitted waveform. According to one example, this may be a matrix of transmitters organized to generate a beam that is focused on a point. The power may be set to burn tissue by generating a sound pressure on the surface of the skin or to measure the elasticity of a singular point on the skin covering.

[0219] According to one embodiment, the transmitter is configured to treat the subcutaneous region. TARGET An image point may be defined with respect to the target point P, the point image being located on the surface of the body model. TARGET The model is located a few millimeters or even centimeters below the surface of the skin, following a normal to the surface of the model that passes through

[0220] The transmitter is guided to the target point P TARGET Alternatively, the guidance of the transmitter may be performed directly from the definition of a point in space to be reached, defining a point image, said point image being located under the patient's skin. This point is called the target point P TARGET If not defined with respect to TARGET is used to acquire images or to plan a corrected trajectory of the robot.

[0221] The head of the robotic arm may be equipped with sensors making it possible to probe the mechanical properties of the skin such as skin elasticity, firmness, moisture or alternatively roughness.

[0222] According to another example, a fluorescence sensor is placed at the distal end of the robot arm 6. Such a sensor allows monitoring bioactive substances, for example for cosmetic applications. According to another example, spectroscopic techniques may be employed to perform molecular characterization.

[0223] Dermatology Applications According to an embodiment, the head of the robotic arm 6, also referred to as the "operator device" 8, is equipped with an optical system for dermatological applications. In this case, a pair of optical devices may be arranged. The first optical device 83, 84 comprises a camera that performs a so-called "macroscopic" scan due to the movements made by the robotic arm 6. These cameras perform the imaging of the target image of the surface of the skin. According to an example, a light source makes it possible to illuminate the imaging zone. The transmitted light may be visible or infrared light or alternatively for example a fluorescent source. According to an example, an LED system including a polarizer is configured to transmit polarized light. In the case of polarized light, a light source of the LED type is covered with a polarizing filter. In this example, a camera arranged at the end of the head of the robotized arm is equipped with a polarizing filter. The filters of the light source and the camera are configured orthogonally according to an example.

[0224] When a singularity is detected, the robot arm 6 is controlled to correct its trajectory: the positioning of the head is performed at one of the detected singularities, for example at the center or center of gravity of the zone forming the singularity, so that the guidance is stopped and the initial trajectory is temporarily interrupted.

[0225] The controller activates a second optical device 85 that allows taking millimeter or microscopic images with dermatological resolution. To obtain good resolution, the optical device 85 may be controlled according to a movement 86 along an axis perpendicular to the plane of one of the optical elements 83, 84 or 85.

[0226] Fig. 5 represents a portion 9 of the body surface of the patient 1 on which a trajectory 30 is represented. The trajectory 30 may be obtained from a scanning strategy of the robot arm 6. According to an example, the generated trajectory 30 is obtained by a planning module PLAN_TRAJ. In the example of Fig. 4, three dermatological singular points 91 are displayed on the portion 9 of the skin of the patient 1. These may be, for example, scars, melanomas, moles or any other skin surface singular points. If the optical range 87 of the first optical device 83, 84 captures the singular points 91, the acquired images are dynamically analyzed to govern the control of the robot arm 6.

[0227] Thus, the second optical device makes it possible to take an image of a close-up zone of the surface of the skin of the patient 1. According to one example, the second optical device 85 is a dermatoscope. The second optical device 85 may comprise a glass plate in contact with the skin. According to an exemplary embodiment, the second optical device may comprise a polarized light transmitter to obtain detailed image information of the epidermis.

[0228] For this purpose, the last branch 66 of the robot arm 6 or the operator device 81 may be equipped with a force feedback sensor in order to brake or even stop and stabilise the progress of the second optical device 85. Triggering of the ejection may be performed when the second optical device 85 is in contact with the skin of the patient 1.

[0229] According to another example, each axis of the robot is equipped with a force sensor. The analysis of the force values ​​indicated by the sensors is used to control the pressure exerted on the surface. These torque sensors in each axis thus allow the robot to control the contact pressure between the tool and the working surface, in this case the patient's skin. This may be used to perform a "sliding" of the tool over the working surface.

[0230] According to an embodiment, a scanning device 87 is activated to move hair away from the optical element 85. Such a scanning device 87 may be automatically implemented during stabilization of the optical element of the second optical device 85. The advantage is to avoid imaging hairs located close to the singular point where imaging is desired. The scanning device 87 includes, according to an embodiment, a rotating rod that activates a flexible forceps close to the optical element 85. The advantage is to turn over the hair so that no hair remains between the optical element 85 and the skin. According to an exemplary embodiment, several scanning devices 87 are arranged around the optical element 85. Figure 6B represents two of them.

[0231] When the second optical element has captured one or more images with finer resolution of the singularity, the second device 85 may perform a retraction movement to return to its initial position. The robotic arm may then continue its progression and return to the planned guided trajectory.

[0232] According to an exemplary embodiment, the guiding is performed at a distance comprised between 1 cm and 20 cm from the surface of the body of the patient 1. According to a preferred embodiment, the guiding is performed at a distance comprised between 2 cm and 8 cm from the surface of the skin.

[0233] According to an embodiment, at a given distance from the surface of the robot, the field of view of the optical elements may be obtained, the properties of which are known. These properties make it possible to know the part that the image to be acquired covers on the complex surface of the patient's body. The advantage is that it is possible to accurately correspond the image to the three-dimensional surface of the patient's body. Furthermore, a possible indexing of the image to be acquired on the generated parameterized body model MOD_P, the image may be placed at a given position of said model. Thus, a three-dimensional representation of the body may be archived together with a user-accessible image capable of representing the skin singularities. The association of the generated parameterized body model with the acquired image may be performed from the mesh M1 or M2. If the mesh contains for example triangles, the image may be subdivided to correspond said triangles with the surface of the image.

[0234] According to one embodiment, the invention includes a step of referencing the captured image to a set of nodes of a mesh. According to an exemplary case, each triangle of the mesh includes an identifier. The image may then be stored together with data associating said image with the set of identifiers.

[0235] According to one embodiment, the images acquired by the optical elements 83, 84 and 85 are stored in a memory, for example the memory of a movement controller of the robot arm 6. According to an alternative, the operator device 8 comprises an electronic board with at least one calculator and a memory. The images acquired may first be stored locally and then transmitted to a remote electronic device.

[0236] According to an exemplary embodiment, the pictures taken simultaneously by the optical elements 83 and 84 are related to each other. The acquired images may, for example, be time stamped and related to points in space at the reference point R0. They may be related to points located with respect to a reference frame R3 related to the operator device of the robot arm 6, or alternatively referenced with respect to the trajectory 30.

[0237] When an image is acquired with the second optical device 85, this may be associated with the image acquired by the first optical device 83, 84, in particular at the same position on the trajectory 30 as the position at which the operator device 8 activated the movement of the second optical device 85.

[0238] An advantage of the invention is that it is possible to index each acquired image on the parametric model generated from the regression step REG. The method thus allows to identify the location of a portion of the image on the skin and to index it on the generated model. An advantage is that it stores a highly accurate record of the photographed zones with respect to the geometry of the body. Thus, if the patient changes position or if said patient returns for a later visit, the indexing of the stored photographs makes it possible to directly generate a new representation of the already indexed zone of interest according to the new image acquisition.

[0239] The advantage is to construct images of different resolutions of different parts of the surface of a body object.

[0240] In order to obtain a good response of the robot arm 6 during the real-time acquisition of images taken by the operator device, the acquired images may be compared with a group of characteristic images. According to another example, a trained neural network is capable of carrying out a classification of the acquired images to classify them and to generate commands according to the result of the classification. As an example, the detection of certain singular points does not require the activation of the second optical device 85, while the detection of other singular points requires imaging with a more precise resolution.

[0241] According to another exemplary embodiment, the robot arm 6 executes a first planned trajectory 30 with an overlap of images acquired by the first devices 83, 84. The robot arm 6 then executes a second acquisition of images at the points where the singular points are detected. When the first trajectory 30 is executed to guide the robot arm 6 over the part selected by the operator and when a set of images acquired by the first optical device is realized, the robot arm 6 enters a second trajectory 31. This trajectory may be generated automatically from the singular points identified during the course of the first trajectory 30. The second device 85 is automatically activated at the points specific to the second trajectory 31 in order to collect a second image sequence at the identified points.

[0242] Applicable Laser According to another possible application, the robot arm 6 includes a laser transmitter at its distal end. The laser transmitter may, for example, be located on the distal part of the operator device 8. This may be used for hair removal, treatment of certain skin diseases or removal of tattoos. Furthermore, ultherapy treatments or any medical or aesthetic dermatological treatment with a laser may be implemented with the robot arm including a laser source. For this purpose, the configuration of the laser source may be adapted according to the application, depending on parameters such as power, wavelength, pulsed or continuous mode.

[0243] In the case of managing a laser source, the mesh M2 can advantageously be used to optimize the settings of the laser emission, in particular to manage the emission power. Indeed, a homogeneous mesh, for example comprising an equilateral triangle, allows to define a mesh of circles circumscribing or inscribing said triangle. The advantage of a circular mesh circumscribing said triangle is to define the overlap between the circles and thus between the emissions. This configuration allows to define an emission target zone adjacent to the target zone of interest. The advantage of the construction of a tangent circle derived directly from the homogeneous mesh is to generate a target laser emission zone, while allowing to manage the parameter of the amount of homogeneous overlap between two successive emissions to adjacent targets.

[0244] The present invention allows to manage the overlap amount to compensate for the non-uniformity of the energy content of the laser beam around the main emission axis, thus allowing to place a substantially homogenous amount of energy in a given zone with a homogenous type of mesh.

[0245] In case of multiple laser transmissions to be performed to achieve the treatment goal of a skin zone, the guidance of the robot arm 6 may be configured to be fixed at a set of points to perform successive injections according to a given trajectory plan. The trajectory plan PLAN_TRAJ is determined using the user interface UI. The trajectory of the robot arm 6 may be adapted in real time to take into account, for example, data related to the settings of the laser source, the patient's movements or alternatively the detection of arm collisions.

[0246] The system of the present invention includes means for defining the parameters of the laser firing, i.e. firing power and / or diameter to be reached on the surface of the skin and amount of overlap. The laser transmitter and the guidance track may be configured to achieve these objectives.

[0247] According to one embodiment, the resolution of the homogeneous mesh may be adapted according to a given injection setting: typically, if the surface of the laser injection is defined to have a maximum radius on the surface of the skin to be reached, the mesh is configured to define a circle whose dimensions correspond to the injection constraints.

[0248] In each of the applications, the robot arm 6 covering a given trajectory during real-time guidance is made to evaluate the trajectory in real-time. This evaluation is carried out using a computer and an implementation of the method of the invention.

[0249] If the body does not move, the image of the body surface that is recalculated to position the robot arm 6 only requires a rotation of the already calculated surface, i.e. the transformation is isometric and corresponds to one or more rotations of the robot arm according to the portion of the trajectory that is covered. The generated positions of the body surface are generated with respect to the position of the head of the robot arm.

[0250] In case of body movements, the image of the body surface is recalculated to position the robot arm 6, requiring an anisotropic transformation, and the surface is recalculated from the method of the invention. Apart from potential rotations to be taken into account, the method of the invention makes it possible to perform a regression on the shape descriptors and reconstruct the body surface in the perspective of the position of the head of the robot arm.

[0251] According to an embodiment, in particular in the context of the placement of transmitters, of the laser or ultrasound type, on the head 8 of the robot arm 6, a step of optimization of the mesh of the surface of the object may be implemented if the region of the object is below a given surface threshold. This optimization may be performed in order that the transformation in this region is considered isometric. According to an exemplary embodiment, the method of the invention makes it possible to determine key points on the surface of the body object in order to define the regions of the object that have undergone isometric geometric transformations.

[0252] An advantage of the present invention is that it can be adapted for dermatological, surgical, cardiology applications, etc.

Claims

1. 1. A method for guiding a robotic arm in real time for processing data of all or part of a body surface, comprising: generating a body model of all or a portion of a surface of a human body from a parametric human body model, the body model comprising a mesh of points; planning a treatment trajectory on a surface of a body model using a computer; calculating a guided trajectory for a distal end of the robotic arm; generating a kinematic of the robot arm such that a point of an operator device disposed at the distal end of the robot arm is controlled on the guided trajectory; calculating the kinematics of the robot arm to cover the guided trajectory calculated in real time from the body model of all or part of the surface of the human body; - generating a new body model of all or part of the surface of the human body at regular intervals, ensuring the calculation of new kinematics of the robot arm for automatically managing the displacements of the robot arm in real time, the new calculated surface of the body model leading to the calculation of a new treatment trajectory and to the calculation of a new guidance trajectory; receiving at least one location of a target point in space; activating a single acquisition of a sensor of an operator device, said operator device being placed at the distal end of said robot arm, said activation being performed when an axis of said sensor is oriented perpendicular to a line passing through said target point, said target point being referenced on the generated body model; The method includes:

2. The method of guiding according to claim 1 , characterized in that the treatment trajectory is generated within a treatment zone of a surface of the body model.

3. The method of guiding according to claim 2 , characterized in that the treatment trajectory is generated automatically from a definition of a treatment zone defined on a surface of the body model.

4. The guiding method according to claim 1, characterized in that the guiding trajectory is calculated from predetermined computer instructions regarding distances and orientations, which make it possible to generate a relative position of the distal end of the robot arm with respect to the treatment trajectory, the computer instructions including distance values ​​and orientation angle values, and guiding the robot arm in real time.

5. 2. The method of guiding according to claim 1, wherein the robot arm includes at least three branches articulated in pairs to each other by pivot links, the set of pivot links including kinematics defined by kinematic data configured to impart a given guidance trajectory and a generated body model.

6. 2. The method of claim 1, further comprising the generation of a new body model at regular intervals so that the robot arm can be controlled in real time from the generation of a new trajectory of the robot arm, the new calculated surface of the body model leading to a modification of the treatment trajectory, the method further comprising the calculation of new kinematics of the robot arm controlled on the defined area of ​​the new trajectory.

7. A method of guiding according to any one of claims 1 to 6, characterized in that it comprises a step of calibrating the position of the robot arm relative to a reference point referred to an acquisition system located at a point in space and configured to acquire data making it possible to generate a body model.

8. 2. The method of guidance of claim 1, wherein a set of images is produced from an optical device disposed on the operator device, each image indexing at least one node or mesh of the mesh of the body model as a function of the position of the target point on the body model and the viewing angle of the optical device.

9. 9. The method of guiding according to claim 8, characterized in that at least a pair of images are simultaneously acquired from an optical device including two sensors that are integral and arranged on different axes of the operator device, each image indexing at least one node or mesh of the mesh of the body model depending on the position of the target point of the body model and the viewing angle of the optical device, and the images are processed to reconstruct a three-dimensional local image.

10. The method of guiding according to claim 9, characterized in that the three-dimensional local images are superimposed in a 3D representation on a generated body model.

11. 11. The method of guiding according to claim 1, characterized in that a first image acquired by a first optical device of the operator device is compared in real time with a set of images, said comparison operation including, if necessary, the generation of an indicator of an object, said indicator of an object making it possible to generate a digital control for managing the robot arm.

12. 12. The method of guiding according to claim 11, characterized in that the digital control causes a first management of the robot arm at a target position from which the first image is extracted, and the digital control causes a second management of the operator device for activating a second optical device, the activation of the second optical device making it possible to create a second image in an area near a zone including the target point.

13. 12. The method of claim 11, wherein the step of comparing is a step of classifying the obtained image from a neural network trained on a set of images.

14. A system for guiding a robotic arm, comprising: an acquisition system for acquiring images of a patient's body to create the body model; and a robotic arm equipped with an operator device for performing the method described in any one of claims 1 to 13.

Images