Method for generating a point for penetrating the body of a subject, and associated device

Abstract

The invention relates to a computer-implemented method for generating a penetration point on the surface of the body of a subject, comprising the following steps:  receiving an optical image (10) of a predetermined portion of the body surface of a subject, said image being acquired by an acquisition module comprising at least one optical system;  selecting (SEL), on the acquired image (10), at least one zone of interest (24) on the surface of the body;  generating (GEN) a penetration point (21) on the surface of the body from the position of the at least one zone of interest (24) detected on the portion of the body surface of the user.

Description

[0001] Description

[0002] Title: METHOD FOR GENERATING A PENETRATION POINT OF A SUBJECT'S BODY AND ASSOCIATED DEVICE

[0003] Field of invention

[0004] The invention relates to a computer-implemented method for determining, from an image of a body portion, a point of penetration, in particular for the injection of an anesthetic product or for the sampling of liquid or tissue from the body. The invention also relates to a device and an associated computer program product as well as a memory comprising such a computer program.

[0005] State of the art

[0006] During certain hand amputations or surgeries on localized areas of a patient's hand, it is necessary to apply local anesthesia to the area of ​​the patient's hand to be treated.

[0007] In the example of the hand, the injection points of the anesthetic agent are crucial for achieving effective and localized anesthesia. In fact, depending on the area to be anesthetized, the points where the injection of anesthetic agent must be performed are often located away from the area to be treated, generally following the nerve pathway of the hand.

[0008] We are familiar with the document "Wide Awake Hand Surgery" by Donald Lalonde, published by Thieme, which describes the different injection points depending on the injured areas of the hand. The surgeon can therefore refer to it to determine the injection points.

[0009] However, this comparison work to determine the injection points can be lengthy and must be carried out by experienced health personnel to identify the injured and uninjured areas on the patient's hand.

[0010] There is therefore a need to enable the surgeon to automatically generate an indication of the location of the point(s) on the surface of the patient's hand where the anesthetic agent should be injected.

[0011] Summary of the Invention According to one aspect, the invention relates to a computer-implemented method of generating a penetration point on the surface of a subject's body comprising the steps of:

[0012] ■ receiving an optical image of a predetermined portion of a subject's body surface acquired by an optical acquisition module;

[0013] ■ the selection, on the acquired image, of at least one area of ​​interest on the surface of the body;

[0014] ■ generating coordinates of a penetration point on the surface of the body from the position of the at least one area of ​​interest detected on the portion of the user's body surface.

[0015] The invention thus allows the automatic generation of a penetration point on the body surface of a subject from an optical image. One advantage is to give indications to the surgeon on the location on the hand where the injection must be performed more reliably and more quickly. The penetration points preferably comprise two-dimensional spatial coordinates on the acquired image. In one embodiment, these coordinates may comprise spatial coordinates on a point cloud acquired from the acquired optical image.

[0016] The term "body surface" can be understood as the superficial part of the body such as the skin surface, a deep surface such as the surface of an organ, the surface of tissues accessible during surgery when the skin is open or the surface of a body wall, for example during a fibroscopy.

[0017] In one embodiment, said selection step comprises selecting on the acquired image at least one area of ​​interest via a user interface.

[0018] An advantage is that it allows the surgeon to provide input to the computer program with indications about an area of ​​the predetermined body portion on which a surgical procedure is planned. The method, based on the acquired image and the selection, can then generate the appropriate penetration point to anesthetize the area selected via the user interface. In one execution mode, the selection step is implemented by a learning function configured to:

[0019] ■ receiving as input an image of a predetermined body surface portion including an injured area and

[0020] ■ generate as output at least one area of ​​interest comprising the injured area on the body surface portion.

[0021] An advantage is to allow the automatic generation of a penetration point from an optical image by identifying a damaged area on said portion and generating the penetration point so as to anesthetize the identified damaged area by injecting an anesthetic agent at the penetration point. Preferably, the penetration point is generated on the acquired optical image.

[0022] In one embodiment, the learning function has been trained from a plurality of images of the predetermined portion of the body surface of labeled subjects, each label comprising information relating to the area of ​​interest present on said body surface such as size and / or position information.

[0023] In one embodiment, the penetration point is generated based on the position and dimensions of said area of ​​interest on the body surface portion.

[0024] In one embodiment, the method further comprises displaying on a display an image comprising the superposition of the predetermined portion of the body surface and the penetration point generated on said portion and optionally a visualization indicator of the area of ​​interest superimposed on said portion of body surface.

[0025] This step advantageously allows the surgeon to become aware of the point of penetration generated by the method. He can then confirm this point and / or inject the anesthetic product into the subject's body at the indicated coordinates. The visual indicator may also include instructions for orienting the needle to promote anesthesia of the subject's area of ​​interest.

[0026] In one embodiment, the method comprises generating a guidance instruction for guiding a needle head arranged in the distal portion of a movable element so as to reach said penetration point. The generation of the guidance instructions thus makes it possible to guide a robotic arm to give the trajectory to be carried out to introduce an injection device into the subject's body at the coordinates of the previously generated penetration point.

[0027] In an execution mode, said selection step is implemented by a learning function configured to:

[0028] ■ receive as input images of a predetermined portion of the body surface and

[0029] ■ output the coordinates of at least one penetration point on the body surface.

[0030] In one embodiment, said learning function is trained from images of the predetermined body surface portion; said images each being labeled by information comprising the areas of the body surface portion damaged and at least one penetration point associated with said areas on the body surface portion.

[0031] In one embodiment, the amount of a drug or anesthetic substance is generated based on said acquired image and / or the location of the injured areas on the body surface portion.

[0032] An advantage is that it allows the generation of a quantity of substance to be injected associated with the generated penetration point. Thus, the surgeon knows directly how much must be injected at each generated penetration point.

[0033] According to another aspect, the invention relates to a medical device comprising software and / or hardware means for implementing the method according to one of the preceding claims.

[0034] In one embodiment, the hardware means comprise an optical acquisition device, a computer module comprising a computer / processor and a memory, a display and / or an injection device.

[0035] According to another aspect, the invention relates to a medical device comprising software and / or hardware means for implementing the steps of the method according to the invention and an actuation module comprising a needle holder support and a movable element such as an articulated robotic arm for moving said needle holder support. These means are configured to generate the automatic movement of the movable element until the needle holder support reaches the generated coordinates of the penetration point. An advantage is to automatically inject the anesthetic product into a subject from an image of a portion of the latter's body. The number of patients that can be operated on by a surgeon is therefore improved.

[0036] In one embodiment, the needle holder includes a stop surface and at least one retractable needle movable between two positions, a first retracted position in which the tip of the needle is protected behind the stop surface and an injection position in which the tip of the needle protrudes relative to the stop surface. One advantage is to allow verification that the needle is correctly positioned to inject at the point of penetration before pricking the subject. Another advantage is to reduce patient pain.

[0037] In one embodiment, the abutment surface comprises a pressure sensor and control means for moving the needle from the retracted position to the injection position when pressure is detected by said pressure sensor.

[0038] In one embodiment, the hardware means comprise a computer module, optionally comprising at least one computer and at least one memory; at least one optical acquisition module, and / or needle-holding equipment and a movable element such as an articulated robotic arm for moving said needle-holding equipment; and / or a display screen.

[0039] The invention also relates to a computer program product, comprising instructions which, when the program is executed by a computer, cause the device to implement the method according to the invention. The invention also relates to a computer-readable medium (for example, a non-transitory memory) on which said computer program product is recorded.

[0040] Brief description of the figures

[0041] Other characteristics and advantages of the invention will emerge on reading the detailed description which follows, with reference to the appended figures, which illustrate: Fig. 1: a schematic view of a device according to one embodiment of the invention.

[0042] Fig. 2A: A schematic view of an image of a subject's hand on a support including an injured area.

[0043] Fig. 2B: A schematic view of an image of a subject's hand on a support comprising an area of ​​interest generated on said image.

[0044] Fig. 2C: A schematic view of an image of a subject's hand on a support comprising an area of ​​interest and two penetration points generated on said image.

[0045] Fig. 3: a schematic sectional view of a needle holder support of a device according to an embodiment of the invention in which the needle is in the retracted position.

[0046] Fig. 4: A schematic sectional view of a needle holder of a device in Fig. 3 in which the needle is in the injection position.

[0047] Fig. 5: A bottom view of a needle-holding device according to Figure 3.

[0048] Fig. 6: a flowchart representing the steps of the method according to one embodiment of the invention.

[0049] Fig. 7: A schematic image of a training image comprising a plurality of landmarks each corresponding to a predetermined point of a hand.

[0050] Description of the invention

[0051] The following description essentially concerns an example for the generation of penetration points of the hand of a subject in order to inject an anesthetic agent. The described device and the described method can also be applied to other portions of the human or animal body (elbow, arm, lower limbs, etc.) and for other applications. Similarly, the invention can be carried out on a body portion during a surgical operation, for example, the body portion can be an organ of the body of a patient during an operation in which the patient's body is open and where said organ is directly accessible.

[0052] The system and the proposed method can be adapted, for example, to other types of surgery requiring precise determination of penetration points for the injection of medicinal agents, for sampling at specific locations or for surgical treatment (suture, incision, bone repair, tendon, nerve or vascular repair, etc.).

[0053] In the example described, a device 1 is provided which makes it possible to generate, on an image of the hand, injection points of anesthetic agent according to the injured areas of the hand. This result is obtained by implementing two successive algorithms, the first making it possible to identify injured areas of the hand, the second making it possible to determine the position of the injection points on the hand according to the position of the injured areas on the hand.

[0054] The device is, in this example, part of a surgical system 2 which allows an automatic injection of an anesthetic agent to be carried out on a patient's hand.

[0055] Acquisition and support module

[0056] The device comprises at least one optical image acquisition module. The optical acquisition module is designed to acquire optical images of a surface portion of the subject. The optical acquisition module is designed to acquire images at least in the visible wavelengths. In an alternative example, the optical acquisition module is designed to acquire images in the infrared range.

[0057] The optics preferably includes a photographic device or a camera.

[0058] In an alternative or cumulative embodiment, the device comprises a module for acquiring a depth map such as a stereoscopic camera.

[0059] The device 1 may further comprise a support 25 intended to receive the hand 20 of the subject.

[0060] Preferably, the optical acquisition module OPT is arranged so as to capture an image 10 of a portion of the subject's body 20 fixed on the support.

[0061] Acquiring the image 10 of the hand 20 may comprise placing a hand in a support 25. The support 25 comprises a substantially planar surface intended to receive the hand 20 of a subject.

[0062] The support 25 preferably comprises means 29 for fixing the subject's hand in a predetermined position. As such, the support 25 may comprise visual indications intended to indicate to the subject how and where to place his hand on the support. In another example, the support 25 comprises stop walls 29' or reversible attachment means 29 designed and arranged on the support 25 to fix the hand on the support 25 in a predetermined position.

[0063] An advantage is to allow reproducibility of the position of the hand 20 on the acquired image 10, thus facilitating the training of a machine learning model.

[0064] A second advantage is that it allows the hand position to be secured when injecting the anesthetic product.

[0065] Display and user interface

[0066] In one embodiment, the device 1 comprises a user interface INT. The user interface INT is designed to allow a user to transmit information or commands 15. The user interface INT, also called a “human-machine interface” may comprise a keyboard, a mouse, a touch surface or any other means allowing the user to communicate with a machine or a CALC computer module.

[0067] In one embodiment, the device 1 or the medical system 2 further comprises an AFF display such as a display screen. The display screen is intended to display the acquired images 10 and visual information such as the generated areas of interest 24 and / or the penetration points 21 generated by the method according to the invention and described below.

[0068] Robotic arm

[0069] The surgical system preferably includes IFG penetration equipment.

[0070] The penetration equipment preferably comprises an IFG needle holder equipment (referred to as “equipment” in the remainder of the description). This IFG equipment serves as a gripper for the injection needle 103.

[0071] In the illustrated example, the IFG equipment is mounted on a movable element. The movable element preferably comprises a multidirectional guidance RGB robot (referred to as “robot” in the remainder of the description).

[0072] The ROB robot may comprise an articulated arm that allows the needle 103 to be guided according to guidance instructions 13 received from a CALC computer module. The ROB robot may comprise a 6-axis robot, offering 6 degrees of freedom. The robotic arm allows the needle 103 to be moved in the three directions of space and the inclination and orientation of the penetration equipment or the needle 103 to be changed thanks to the three rotations around these three axes. The robotic arm may also be redundant with 7 axes in order to have redundancy on the robot's posture depending on the position of the subject's hand.

[0073] In another example, the multidirectional guidance robot ROB comprises a robot mounted on a rail in two directions of space and movable in translation according to the third direction of space.

[0074] The ROB robot comprises one or a plurality of motors for generating the needle's mobility in one or more degrees of freedom according to the guidance instructions. The motor(s) are controlled by a motion controller capable of activating and controlling each motor independently.

[0075] The guidance instructions 13 include information relating to the movements as well as the inclination and orientation of the penetration equipment or the needle relative to the patient or the patient's hand in the frame of reference of the robot ROB or in the frame of reference of the fixed support 25.

[0076] The guidance instructions 13 are generated by the CALC computer module and transmitted to the robot's motion controller.

[0077] Needle holder equipment

[0078] The IFG needle-holding equipment is preferably arranged on the distal part of the robotic arm. An example of IFG needle-holding equipment equipped with a needle 103 is now described with reference to FIGS. 3 to 5.

[0079] The IFG equipment comprises means for cooperating integrally with the needle 103. The embodiment described and illustrated represents a single needle 103. However, it is understood that the IFG equipment may be equipped with at least two needles. The needles may have a substantially similar size or different sizes.

[0080] The needle 103 is removably attached to the IFG equipment so that the needle 103 can be replaced and the IFG equipment can be reused indefinitely by replacing the old needle with a new needle. The IFG equipment includes means for cooperating integrally with the needle.

[0081] In this example, the needle 103 is an injection needle connected to a reservoir 101 for containing a medicinal or anesthetic agent for injection into the subject's hand 20. The needle 103 is fluidically connected to the reservoir 101 by a fluidic hose 102. Preferably, the IFG equipment comprises a means for controlling the flow through the needle 103. In one example, the equipment comprises a fluidic pump for controlling the passage of a predetermined quantity of liquid from the reservoir into the subject's hand through the needle. In another example, the reservoir comprises a syringe and the IFG equipment comprises a remotely controlled piston system 107 for controlling the passage of a predetermined quantity of liquid from the reservoir 101 into the subject's hand 20 through the needle 103.

[0082] In one embodiment, the needle 103 is movable in the IFG equipment between two positions: a first retracted position and a second injection position.

[0083] In the first retracted position illustrated in Figure 3, the distal end of the needle 103 is located on the proximal side of the wall 100 of the IFG equipment intended to be in direct contact with the subject's skin. In this way, when the wall 100 of the IFG equipment is in direct contact with the subject's skin, the distal end of the needle 103 is not in direct contact with the subject's skin.

[0084] In the second injection position illustrated in Figure 4, the distal end of the needle 103 protrudes from the wall 100 of the IFG equipment. In this way, the needle 103 penetrates the skin of the subject when said wall 100 is in direct contact with the skin of the subject.

[0085] The wall 100 may comprise an orifice 105 for the passage of the needle 103 from the retracted position to the injection position. The IFG equipment may comprise a motor or an actuating means for causing the needle 103 to move between the first and second positions.

[0086] In the illustrated example, said wall 100 comprises at least one detector 104 making it possible to detect contact between the wall 100 and the subject's skin. The detector 104 preferably comprises a pressure sensor arranged to emit a signal when the wall is in contact with the subject's skin. The detector 104 may also comprise an impedance detector or any other means for detecting contact between the subject's skin and the wall 100.

[0087] In another example not shown, the IFG equipment comprises means for measuring the distance between the wall 100 of the IFG equipment and the skin surface of the subject. These means may comprise a laser emission / detection pair making it possible to measure a distance based on the time of flight of the laser reflected on the skin surface of the subject.

[0088] In a first example, the detector 104 is connected to the CALC computer module or another computing unit. In a second example, the detector 104 is connected to the motor or actuating means to cause the needle 103 to move from the retracted position to the injection position automatically when contact is detected.

[0089] In another embodiment, the penetration equipment comprises means for sampling bodily fluids or tissues from the body. The penetration equipment may comprise a sampling needle connected to a sampling reservoir for receiving the sampled tissues or fluids. The penetration equipment may comprise a pump such as a peristaltic pump for moving the tissues or fluids from the distal end of the needle to the reservoir.

[0090] In another embodiment, the penetration equipment comprises means for holding a tissue cutting tool of the subject such as a scalpel or any cutting tool intended to be inserted through the body surface of the subject.

[0091] Acquisition An example of execution of the method according to the invention is now described, in particular with reference to figure 6.

[0092] An image 10 of the hand 25 is acquired CAPT by the image acquisition module OPT.

[0093] Preferably, the image 10 of the hand is obtained when the latter is arranged in a predetermined position, for example placed flat on the support. Preferably, the support 25 comprises a solid-colored surface.

[0094] The acquired optical image 10 is then transmitted REC to a computer module CALC described later which receives said acquired image.

[0095] The acquired optical image 10 may then be processed prior to its use. For example, the method may comprise applying a denoising filter to the image.

[0096] In another example, the acquired optical image 10 is processed so as to unify the background of the image, that is to say that the portions of the image not including the hand are homogenized so as to obtain a plain background.

[0097] These treatments advantageously facilitate the following steps by providing the CALC computer module with images 10 comprising as few variations as possible.

[0098] Preferably, the optical image is a conventional image of acquisition of light rays in the visible range. In another embodiment, the optical image comprises an infrared image of a portion of a subject's body, for example to highlight the venous network. In another embodiment, the term "optical image" means an image pair comprising an image in the visible range and an infrared image of the same portion of the body, preferably acquired simultaneously.

[0099] Automatic selection of areas of interest.

[0100] The method comprises a step of determining or selecting SEL at least one area of ​​interest 24 of the hand 25 of the subject.

[0101] From the acquired image 10, a set of information is generated comprising the dimensions of an area of ​​interest 24 and the coordinates of said area of ​​interest 42. In an example illustrated in FIG. 2B, the area of ​​interest 24 is characterized by a rectangular geometric shape generated on the image 10 framing an injured area 22 detected on the acquired image 10 of the hand 25.

[0102] In a first example, this SEL step is implemented by an algorithm configured to receive as input an image of the subject's hand and to generate as output a region of interest comprising a damaged area 22. A damaged area 22 designates a portion of the surface of the subject's body comprising wounds or other visible lesions on the body surface.

[0103] The at least one area of ​​interest 24 then preferably comprises an area of ​​the injured body surface whose lesions are visible on said body surface. The generated area of ​​interest 24 preferably comprises information relating to the position coordinates of said area of ​​interest and information relating to the dimensions and / or the shape of the area of ​​interest. The coordinates of the area of ​​interest may be relative to the position of the hand. In one example, a point of interest of the acquired image 10 is identified and the coordinates of the generated area of ​​interest 24 are relative to the position of said point of interest which serves as a spatial reference.

[0104] In another embodiment, the area of ​​interest comprises an area including an anatomical element of interest such as a predefined portion of the venous system. The area of ​​interest may comprise an area including pathological tissues such as necrotic or cancerous tissues.

[0105] The point of interest may comprise a specific point on the subject's hand recognized by the algorithm. In another example, the recognized point of interest comprises an element of the support on which the hand is positioned during image acquisition, for example, a point on a 29' stop.

[0106] The dimensions and / or position of the area of ​​interest 24 are automatically determined so as to partially or totally frame the damaged area 22 visible on the acquired image.

[0107] In one embodiment, the selection of an area of ​​interest 24 is implemented by a first image recognition algorithm.

[0108] The image recognition algorithm preferably comprises a first trained machine learning module configured to generate a position and the dimensions of at least one area of ​​interest 24 from an image acquired of a subject's hand. The image recognition algorithm may be configured to generate an area of ​​interest 24 of rectangular shape or of any other predetermined shape.

[0109] In one example, the first machine learning module is implemented using a learning function trained using supervised and / or machine learning. The learning function preferably comprises a neural network. The learning function is preferably trained using a series of labeled hand images.

[0110] The learning function of the first machine learning module was trained from a series of hand images comprising at least one injured area 22 and labeled with the dimensions and positions of areas of interest totally or partially framing at least one injured area 22 visible on the associated image.

[0111] Assisted selection of areas of interest.

[0112] In an alternative example, the at least one area of ​​interest 24 is selected from a user command. This alternative is particularly advantageous when the envisaged surgical operation is not intended to repair visible lesions on the subject's hand. For example, during a fracture of a carpal bone, it happens that no lesion is visible on the acquired optical image 10.

[0113] In this example, the at least one area of ​​interest is generated from a command 15 generated by the human-machine interface INT. As such, several alternative or cumulative modes are possible.

[0114] In a first mode, the area of ​​interest 24 can be directly generated by the user by entering, using the human-machine interface INT, the dimensions and / or the position on the acquired image of the area of ​​interest. The user can enter, using the human-machine interface INT, the coordinates and / or the dimensions of the area of ​​interest 24. The user can also draw directly on a touch-sensitive surface displaying the area(s) of interest 24 on the acquired image 10.

[0115] In a second mode, the instruction 15 provided by the user via the human-machine interface INT includes pre-operative information. The method then includes a computer-implemented step of selecting a predetermined area of ​​interest 24 from the information provided via the human-machine interface INT.

[0116] Preoperative information may include pathology or injury. Preoperative information may also include a portion of the hand to be operated on.

[0117] In this mode, at least one area of ​​interest 24 is selected from a library recorded in a memory MEM from said preoperative information. Preferably, the library comprises a plurality of areas of interest 24 or groups of areas of interest 24 and each area or group of areas is associated with one or more predetermined preoperative information.

[0118] Generation of injection points

[0119] The method further comprises generating GEN at least one penetration point 21 on the subject's hand 25 from the at least one previously selected area of ​​interest 24.

[0120] The step of generating GEN of the penetration points 21 is preferably implemented by computer by a second trained machine learning module.

[0121] The second trained machine learning module is configured to generate at least one penetration point 21 from the acquired image 10 and from the at least one generated area of ​​interest 24.

[0122] The penetration point 21 corresponds to the point on the subject's hand 25 where the anesthetic product must be injected to anesthetize the injured area 22 or the area on which the surgical procedure is to be performed.

[0123] The penetration point 21 preferably comprises coordinates on the acquired image 10. Preferably, the penetration point 21 comprises coordinates relative to the point of interest on the acquired image 10 as described previously.

[0124] In one example, the second machine learning module is implemented using a second learning function trained using supervised and / or machine learning. The second learning function preferably comprises a neural network. The second learning function is preferably trained using a labeled training data set. The training data set comprises a plurality of images (preferably optical images) of the hand.

[0125] The data series comprises a series of images acquired from the hand of a plurality of subjects, each associated with at least one area of ​​interest 24. The areas of interest 24 are preferably generated according to one of the methods described above. Each of the images is also associated with a label. The label corresponds to the coordinates of at least one penetration point on the image of the associated hand.

[0126] In one embodiment, the training images are also labeled with landmarks 26, 28. An example of a training image including such landmarks is illustrated in Figure 7.

[0127] The landmarks may include centers of mass 28 of the hand or a portion of the hand. The centers of mass may be generated at coordinates substantially corresponding to the center of a predetermined area of ​​the hand. For example, a landmark may correspond to the center of mass of the palm of the hand, a phalanx, or a finger.

[0128] The landmarks may comprise deflection points 26. The deflection points 26 are generated at the boundary between two adjacent portions of the hand. For example, a landmark may be generated between the portions of the hand corresponding to two adjacent phalanges of the same finger. The location of such a landmark may then correspond to the location of a joint, for example between two phalanges.

[0129] The landmarks may include the tip or end of a finger. Such a point of interest may be generated at the distal end of the region of interest corresponding to the last phalanx of a finger, or corresponding to a center of mass of the region of interest corresponding to the last phalanx of a finger. The different landmarks may be connected to each other by segments 27.

[0130] In one embodiment, the step of generating points of interest on the training images is carried out manually using a human-machine interface.

[0131] In one embodiment, the step of generating the points of interest comprises generating the two-dimensional coordinates of each landmark point on the training image. Each landmark point may be associated with a label comprising an identifier of the region of the hand associated with said point.

[0132] An advantage of these landmarks on the training image is that they allow the second machine learning module to learn the different regions of the hand more quickly and accurately. The second learning function learns the relative positions of the areas of interest and penetration points more easily by comparing them to the positions of landmarks corresponding to specific points of the hand such as joints.

[0133] Landmarks include coordinates of a point or area on the acquired image and labeling information corresponding to identification information. For example, identification information makes it possible to find, from one image to another, a predetermined joint.

[0134] A second machine learning module trained with such input data is advantageously more efficient in determining, from an optical image 10 and areas of interest 24, penetration points 21.

[0135] In one embodiment, the penetration points 21 are associated with a quantity of anesthetic agents to be injected. In this mode, the penetration point 21 generated by the learning function comprises a quantity or volume of anesthetic agent to be injected. To generate this quantity associated with the penetration point 21, the penetration points associated with the training images are each associated with a predetermined quantity.

[0136] In an alternative or cumulative embodiment, the method comprises acquiring an infrared image of the subject's hand. Such acquisition may be performed by an infrared image acquisition device such as an infrared camera.

[0137] An advantage of such an infrared image is that it allows the venous mapping of the subject's hand or body part to be highlighted on this image.

[0138] In one embodiment, the acquired infrared image is transmitted and received by the computer module. The method then comprises a step of identifying the venous channels of the subject's hand. This identification can be implemented by an image recognition algorithm or by an algorithm for detecting the light intensity of the acquired infrared image.

[0139] In an acquisition mode, the method comprises a step of adjusting the generated penetration point if said generated penetration point is located on or near a venous channel of the subject's hand. An advantage is to avoid the injection of therapeutic or anesthetic agents into the subject's venous network.

[0140] This step may include a comparison of the penetration point with the coordinates of the different venous channels identified on the infrared image. If the difference between the generated penetration point and an identified venous channel is below a predetermined threshold, the method includes a step of generating a new penetration point. In one embodiment, the coordinates of the new penetration point are generated such that the distance between the new penetration point and the venous channel is greater than a predetermined threshold value.

[0141] Increased training data

[0142] Preferably, the device 1 comprises image processing means for processing the received images before providing them to the first or second machine learning module. These processing means may comprise image filters or contrast functions.

[0143] In one mode, the images of the plurality of hand images for training the first machine learning module and / or the second machine learning module have been generated from acquired images preprocessed by denoising and / or background unification processes.

[0144] In one mode, the images of the plurality of hand images for training the first machine learning module were generated from acquired images on which different operations were implemented.

[0145] This advantageously makes it possible, from an acquired image, to generate a plurality of images of the hand in order to increase the training data of the first machine learning module. Said operations may include random zoom operations, Gaussian blur and / or variation of hue or brightness of the acquired image.

[0146] Said operations may also include the application of an image inversion with respect to an axis of symmetry. This operation advantageously makes it possible to double the number of training images since with a right-hand training image, a left-hand training image is generated and vice versa.

[0147] In a particular embodiment, the method comprises acquiring a point cloud of the subject's hand. The acquisition of the point cloud may be implemented cumulatively or as an alternative to optical image acquisition of the subject's hand. The point cloud may then be acquired from a stereoscopic camera or a depth camera.

[0148] Computer module

[0149] The medical device also includes a CALC computer module.

[0150] In a first example illustrated in Figure 1, a computer module comprises at least one PRO processor configured to implement the method according to the invention and at least one data storage medium such as a MEM memory, preferably a non-transitory memory, for storing data and / or for storing a computer program product allowing the processor to execute the method according to the invention. In one embodiment, the computer and / or the memory may be located in a remote device and connected to the medical device to transmit information in a wired or wireless manner, for example by Bluetooth, Wi-Fi, via a network connection or any other type of wireless connection known to those skilled in the art.

[0151] Preferably, the first trained machine learning module and / or the second trained machine learning module are stored in the memory. The processor is configured to implement the first trained machine learning module and / or the second trained machine learning module. In a second example not shown, the computer module CALC comprises communication means for transmitting and receiving data from a remote server. The remote server is then configured to implement the step of selecting an area of ​​interest 24 and / or the step of generating at least one penetration point described above.

[0152] The computer module CALC is connected to the optical acquisition device OPT so as to receive the images acquired 10 by the optical acquisition device.

[0153] In this respect, the computer module CALC comprises information reception means for receiving optical images 10, in particular optical images acquired by the acquisition device OPT.

[0154] The CALC computer module may be connected to an AFF display. In one example, the computer module is configured to transmit the acquired image 10 to the display.

[0155] In another example, the computer module is configured to display on the display the acquired image as well as the generated penetration points. Preferably, the penetration points 21 are displayed superimposed on the acquired image on the display at the position corresponding to the generated coordinates of said penetration point.

[0156] In one embodiment, the computer module CALC is connected to the human-machine interface INT and is configured to receive commands 15 generated by the human-machine interface INT.

[0157] In the present application, the term “trained learning function” preferably means a function implemented by a neural network trained by a training database.

[0158] Robotic arm guidance

[0159] In one embodiment, the method further comprises, from the coordinates of the at least one penetration point 21 generated, the GUI generation of a guidance instruction 13. The guidance instruction 13 advantageously makes it possible to guide the needle 103 from a starting position to the penetration point 21 on the surface of the hand previously generated.

[0160] The guidance instruction 13 is preferably transmitted to the robot's motion controller. The robot ROB thus moves based on the guidance instructions 15 received until the needle 103 is in position to reach the penetration point 21 generated on the subject's hand 20. Preferably, the subject's hand 20 is attached to the support 25 in a predetermined position, which advantageously allows the robot to reach a penetration point 21 on the hand since the position of the latter can be located in the reference frame of the support 25.

[0161] In another mode, the ROB robot includes a sensor to detect the position of the hand and the guidance instruction is adapted according to the position of the hand in the robot's frame of reference and according to the coordinates on the hand of the penetration point.

[0162] Preferably, the guidance instructions 13 comprise trajectory and inclination and orientation instructions for the equipment so as to place the orifice 105 of the equipment in contact with the subject's skin on the penetration point. It is understood that the guidance trajectory makes it possible to move the equipment until the wall 100 of the latter is in contact with the subject's skin in such a way that the penetration point 21 generated is at least partially covered by the orifice 105 of the wall 100 of said equipment.

[0163] Injection of the anesthetic product

[0164] In one embodiment, the equipment is configured to, when the detector 104 detects contact with the subject's skin, implement a penetration step INJ comprising moving the needle into the injection position. The needle 103 thus penetrates the subject's skin at the injection point 21.

[0165] A command to inject the drug agent is then transmitted to the IFG equipment which, in response, causes the drug agent to be injected from the reservoir into the patient's skin through the needle.

[0166] Preferably, the injection command comprises a predetermined quantity or volume of anesthetic agent to be injected. In one embodiment, the CALC computer module is connected to the flow control means (such as the piston control means 107 described previously). The CALC computer module is then configured to activate the delivery of the predetermined quantity of agents from the reservoir through the needle 103. This predetermined quantity or volume is preferably a function of the quantity generated by the CALC computer module from the optical image 10 and the at least one area of ​​interest 24 as described previously.

[0167] Operator control

[0168] In one execution mode, the computer module CALC is configured to display on the display AFF the at least one generated penetration point 21 and the operator can then launch the guidance command by an action on the human-machine interface. In the same way, once the orifice of the equipment is in contact with the subject's skin at the penetration point, the operator can manually launch the injection by a confirmation action on the human-machine interface.

[0169] In an alternative embodiment, the operator may manually move an IFG device as described above over the subject's skin such that the orifice 105 of the IFG device at least partially covers the generated penetration point 21.

[0170] It is understood that the example described with reference to the figures does not limit the invention to the generation of penetration points 21 of the hand, but can be applied to any limb, in particular the lower and upper limbs. Similarly, the invention is not limited to the generation of penetration points for the injection of an anesthetic product.

[0171] Implementation variants

[0172] An embodiment and use of the invention has been described with reference to the non-limiting application of the hand for generating penetration points for the injection of anesthetic product.

[0173] However, other cases of applications of the invention are conceivable, some of which are cited below as non-limiting examples.

[0174] The term "body surface" can be understood as the superficial part of the body such as the surface of the skin. Said "body surface" can also be understood as a deep surface such as the surface of an organ or the surface of tissues accessible during surgery. In this case, the optical image is produced after opening the subject's body. The term "body surface" can also refer to a body wall, such as the internal wall of the digestive or respiratory tract, for example during a fibroscopy.

[0175] The penetration point 21 may also correspond to a point on the body surface where a sample is to be taken, for example the taking of healthy or pathological tissues or body fluids. Said sampling may be for therapeutic purposes or may be carried out for the purpose of analyzing sampled tissues or fluids, for example with the intention of making a diagnosis.

Claims

CLAIMS A computer-implemented method of generating a penetration point on the surface of a subject's body comprising the steps of: ■ receiving an optical image (10) of a predetermined portion (20) of the body surface of a subject acquired by an optical acquisition module (OPT); ■ the selection (SEL), on the acquired image (10), of at least one area of ​​interest (24) on the surface of the body; ■ the generation (GEN) of coordinates of a penetration point (21) on the surface of the body from the position of the at least one area of ​​interest (24) selected on the portion of the user's body surface (20). Method according to claim 1 wherein said selection step (SEL) comprises the selection on the acquired image (10) of at least one area of ​​interest (24) via a user interface (INT). Method according to claim 1 wherein the selection step (SEL) is implemented by a learning function configured to: ■ receiving as input an image (10) of a predetermined body surface portion (20) comprising an injured area (22), and ■ generating as output at least one area of ​​interest (24) comprising the image of the injured area (22) on the portion of body surface. Method according to claim 3, in which the learning function has been trained from a plurality of images of the predetermined portion of body surface of labeled subjects, each label comprising information relating to the area of ​​interest (24) present on said body surface such as size and / or position information. Method according to one of claims 1 to 4, wherein the coordinates of the penetration point (21) are generated as a function of the position and dimensions of said area of ​​interest (24) on the portion of body surface (20). Method according to one of the preceding claims further comprising the display on a display (AFF) of an image comprising the superposition of the predetermined portion of the body surface and the penetration point (21) generated on said portion and, optionally, a display indicator of the area of ​​interest (24) superimposed on said portion of body surface (20). Method according to one of the preceding claims comprising the generation of a guidance instruction (13) for guiding a needle head arranged in the distal part of a movable element so as to reach said penetration point.Medical device (1) comprising software and / or hardware means for implementing the method according to one of the preceding claims. Computer program product, comprising instructions which, when the program is executed by a computer, cause the device according to claim 8 to implement the method according to one of claims 1 to 7.

10. Computer-readable medium (MEM) on which the computer program product according to claim 9 is recorded.

Images