Surgical robot system and method for controlling surgical robot system

Abstract

A surgical robot system according to one embodiment of the present invention comprises: a robot arm having at least one joint and accessible to an affected area by using an end effector mounted on one end thereof; an image capturing unit for acquiring image information including the affected area; and a control unit for generating a working path of the robot arm on the basis of the image information, wherein the image capturing unit may comprise: a light emitting unit for irradiating first light of a first wavelength band onto the affected area; and a light receiving unit for collecting reflected light reflected from the affected area to acquire the image information on the affected area.

Description

Surgical robot system and surgical robot system control method

[0001] The present invention relates to a surgical robot system and a surgical robot system control method.

[0002] With recent advancements in medical technology, surgical robots are being widely utilized in various surgical fields that require precision. Surgical robots assist or replace the surgeon's manual skills, enabling or facilitating more precise surgical procedures such as incision, fixation, and suturing. Their particular advantage lies in their ability to perform precise surgeries even in areas difficult for surgeons to access directly.

[0003] Therefore, various attempts are being made to obtain information about the affected area and use this to precisely control the robotic arm to perform surgery.

[0004] The surgical robot system and the surgical robot system control method according to embodiments of the present invention can acquire image information including a wound, and generate a working path of a robot arm using the result of matching modeling information and image information.

[0005] However, the above-described tasks are according to embodiments of the present invention, and the purpose and tasks to be solved by the present invention are not limited thereto.

[0006] A surgical robot system according to one embodiment of the present invention includes a robot arm having at least one joint and capable of approaching a lesion using an end effector mounted on one end thereof, a photographing unit for obtaining image information including the lesion, and a control unit for generating a working path of the robot arm based on the image information, wherein the photographing unit may include a light emitting unit for irradiating the lesion with a first light of a first wavelength band, and a light receiving unit for collecting reflected light reflected from the lesion to obtain the image information for the lesion.

[0007] According to one embodiment of the present invention, a surgical robot system and a surgical robot system control method can acquire image information containing accurate information about a lesion or a measurement unit by using light of a preset wavelength band or controlling a photographing unit. In addition, the surgical robot system and the surgical robot system control method can correct scan data acquired from image information using cartilage data and quickly reflect the movement of the lesion over time to update the working path of the robot arm. Of course, the scope of the present invention is not limited by these effects.

[0008] FIG. 1 is a drawing illustrating a surgical robot system according to one embodiment of the present invention.

[0009] Figure 2 is a drawing showing in detail some of the configurations of Figure 1.

[0010] Figure 3 is a diagram illustrating a control system of the surgical robot system of Figure 1.

[0011] FIG. 4 is a diagram for explaining a process for generating a matching result according to one embodiment of the present invention.

[0012] FIG. 5 is a diagram for explaining a process of obtaining landmark data according to one embodiment of the present invention.

[0013] Figure 6 is a drawing showing in detail some of the configurations of Figure 1.

[0014] Figure 7 is image information of a wound captured in different wavelength bands using a photographing unit.

[0015] Figure 8 is a drawing briefly illustrating image information acquired from the photographing unit of Figure 6.

[0016] Figure 9 is a drawing showing that the position of the shooting section of Figure 6 has been partially changed.

[0017] FIG. 10 is a control timing graph for some configurations of a surgical robot system according to one embodiment of the present invention.

[0018] Figure 11 is a drawing showing scan data acquired from the control unit.

[0019] Figure 12 is a diagram showing scan data before and after changing the posture of the refund.

[0020] Figure 13 is a drawing showing the process of acquiring scan data after changing the posture of the refund.

[0021] FIG. 14 is a drawing illustrating a measuring unit according to one embodiment of the present invention.

[0022] Figures 15 and 16 are drawings illustrating a process of measuring the cartilage thickness of a wound using the measuring unit of Figure 14.

[0023] FIG. 17 is a drawing illustrating a process of measuring the cartilage thickness of a wound using a measuring unit according to another embodiment of the present invention.

[0024] FIG. 18 is a drawing illustrating a process of measuring the cartilage thickness of a wound using a measuring unit according to another embodiment of the present invention.

[0025] Figure 19 is a drawing showing measurement points for measuring cartilage thickness.

[0026] FIG. 20 is a diagram illustrating a process of measuring a gap between body structures constituting a refund using a gap measuring sensor according to one embodiment of the present invention.

[0027] Figure 21 is a drawing showing that a reference point has been created on scan data.

[0028] FIG. 22 is a drawing illustrating a surgical robot control method according to one embodiment of the present invention.

[0029] Figure 23 is a drawing illustrating in detail some steps of Figure 22.

[0030] Figure 24 is a drawing detailing some other steps of Figure 22.

[0031] FIG. 25 is a drawing illustrating a surgical robot control method according to another embodiment of the present invention.

[0032] FIG. 26 is a drawing illustrating a surgical robot control method according to another embodiment of the present invention.

[0033] Figure 27 is a drawing illustrating in detail some steps of Figure 26.

[0034] FIG. 28 is a drawing illustrating a surgical robot control method according to another embodiment of the present invention.

[0035] FIG. 29 is a drawing illustrating a surgical robot control method according to another embodiment of the present invention.

[0036] Figure 30 is a drawing showing an enlarged view of part A of Figure 1.

[0037] Figure 31 is a drawing that briefly illustrates the process of performing surgery using a surgical robot system.

[0038] A surgical robot system according to one embodiment of the present invention includes a robot arm having at least one joint and capable of approaching a lesion using an end effector mounted on one end thereof, a photographing unit for obtaining image information including the lesion, and a control unit for generating a working path of the robot arm based on the image information, wherein the photographing unit may include a light emitting unit for irradiating the lesion with a first light of a first wavelength band, and a light receiving unit for collecting reflected light reflected from the lesion to obtain the image information for the lesion.

[0039] Additionally, the surgical robot system may further include a lighting unit that irradiates the affected area with a second light of a second wavelength band different from the first wavelength band.

[0040] Additionally, the first minimum wavelength value and the first maximum wavelength value of the first wavelength band may have values ​​greater than the second minimum wavelength value and the second maximum wavelength value of the second wavelength band, respectively.

[0041] Additionally, the photographing unit may further include a filter unit that filters the reflected light incident on the light receiving unit into light of a third wavelength band.

[0042] Additionally, the filter unit can filter the light emitted from the light emitting unit into light of a third wavelength band.

[0043] Additionally, the third wavelength band may correspond to the first wavelength band.

[0044] In addition, the photographing unit further includes a filter unit that filters the reflected light incident on the light receiving unit into light of a third wavelength band, and a third minimum wavelength value of the third wavelength band may have a value greater than a second maximum wavelength value of the second wavelength band.

[0045] In addition, the control unit may obtain modeling data, which is a three-dimensional model of the affected area, from an external photographing device, obtain scan data, which is three-dimensional information of the affected area, based on the image information, match the modeling data and the scan data to generate a matching result for the affected area, and generate the work path using the matching result.

[0046] In addition, the control unit can generate at least one first reference point on the modeling data, acquire a second reference point on the scan data corresponding to the first reference point, and match the first reference point and the second reference point to generate a matching result for the affected area.

[0047] In addition, the photographing unit repeatedly acquires the image information according to a preset first cycle, and the control unit can correct the work path using the image information acquired for each first cycle.

[0048] In addition, the photographing unit obtains the image information including a marker for the affected area that reflects the movement of the affected area, and the control unit can correct the work path based on the image information including the marker for the affected area according to the first cycle.

[0049] In addition, the control unit may obtain modeling data, which is a three-dimensional model of the affected area, from an external photographing device, obtain first scan data for the affected area corresponding to the first time point and first affected area marker data for the affected area marker based on the image information of the first time point, obtain second affected area marker data corresponding to the second time point based on the image information of the second time point, and obtain second scan data for the affected area corresponding to the second time point using the first scan data, the first affected area marker data, and the second affected area marker data.

[0050] In addition, the surgical robot system further includes a lighting unit that irradiates a second light of a second wavelength band different from the first wavelength band to the affected area according to a preset second cycle, and the control unit can control at least one of the photographing unit and the lighting unit to turn off the lighting unit during a time period in which the photographing unit acquires the image information.

[0051] Additionally, the control unit can generate the matching result based on the modeling data and the scan data using the first model that has been previously learned.

[0052] In addition, the first model may be learned by using learning modeling data corresponding to the modeling data and learning scan data corresponding to the scan data as input data, and using a learning matching result generated by matching the modeling data and the scan data as output data.

[0053] Additionally, the control unit can obtain a medical image including the affected area from the input unit, and extract a region of interest from the medical image to obtain the modeling data.

[0054] Additionally, the control unit can obtain the modeling data from the medical image using the second model that has been previously learned.

[0055] Additionally, the control unit can obtain landmark data for feature points on the affected area based on the modeling data, and generate the work path based on the landmark data and the matching result.

[0056] Additionally, the control unit can obtain the landmark data from the modeling data using the learned third model.

[0057] A surgical robot system according to another embodiment of the present invention includes a robot arm having at least one joint, an end effector mounted on one end of the robot arm, and a photographing unit disposed between the robot arm and the end effector, wherein the photographing unit may include a light emitting unit that irradiates the affected area with first light of a first wavelength band, and a light receiving unit that collects reflected light reflected from the affected area to obtain image information about the affected area.

[0058] Additionally, the surgical robot system may further include a lighting unit that irradiates the affected area with a second light of a second wavelength band different from the first wavelength band.

[0059] Additionally, the first minimum wavelength value and the first maximum wavelength value of the first wavelength band may have values ​​greater than the second minimum wavelength value and the second maximum wavelength value of the second wavelength band, respectively.

[0060] Additionally, the photographing unit may further include a filter unit that filters the reflected light incident on the light receiving unit into light of a third wavelength band.

[0061] In another embodiment of the present invention, a surgical robot system control method may include a step of obtaining modeling data, which is a three-dimensional model of a affected area, from an external photographing device, a step of obtaining scan data, which is three-dimensional information about the affected area, based on image information about the affected area, a step of matching the modeling data and the scan data to generate a matching result for the affected area, and a step of generating a working path of a robot arm that can access the affected area using an end effector mounted on one end using the matching result.

[0062] In addition, in the step of generating the above matching result, the matching result can be generated based on the modeling data and the scan data using the first model that has been previously learned.

[0063] In addition, the surgical robot system control method further includes a step of obtaining landmark data for feature points on the affected area based on the modeling data, and the step of generating the work path can generate the work path based on the landmark data and the matching result.

[0064] In addition, the step of generating the matching result may include a step of generating at least one first reference point on the modeling data, a step of acquiring a second reference point on the scan data corresponding to the first reference point, and a step of generating a matching result for the affected area by matching the first reference point and the second reference point.

[0065] In addition, the step of acquiring the scan data may include a step of acquiring first scan data for the affected area corresponding to the first time point and first affected area marker data for the affected area marker based on the image information of the first time point, a step of acquiring second affected area marker data corresponding to the second time point based on the image information of the second time point, and a step of acquiring second scan data for the affected area corresponding to the second time point using the first scan data, the first affected area marker data, and the second affected area marker data.

[0066] A surgical robot system according to another embodiment of the present invention may include a measuring unit including a probe that can be inserted into cartilage at one or more locations set in advance among the affected area, a photographing unit including a light emitting unit that irradiates the measuring unit with first light of a first wavelength band, and a light receiving unit that collects reflected light reflected from the measuring unit to obtain image information including the affected area and the measuring unit, and a control unit that generates cartilage data for the cartilage based on the image information.

[0067] In addition, the measurement unit measures the insertion depth of the probe into the cartilage and transmits the measurement to the control unit, and the control unit obtains measurement position data for the position where the probe is inserted on the affected area based on the image information, and can generate cartilage data using the insertion depth and the measurement position data.

[0068] In addition, the measuring unit may include a case connected to the probe, a post arranged parallel to the probe on one side of the case, and a guide cover connected to the probe, one side of which is penetrated by the post to guide the direction of movement of the probe.

[0069] In addition, the surgical robot system may further include an elastic member disposed between the probe and the case and a detection sensor that detects the insertion depth and transmits the detection to the control unit.

[0070] Additionally, the control unit can generate cartilage data for the cartilage using the image information before the measuring unit is inserted into the affected area and the image information after the measuring unit is inserted into the affected area.

[0071] In addition, the device may further include a lighting unit that irradiates the second light of a second wavelength band different from the first wavelength band to the affected area.

[0072] Additionally, the second minimum wavelength value and the first maximum wavelength value of the first wavelength band may have values ​​greater than the second minimum wavelength value and the second maximum wavelength value of the second wavelength band, respectively.

[0073] In addition, the device may further include a filter unit that filters the reflected light incident on the light receiving unit into light of a third wavelength band.

[0074] Additionally, the filter unit can filter the light emitted from the light emitting unit into light of a third wavelength band.

[0075] Additionally, the third wavelength band may be the same as the first wavelength band.

[0076] In addition, the surgical robot system further includes a filter unit that filters the reflected light incident on the light receiving unit into light of a third wavelength band, and a third minimum wavelength value of the third wavelength band may have a value greater than a second maximum wavelength value of the second wavelength band.

[0077] In addition, the surgical robot system further includes a robotic arm having at least one joint and capable of accessing the affected area using an end effector mounted on one end, and the control unit can generate a working path of the robotic arm based on the cartilage data and the image information.

[0078] In addition, the control unit may obtain modeling data, which is a three-dimensional model of the affected area, from an external photographing device, obtain scan data, which is three-dimensional information of the affected area, based on the image information, correct the scan data using the cartilage data, match the modeling data with the corrected scan data to generate a matching result for the affected area, and generate the work path using the matching result.

[0079] A surgical robot system according to another embodiment of the present invention may include a sub-robot arm having at least one joint, a measuring unit mounted on an end of the sub-robot arm and including a probe that can be inserted into one or more cartilages set in advance among the affected areas, a joint sensor mounted on each of the joints and detecting movement of the joints to obtain position information of the measuring unit, and a control unit that generates cartilage data for the cartilage using the position information.

[0080] A surgical robot system control method according to another embodiment of the present invention may include a step of obtaining modeling data, which is a three-dimensional model of a affected area, from an external photographing device, a step of obtaining scan data, which is three-dimensional information about the affected area, based on image information about the affected area, a step of obtaining cartilage data based on the image information and correcting the scan data using the cartilage data, a step of matching the modeling data with the corrected scan data to generate a matching result for the affected area, and a step of generating a working path of a robot arm using the matching result.

[0081] Other aspects, features and advantages other than those described above will become apparent from the following drawings, claims and detailed description of the invention.

[0082] The present invention is capable of various modifications and embodiments. Specific embodiments are illustrated in the drawings and described in detail in the detailed description. The effects and features of the present invention, as well as the methods for achieving them, will become clearer with reference to the embodiments described in detail below, along with the drawings. However, the present invention is not limited to the embodiments disclosed below and can be implemented in various forms.

[0083] Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings. When describing with reference to the drawings, identical or corresponding components are given the same reference numerals and redundant descriptions thereof will be omitted.

[0084] In the examples below, singular expressions include plural expressions unless the context clearly indicates otherwise.

[0085] In the following examples, terms such as “include” or “have” mean that a feature or component described in the specification is present, and do not preclude the possibility that one or more other features or components may be added.

[0086] In some embodiments, where implementations are otherwise feasible, specific process sequences may be performed in a different order than described. For example, two processes described in succession may be performed substantially simultaneously, or in a reverse order from the described order.

[0087] For convenience of explanation, the sizes of components in the drawings may be exaggerated or reduced. For example, the sizes and thicknesses of each component shown in the drawings are arbitrarily indicated for convenience of explanation, and thus the following embodiments are not necessarily limited to those shown.

[0088] FIG. 1 is a drawing illustrating a surgical robot system according to one embodiment of the present invention.

[0089] Referring to FIG. 1, a surgical robot system (1) according to one embodiment of the present invention may include a robot arm (100), a photographing unit (200), a lighting unit (300), a measuring unit (400), a gap measuring sensor (GM), a control unit (500), and a lower limb assist device (600).

[0090] The robot arm (100) may include at least one link and joint. The robot arm (100) can change the position and posture of a group of objects using the rotational motion of the joints. Although FIG. 1 illustrates the robot arm (100) as including two joints and two links, the configuration of the robot arm (100) is not limited thereto.

[0091] A photographing unit (200) and an end effector (EF) can be connected to one end of a robot arm (100). The robot arm (100) can operate to change the position and posture of the photographing unit (200) and the end effector (EF). The robot arm (100) can approach the affected area (W) using the end effector (EF) and the photographing unit (200) mounted on one end. The robot arm (100) can change the posture of the photographing unit (200) and the end effector (EF) with respect to the affected area (W) using the rotational movement of the joints.

[0092] An end effector (EF) can be any type of surgical tool. For example, an EF could be an electrocautery device used to cut tissue and control bleeding, or a cannula inserted into the body. However, the types of EFs are not limited to these.

[0093] The robot arm (100) can be connected to a base robot (BR). The robot arm (100) can be mounted on one side of the base robot (BR) and supported by the base robot (BR).

[0094] The base robot (BR) can move within the surgical space where the surgery is performed. For example, the base robot (BR) may be equipped with wheels on its underside. The robot arm (100) can move to various positions within the surgical space, allowing the imaging unit (200) or end effector (EF) to be appropriately positioned to the surgical site.

[0095] When the motion of the robot arm (100) is controlled so that the end effector (EF) approaches the affected area (W), the movement of the end effector (EF) can be guided by a guide (not shown). For example, when sawing the affected area (W) using the end effector (EF), the movement direction of the end effector (EF) can be guided by a cutting guide (not shown) that guides the cutting path. Thus, the precision of surgery using the robot arm (100) can be improved.

[0096] In one embodiment, a guide (not shown) for guiding the movement of the end effector (EF) may be a mechanical configuration. The guide (not shown) may guide the movement direction of the end effector (EF) by contacting and supporting the end effector (EF), or may guide the movement direction of the robot arm (100) by contacting and supporting the robot arm (100).

[0097] In another embodiment, a guide (not shown) for guiding the movement of the end effector (EF) may be implemented by the control logic or algorithm of the robot arm (100). For example, the control unit (500) may guide the movement direction of the end effector (EF) by limiting the range of motion of the robot arm (100) using a virtual wall algorithm, etc., as described below.

[0098] The robot arm (100) and the base robot (BR) can form a slave robot. The robot arm (100) and the base robot (BR) can operate by receiving control signals from the master robot. In other words, the robot arm (100) and the base robot (BR) can operate based on signals input by the surgeon to the master robot. Thus, the robot arm (100) and the base robot (BR) can be precisely controlled by the surgeon.

[0099] The photographing unit (200) can obtain shape and location information about the affected area (W). The photographing unit (200) can obtain information about the affected area (W) and provide all information necessary to create a working path of the robot arm (100) to the control unit (500) as described below.

[0100] In one embodiment, the photographing unit (200) can obtain information about the affected area (W) using light. For example, the photographing unit (200) can obtain shape and position information about the affected area (W) by projecting visible light or infrared light onto the affected area (W) and photographing the projected pattern or by using stereo vision or ToF (Time of Flight). The present invention is not limited thereto, and the photographing unit (200) can obtain information about the shape and position of the affected area (W) by projecting X-rays onto the affected area (W) and utilizing the difference in absorption rate of the X-rays, or by projecting ultrasound and utilizing the time and angle at which the ultrasound is reflected. Alternatively, the photographing unit (200) may have a configuration such as LiDAR that fires laser pulses in all directions and obtains information about the affected area (W) by utilizing the time it takes for the laser pulses to be reflected and return.

[0101] In another embodiment, the photographing unit (200) can obtain information about the affected area (W) using mechanical interaction. For example, the photographing unit (200) may be configured as a robot arm comprising a multi-joint arm and an encoder, and can obtain shape and position information about the affected area (W) by contacting the affected area (W) through an end of the robot arm and tracking the position of the end through the encoder. Alternatively, the photographing unit (200) can obtain information about the shape and position of the affected area (W) by measuring a resistance force that changes according to the contact force of one end using the piezoelectric effect.

[0102] In addition, the photographing unit (200) may have various configurations that can be adopted by a person skilled in the art to obtain information about the affected area (W). However, for convenience of explanation, the following description will focus on an example in which the photographing unit (200) irradiates light to the affected area (W) to obtain image information about the affected area (W).

[0103] That is, the photographing unit (200) can obtain image information including the affected area (W) by irradiating light onto the affected area (W). The image information obtained by the photographing unit (200) can be provided to the surgeon (ST) through the display (DP) or used by the control unit (500) to obtain various information for controlling the robot arm (100).

[0104] In one embodiment, the photographing unit (200) can acquire multiple pieces of image information. Specifically, the photographing unit (200) can acquire multiple pieces of image information at various positions and postures with respect to the affected area (W).

[0105] That is, the photographing unit (200) can obtain multiple image information with different points of view for the affected area (W). Thus, as described below, when obtaining a matching result for the affected area (W), the accuracy of the matching result can be improved.

[0106] Image information may include information about the affected area (W), such as its location and posture. The image information may be two-dimensional image information or three-dimensional image information generated from multiple two-dimensional images. The type of image information is not limited thereto, and the image information may be three-dimensional model data for expressing the three-dimensional shape or structure of an object, or spatial data for expressing the location and arrangement of an object within space.

[0107] However, for convenience of explanation, the following description will focus on an example in which the image information is three-dimensional image information. For example, the image information acquired from the photographing unit (200) may include the coordinate values ​​of a specific point and depth data of an object located at the point.

[0108] Hereinafter, the 'wavelength band of light' may be defined as a set of wavelengths having an intensity greater than a preset ratio compared to the maximum intensity of the wavelength having the maximum intensity in the light spectrum. In turn, the 'wavelength band of light' may be defined as a continuous wavelength region in which the intensity of light is maintained at a preset reference intensity value.

[0109] For example, the reference intensity value may be 30% of the maximum intensity value in the light spectrum. The size of the reference intensity value is not limited thereto, and the size of the reference intensity value that determines the wavelength band of light may be appropriately selected depending on the measurement environment, the characteristics of the light source, or the type of tissue to be detected.

[0110] The photographing unit (200) can irradiate light (LE) to the affected area (W). The photographing unit (200) can irradiate light (LE) of a preset wavelength band to the affected area (W). The photographing unit (200) can adjust the wavelength band of the irradiated light to image tissues that react to the irradiated light (LE).

[0111] The light (LE) emitted from the photographing unit (200) can be reflected from the affected area (W). The light (LE) irradiated and reflected from the affected area (W) can propagate toward the photographing unit (200). That is, the light (LE) emitted from the photographing unit (200) can form a part of the reflected light (LR) collected by the photographing unit (200).

[0112] The light emitted (LE) may be reflected from one or more components. For example, the light emitted (LE) may be reflected from a measuring unit (400) that contacts the affected area (W) and measures the cartilage thickness of the affected area (W), or may be reflected from a affected area marker (WM) attached to the affected area (W).

[0113] The photographing unit (200) can collect reflected light (LR). The photographing unit (200) can collect light reflected from the target area (W), the measuring unit (400), and any other arbitrary components to obtain image information. That is, the photographing unit (200) can collect reflected light (LR) to obtain information necessary for controlling the robot arm (100).

[0114] The reflected light (LR) may include light reflected from one or more light sources. For example, the reflected light (LR) may be formed by the light (LE) emitted from the photographing unit (200) and the second light (L2) emitted from the lighting unit (300) being reflected in any configuration.

[0115] The type of light included in the reflected light (LR) is not limited thereto, and for example, the reflected light (LR) may include light emitted from the display (DP) and reflected in any configuration. However, for convenience of explanation, the following description will focus on an embodiment in which the reflected light (LR) is light formed by reflecting the emitted light (LE) and the second light (L2).

[0116] The reflected light (LR) may contain information about the reflected component. For example, the reflected light (LR) may contain information about the shape, surface condition, reflective properties, etc. of the visual information, such as the target area (W) or the measuring unit (400).

[0117] The photographing unit (200) can collect reflected light (LR) to obtain image information about the reflected component. In other words, the photographing unit (200) can capture a component that reflects light by collecting reflected light (LR). Thus, the photographing unit (200) can obtain image information including at least one of the affected area (W) and the measuring unit (400) to which light is irradiated and reflected. Although the following description focuses on an embodiment in which image information about the affected area (W) is obtained by the photographing unit (200) for convenience of explanation, it can be considered that the image information includes information about the measuring unit (400).

[0118] The lighting unit (300) can emit second light (L2). The lighting unit (300) can irradiate the second light (L2) having a preset wavelength band to the affected area (W), the measuring unit (400), etc. In one embodiment, the lighting unit (300) can emit the second light (L2) toward the affected area (W) so that the surgeon (ST) can visually check the affected area during surgery.

[0119] The second light (L2) emitted from the lighting unit (300) may be reflected from the return unit (W), the measuring unit (400), etc., and collected by the photographing unit (200). In other words, the second light (L2) emitted from the lighting unit (300) may form part of the reflected light (LR).

[0120] In one embodiment, the lighting unit (300) may be a spotlight, a ring light, etc. However, the type of the lighting unit (300) is not limited thereto, and for example, the lighting unit (300) may be configured with various light sources that provide light so that the surgeon (ST) can visually check the affected area (W) and other devices for the surgery, such as an LED array or a fluorescent lighting device. In the following, for the convenience of explanation, the description will focus on an embodiment in which the lighting unit (300) is a spotlight provided for the smooth progress of the surgery.

[0121] The lighting unit (300) can be controlled by receiving a control signal from the control unit (500). The lighting unit (300) can receive a control signal from the control unit (500) to adjust the timing of emission of the second light (L2).

[0122] One side of the measuring unit (400) can be inserted into the affected area (W). For example, the probe (PR) of the measuring unit (400) can be inserted into the affected area (W). The measuring unit (400) can measure the thickness of the cartilage (C) in the affected area (W) using the probe (PR) inserted into the affected area (W).

[0123] The measuring unit (400) can measure the insertion depth of the probe (PR) into the affected area (W) or provide information for other components to measure. The specific process of measuring the cartilage thickness of the affected area (W) using the measuring unit (400) will be described in detail later.

[0124] The gap measurement sensor (GM) can measure the gap between body structures in the affected area (W). The gap measurement sensor (GM) can measure the gap between body structures constituting the affected area (W) and provide general information for establishing a surgical plan or generating a working path of the robot arm (100).

[0125] In one embodiment, a gap measurement sensor (GM) can be inserted into a wound (W) to measure a gap between body structures constituting the wound (W). The gap measurement sensor (GM) can be inserted between body structures constituting the wound (W) to measure a gap between the body structures. Although the method by which the gap measurement sensor (GM) measures a gap between body structures is not limited thereto, for the convenience of explanation, the following description will focus on an embodiment in which the gap measurement sensor (GM) is inserted between body structures to measure a gap between the body structures.

[0126] The control unit (500) can be electrically connected to other components of the surgical robot system (1). The control unit (500) can receive information from other electrically connected components and generate control signals for each component based on the received information. The control unit (500) can transmit the generated control signals to the corresponding components, thereby ensuring that surgery using the surgical robot system (1) is performed with precision.

[0127] The control unit (500) can transmit a control signal to the robot arm (100). For example, the control unit (500) can determine a movement path of the end effector (EF) for performing a surgery, and can determine a working path of the robot arm (100) that allows the end effector (EF) to move along the movement path. The control unit (500) can generate a control signal to cause the robot arm (100) to move along the working path. Thus, the operation of the robot arm (100) can be controlled by the control unit (500).

[0128] The control unit (500) can transmit a control signal to the base robot (BR). The control unit (500) can control the operation of the base robot (BR) so that the robot arm (100) moves along a work path. The control unit (500) can control the position of the robot arm (100) connected to the base robot (BR) by adjusting the position of the base robot (BR) in the surgical space.

[0129] That is, the control unit (500) can adjust the position of the end effector (EF) using the robot arm (100) and the base robot (BR). By organically linking the movements of the robot arm (100) and the base robot (BR) and controlling them, the position and posture of the end effector (EF) can be precisely adjusted.

[0130] In one embodiment, the robot arm (100) and the base robot (BR) can be controlled by the control unit (500). The robot arm (100) and the base robot (BR) receive operation signals from the control unit (500) and operate, thereby allowing surgery to be performed using the end effector (EF).

[0131] The robot arm (100) and the base robot (BR) can be operated by motion signals and move along a work path generated by the control unit (500). For example, the motion signal of the robot arm (100) can be generated by applying an impedance and admittance control technique as described below, or by applying a virtual wall algorithm. Alternatively, the motion signal of the robot arm (100) can be generated by a combination of basic surgical unit motions, so that the precision and accuracy of the surgery by controlling the robot arm (100) can be improved.

[0132] In another embodiment, the robot arm (100) and the base robot (BR) can be directly manipulated by the surgeon (ST). The robot arm (100) and the base robot (BR) can have their positions and postures changed by the surgeon (ST), or the end effector (EF) can be positioned adjacent to the affected area (W) by the control unit (500) and then manipulated by the surgeon (ST) to perform surgery using the end effector (EF).

[0133] When the robot arm (100) is operated by the surgeon (ST), the operation of the robot arm (100) can be assisted by the control unit (500). For example, the operation of the robot arm (100) by the surgeon (ST) can be controlled by an impedance and admittance control technique corresponding to an external force. Alternatively, the range of motion of the robot arm (100) can be limited by a virtual wall algorithm to prevent the end effector (EF) from being displaced from the surgical area. In other words, when the operation of the robot arm (100) is assisted by the control unit (500) and the surgery is performed by the surgeon (ST), the accuracy and precision of the surgery can be improved.

[0134] The control unit (500) can control the robot arm (100) and the base robot (BR) to determine the position and posture of the photographing unit (200). The control unit (500) can determine the position and posture of the photographing unit (200) according to optimal photographing conditions for obtaining image information about the affected area (W) or the measuring unit (400), and transmit a control signal to the robot arm (100) and the base robot (BR) so that the photographing unit (200) has the corresponding position and posture. Thus, the control unit (500) can improve the quality of image information obtained from the photographing unit (200).

[0135] The control unit (500) can receive image information from the photographing unit (200). The control unit (500) can receive image information obtained by photographing the affected area (W) from the photographing unit (200). The control unit (500) can process the received image information and generate control signals regarding the operation of each electrically connected component.

[0136] The control unit (500) can control the operation of the photographing unit (200) by transmitting a control signal to the photographing unit (200). For example, the control unit (500) can control the operation of the photographing unit (200) to determine whether the photographing unit (200) takes a picture. The control unit (500) can control the photographing unit (200) to determine whether to emit light (LE), thereby determining whether the photographing unit (200) takes a picture.

[0137] In one embodiment, the control unit (500) can control the shooting timing of the photographing unit (200). The control unit (500) can control the shooting interval or the number of repetitions, etc., by controlling the first cycle in which the photographing unit (200) repeatedly takes pictures of the affected area (W) or the measuring unit (400). In other words, the control unit (500) can control the frequency of image information collection.

[0138] The control unit (500) can control the operation of the lighting unit (300) by transmitting a control signal to the lighting unit (300). The control unit (500) can transmit a control signal to the lighting unit (300) so that the lighting unit (300) turns on or off.

[0139] In one embodiment, the control unit (500) can determine whether the lighting unit (300) emits the second light (L2). The control unit (500) can control the frequency and time of light irradiation by adjusting the second cycle in which the lighting unit (300) emits the second light (L2). That is, the control unit (500) can improve the quality of acquired image information by synchronizing or desynchronizing the first cycle of the photographing unit (200) and the second cycle of the lighting unit (300).

[0140] The control unit (500) may be electrically connected to the measuring unit (400). The control unit (500) may receive information about the insertion depth at which the measuring unit (400) is inserted into the return part (W) from the measuring unit (400). The control unit (500) may process the information received from the measuring unit (400) and use it to generate a working path of the robot arm (100).

[0141] The control unit (500) can transmit various information for surgical progress to the display (DP). The control unit (500) can transmit image information received from the photographing unit (200), position information of the end effector (EF) and the photographing unit (200), insertion depth received from the measuring unit (400), etc. to the display (DP). Thus, the surgeon (ST) can visually confirm information necessary for surgical progress.

[0142] The control unit (500) can receive information for generating a work path of the robot arm (100) from the input unit (IM). In one embodiment, the control unit (500) can receive modeling data, which is three-dimensional information about the affected area (W), from the input unit (IM). For example, the control unit (500) can receive modeling data stored in the input unit (IM) or modeling data about the affected area (W) input by the surgeon (ST).

[0143] In another embodiment, the control unit (500) may receive a medical image including the affected area (W) from the input unit (IM). For example, the control unit (500) may receive a medical image including the affected area (W) obtained through computed tomography (CT) from the input unit (IM). That is, the control unit (500) may be configured to generate a three-dimensional image by reconstructing a plurality of two-dimensional images obtained by tomography. In this case, the medical image may be configured with a plurality of two-dimensional images obtained from at least one viewpoint or angle.

[0144] The control unit (500) can extract (segment) a region of interest from a medical image to obtain modeling data. For example, if the affected area (W) is the knee, the control unit (500) can segment and process images of the femur and tibia to obtain detailed information about the relevant area.

[0145] The control unit (500) can establish a surgical plan for the affected area (W) and generate a work path corresponding to the surgical plan. For example, the control unit (500) can obtain scan data from image information received from the photographing unit (200), and generate a matching result by matching the scan data and modeling data. Based on the matching result, the control unit (500) can generate a surgical plan and a work path corresponding to the surgical plan. The control unit (500) can convert the control signal into a control signal for driving the robot arm (100) and transmit it to the robot arm (100) so that the robot arm (100) moves along the generated work path.

[0146] The control unit (500) can receive an operation signal input by the operator (ST) from the input unit (IM). The control unit (500) can receive an operation signal for each component of the surgical robot system (1) input by the operator (ST) to the input unit (IM).

[0147] For example, the control unit (500) can receive a manipulation signal input by the operator (ST) to the input unit (IM) regarding the movement of the end effector (EF). The control unit (500) can convert the signal input by the operator (ST) into a control signal regarding the operation of the robot arm (100) and transmit it to the robot arm (100). The robot arm (100) can be controlled by the control unit (500) and operated according to the intention of the operator (ST).

[0148] The input unit (IM) can receive manipulation signals regarding the operation of the surgical robot system (1) from the surgeon (ST). The input unit (IM) can transmit the manipulation signals received from the surgeon (ST) to the control unit (500). The input unit (IM) can receive manipulation signals regarding the operation of the surgical robot system (1) from various components that can be directly operated by the surgeon, such as buttons, a mouse, or a joystick.

[0149] Although Fig. 1 illustrates that the input unit (IM) and the display (DP) are separate components, the configuration of the input unit (IM) and the display (DP) is not limited to this. For example, the input unit (IM) and the display (DP) are configured as an integrated unit, so that the surgeon (ST) can check the surgical status through the display (DP) and control the operation of the surgical robot system (1) by manipulating the display (DP).

[0150] The input unit (IM) can obtain three-dimensional information including the affected area (W) from the outside. The input unit (IM) can transmit the three-dimensional information including the affected area (W) obtained from the outside to the control unit (500).

[0151] In one embodiment, the input unit (IM) may receive modeling data from a surgeon (ST). Alternatively, the input unit (IM) may request information about a patient (P), a lesion (W), etc. from an external server to receive modeling data.

[0152] In another embodiment, the input unit (IM) may receive medical images from an external device (not shown). For example, the input unit (IM) may receive medical images acquired by photographing a target area from a CT scan device. The input unit (IM) may transmit the acquired medical images to the control unit (500).

[0153] A lower extremity assist device (600) can assist the lower extremities of a patient (P). The lower extremity assist device (600) can contact and support the lower extremities of a patient (P) lying on an operating table (OT) for surgery. The lower extremity assist device (600) is connected to the operating table (OT) on one side, and can transmit the load from the lower extremities of the patient (P) to the operating table (OT). Thus, the lower extremity assist device (600) can stably support the position and posture of the affected part (W) during surgery.

[0154] The lower extremity assist device (600) can move relative to the operating table (OT). The lower extremity assist device (600) can be moved relative to the operating table (OT) during surgery to change its position and posture. That is, the position and posture of the affected part (W) supported by the lower extremity assist device (600) can be changed. The position and posture of the affected part (W) can be changed as needed during surgery, thereby improving the precision of the surgery.

[0155] The lower extremity assist device (600) is electrically connected to the control unit (500) and can be controlled by the control unit (500). For example, the lower extremity assist device (600) can be controlled by the control unit (500) to change its position and posture with respect to the operating table (OT). Thus, the lower extremity assist device (600) can appropriately adjust the position of the affected part (W) of the patient (P) with respect to the operating table (OT) according to the request of the operator (ST) or the surgical situation.

[0156] Figure 2 is a drawing showing in detail some of the configurations of Figure 1.

[0157] Referring to FIG. 2, the end effector (EF) and the shooting unit (200) can be mounted on the end of the robot arm (100).

[0158] The end effector (EF) and the photographing unit (200) can be installed together at one end of the robot arm (100). The end effector (EF) and the photographing unit (200) are arranged adjacent to each other, so that their positions and postures can be changed together when the robot arm (100) operates. Thus, alignment of the end effector (EF) and the photographing unit (200) using the robot arm (100) can be facilitated.

[0159] In one embodiment, the photographing unit (200) may be positioned between the robot arm (100) and the end effector (EF). The end effector (EF) may be mounted on one side of the photographing unit (200) connected to the end of the robot arm (100). The end effector (EF) is connected to the robot arm (100) via the photographing unit (200), thereby performing a motion structurally separated from the robot arm (100). That is, the surgeon (ST) can easily access the end effector (EF) without structurally changing the photographing unit (200) or the robot arm (100). Thus, maintenance and replacement of the end effector (EF) may be facilitated.

[0160] The end effector (EF) and the photographing unit (200) may be arranged side by side. Specifically, the direction in which the end of the end effector (EF) that interacts with the affected area (W) faces and the direction in which the light is emitted and collected from the photographing unit (200) faces may be parallel. The photographing unit (200) can simultaneously photograph the affected area (W) and the end effector (EF). The surgeon (ST) can check the condition of the affected area (W) while simultaneously manipulating the end effector (EF), thereby maximizing the precision of the surgery.

[0161] Figure 3 is a diagram illustrating a control system of the surgical robot system of Figure 1.

[0162] Referring to FIG. 3, the control unit (500) can be electrically connected to each component of the surgical robot system (1).

[0163] The control unit (500) can receive information from each component of the surgical robot system (1). For example, the control unit (500) can receive operation signals, image information, and information on insertion depth input from the operator (ST) through the input unit (IM), the photographing unit (200), and the detection sensor (470). The control unit (500) can store the received information or generate a working path of the robot arm (100) using the received information.

[0164] The control unit (500) can transmit control signals to each component of the surgical robot system (1). For example, the control unit (500) can transmit control signals to the robot arm (100), the lower extremity assist device (600), and the display (DP). The control unit (500) can transmit a control signal to the robot arm (100) so that the robot arm (100) moves along a work path, or transmit a control signal to the lower extremity assist device (600) for adjusting the posture of the affected part (W). Alternatively, the control unit (500) can transmit various information for the surgery to the display (DP) so that the surgeon (ST) can visually check it.

[0165] In one embodiment, the control unit (500) may include a memory (510), a processor (520), and a communication unit (530).

[0166] The memory (510) can store all information necessary for generating a working path of the robot arm (100). For example, the memory (510) can store data regarding a pre-surgical plan.

[0167] In one embodiment, the memory (510) can store modeling data. Specifically, the memory (510) can receive modeling data input by the surgeon (ST) into the input unit (IM), or can receive and store modeling data stored in the input unit (IM). Alternatively, the memory (510) can store modeling data acquired from a medical image obtained by photographing the affected area (W) using an external device.

[0168] In another embodiment, the memory (510) may store surgical planning information generated based on modeling data. For example, the memory (510) may store surgical planning information including information regarding the surgical procedure, such as the incision location, incision range, and tissue removal order of the affected area (W).

[0169] The memory (510) can receive and store surgical plan information generated based on modeling data from a processor (520) to be described later, or can receive and store surgical plan information generated from modeling data from a separate computing device.

[0170] In one embodiment, the memory (510) can receive and store information collected during surgery. The memory (510) can receive and store information about the affected area (W), the measuring unit (400), etc., obtained from each component of the surgical robot system (1).

[0171] For example, the memory (510) can store image information on the affected area (W) acquired through the photographing unit (200), scan data acquired from the image information in the processor (520), insertion depth measured by the detection sensor (470) of the measuring unit (400), etc. The type of information stored in the memory (510) is not limited thereto, and the memory (510) can store various information for generating a work path of the robot arm (100), such as affected area marker data generated by the processor (520), as described below.

[0172] The processor (520) can generate a work path of the robot arm (100) based on image information.

[0173] In one embodiment, the processor (520) may obtain scan data, which is three-dimensional information about the affected area (W), from image information, and use the modeling data and the scan data to generate a working path of the robot arm (100). For example, the processor (520) may match the modeling data and the scan data to generate a matching result, and use the matching result to generate a working path.

[0174] The processor (520) can obtain scan data from multiple pieces of image information. The processor (520) can obtain scan data from multiple pieces of image information with different viewpoints with respect to the affected area (W). That is, the processor (520) can obtain multiple pieces of scan data with different viewpoints with respect to the affected area (W).

[0175] The processor (520) can obtain scan data, which is three-dimensional information about the affected area (W), based on the image information obtained from the photographing unit (200). For example, the processor (520) can recognize the affected area (W) from the image information, which is three-dimensional information, and obtain scan data. The processor (520) can obtain scan data, which is a set of three-dimensional coordinate values ​​for each point of the affected area (W), based on the image information.

[0176] In one embodiment, the processor (520) may receive modeling data from an external source. For example, the processor (520) may receive modeling data input by a user from an input unit (IM) and use the received modeling data to generate a matching result.

[0177] In another embodiment, the processor (520) may receive medical images from an external source and generate modeling data from the medical images. For example, the processor (520) may receive medical images from the input unit (IM) and obtain modeling data obtained by separating and processing a region of interest from the medical images. For example, if the affected area (W) is the knee, the modeling data may be data obtained by separating the region including the tibia and femur from the medical image.

[0178] The processor (520) can correct the modeling data to match the scan data. For example, the processor (520) can obtain cartilage data, which is three-dimensional information about the cartilage (C) in the affected area (W), and correct the modeling data.

[0179] In one embodiment, the processor (520) can generate cartilage data based on image information acquired from the photographing unit (200).

[0180] For example, the processor (520) can obtain a measurement point at which the thickness of the cartilage (C) is measured and the thickness of the cartilage (C) at the measurement point using image information including the measurement unit (400). The processor (520) can generate cartilage data for the cartilage (C) by comparing image information before the measurement unit (400) is inserted into the affected area (W) with image information after the measurement unit (400) is inserted into the affected area (W).

[0181] Alternatively, the processor (520) may receive information on the thickness of the cartilage (C) from the detection sensor (470) and information on the measurement point from the image information. Thus, the processor (520) may generate cartilage data on the location and thickness of the cartilage (C).

[0182] The processor (520) can correct scan data using cartilage data. The processor (520) can delete a portion corresponding to cartilage (C) from the scan data using the cartilage data.

[0183] In other words, the processor (520) can correct the scan data so that the scan data includes three-dimensional information about the shape of the bone (B). Thus, the processor (520) can generate information for generating a precise work path of the robot arm (100).

[0184] The alignment result obtained from the processor (520) may include information such as differences in position and posture between the modeling data and the scan data, or the reliability of the alignment. Alternatively, the alignment result may include information regarding the shape, position, and posture of the actual affected area estimated based on the differences between the modeling data and the scan data. The alignment result is not limited thereto, and may include various information for generating a working path of the robot arm (100).

[0185] In one embodiment, the processor (520) may generate a matching result using a learning model. The processor (520) may match modeling data and scan data using a previously learned learning model to generate a matching result. The specific process by which the processor (520) generates a matching result using the learning model will be described below.

[0186] In another embodiment, the processor (520) may match the modeling data and the scan data using a 6D pose estimation technique or an Iterative Closest Point (ICP) algorithm. The processor (520) may estimate the position and pose of the affected area (W) using the scan data, align the 6D pose information to the modeling data, and generate a matching result, or may apply the Iterative Closest Point (ICP) algorithm to minimize the shape difference between the modeling data and the scan data to generate a matching result.

[0187] The processor (520) can match multiple scan data and modeling data. The processor (520) can match each scan data and modeling data to obtain more accurate matching results. For example, the processor (520) can select the scan data with the highest 3D coordinate value match with the modeling data and perform matching with the modeling data. Thus, the accuracy of the matching results can be improved.

[0188] The method by which the processor (520) matches modeling data and scan data to generate a matching result is not limited to what has been described above, and it is reasonable to consider that all methods that can be used by a person skilled in the art to match modeling data and scan data fall within the scope of the present invention.

[0189] The processor (520) can generate a working path of the robot arm (100) using the matching result. For example, the processor (520) can match the position and posture of the actual affected area (W) with the surgical planning information based on the matching result. The processor (520) can generate a working path of the robot arm (100) for performing the surgery according to the surgical planning information.

[0190] For example, the processor (520) can generate a work path including a movement path and posture change of the robot arm (100) so that the end effector (EF) reaches a specific part of the affected area (W) or performs tasks such as incision or fixation. Thus, the precision and safety of the surgery can be improved and the workload of the surgeon can be reduced.

[0191] In one embodiment, the processor (520) may generate a work path by considering landmark data on body features including the alignment result and the affected area (W). The processor (520) may derive data on the posture or position of the affected area (W) from the landmark data and generate a work path based thereon.

[0192] The processor (520) can obtain landmark data for the affected area (W) based on modeling data. The processor (520) can extract coordinate values ​​for feature points on the body including the affected area (W) from the modeling data, which is three-dimensional coordinate data for the affected area (W), and define them as landmark data.

[0193] In one embodiment, when the affected area (W) is the knee, the processor (520) may select one or more coordinate values ​​on the modeling data for calculating the angle formed by the femur and the tibia and define them as landmark data. For example, the processor (520) may define coordinate values ​​for preset points on the hip center, the femur knee center, the tibia knee center, and the medial malleolus as landmark data.

[0194] In one embodiment, the processor (520) may generate a work path using the alignment results and landmark data. The processor (520) may obtain information about the posture of the affected area (W) based on the landmark data and generate a work path based on this information. For example, if the affected area (W) is the knee, the processor (520) may calculate the angle between the femur and tibia and generate a work path based on this information.

[0195] In another embodiment, the processor (520) may generate a work path using the alignment results and the spacing. For example, the processor (520) may establish a surgical plan for an implant based on information regarding the spacing between body structures obtained from a spacing measurement sensor (GM).

[0196] The processor (520) can generate a working path of the robot arm (100) by considering the matching result and the surgical plan for the implant, or can modify the working path generated based on the matching result by reflecting the surgical plan for the implant.

[0197] The processor (520) can generate an operation signal of the robot arm (100). Specifically, the processor (520) can generate an operation signal of the robot arm (100) to cause the robot arm (100) to move along a work path.

[0198] In one embodiment, the control unit (500) may generate an operation signal of the robot arm (100) by applying an impedance and admittance control technique. The processor (520) may control the dynamic relationship between position, speed, and force in response to an external force generated at one end of the robot arm (100), or may control the position of the robot arm (100) based on the system's response to the external force. Thus, the processor (520) may precisely control the robot arm (100) according to changes in the operating force of the surgeon (ST).

[0199] In another embodiment, the processor (520) may apply a virtual wall algorithm to limit the range of motion of the robot arm (100) or set a work area. The processor (520) may prevent the robot arm (100) from leaving the surgical area by applying a resistance force to the motion of the robot arm (100) that exceeds the preset spatial boundary.

[0200] In another embodiment, the processor (520) may set the basic surgical unit motion of the robot arm (100) and generate an operation signal of the robot arm (100) by combining the basic surgical unit motions. The control unit (500) may generate an operation signal of the robot arm (100) to sequentially execute the basic surgical unit motions or selectively perform them according to the generated work path of the robot arm (100).

[0201] The processor (520) can generate operation signals for the photographing unit (200) and the lighting unit (300). For example, the processor (520) can generate operation signals for the operation timing of the photographing unit (200) and the lighting unit (300). The processor (520) can generate the operation signals so that the photographing timing of the photographing unit (200) and the emission timing of the lighting unit (300) do not overlap. Thus, the accuracy of image information acquired from the photographing unit (200) can be improved.

[0202] The communication unit (530) can perform data transmission and reception between each component of the surgical robot system (1).

[0203] The communication unit (530) can receive information from the input unit (IM), the photographing unit (200), or the detection sensor (470) and transmit it to the memory (510) or the processor (520). The communication unit (530) can convert the information generated by the processor (520) into a signal regarding the operation of the robot arm (100), the lower limb auxiliary device (600), or the display (DP) and transmit a control signal to each component.

[0204] The communication unit (530) can process information generated by the processor (520) and convert it into a control signal. In one embodiment, the communication unit (530) can generate a control signal of the robot arm (100) based on the operation signal of the robot arm (100) generated by the processor (520). The communication unit (530) can convert the operation signal of the robot arm (100) generated by the processor (520) into a control signal regarding the operation of each joint of the robot arm (100), thereby allowing the robot arm (100) to move along a work path.

[0205] FIG. 4 is a diagram for explaining a process for generating a matching result according to one embodiment of the present invention.

[0206] Referring to FIG. 4, the processor (520) can generate a matching result using the pre-learned first model (M1). Specifically, the processor (520) can generate a matching result based on modeling data and scan data using the pre-learned first model (M1). That is, the processor (520) can generate all information for generating a work path using the first model (M1).

[0207] The first model (M1) can be trained using information corresponding to modeling data, scan data, and matching results. For example, the first model (M1) can be trained using learning modeling data corresponding to modeling data and learning scan data corresponding to scan data as input data, and using the learning matching results generated by matching the modeling data and scan data as output data.

[0208] The first model (M1) can be trained based on the difference between the matching result generated using the learning modeling data and the learning scan data as input data and the learning matching result. For example, training the first model (M1) may include a process of updating at least one parameter constituting the first model (M1) based on the difference between the matching result generated using the learning modeling data and the learning scan data and the learning matching result.

[0209] In one embodiment, the processor (520) can obtain modeling data using a pre-learned second model (M2). The processor (520) can obtain modeling data from a medical image using the pre-learned second model (M2).

[0210] The second model (M2) can be trained using information corresponding to medical images and modeling data. For example, the second model (M2) can be trained using training medical images corresponding to medical images as input data and training modeling data corresponding to modeling data as output data.

[0211] The learning modeling data used to train the second model (M2) may be data from medical images with regions of interest separated. For example, if the affected area (W) is the knee, the learning modeling data may be data from medical images with regions containing the tibia and femur separated.

[0212] The second model (M2) can be trained based on the differences between the modeling data generated using the training medical images as input data and the resulting training modeling data. For example, training the second model (M2) may include updating at least one parameter constituting the second model (M2) based on the differences between the modeling data generated using the training medical images and the training modeling data.

[0213] In one embodiment, the first model (M1) and the second model (M2) may include a learning model based on at least one of various artificial neural network architectures, such as a multilayer perceptron (MLP), a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a long short-term memory (LSTM), a gated recurrent unit (GRU), a transformer, or an autoencoder. In addition, the first model (M1) and the second model (M2) may include a learning model based on at least one of various data analysis techniques, such as a principal component analysis (PCA) or an independent component analysis (ICA).

[0214] FIG. 5 is a diagram for explaining a process of obtaining landmark data according to one embodiment of the present invention.

[0215] Referring to FIG. 5, the processor (520) can obtain landmark data using a pre-learned third model (M3). The processor (520) can generate modeling data from a medical image using the pre-learned third model (M3).

[0216] The third model (M3) can be trained using information corresponding to medical images and landmark data. For example, the third model (M3) can be trained using learning modeling data corresponding to the modeling data as input data, and learning landmark data corresponding to the landmark data as output data.

[0217] The learning landmark data used for learning the third model (M3) may be information obtained by extracting coordinate values ​​for feature points on the body including the affected area (W). For example, if the affected area (W) is the knee, the learning landmark data may include coordinate values ​​for the femoral knee joint center, the tibial knee joint center, or a preset point on the medial malleolus.

[0218] The third model (M3) can be trained based on the differences between the landmark data generated from the learning modeling data and the learning landmark data. For example, training the third model (M3) may include updating at least one parameter constituting the third model (M3) based on the differences between the landmark data generated from the learning modeling data and the learning landmark data.

[0219] In one embodiment, the third model (M3) may include a learning model based on at least one of various artificial neural network architectures, such as a multilayer perceptron (MLP), a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a long short-term memory (LSTM), a gated recurrent unit (GRU), a transformer, or an autoencoder. In addition, the third model (M3) may include a learning model based on at least one of various data analysis techniques, such as a principal component analysis (PCA) or an independent component analysis (ICA).

[0220] Figure 6 is a drawing showing in detail some of the configurations of Figure 1.

[0221] Referring to Fig. 6, the affected area (W) can be irradiated with light (LE) emitted from the photographing unit (200) and second light (L2) emitted from the lighting unit (300). The reflected light (LR) reflected from the affected area (W) can be transmitted to the photographing unit (200).

[0222] Although FIGS. 6 and 7 illustrate that the emission light (LE) and the second light (L2) are irradiated only to the affected area (W), the target irradiated with the emission light (LE) and the second light (L2) is not limited thereto. For example, the emission light (LE) and the second light (L2) may be irradiated to and reflected by the affected area marker (WM) or the measuring unit (400) described below. However, for the convenience of explanation, the following description will focus on an example in which the emission light (LE) and the second light (L2) are irradiated to the affected area (W).

[0223] Hereinafter, the light receiving unit (230) may be referred to as each of the first light receiving unit (230a) and the second light receiving unit (230b), or as a component encompassing the first light receiving unit (230a) and the second light receiving unit (230b).

[0224] In one embodiment, the photographing unit (200) may include a main body (210), a light emitting unit (220), and a light receiving unit (230).

[0225] The main body (210) can form the exterior of the photographing unit (200). The main body (210) can provide a space within which a light emitting unit (220) and a light receiving unit (230) can be placed. The main body (210) can spatially separate the internal space and the external space to protect the components placed within.

[0226] The light emitting unit (220) can irradiate the first light (L1) to the affected area (W). The first light (L1) emitted from the light emitting unit (220) can be reflected from the affected area (W). That is, the first light (L1) can form a part of the reflected light (LR).

[0227] The light emitting unit (220) can emit the first light (L1) of the first wavelength band. That is, the light emitting unit (220) can cause the first light (L1) of the first wavelength band to be reflected from the light receiving unit (230) to form a portion of the reflected light (LR). Thus, the light emitting unit (220) can determine the wavelength band of the light collected by the light receiving unit (230).

[0228] In one embodiment, the first wavelength band may be a near-infrared wavelength band. The light emitting unit (220) can safely protect the affected area (W) being irradiated with light by using the first light (L1) in the infrared wavelength band with low energy. In other words, even if the affected area is irradiated with light for a long period of time during surgery, the safety of the affected area (W) can be ensured.

[0229] Preferably, the first wavelength band may be a wavelength band of 800 nm or more. In other words, the light emitting unit (220) may irradiate light of a wavelength band different from the visible light band to the affected area (W).

[0230] When the light emitting unit (220) irradiates a wavelength band of less than 800 nm to the affected area (W), interference may occur due to visible light irradiated from another light source irradiating the affected area (W). That is, the reflected light collected by the light receiving unit (230) may be affected, and there is a concern that noise may occur in the image information acquired by the photographing unit (200).

[0231] On the other hand, when the light emitting unit (220) irradiates light of a wavelength band of 800 nm or more to the affected area (W), interference by visible light irradiated to the affected area (W) can be prevented. The light emitting unit (220) can improve the accuracy of image information acquired from the photographing unit (200) by irradiating the first light of a wavelength band that does not overlap with visible light to the affected area (W).

[0232] More preferably, the first wavelength band may be a wavelength band of 830 nm to 850 nm. The first light (L1) having a wavelength of 830 nm to 850 nm can effectively obtain image information about the inside of the tissue of the affected area (W) by penetrating the skin, muscles, blood vessels, etc. to a certain depth. For example, the first light (L1) having a wavelength of 830 nm to 850 nm can detect a difference in absorption rate between blood and tissue, thereby allowing changes in blood flow to be identified during surgery.

[0233] Meanwhile, for the first light (L1) having a wavelength of 830 nm to 850 nm, the bone may have a relatively high reflectivity compared to soft tissue. That is, the light emitting unit (220) may allow the reflected light (LR) to include light reflected with a high reflectivity from the outer surface of the bone. Thus, the light receiving unit (230) may collect the reflected light (LR) containing more accurate information about the outer surface of the bone, and the photographing unit (200) may obtain accurate image information about the affected area (W) and provide it to the control unit (500).

[0234] The light receiving unit (230) can collect reflected light (LR) reflected from the affected area (W). The light receiving unit (230) can collect reflected light (LR) emitted from the light emitting unit (220) and the lighting unit (300) and reflected from the affected area (W). The light receiving unit (230) can collect reflected light (LR) to obtain image information about the affected area (W).

[0235] A plurality of light-receiving units (230) may be provided. In FIGS. 4 and 7, two light-receiving units (230) are illustrated as being included in the photographing unit (200), but the number of light-receiving units (230) is not limited thereto. However, for convenience of explanation, the following description will focus on an embodiment in which two light-receiving units (230) are included in the photographing unit (200).

[0236] In one embodiment, the photographing unit (200) may include a first light receiving unit (230a) and a second light receiving unit (230b).

[0237] The first light-receiving unit (230a) and the second light-receiving unit (230b) may be positioned at different locations within the case (410) and may be positioned to face different directions. The first light-receiving unit (230a) and the second light-receiving unit (230b) may observe the affected area (W) from different viewpoints, thereby clearly identifying the three-dimensional shape of the affected area (W).

[0238] The first light receiving unit (230a) and the second light receiving unit (230b) can obtain depth information or three-dimensional shape information of the affected area (W) using images acquired at different points in time. For example, the light receiving unit (230) can obtain depth information of the affected area (W) using the principle of stereo vision or the parallax between images.

[0239] In one embodiment, the first light-receiving unit (230a) and the second light-receiving unit (230b) may be arranged symmetrically with respect to the light-emitting unit (220). The first light-receiving unit (230a) and the second light-receiving unit (230b) may uniformly detect the affected area (W), and the accuracy of the acquired image information may be improved.

[0240] The lighting unit (300) can irradiate the second light to the affected area (W). The second light (L2) emitted from the lighting unit (300) can be reflected from the affected area (W) to form a portion of the reflected light (LR).

[0241] The lighting unit (300) can emit second light (L2) of a second wavelength band. The lighting unit (300) can cause the second light (L2) of the second wavelength band to be reflected from the ring (W) and form part of the reflected light (LR).

[0242] The second wavelength band of the second light (L2) may be a visible light wavelength band. The second light (L2) may be a visible light band that can be visually recognized by the surgeon (ST) and may be irradiated to the affected area (W). Thus, the surgeon (ST) may obtain visual information about the affected area (W), such as the shape or outline of the affected area (W), through the reflected light (LR) reflected from the affected area (W) and entering the field of view.

[0243] The second wavelength band may be different from the first wavelength band. The wavelength bands of the first light (L1) emitted from the light emitting unit (220) and the second light (L2) emitted from the lighting unit (300) may be different from each other.

[0244] The first light (L1) and the second light (L2) have different wavelength bands, so that light containing different physical and physiological information about the affected area (W) can be included in the reflected light (LR). Thus, the surgeon (ST) can be provided with precise and rich information about the affected area (W).

[0245] The first minimum wavelength value and the first maximum wavelength value of the first wavelength band may be greater than the second minimum wavelength value and the second maximum wavelength value of the second wavelength band, respectively. That is, the first wavelength band may be located in a wavelength region higher than the second wavelength band.

[0246] The first light (L1) can perform a different function from the second light (L2) by irradiating light of a higher wavelength band than the second light (L2). For example, the first light (L1) of the first wavelength band can be used to obtain image information about the tissue of the affected area (W), and the second light (L2) of the second wavelength band can be used by the surgeon (ST) to visually observe the affected area (W).

[0247] In one embodiment, the first minimum wavelength value of the first wavelength band may be greater than the second maximum wavelength value of the second wavelength band. That is, the first light (L1) may have a wavelength band higher than the second light (L2), but may have a separate wavelength band that does not overlap with the second light (L2). Thus, interference between the first light (L1) and the second light (L2) can be prevented, and the accuracy of image information acquired by the photographing unit (200) can be improved.

[0248] Referring to FIG. 4, the photographing unit (200) may further include a filter unit (240). The filter unit (240) may adjust the wavelength band, such as an ND filter, a UV filter, or a visible light filter, or may adjust the vibration direction of light, such as a polarizing filter. However, for convenience of explanation, the following description will focus on an embodiment in which the filter unit (240) adjusts the wavelength band of light passing through it.

[0249] The filter unit (240) can only pass light of a preset wavelength band among the incident light. The light passing through the filter unit (240) can be filtered into light of a preset third wavelength band. That is, the intensity of light outside the third wavelength band among the light passing through the filter unit (240) can be reduced.

[0250] The filter unit (240) can filter the reflected light (LR) reflected from the target area (W) and incident on the light receiving unit (230). The filter unit (240) can selectively pass light of a third wavelength band among the reflected light (LR). That is, the filter unit (240) can filter the reflected light (LR) so that the third light (L3) having the third wavelength band can be collected by the light receiving unit (230). Thus, the filter unit (240) can minimize the influence of noise or background light on the image information acquired by the photographing unit (200), and enable accurate image information about the target area (W) to be acquired.

[0251] The filter unit (240) can filter the first light (L1) emitted from the light emitting unit (220). The filter unit (240) can selectively pass light of a third wavelength band among the first light (L1). The filter unit (240) can filter the first light (L1) so that the emitted light (LE) having the third wavelength band is irradiated to the affected area (W). The filter unit (240) can precisely control the wavelength band of the light irradiated to the affected area (W).

[0252] The filter unit (240) can simultaneously filter the first light (L1) emitted from the light emitting unit (220) and the reflected light (LR) incident on the light receiving unit (230). In detail, the first light (L1) emitted from the light emitting unit (220) is filtered in the filter unit (240) so that the emitted light (LE) having a third wavelength band can be irradiated to the affected area (W). In addition, the reflected light (LR) including the emitted light (LE) reflected from the affected area (W) can be filtered in the filter unit (240) so that the third light (L3) of the third wavelength band can be collected in the light receiving unit (230).

[0253] In other words, the first light (L1) emitted from the light emitting unit (220) can be filtered multiple times by the filter unit (240) and collected by the light receiving unit (230). Thus, the filter unit (240) can enable light of a precisely controlled wavelength band to be collected by the light receiving unit (230).

[0254] Although FIGS. 6 and 9 illustrate that a single filter unit (240) is mounted on one side of the case (410) to filter the first light (L1) emitted from the light emitting unit (220) and the reflected light (LR) incident on the light receiving unit (230), the shape and arrangement of the filter unit (240) are not limited thereto. For example, a plurality of filter units (240) may be provided and mounted on each light emitting unit (220) and light receiving unit (230).

[0255] In one embodiment, the third minimum wavelength value of the third wavelength band may have a value greater than the second maximum wavelength value of the second wavelength band. That is, the minimum wavelength value of the light that the filter unit (240) passes through may be greater than the maximum wavelength value of the second light (L2). Thus, the second light (L2) irradiated from the lighting unit (300) may not pass through the filter unit (240).

[0256] In other words, the reflected light (LR), which includes the light reflected from the light source (LE) and the second light (L2) from the target area (W), includes light of the second wavelength band, but in the process of the reflected light (LR) being filtered in the filter unit (240), the light of the second wavelength band may not pass through the filter unit (240). Therefore, the third light (L3) may not include light having the second wavelength band.

[0257] Since the light of the second wavelength band of the second light (L2) emitted from the lighting unit (300) is not included in the third light (L3), the influence of the second light (L2) on the light collected by the light receiving unit (230) can be minimized. The light receiving unit (230) can collect the third light (L3) with the influence of the lighting unit (300) minimized, thereby preventing the image information collected by the photographing unit (200) from being affected by optical interference or noise caused by the lighting unit (300).

[0258] In one embodiment, the third wavelength band may correspond to the first wavelength band. Again, the third wavelength band may be substantially the same as the first wavelength band. The filter unit (240) may pass a wavelength band that is substantially the same as the wavelength band of the first light (L1) emitted from the light emitting unit (220). The filter unit (240) may pass the first light (L1) emitted from the light emitting unit (220) so that the emitted light (LE) has a wavelength band that is substantially the same as the first light (L1).

[0259] That is, the filter unit (240) can minimize the influence on the first light (L1) emitted from the light emitting unit (220), and can allow the first light (L1) among the reflected light (LR) reflected from the target area (W) to pass through and be incident on the light receiving unit (230). Meanwhile, the filter unit (240) can not allow the second light (L2) among the reflected light (LR) reflected from the target area (W) to pass through. The filter unit (240) can improve the accuracy and reliability of the captured image information by ensuring that only information based on the first light (L1) is included in the reflected light (LR) incident on the light receiving unit (230).

[0260] In one embodiment, the third wavelength band may be a near-infrared wavelength band. The filter unit (240) can reduce the noise ratio of image information by allowing the third light (L3) in the near-infrared wavelength band, which has less light scattering, to be collected by the light receiving unit (230). Thus, the accuracy of image information acquired by the photographing unit (200) can be improved.

[0261] Preferably, the third wavelength band may be a wavelength band of 800 nm or more. That is, the light receiving unit (230) may collect light of a wavelength band separate from the visible light band.

[0262] When the filter unit (240) passes a wavelength band less than 800 nm, the visible light irradiated on the affected area (W) is included in the reflected light (LR), so there is a risk that the visible light may affect the third light (L3). That is, noise may be included in the third light (L3) collected by the light receiving unit (230), so noise may be generated in the image information acquired by the photographing unit (200).

[0263] On the other hand, when the filter unit (240) passes a wavelength band of 800 nm or more, the influence of visible light irradiated on the affected area (W) on the third light (L3) incident on the light receiving unit (230) can be minimized. That is, the photographing unit (200) can obtain accurate image information on the affected area (W) by collecting the third light (L3) in a wavelength band that does not overlap with visible light.

[0264] More preferably, the third wavelength band may be a wavelength band of 830 nm to 850 nm. As described above, light having a wavelength band of 830 nm to 850 nm may have a predetermined transmittance to skin, muscles, etc. That is, the photographing unit (200) can collect the third light (L3) of the third wavelength band to effectively obtain image information about the inside of the tissue of the affected area.

[0265] In addition, since bones can have a high reflectivity for light having a wavelength of 830 nm or more and 850 nm or less, the photographing unit (200) can collect light reflected from the outer surface of the bone with a high reflectivity. Thus, the photographing unit (200) can obtain accurate image information about the outer surface of the bone.

[0266] Figure 7 is image information of a wound captured in different wavelength bands using a photographing unit.

[0267] Figure 7(a) is image information obtained by photographing the affected area (W) using light in the visible light band, and Figure 7(b) is image information obtained by photographing the affected area (W) using light in the near-infrared band.

[0268] In general, the lighting unit (300) of the operating room uses a visible light source to secure the operator's field of vision. Therefore, when photographing the affected area (W) using light in the visible light band, interference between the emitted light (LE) and the second light (L2) may occur, resulting in a phenomenon in which the reflected light (LR) is strengthened. In other words, when photographing the affected area (W) using light in the visible light band, noise may occur in the image information, as shown in Fig. 5(a).

[0269] Meanwhile, when photographing the affected area (W) using light in the infrared band, interference between the emitted light (LE) and the second light (L2) can be minimized. In other words, interference between the emitted light (LE) and the second light (L2) can be prevented, thereby eliminating the phenomenon of the reflected light (LR) being enhanced. Thus, when photographing the affected area (W) using light in the infrared band, accurate image information, such as that shown in Fig. 5(b), can be obtained.

[0270] FIG. 8 is a drawing that briefly illustrates image information acquired from the photographing unit of FIG. 6, and FIG. 9 is a drawing that illustrates that the position of the photographing unit (200) of FIG. 6 has been partially changed.

[0271] Hereinafter, the 'light pattern' may be defined as a light pattern formed when the emitted light (LE) is irradiated onto the affected area (W). In addition, hereafter, the first reflected light (LR) may be defined as light propagated toward the first light-receiving portion (230a) among the reflected lights (LR), and the second reflected light (LR) may be defined as light propagated toward the second light-receiving portion (230b) among the reflected lights (LR).

[0272] Hereinafter, the 'first position' may be defined as the position of the photographing unit (200) aligned on the first axis (AX1) of FIGS. 4 and 7, and the 'second position' may be defined as the position of the photographing unit (200) aligned on the first axis (AX1') of FIG. 7.

[0273] Referring to FIGS. 6, 8, and 9, the light emitting unit (220) can irradiate light (LE) having a predetermined pattern to the affected area (W). That is, a light pattern (Lp) is projected onto the affected area (W), and the reflected light (LR) transmitted to the light receiving unit (230) can include information about the light pattern (Lp).

[0274] Referring to FIG. 6, the intensities of the first reflected light (LRa) and the second reflected light (LRb) transmitted to the first light receiving unit (230a) and the second light receiving unit (230b) may be different depending on the angle at which the photographing unit (200) photographs the affected area (W).

[0275] In one embodiment, when the photographing unit (200) is at the first position, the amount of light of the second reflected light (LRb`) reflected from the second affected area (W2) may be greater than the amount of light of the first reflected light (LRa) reflected from the first affected area (W1) depending on the characteristics of the affected area (W) or the environment of the surgical space.

[0276] When the light quantity of the third b light (L3b) filtered by the second reflected light (LRb) in the filter unit (240) exceeds a preset light quantity reference value, a light saturation area (SA) may be generated in the image information acquired by the light receiving unit (230). Again, when the second light quantity collected by the second light receiving unit (230b) exceeds a preset light quantity, a light saturation area (SA) may be generated in the image information acquired by the light receiving unit (230). At this time, the control unit (500) may have difficulty acquiring accurate scan data for the affected area (W) from the image information.

[0277] Referring to FIG. 9, the position and posture of the photographing unit (200) obtained from the photographing unit (200) may be changed. The position and posture of the photographing unit (200) may be changed so that the light saturation area (SA) on the acquired image information is minimized.

[0278] The control unit (500) can change the position of the photographing unit (200) using the first light reception amount detected by the first light reception unit (230a) and the second light reception amount detected by the second light reception unit (230b). The control unit (500) can adjust the position of the photographing unit (200) when the first light reception amount and / or the second light reception amount exceeds a preset reference value.

[0279] The control unit (500) can adjust the position of the photographing unit (200) in a direction in which the first light reception amount and the second light reception amount have corresponding values. For example, the control unit (500) can change the photographing unit (200) from the first position to the second position.

[0280] In one embodiment, the control unit (500) may move the photographing unit (200) in a direction in which the first light reception amount and the second light reception amount are each detected to be below a preset reference value. In turn, the control unit (500) may determine a position in which the first light reception amount and the second light reception amount are each detected to be below a preset reference value as a second position, and move the photographing unit (200) to the second position. Thus, the image saturation phenomenon due to excessive light entering the first light reception unit (230a) and the second light reception unit (230b) at the second position may be prevented.

[0281] In another embodiment, the control unit (500) can move the photographing unit (200) in a direction in which the ratio of the first light reception amount and the second light reception amount is within a preset range. The control unit (500) can determine a position in which the ratio of the first light reception amount and the second light reception amount is within a preset range as a second position, and move the photographing unit (200) to the second position. That is, alignment between the images acquired from the first light reception unit (230a) and the second light reception unit (230b) at the second position becomes easier, so that the accuracy of the image information acquired from the photographing unit (200) can be improved.

[0282] Meanwhile, the control unit (500) can adjust the position of the photographing unit (200) so that the light saturated area (SA) on the image information has a size that does not affect the generation of scan data. For example, the control unit (500) can calculate the area value of the light saturated area (SA) on the image information and move the photographing unit (200) in a direction in which the area value of the light saturated area (SA) decreases. The control unit (500) can determine a position where the area value of the light saturated area (SA) is less than a preset value as a second position and move the photographing unit (200) to the second position.

[0283] The control unit (500) can calculate the area value of the light saturated area (SA) based on the number of pixels in the image information, but the method for calculating the area value of the light saturated area (SA) is not limited to this.

[0284] The control unit (500) can control the operation of the robot arm (100) to change the position and posture of the photographing unit (200). For example, the control unit (500) can determine the second position of the photographing unit (200) and control the robot arm (100) so that the photographing unit (200) is placed at the second position.

[0285] FIG. 10 is a control timing graph for some configurations of a surgical robot system according to one embodiment of the present invention.

[0286] Fig. 10(a) is a graph of a clock signal (SG) of a control unit (500) over time, Fig. 10(b) is a graph of a change in the amount of light of a lighting unit (300) over time, and Fig. 10(c) is a control graph of a photographing unit (200) over time.

[0287] Referring to FIG. 10, the control unit (500) can generate a clock signal (SG). The control unit (500) can control the photographing unit (200) and the lighting unit (300) based on the clock signal (SG). The clock signal (SG) can be a reference signal for the control unit (500) to control the photographing unit (200) and the lighting unit (300). For example, the clock signal (SG) can be a pulse signal with a preset cycle.

[0288] The photographing unit (200) can repeatedly acquire image information according to a preset first cycle (T1). The control unit (500) can control the photographing unit (200) so that the photographing unit (200) repeatedly acquires image information according to the first cycle (T1).

[0289] The photographing unit (200) can photograph the affected area (W) for a photographing time (TS) every first cycle (T1). In turn, the photographing unit (200) can obtain image information including the affected area (W) for a photographing time (TS) every first cycle (T1).

[0290] In one embodiment, the shooting time (TS) may be the time during which the shooting unit (200) irradiates light to the affected area (W) using the light emitting unit (220) to obtain image information including the affected area (W). The shooting time (TS) may be divided into a start time (Ti) required for the output light (LE) to return to a normal light quantity, an effective shooting time (Tfilm) during which the output light (LE) is maintained at a normal light quantity, and an end time (To) required for the light quantity of the output light (LE) to return to zero.

[0291] The photographing unit (200) can obtain image information by collecting reflected light (LR) from the light receiving unit (230) during the photographing time (TS). The photographing unit (200) can obtain image information while the emission light (LE) is emitted, and can obtain image information from the reflected light (LR) that includes the emission light (LE).

[0292] In one embodiment, the photographing unit (200) can acquire image information by collecting reflected light (LR) from the light receiving unit (230) during the effective photographing time (Tfilm). The photographing unit (200) can collect reflected light (LR) while emitting light (LE) having a sufficient amount of light is emitted. Thus, the quality of image information acquired from the photographing unit (200) can be improved.

[0293] The lighting unit (300) can emit the second light (L2) according to a preset second cycle (T2). The control unit (500) can control the lighting unit (300) so that the lighting unit (300) repeatedly emits the second light (L2) according to the second cycle (T2).

[0294] The second cycle (T2) of the lighting unit (300) can be divided into an on time (Ton) during which the lighting unit (300) is driven to emit the second light (L2), and an off time (Toff) during which the lighting unit (300) is driven to stop emitting the second light (L2). While the lighting unit (300) is operating, the on time and the off time can be alternately repeated. The on time (Ton) and the off time (Toff) can be divided based on the clock signal (SG).

[0295] On time (Ton) can be divided into return time (Tr) and main on time (Tmon).

[0296] The return time (Tr) may be the time required for the third light (L3) emitted by the lighting unit (300) to return to normal light quantity. The lighting unit (300) may be controlled by the control unit (500) to be turned on at the start of the return time (Tr).

[0297] The main on time (Tmon) may be the time during which the light quantity of the second light (L2) emitted from the lighting unit (300) is maintained at a preset value or higher. The main on time (Tmon) may be sufficient to ensure that the surgery is not interrupted.

[0298] Off time (Toff) can be divided into reserve time (Tp) and main off time (Tmoff).

[0299] The standby time (Tp) may be the time required for the light quantity of the second light (L2) to return to zero, and the main off time (Tmoff) may be the time required for the light quantity of the second light (L2) to remain zero.

[0300] When the lighting unit (300) is an LED, the return time (Tr) and the reserve time (Tp) can be determined by considering the output characteristics (pulse rise-time and pulse fall-time) that depend on the LED current when a predetermined power is applied to the LED. At this time, the return time (Tr) and the reserve time (Tp) of the lighting unit (300) can each be 40 ns to 100 ns.

[0301] Additionally, the main on time (Tmon) of the lighting unit (300) may be within the range of 200 ns to 4.9 sec.

[0302] The main on time (Tmon) may be equal to or longer than the main off time (Tmoff). In other words, the time during which the second light (L2) is irradiated to the affected area (W) may be longer than the time during which the second light (L2) is not irradiated to the affected area (W). Thus, the surgeon (ST) can continuously observe the affected area (W) in an environment where the lighting is maintained stably.

[0303] The control unit (500) can adjust the operation timing of the shooting unit (200) and the lighting unit (300) so that the light (LE) emitted from the shooting unit (200) and the second light (L2) emitted from the lighting unit (300) are alternately emitted to the target area (W).

[0304] For example, the first cycle (T1) and the second cycle (T2) may correspond to each other. The first cycle (T1) and the second cycle (T2) may have substantially the same value. In other words, the operating cycle of the photographing unit (200) and the operating cycle of the lighting unit (300) may be synchronized with each other, so that the photographing unit (200) and the lighting unit (300) may operate alternately.

[0305] The control unit (500) can control at least one of the photographing unit (200) and the lighting unit (300) so that the lighting unit (300) is turned off during the time that the photographing unit (200) acquires image information. The photographing unit (200) can acquire image information by collecting reflected light (LR) during the time that the lighting unit (300) is driven to stop emitting the second light (L2).

[0306] In one embodiment, the effective shooting time (Tfilm) may proceed during the off time (Toff). The effective shooting time (Tfilm) may start and end during the off time (Toff). The shooting unit (200) may collect reflected light (LR) to obtain image information while the lighting unit (300) operates to stop emitting the second light (L2). Thus, the shooting unit (200) may collect reflected light (LR) in a state where the influence of the second light (L2) is minimized to obtain image information.

[0307] The effective shooting time (Tfilm) can be performed during the main off time (Tmoff). The effective shooting time (Tfilm) can be started and ended during the main off time (Tmoff). The shooting unit (200) can obtain image information by collecting reflected light (LR) while the amount of light of the second light (L2) emitted from the lighting unit (300) remains zero. In this case, the influence of the second light (L2) on the reflected light (LR) incident on the shooting unit (200) can be excluded. Thus, image information can be obtained in a state where interference by the second light (L2) is excluded.

[0308] The start time (Ti) and the return time (Tr) may overlap each other. The start time (Ti) may start during the return time (Tr) and end during the main off time (Tmoff). That is, the photographing unit (200) may start emitting the output light (LE) while the light quantity of the second light (L2) in the lighting unit (300) is decreasing. Thus, the output light (LE) may have a normal light quantity immediately after the main off time (Tmoff) begins.

[0309] The end time (To) and the standby time (Tp) may overlap each other. Specifically, the end time (To) may start during the main off time (Tmoff) and end during the standby time (Tp). The photographing unit (200) may stop emitting the output light (LE) while the light quantity of the second light (L2) in the lighting unit (300) returns to the normal light quantity. Thus, the output light (LE) may have the normal light quantity until just before the standby time (Tp) starts.

[0310] The start time (Ti) and the end time (To) can overlap with the return time (Tr) and the reserve time (Tp), respectively, to advance the start time of the effective shooting time (Tfilm) and delay the end time of the effective shooting time (Tfilm). Thus, the effective shooting time (Tfilm) can be secured, and the accuracy of the image information acquired from the shooting unit (200) can be improved.

[0311] Figure 11 is a drawing showing scan data acquired from the control unit.

[0312] Referring to FIG. 11, the control unit (500) can obtain scan data for the affected area (W) from image information obtained from the photographing unit (200).

[0313] As described above, the control unit (500) can obtain scan data, which is three-dimensional information about the affected area (W), based on the image information obtained from the photographing unit (200). For example, the control unit (500) can recognize the shape of the affected area (W) on the image information and obtain scan data, which is a set of three-dimensional coordinate values ​​for each point of the affected area (W).

[0314] The control unit (500) can obtain a 3D point (WD) of a lesion (W) designated by three-dimensional information on the image information. The control unit (500) can generate a lesion point (WD) for all points of the lesion (W) recognized on the image information.

[0315] The control unit (500) can generate scan data including 3D point cloud information generated using each of the affected area points (WD). The control unit (500) can obtain shape and location information of the affected area (W) using the 3D point cloud information for the affected area (W).

[0316] For example, the control unit (500) can extract detailed shape information about the shape of the affected area (W), such as the curvature, height, and boundary of the affected area (W), by using 3D point cloud information about the affected area (W). Thus, the control unit (500) can obtain basic information for generating a work path of the robot arm (100).

[0317] A wound marker (WM) may be further included in the vicinity of the wound (W) to assist in acquiring scan data. When the wound marker (WM) is included in the image information, the control unit (500) may provide wound marker data, which is three-dimensional information about the wound marker (WM).

[0318] In one embodiment, the affected area marker (WM) may be configured to have a QR (Quick Response) code, Aruco marker, reflective marker, reflective sticker, or barcode attached that is visually recognizable by an optical camera. However, the present invention is not limited thereto, and the affected area marker (WM) may be configured to have a marker with fluorescent properties attached, or may not be provided as a separate component but may be a pattern drawn or engraved on the affected area (W). Alternatively, the affected area marker (WM) may not have a separate marker attached and may have a preset shape so that the shape can be recognized by the control unit (500). However, for the convenience of explanation, the following description will focus on an embodiment in which the affected area marker (WM) is provided as a separate component whose shape can be recognized by the photographing unit (200).

[0319] The refund marker data for the refund marker (WM) can be obtained by providing image information including the refund marker (WM) to the control unit (500).

[0320] The WM may be captured by a separate WM recognition camera or scanner (hereinafter, “WM camera”), and image information including the WM marker may be provided to the control unit (500). The WM marker camera may be built into the photographing unit (200) or may be provided as a separate component from the photographing unit (200).

[0321] When the camera for the lesion marker is positioned adjacent to the photographing unit (200), the camera for the lesion marker acquires image information about the lesion marker (WM) at a location adjacent to the lesion marker (WM), thereby improving the accuracy of the lesion marker data. On the other hand, when the camera for the lesion marker is positioned away from the photographing unit (200), the lesion marker (WM) can be prevented from leaving the field of view of the camera for the lesion marker.

[0322] However, for the convenience of explanation, the following description will focus on an embodiment in which a camera or scanner for recognizing a lesion marker (WM) is formed integrally with a photographing unit (200), and scan data and lesion marker data are acquired simultaneously when the image information acquired from the photographing unit (200) includes both the lesion (W) and the lesion marker (WM).

[0323] The affected area marker (WM) can be positioned to reflect the movement of the affected area (W). Although FIGS. 11 to 13 illustrate that the affected area marker (WM) is attached to the affected area (W), the position of the affected area marker (WM) is not limited thereto. For example, the affected area marker (WM) can be attached to another part of the patient (P) where the movement of the affected area (W) can be reflected, such as adjacent skin, bone, or a fixable anatomical part. However, for convenience of explanation, the following description will focus on an embodiment in which the affected area marker (WM) is attached to one side of the affected area (W).

[0324] The control unit (500) can obtain the 3D information of the lesion marker (WM), which is the lesion marker data, when the image information includes the lesion marker (WM). When the lesion (W) and the lesion marker (WM) are included simultaneously in the image, the control unit (500) can obtain the scan data and the lesion marker data simultaneously.

[0325] The control unit (500) can obtain marker points (WMD), which are three-dimensional information about the lesion marker (WM). For example, the control unit (500) can generate marker points (WMD) for all points of the lesion marker (WM) recognized on the image information.

[0326] The control unit (500) can generate target marker data including 3D point cloud information generated using each marker point (WMD). The control unit (500) can extract specific information regarding the shape of the target marker (WM) using the 3D point cloud information regarding the target marker (WM).

[0327] The control unit (500) can improve the accuracy of the work path for the robot arm (100) by using the return marker data.

[0328] In one embodiment, the control unit (500) may correct the matching result using the affected area marker data. For example, the control unit (500) may match the affected area marker data with a three-dimensional model of the affected area marker (WM) obtained externally to generate a matching result for the affected area marker (WM), and may use this to correct the matching result obtained by matching modeling data and scan data.

[0329] However, the specific method for correcting the alignment result using the return marker data is not limited to the above-described method. For example, the control unit (500) may improve the accuracy of the alignment result by correcting the scan data using the return marker data. Thus, the precision of the work path of the robot arm (100) generated by the control unit (500) can be improved.

[0330] Figure 12 is a diagram showing scan data before and after a change in the posture of the affected area. Figure 12(a) is a diagram showing the first scan data of the affected area acquired at a first time point, and Figure 12(b) is a diagram showing the second scan data of the affected area acquired at a second time point.

[0331] Referring to Fig. 12, the position of the affected area (W) may be changed during surgery. The photographing unit (200) may repeatedly acquire image information by photographing the affected area (W) according to a first cycle. The photographing unit (200) may acquire first image information about the affected area (W) at a first time point, and second image information about the affected area (W) at a second time point. At this time, the time interval between the first time point and the second time point may correspond to the first cycle.

[0332] The control unit (500) can correct the working path of the robot arm (100) using the image information acquired for each first cycle. In detail, the control unit (500) can repeatedly acquire scan data for each first cycle and update the working path of the robot arm (100) based on the newly acquired scan data.

[0333] Hereinafter, 'image information' may be referred to as first image information and second image information, or an element encompassing the first image information and the second image information, and 'scan data' may be referred to as first scan data and second scan data, or an element encompassing the first scan data and the second scan data. In addition, 'revenue marker data' may be referred to as first return marker data and second return marker data, or an element encompassing the first return marker data and the second return marker data.

[0334] The control unit (500) can update the first scan data acquired at the first point in time with the second scan data acquired at the second point in time. The control unit (500) can acquire the first return point (WD1) for the return area (W) at the first point in time from the first image information, and can acquire the second return point (WD2) for the return area (W) at the second point in time from the second image information. The control unit (500) can update the scan data by updating the first return point (WD1) for the return area (W) at the first point in time to the second return point (WD2) for the return area (W) at the second point in time.

[0335] The control unit (500) can generate a working path of the robot arm (100) based on second scan data including information about the second return point (WD2). The control unit (500) can update the working path of the first point in time with the newly generated working path of the second point in time. Thus, the control unit (500) can update the working path of the robot arm (100) according to the movement of the return point (W).

[0336] The control unit (500) can correct scan data using a return marker (WM). Specifically, the control unit (500) can obtain return marker data for the return marker (WM) from image information and update the scan data.

[0337] The control unit (500) can update the first marker data acquired at the first time point with the second marker data acquired at the second time point. The control unit (500) can update the marker data by updating the first marker point (WMD1) for the marker (WM) at the first time point with the second marker point (WMD2) for the marker (WM) at the second time point.

[0338] The control unit (500) can improve the accuracy of the working path of the robot arm (100) by using the updated second return marker data.

[0339] In one embodiment, the control unit (500) may correct the matching result using the second affected area marker data. For example, the control unit (500) may match a three-dimensional model of the affected area marker (WM) obtained externally with the second affected area marker data to generate a matching result for the affected area marker (WM), and may use this to correct the matching result obtained by matching modeling data and scan data.

[0340] Figure 13 is a diagram illustrating a process for acquiring scan data after a change in the posture of a wound. Figure 13(a) is a diagram illustrating scan data of a wound acquired at a first time point, Figure 13(b) is a diagram illustrating second wound marker data of a wound marker acquired at a second time point, and Figure 13(c) is a diagram illustrating second wound marker data and second scan data at a second time point.

[0341] Referring to FIG. 13, the control unit (500) can obtain second scan data using the return marker data for the return marker (WM). The control unit (500) can estimate the second scan data based on the first return marker data and the second return marker data.

[0342] Referring to FIG. 13(a), the control unit (500) can obtain a first point (WD1) for the affected area (W) and a first marker point (WMD1) for the affected area marker (WM) from the first image information acquired at the first point in time. The control unit (500) can generate first scan data using the first point (WD1) and generate first marker data using the first marker point (WMD1).

[0343] Referring to FIG. 13(b), the control unit (500) can obtain a second marker point (WMD2) for the affected area marker (WM) from the second image information acquired at the second point in time. The control unit (500) can generate second affected area marker data using the second marker point (WMD2).

[0344] Referring to FIG. 13(c), the control unit (500) can obtain a second return point (WD2) using the return marker data. The control unit (500) can compare the first return marker data and the second return marker data to calculate a relationship between the first scan data and the second scan data.

[0345] In one embodiment, the control unit (500) may compare the first marker point (WMD1) and the second marker point (WMD2) to calculate transformation information between the first marker point (WMD1) and the second marker point (WMD2). For example, the control unit (500) may calculate a transformation matrix using the first marker point (WMD1) and the second marker point (WMD2).

[0346] The control unit (500) can obtain second scan data using the transformation information calculated using the first and second return marker data and the first scan data. For example, the control unit (500) can estimate the second return point (WD2') by applying a transformation matrix to the first return points (WD1). The control unit (500) can generate the second scan data based on the set of second return points (WD2') generated by applying the transformation matrix.

[0347] That is, if there is a change in the posture of the affected area (W) over time, the control unit (500) can estimate the change in posture of the affected area (W) based on the affected area marker data without directly acquiring scan data for the affected area (W). The control unit (500) can track the change in position and posture of the affected area marker (WM) over time, and predict the change in position of the affected area points (WD) linked to the affected area marker (WM).

[0348] The control unit (500) can quickly reflect real-time changes in the affected area (W) and update the scan data for the affected area (W) by using only information on a relatively small number of marker points without repeatedly acquiring scan data for the affected area (W). Thus, the control unit (500) can shorten the scan data acquisition time and improve the response speed of the surgical robot system (1).

[0349] FIG. 14 is a drawing illustrating a measuring unit according to one embodiment of the present invention.

[0350] Referring to FIG. 14, a measuring unit (400) according to one embodiment of the present invention may include a probe (PR), a case (410), a post (420), a support (430), a guide cover (440), and a measuring marker (MM).

[0351] A probe (PR) is inserted into one side of the human body to measure internal information of the tissue, such as the depth and thickness of cartilage (C). The probe (PR) can be designed to have a predetermined length and diameter so that it can be inserted while minimizing damage to the cartilage (C).

[0352] The case (410) may form the outer shape of the measuring unit (400). The case (410) may provide an internal space in which the probe (PR) and each component supporting the probe (PR) are accommodated. The case (410) may allow the operator (ST) to hold the measuring unit (400) to control the insertion direction of the probe (PR) or to control the measuring position (MP) described below.

[0353] The post (420) is positioned on one side of the case (410) and can extend in one direction. The post (420) can be inserted into a guide hole (441) formed in a guide cover (440) to be described later, and can guide the movement direction of the guide cover (440). That is, the post (420) can support the posture and movement direction of the probe (PR) connected to the guide cover (440).

[0354] In one embodiment, a plurality of posts (420) may be provided. The plurality of posts (420) may guide the direction of movement of the guide cover (440) and support the support member (430). The plurality of posts (420) may firmly support the position of the case (410) relative to the support member (430) that is in contact with and supported by the return part (W).

[0355] Although FIG. 14 illustrates that the measuring unit (400) includes four posts (420), the number of posts (420) is not limited thereto.

[0356] The support member (430) can be connected to one end of the post (420). The support member (430) can be in contact with the affected area (W) with the other side opposite to the side connected to the post (420). The support member (430) can be in contact with and supported by the affected area (W) to support the position and posture of the measuring unit (400) with respect to the affected area (W). Thus, the support member (430) can stably fix the direction and position in which the probe (PR) is inserted into the affected area (W).

[0357] An insertion hole (431) may be formed in the support member (430). The insertion hole (431) may be formed at a position corresponding to the probe (PR). The probe (PR) may pass through the insertion hole (431) and be inserted into the affected area (W).

[0358] The guide cover (440) can cover one side of the case (410). The guide cover (440) is arranged to surround the periphery of the case (410) and can slide relative to the outer surface of the case (410).

[0359] A guide hole (441) may be formed in the guide cover (440). The guide hole (441) may have a position corresponding to the post (420) and a shape corresponding to the cross-section of the post (420). That is, the post (420) may be inserted into the inside of the guide hole (441), and the inner surface of the guide hole (441) may be in contact with the post (420). Thus, the guide cover (440) may be supported on the post (420) through the guide hole (441).

[0360] In other words, the direction of movement of the guide cover (440) can be determined by the post (420). In one embodiment, the post (420) can be arranged in a direction parallel to the probe (PR). At this time, the post (420) can guide the direction of movement of the guide cover (440) so that the guide cover (440) moves along the direction in which the probe (PR) extends. Thus, the direction in which the probe (PR) is inserted into the affected area (W) can be aligned, and errors in measuring the thickness of the cartilage (C) can be minimized.

[0361] A measurement marker (MM) may be mounted on one side of the case (410). The measurement marker (MM) may be measured by the photographing unit (200) as described below. The measurement marker (MM) may be measured by the photographing unit (200) so that information about the position where the measurement unit (400) is placed and the posture of the measurement unit (400) may be included in the image information. Thus, the control unit (500) may obtain measurement position data, which is information about the measurement position (MP) on the affected area (W) into which the probe (PR) is inserted, from the image information.

[0362] In one embodiment, the measurement marker (MM) may be configured to have a QR (Quick Response) code, Aruco marker, or barcode attached that can be visually recognized by an optical camera. However, the present invention is not limited thereto, and the measurement marker (MM) may also have a preset shape without a separate marker drawn thereon, enabling shape recognition by the control unit (500).

[0363] Figures 15 and 16 are drawings illustrating a process of measuring the cartilage thickness of a wound using the measuring unit of Figure 14.

[0364] Referring to FIGS. 15 and 16, a measuring unit (400) according to one embodiment of the present invention can measure the insertion depth using the depth at which the probe (PR) is inserted into the affected area (W).

[0365] In one embodiment, the measuring unit (400) may further include a guide block (450), an elastic member (460), and a detection sensor (470) disposed inside the case (410).

[0366] The guide block (450) can be connected to the probe (PR). One end of the guide block (450) can be connected to the guide cover (440) and the other end can be connected to the probe (PR). The guide block (450) can connect the guide cover (440) and the probe (PR) to each other, so that the probe (PR) can move along the guide cover (440).

[0367] The guide block (450) can be connected to a guide slot (411) formed inside the case (410) so that the direction of movement can be guided. For example, the guide block (450) can be inserted into the guide slot (411) and slide. The probe (PR) can be connected to the guide block (450) so that it can move in parallel with the direction of movement of the guide block (450). Thus, the insertion direction of the probe (PR) can be stably aligned.

[0368] An elastic member (460) may be placed between the probe (PR) and the case (410). The elastic member (460) may connect the guide block (450) connected to the probe (PR) and the inside of the case (410). The elastic member (460) may apply an elastic restoring force to the probe (PR) inserted into the affected area (W). That is, the probe (PR) inserted into the affected area (W) by moving forward may return to its initial position by the elastic member (460).

[0369] The probe (PR) can be advanced from the measurement position (MP) and inserted into the affected area (W). The probe (PR) passes through the cartilage (C) arranged on the outer side of the affected area (W), but its advancement may be restricted when it reaches the bone (B). The probe (PR) can be inserted into the affected area (W) by moving from the outer surface of the cartilage (C) to the outer surface of the bone (B). That is, the movement distance of the probe (PR) can correspond to the thickness of the cartilage (C). As described below, the control unit (500) can obtain information on the thickness of the cartilage (C) by using the insertion depth at which the probe (PR) is inserted into the affected area (W).

[0370] The detection sensor (470) can detect the insertion depth at which the probe (PR) is inserted into the affected area (W). For example, the detection sensor (470) can measure the insertion depth by detecting the distance the probe (PR) moves.

[0371] The detection sensor (470) is electrically connected to the control unit (500) and can transmit information to the control unit (500). The detection sensor (470) can transmit information on the measured insertion depth to the control unit (500). Thus, the detection sensor (470) can transmit information for forming cartilage data to the control unit (500).

[0372] The photographing unit (200) can photograph a measurement marker (MM) arranged on one side of the case (410). The photographing unit (200) can irradiate light (LE) to the measurement marker (MM) and collect reflected light (LR) reflected from the measurement marker (MM). For example, the photographing unit (200) can include a light emitting unit (220) that irradiates first light (L1) of a first wavelength band, a light receiving unit (230) that collects reflected light (LR) to obtain image information, and a filter unit (240) that filters the first light (L1) and reflected light (LR) into a third wavelength band. Thus, the photographing unit (200) can obtain image information about the measurement marker (MM).

[0373] The control unit (500) can obtain image information including a measurement marker (MM) from the photographing unit (200) to obtain three-dimensional information about a measurement position (MP) on a wound (W) into which a probe (PR) is inserted.

[0374] In one embodiment, the control unit (500) may compare the coordinate system in which shape information about the measuring unit (400) is acquired with the coordinate system containing scan data to align the respective coordinate systems. For example, the control unit (500) may compare the coordinate system of image information in which shape information about the measuring unit (400) is acquired with the coordinate system containing scan data to derive a transformation matrix, and may align the respective coordinate systems using the transformation matrix. Thus, as described below, when cartilage data is acquired by comparing shape information and insertion depth about the measuring unit (400) with scan data, the accuracy of the cartilage data, which is a vector value, may be improved.

[0375] The control unit (500) can obtain shape information about the measuring unit (400). For example, the control unit (500) can obtain shape information about the measuring unit (400) by obtaining three-dimensional cloud information about the measuring unit (400) from image information including the measuring unit (400). Alternatively, the control unit (500) can obtain shape information about the measuring unit (400) by using CAD information about the measuring unit (400) that has been previously input.

[0376] The control unit (500) can estimate the position and attitude of the measurement unit (400) using the shape information of the measurement unit (400) and information obtained from the measurement marker (MM). The control unit (500) can generate measurement position data including three-dimensional information about the measurement position (MP) and the insertion direction in which the probe (PR) is inserted using the shape information of the measurement unit (400) and the information about the position and attitude of the measurement unit (400).

[0377] The control unit (500) can generate cartilage data by acquiring measurement position data and an insertion depth corresponding to the measurement position (MP). In detail, the control unit (500) can compare the measurement position data and the insertion depth corresponding to the measurement position (MP) with scan data to acquire cartilage data representing the shape of the cartilage at the measurement position (MP) as a vector value.

[0378] The control unit (500) can correct the scan data using cartilage data. The control unit (500) can correct the scan data by applying the cartilage data to the scan data. In one embodiment, the control unit (500) can apply each vector value of the cartilage data to the 3D point cloud for the affected area (W) forming the scan data, so that the scan data includes a 3D point cloud for the bone excluding the cartilage (C).

[0379] The control unit (500) can improve the precision of surgery using the robot arm (100) by generating a working path of the robot arm (100) based on a 3D point cloud for the bone.

[0380] FIG. 17 is a drawing illustrating a process of measuring the cartilage thickness of a wound using a measuring unit according to another embodiment of the present invention.

[0381] The measuring unit (400`) according to another embodiment of the present invention has only some differences in the configuration and method for acquiring cartilage data compared to the measuring unit (400) according to one embodiment of the present invention, and is the same as the measuring unit (400`) according to one embodiment of the present invention in the method for acquiring shape data of the measuring unit (400`) or correcting scan data using cartilage data in the control unit (500). Therefore, the following redundant description will be omitted and the differences will be mainly described.

[0382] Referring to FIG. 17, one side of a measuring unit (400`) according to another embodiment of the present invention can be inserted into a return portion (W).

[0383] In one embodiment, the measuring unit (400`) can advance with the probe (PR`) inserted into the cartilage (C). The advancement of the measuring unit (400`) can be limited when the probe (PR) comes into contact with the bone (B). That is, the measuring unit (400`) can move by the thickness of the cartilage (C). The movement distance (D) of the measuring unit (400`) can correspond to the thickness of the cartilage (C).

[0384] The photographing unit (200) can obtain image information by photographing the measuring unit (400). The photographing unit (200) can obtain image information about the measuring unit (400`) just before being inserted into the affected area (W) and the measuring unit (400`) in a state where the probe (PR`) is in contact with the bone (B) and its forward movement is restricted.

[0385] The control unit (500) can obtain information about the movement of the measuring unit (400`) using image information about the measuring unit (400`). The control unit (500) can obtain image information about the measuring unit (400`) just before being inserted into the return part (W) and image information about the measuring unit (400`) in a state where its forward movement is restricted due to contact with the goal (B).

[0386] The control unit (500) can obtain measurement position data for a measurement position (MP`) where a probe (PR) is inserted on a target area (W) from each image information. The control unit (500) can obtain measurement position data for a measurement position (MP`) by recognizing the shape of a measurement unit (400) included in the image information or recognizing a measurement marker (MM`) to estimate the position of the probe (PR`).

[0387] The control unit (500) can obtain the insertion depth at which the measuring unit (400`) is inserted into the affected area (W) at the measuring position (MP`) from each image information.

[0388] In one embodiment, the control unit (500) can recognize the measuring unit (400`) from each image information and obtain three-dimensional information about the measuring unit (400`). The control unit (500) can estimate the movement distance (D) moved by the measuring unit (400`) using the three-dimensional information about the measuring unit (400`) extracted from each image information.

[0389] In another embodiment, the control unit (500) can recognize a measurement marker (MM`) in each image information and obtain marker data, which is three-dimensional information about the measurement marker (MM`). The control unit (500) can estimate the movement distance (D) of the measurement unit (400`) from the marker data by utilizing the spatial relationship between the marker data and the measurement unit (400`).

[0390] The control unit (500) can obtain the insertion depth at which the measuring unit (400`) is inserted into the affected area (W) using the movement distance (D). The control unit (500) can generate cartilage data using the thickness of the cartilage (C) obtained from the measurement position (MP`) and the insertion depth at the measurement position (MP`), and can correct the scan data using the cartilage data.

[0391] The measuring unit (400``) according to another embodiment of the present invention has only some differences in the configuration and method for acquiring cartilage data compared to the measuring unit (400) according to one embodiment of the present invention, and is the same as the measuring unit (400) according to one embodiment of the present invention in the method of correcting scan data using cartilage data in the control unit (500), so the following redundant description will be omitted and the differences will be mainly described.

[0392] FIG. 18 is a drawing illustrating a process of measuring the cartilage thickness of a wound using a measuring unit according to another embodiment of the present invention.

[0393] Referring to FIG. 18, a measuring unit (400``) according to another embodiment of the present invention may include a probe (PR``) and a sub robot arm (SR``).

[0394] The probe (PR``) can be mounted on the end of the sub-robot arm (SR``). The position and posture of the probe (PR``) can be adjusted according to the movement of the sub-robot arm (SR``). For example, the probe (PR``) can be inserted into the affected area (W) by the movement of the sub-robot arm (SR``).

[0395] The sub-robot arm (SR``) may have at least one joint and link. Although the drawing shows that each sub-robot arm (SR``) includes two joints, the number of joints and links included in the sub-robot arm (SR``) is not limited thereto. However, for convenience of explanation, the following description will focus on an embodiment in which the sub-robot arm (SR``) includes a first joint (J1``), a second joint (J2``), a first link (LN1``), and a second link (LN2``).

[0396] A probe (PR) may be connected to one end of the first link (LN1``), and a first joint (J1``) may be connected to the other end. The first link (LN1``) may rotate about the first joint (J1``) as a rotation center to adjust the position and posture of the probe.

[0397] A first joint (J1``) may be connected to one end of the second link (LN2``), and a second joint (J2``) may be connected to the other end. The second link (LN2``) may rotate about the second joint (J2``) as a rotation center to adjust the position and posture of the first link (LN1``).

[0398] That is, the sub-robot arm (SR``) can adjust the position and posture of the first link (LN1``) and the second link (LN2``) by rotating the first joint (J1``) and the second joint (J2``). The sub-robot arm (SR``) can adjust the position and posture of the probe (PR``) mounted on one end of the first link (LN1``) by adjusting the rotation angle of the first joint (J1``) and the second joint (J2``).

[0399] The sub-robot arm (SR``) can be electrically connected to the control unit (500). For example, the sub-robot arm (SR``) can transmit information about the rotation speed or rotation angle of the first joint (J1``) and the second joint (J2``) to the control unit (500). Thus, the sub-robot arm (SR``) can transmit to the control unit (500) a basis for calculating information about the position or movement of the probe (PR).

[0400] The insertion depth at which the probe (PR``) is inserted into the affected area (W) can be obtained by detecting the motions of the first joint (J1``) and the second joint (J2``). In turn, the insertion depth can be obtained by detecting the motions of the first joint (J1``) and the second joint (J2``) and calculating the movement distance (D) of the probe (PR).

[0401] In one embodiment, the control unit (500) can receive information about the motions of the first joint (J1``) and the second joint (J2``) and calculate the movement distance (D). For example, the control unit (500) can detect the motions of the first joint (J1``) and the second joint (J2``) that cause the probe (PR``) to move in a straight line to calculate the movement distance (D) of the probe (PR``), and can detect the angles of the first joint (J1``) and the second joint (J2``) to calculate the measurement position (MP``) on the affected area (W) into which the probe (PR``) is inserted.

[0402] That is, the control unit (500) can obtain the insertion depth at which the probe (PR) is inserted into the affected area (W) using information about the first joint (J1``) and the second joint (J2``) transmitted from the sub-robot arm (SR``), and obtain measurement position data about the measurement position (MP``) on the affected area (W) where the probe (PR``) is inserted. The control unit (500) can generate cartilage data using the insertion depth and measurement position information, and correct the scan data using the cartilage data.

[0403] Figure 19 is a drawing showing measurement points for measuring cartilage thickness.

[0404] Referring to Figure 19, the thickness of the cartilage (C) of the affected area (W) can be measured at multiple measurement locations (MP).

[0405] In one embodiment, the measuring unit (400) can measure the thickness of the cartilage (C) at a plurality of measuring locations (MP) on the affected area (W). That is, the probe (PR) can be inserted into a plurality of measuring locations (MP) on the affected area (W). Although FIG. 17 illustrates that five measuring locations (MP) are created on the affected area (W), the arrangement and number of the measuring locations (MP) are not limited thereto.

[0406] As described above, the control unit (500) can obtain the measurement positions (MP) into which a plurality of probes (PR) are inserted and the insertion depth corresponding to the measurement positions (MP). That is, the control unit (500) can generate cartilage data for the affected area (W) using the plurality of measurement positions (MP) and the insertion depth corresponding to the measurement positions (MP).

[0407] In one embodiment, the control unit (500) can teach the shape of the cartilage (C) of the affected area (W) to artificial intelligence, and match the learned shape of the cartilage (C) with the measurement position (MP) and the insertion depth corresponding to the measurement position (MP) to generate cartilage data.

[0408] The measurement location (MP) on the affected area (W) where the probe (PR) is inserted and the insertion depth is measured can be preset. For example, the surgeon (ST) can specify a point in the affected area (W) that effectively represents the structural characteristics of the cartilage (C) and define this as the measurement location (MP).

[0409] That is, the measurement location (MP) can be set based on predefined anatomical reference points, or optionally designated based on the surgeon's clinical experience or image-based analysis results. Thus, the reliability of cartilage data generated using the measured cartilage thickness (C) can be improved.

[0410] FIG. 20 is a diagram illustrating a process of measuring a gap between body structures constituting a refund using a gap measuring sensor according to one embodiment of the present invention.

[0411] Referring to FIG. 20, a gap measurement sensor (GM) according to one embodiment of the present invention can be inserted between body structures constituting a wound (W) to measure the gap (S) therebetween. For example, the gap measurement sensor (GM) can be inserted between a first bone (B1) and a second bone (B2).

[0412] In Fig. 20, the gap measurement sensor (GM) is illustrated as being inserted between the first bone (B1) and the second bone (B2), but the body structure constituting the affected area (W) is not limited thereto. However, for convenience of explanation, the following description will focus on an embodiment in which the gap measurement sensor (GM) is inserted between the first bone (B1) and the second bone (B2) to measure the gap (S) between the first bone (B1) and the second bone (B2).

[0413] The gap measurement sensor (GM) can detect the gap (S). For example, the gap measurement sensor (GM) can measure the gap (S) by including a tension sensor inside. The tension sensor can measure the gap (S) by detecting the degree of tension of a variable tube inserted between the first groove (B1) and the second groove (B2). The method by which the gap measurement sensor (GM) detects the gap (S) is not limited thereto, and for example, the gap measurement sensor (GM) can also measure the gap (S) using a magnetic field sensor.

[0414] In one embodiment, the gap measurement sensor (GM) may be configured to insert a variable tube between the first bone (B1) and the second bone (B2), and supply fluid to the variable tube to provide a pressure of a preset magnitude to the first bone (B1) and the second bone (B2). In other words, the gap measurement sensor (GM) may provide a pressure of a preset magnitude to the first bone (B1) and the second bone (B2) in a direction that separates them from each other.

[0415] That is, the gap measurement sensor (GM) can measure the separation gap (S) between the first bone (B1) and the second bone (B2) when a predetermined pressure is applied between them. Thus, the gap measurement sensor (GM) can obtain information that can quantitatively analyze the physical reaction characteristics between the first bone (B1) and the second bone (B2) or the behavior according to the pressure between the joints.

[0416] The gap measurement sensor (GM) may include a first sensor unit (GMa) and a second sensor unit (GMb). The first sensor unit (GMa) and the second sensor unit (GMb) may be spaced apart from each other by a preset interval and may be positioned between the first goal (B1) and the second goal (B2).

[0417] In one embodiment, the gap measurement sensor (GM) may be arranged on a rotational path along which the second goal (B2) rotates relative to the first goal (B1). The first sensor unit (GMa) and the second sensor unit (GMb) may be arranged at a preset interval from each other along a rotational path along which the second goal (B2) rotates relative to the first goal (B1).

[0418] For example, if the affected area (W) is the knee, the first sensor unit (GMa) may be positioned to contact the lateral condyle, and the second sensor unit (GMb) may be positioned to contact the medial condyle.

[0419] That is, the first sensor unit (GMa) can measure the first separation distance (S1) between the lateral condyle eminence corresponding to the femur and the first bone (B1) corresponding to the tibia according to the rotation of the second bone (B2). In addition, the second sensor unit (GMb) can measure the second separation distance (S2) between the medial condyle eminence and the first bone (B1) according to the rotation of the second bone (B2). Thus, the separation measuring sensor (GM) can detect the separation distance according to the rotation of the second bone (B2) and the posture of the second bone (B2) with respect to the first bone (B1).

[0420] Hereinafter, the 'rotation angle (A) of the second bone (B2)' can be defined as the angle at which the second bone (B2) rotates with respect to the first bone (B1). For example, when the affected area (W) is the knee, the 'rotation angle (A) of the second bone (B2)' can be defined as the angle formed by the imaginary second line (Lb) connecting the hip center and the femoral knee center with respect to the imaginary first line (La) connecting the preset point on the medial malleolus and the tibial knee center.

[0421] In one embodiment, the gap measurement sensor (GM) can measure a gap (S) corresponding to a rotation angle (A) of the second goal (B2). When the second goal (B2) rotates, the gap measurement sensor (GM) can measure a first gap (S1) and a second gap (S2) for each preset rotation angle (A).

[0422] For example, the gap measurement sensor (GM) can measure the gap (S) when the rotation angle (A) is 0 degrees or 90 degrees. That is, the gap measurement sensor (GM) can obtain the first gap (S1) and the second gap (S2) corresponding to the rotation angle (A) required for establishing a surgical plan or generating a corresponding work path. Thus, the accuracy of the generated work path can be improved.

[0423] However, the rotation angle of the second goal (B2) at which the gap measurement sensor (GM) measures the gap distance (S) is not limited to the above-described range. For example, the gap measurement sensor (GM) may continuously acquire the gap distance (S) in a range where the rotation angle (A) is 0 degrees or more and 120 degrees or less.

[0424] Figure 21 is a drawing showing that a reference point has been created on scan data.

[0425] Referring to FIG. 21, the control unit (500) can match modeling data and scan data centered on a reference point on the return (W). The control unit (500) can match a first reference point created on the modeling data with a second reference point (RF2) on the scan data to generate a matching result.

[0426] The control unit (500) can generate a first reference point on the modeling data. The control unit (500) can specify a point that effectively represents the structural characteristics of the affected area (W) and designate this as the first reference point. For example, the first reference point can be set according to a preset anatomical reference point or can be arbitrarily designated by the surgeon (ST). Thus, a first reference point reflecting the structural and anatomical characteristics of the affected area (W) on the modeling data can be generated.

[0427] Multiple first reference points can be created. The multiple first reference points can be set to correspond to different anatomical features on the affected area (W). Specifically, the multiple first reference points can be created based on factors such as the relative distance, direction, or anatomical features between each first reference point. Thus, when matching modeling data and scan data based on the reference points, the accuracy of the matching results can be improved.

[0428] The control unit (500) can generate a second reference point (RF2) on the scan data corresponding to the first reference point on the modeling data. For example, the control unit (500) can analyze the surface shape, curvature change, and other patterns of the affected area (W) expressed in the scan data to generate a second reference point corresponding to the first reference point. Thus, a second reference point reflecting the structural and anatomical characteristics of the affected area (W) on the scan data can be generated. The second reference points (RF2) can be generated in the same number as the number of first reference points so as to correspond to each of the first reference points.

[0429] The control unit (500) can generate a matching result between modeling data and scan data by matching the first reference point and the second reference point (RF2). The control unit (500) can generate a matching result by matching the first reference point reflecting the characteristics of the affected area (W) in the modeling data with the second reference point (RF2) reflecting the characteristics of the affected area (W) in the scan data. That is, the control unit (500) can generate a matching result without matching all information values ​​about the affected area (W) included in the modeling data and the scan data. Thus, the speed of generating the matching result can be improved, and the response speed of the surgical robot system (1) can be improved.

[0430] A surgical robot system according to embodiments of the present invention can acquire image information including the affected area and generate a working path of the robot arm. The surgical robot system can acquire image information containing accurate information about the affected area or the measuring unit by using light of a preset wavelength band or controlling a photographing unit. The surgical robot system can correct scan data acquired from the image information using cartilage data and quickly reflect the movement of the affected area over time to update the working path of the robot arm.

[0431] FIG. 22 is a drawing illustrating a surgical robot control method according to one embodiment of the present invention.

[0432] Referring to FIG. 22, a surgical robot control method according to one embodiment of the present invention may include a step of acquiring modeling data (S10), a step of acquiring scan data for a wound (S20), a step of generating a matching result (S30), a step of generating a working path of a robot arm (S40), and a step of controlling the robot arm (S50).

[0433] In the step of acquiring modeling data (S10), modeling data, which is a three-dimensional model of the affected area (W), can be acquired. In one embodiment, in the step of acquiring modeling data (S10), modeling data acquired through CT scanning or modeling data generated by a separate external device can be acquired. In the step of acquiring modeling data (S10), the acquired modeling data can be stored in the memory (510).

[0434] In another embodiment, in step S10 of acquiring modeling data, modeling data may be acquired based on medical images received from an external source. For example, in step S10 of acquiring modeling data, modeling data may be acquired by segmenting and processing a region of interest from the received medical image.

[0435] In the step (S20) of acquiring scan data for the affected area, scan data, which is three-dimensional information for the affected area (W), can be acquired from the image information acquired by the photographing unit (200). In the step (S20) of acquiring scan data for the affected area, scan data, which is a set of three-dimensional coordinate values ​​for each point of the affected area (W) on the image information, can be acquired.

[0436] In one embodiment, in step S20 of acquiring scan data for a wound, the scan data may be acquired from multiple pieces of image information. For example, in step S20 of acquiring scan data for a wound, the scan data may be acquired from multiple pieces of image information of different viewpoints for the wound (W). Thus, scan data representing the wound (W) from different viewpoints may be acquired.

[0437] In the step (S30) of generating a matching result, the modeling data and scan data can be matched to generate a matching result for the return (W).

[0438] In one embodiment, in the step (S30) of generating a matching result, the matching result may be generated using a learning model. In the step (S30) of generating a matching result, the matching result may be generated by matching modeling data and scan data using a previously learned learning model.

[0439] In the step (S30) of generating a matching result, the matching result can be generated using the first model (M1) learned using modeling data, scan data, and information corresponding to the matching result. For example, in the step (S30) of generating a matching result, the learning modeling data corresponding to the modeling data and the learning scan data corresponding to the scan data can be used as input data, and the learning matching result generated by matching the modeling data and the scan data can be used as output data to use the learned first model (M1).

[0440] In another embodiment, in the step (S30) of generating the matching result, a 6D pose estimation technique or an ICP algorithm may be used to match the modeling data and the scan data. In the step (S30) of generating the matching result, the position and pose of the affected area (W) may be estimated using the scan data, and the 6D pose information may be aligned with the modeling data to generate the matching result, or the ICP algorithm may be applied to minimize the shape difference between the modeling data and the scan data to generate the matching result.

[0441] In the step (S30) of generating the matching result, multiple scan data and modeling data can be matched. In the step (S30) of generating the matching result, each scan data and modeling data can be matched to obtain a more accurate matching result. For example, in the step (S30) of generating the matching result, the scan data with the highest degree of 3D coordinate value correspondence with the modeling data can be selected and matched with the modeling data. Thus, the accuracy of the matching result can be improved.

[0442] In the step (S40) of generating a work path of the robot arm, the work path of the robot arm (100) can be generated using the matching result. For example, in the step (S40) of generating a work path of the robot arm, the position and posture of the actual affected area (W) and the surgical plan information can be matched based on the matching result. In the step (S40) of generating a work path of the robot arm, the work path of the robot arm (100) can be generated for performing the surgery according to the surgical plan information.

[0443] For example, in the step (S40) of generating a work path of the robot arm, a work path including a movement path and a change in posture of the robot arm (100) can be generated so that the end effector (EF) reaches a specific part of the affected area (W) or performs work such as incision, excision fixation, etc.

[0444] In the step of controlling the robot arm (S50), a motion signal of the robot arm (100) can be generated so that the robot arm (100) moves along the generated work path.

[0445] In one embodiment, in the step (S50) of controlling the robot arm, an impedance and admittance control technique may be applied to generate an operation signal of the robot arm (100). Alternatively, in the step (S50) of controlling the robot arm, a virtual wall algorithm may be applied to limit the operation range of the robot arm (100) or set a work area. In another embodiment, in the step (S50) of controlling the robot arm, a basic surgical unit motion of the robot arm (100) may be set, and a operation signal of the robot arm (100) may be generated by combining the basic surgical unit motions. The control unit (500) may generate an operation signal of the robot arm (100) to sequentially execute the basic surgical unit motions according to the generated work path of the robot arm (100) or to selectively perform them.

[0446] Figure 23 is a drawing illustrating in detail some steps of Figure 22.

[0447] Referring to FIG. 23, the step of acquiring modeling data (S10) may include the step of acquiring a medical image (S11) and the step of acquiring modeling data based on the medical image (S12).

[0448] In the step of acquiring a medical image (S11), a medical image can be received from an external device. In the step of acquiring a medical image (S11), a three-dimensional image including the affected area (W) can be acquired. In the step of acquiring a medical image (S11), multiple two-dimensional images obtained by scanning a tomography scan can be received and reconstructed to create a medical image, or a reconstructed three-dimensional image can be received.

[0449] In the step (S12) of obtaining modeling data based on a medical image, modeling data can be generated from the medical image. In the step (S12) of obtaining modeling data based on a medical image, modeling data can be obtained by separating a region of interest from the medical image. For example, if the affected area (W) is the knee, in the step (S12) of obtaining modeling data based on a medical image, modeling data can be obtained by separating a region including the tibia and femur from the medical image.

[0450] In one embodiment, in step S12 of obtaining modeling data based on medical images, modeling data may be obtained from medical images using a previously learned learning model. For example, in step S12 of obtaining modeling data based on medical images, a second model (M2) learned using information corresponding to the medical images and modeling data may be utilized.

[0451] In the step (S12) of obtaining modeling data based on medical images, a second model (M2) learned can be used by using a learning medical image corresponding to the medical image as input data and learning modeling data corresponding to the modeling data as output data.

[0452] Figure 24 is a drawing detailing some other steps of Figure 22.

[0453] Referring to FIG. 24, the step (S30) of generating a matching result may include a step (S31) of generating a first reference point, a step (S32) of generating a second reference point, and a step (S33) of generating a matching result.

[0454] In the step S31 of generating a first reference point, a first reference point can be generated on modeling data. In the step S31 of generating a first reference point, a point that effectively represents the structural characteristics of the affected area (W) can be specified and designated as the first reference point. In one embodiment, in the step S31 of generating a first reference point, a plurality of first reference points can be generated. The plurality of first reference points are generated by considering the relative distance, direction, or anatomical features between each first reference point, so that the accuracy of the matching result generated by matching the modeling data and the scan data can be improved.

[0455] In the step (S32) of generating a second reference point, a second reference point (RF2) on the scan data corresponding to the first reference point can be generated. For example, in the step (S32) of generating a second reference point, a pattern such as a surface shape or curvature change of the affected area (W) expressed in the scan data can be analyzed to generate a second reference point corresponding to the first reference point. Thus, a second reference point reflecting the structural and anatomical characteristics of the affected area (W) on the scan data can be generated. The second reference points (RF2) can be generated in the same number as the number of first reference points so as to correspond to each of the first reference points.

[0456] In the step (S33) of generating a matching result, a matching result between modeling data and scan data can be generated by matching a first reference point and a second reference point (RF2). In the step (S33) of generating a matching result, a matching result can be generated by matching a first reference point reflecting the characteristics of the affected area (W) in the modeling data with a second reference point (RF2) reflecting the characteristics of the affected area (W) in the scan data. That is, in the step (S33) of generating a matching result, a matching result can be generated without matching all information values ​​for the affected area (W) included in the modeling data and the scan data.

[0457] FIG. 25 is a drawing illustrating a surgical robot control method according to another embodiment of the present invention.

[0458] The surgical robot control method according to another embodiment of the present invention differs from the surgical robot control method according to another embodiment of the present invention only in that some additional steps are included, and therefore, redundant descriptions will be omitted below and the differences will be described.

[0459] Referring to FIG. 25, a surgical robot control method according to another embodiment of the present invention may further include a step (S25A) of correcting scan data using cartilage data.

[0460] In the step (S25A) of correcting scan data using cartilage data, cartilage data for cartilage (C) among the affected area (W) can be acquired.

[0461] In the step (S25A) of correcting scan data using cartilage data, the thickness of the cartilage (C) in the affected area (W) can be measured using the measuring unit (400).

[0462] In one embodiment, in the step (S25A) of correcting scan data using cartilage data, the thickness of the cartilage (C) may be calculated from the insertion depth of the probe (PR) measured using the detection sensor (470) included in the measuring unit (400), and measurement position data for the measurement position where the probe (PR) is inserted may be obtained from the image information obtained from the photographing unit (200). In the step (S25A) of correcting scan data using cartilage data, cartilage data may be obtained using the measurement position data and the insertion depth.

[0463] In another embodiment, in the step (S25A) of correcting scan data using cartilage data, the thickness of the cartilage (C) can be calculated from the image information of the measuring unit (400`) and measurement position data can be obtained.

[0464] For example, in the step (S25A) of correcting scan data using cartilage data, the insertion depth can be estimated using image information about the measuring unit (400`) before the probe (PR) is inserted into the affected area (W) and image information about the measuring unit (400`) in a state where advancement is restricted. In the step (S25A) of correcting scan data using cartilage data, the thickness of the cartilage (C) can be calculated using the estimated insertion depth.

[0465] In the step (S25A) of correcting scan data using cartilage data, measurement position data for the position on the affected area (W) where the probe (PR) is inserted can be obtained from image information including the measuring unit (400). In the step (S25A) of correcting scan data using cartilage data, cartilage data can be obtained using the estimated insertion depth and measurement position data.

[0466] In another embodiment, in the step (S25A) of correcting scan data using cartilage data, the thickness and measurement position data of the cartilage (C) can be obtained from the operation information of the measuring unit (400``).

[0467] For example, in the step (S25A) of correcting scan data using cartilage data, the measurement position data and the movement distance (D) of the probe (PR) can be measured using the motion information for the first joint (J1``) and the second joint (J2``). In the step (S25A) of correcting scan data using cartilage data, the cartilage data can be acquired using the thickness of the cartilage (C) estimated using the measurement position data and the movement distance (D) of the probe (PR).

[0468] In the step (S25A) of correcting scan data using cartilage data, the portion corresponding to cartilage (C) can be deleted from the scan data using the cartilage data. In other words, in the step (S25A) of correcting scan data using cartilage data, the scan data can be corrected so that three-dimensional information regarding the shape of the bone (B) is included in the scan data. Thus, information for generating a more precise working path of the robot arm (100) can be generated from the scan data.

[0469] FIG. 26 is a drawing illustrating a surgical robot control method according to another embodiment of the present invention.

[0470] The surgical robot control method according to another embodiment of the present invention differs from the surgical robot control method according to one embodiment of the present invention only in the specific contents of some steps and the addition of some steps. Therefore, the following description will be omitted and the differences will be explained.

[0471] Referring to FIG. 26, a surgical robot control method according to another embodiment of the present invention can repeatedly acquire image information including a target area (W) according to a preset cycle. The surgical robot control method according to another embodiment of the present invention can update the alignment result using the newly acquired image information and update the work path in real time.

[0472] That is, a surgical robot control method according to another embodiment of the present invention may include a step (S20B) of acquiring first scan data corresponding to a first time point, a step (S25B) of acquiring second scan data corresponding to a second time point, a step (S30) of generating an updated matching result corresponding to the second time point, and a step (S40B) of generating a work path using the updated matching result.

[0473] In the step (S20B) of acquiring first scan data corresponding to a first point in time, the first scan data corresponding to the first point in time can be acquired based on the first image information of the first point in time. In the step (S20B) of acquiring first scan data corresponding to the first point in time, a first return point (WD1) for the return (W) can be generated on the first image information. In the step (S20B) of acquiring first scan data, first scan data including 3D cloud information generated using the first return point (WD1) can be acquired.

[0474] In the step (S25B) of acquiring second scan data corresponding to the second point in time, second scan data corresponding to the second point in time can be acquired based on image information of the second point in time after the first cycle has elapsed from the first point in time.

[0475] In one embodiment, in the step (S30) of generating an updated matching result corresponding to the second point in time, a second return point (WD2) for the return (W) can be generated from the second image information acquired at the second point in time.

[0476] In the step (S25B) of acquiring second scan data corresponding to the second point in time, second scan data including 3D cloud information generated using the second return point (WD2) can be acquired. Thus, in the step (S30) of generating an updated alignment result corresponding to the second point in time, the first scan data of the first point in time can be updated with the second scan data of the second point in time.

[0477] In the step (S30) of generating an updated alignment result corresponding to the second time point, a new alignment result corresponding to the second time point can be generated by matching the modeling data with the second scan data. In the step (S30) of generating an updated alignment result corresponding to the second time point, the alignment result generated using the first scan data can be updated with an updated alignment result generated using the second scan data.

[0478] In the step (S40B) of generating a work path using the updated alignment result, a new work path can be generated using the updated alignment result of the second time point. That is, in the step (S40B) of generating a work path using the updated alignment result, a work path reflecting the second scan data including information on the affected area (W) of the second time point can be generated. Thus, in the step (S40B) of generating a work path using the updated alignment result, the work path can be updated to present a new work path for precise surgical progress.

[0479] Figure 27 is a drawing illustrating in detail some steps of Figure 26.

[0480] Referring to FIG. 27, the step (S25B) of acquiring second scan data corresponding to the second point in time may include the step (S251B) of acquiring first marker data and second marker data and the step (S252B) of acquiring second scan data.

[0481] In the step (S251B) of acquiring first marker data and second marker data, first marker data corresponding to a first time point can be acquired based on first image information, and second marker data corresponding to a second time point can be acquired based on second image information. In the step (S251B) of acquiring first marker data and second marker data, three-dimensional information on the affected area marker (WM) at the first time point and the second time point can be acquired.

[0482] In the step of obtaining the second scan data (S252B), the second scan data can be obtained using the first scan data, the first marker data, and the second marker data. For example, in the step of obtaining the second scan data (S252B), a relationship between the first marker data and the second marker data can be derived, and this relationship can be applied to the first scan data to estimate the second scan data.

[0483] That is, in the step of acquiring the second scan data (S252B), the change in posture of the affected area (W) can be estimated based on the affected area marker data. In the step of acquiring the second scan data (S252B), the change in position and posture of the affected area marker (WM) over time can be tracked, and the change in position of the affected area points (WD) linked to the affected area marker (WM) can be predicted. Thus, in the step of acquiring the second scan data (S252B), the real-time change of the affected area (W) can be quickly reflected using only information on a relatively small number of affected area markers. Thus, in the step of acquiring the second scan data (S252B), the scan data acquisition time can be shortened, and the response speed of the surgical robot system (1) can be improved.

[0484] A surgical robot control method according to embodiments of the present invention can acquire image information including the affected area and generate a working path of the robot arm. The surgical robot control method can use cartilage data to correct scan data acquired from the image information and quickly update the working path of the robot arm by reflecting the movement of the affected area over time.

[0485] FIG. 28 is a drawing illustrating a surgical robot control method according to another embodiment of the present invention.

[0486] Since the surgical robot control method according to another embodiment of the present invention differs only in some steps when compared with the surgical robot control method according to one embodiment of the present invention, redundant descriptions will be omitted below and the differences will be explained.

[0487] Referring to FIG. 28, a surgical robot control method according to another embodiment of the present invention may further include a step (S35C) of generating landmark data.

[0488] In the step of generating landmark data (S35C), landmark data for feature points on the affected area (W) can be generated based on modeling data. For example, in the step of generating landmark data (S35C), coordinate values ​​for feature points on the body including the affected area (W) can be extracted from the modeling data, which is 3D coordinate data for the affected area (W), and defined as landmark data.

[0489] For example, if the affected area (W) is the knee, in the step of generating landmark data (S35C), coordinate values ​​for bone feature points such as the hip joint center, the femoral knee joint center, and the tibial knee joint center can be defined as landmark data.

[0490] In one embodiment, in the step of generating landmark data (S35C), landmark data can be acquired using a third model (M3) that has been previously learned. In the step of generating landmark data (S35C), landmark data can be acquired using a third model (M3) that has been learned based on information corresponding to medical images and landmark data.

[0491] For example, the third model (M3) used in the step (S35C) of generating landmark data may be a learning model that uses learning modeling data corresponding to modeling data as input data and learning landmark data corresponding to landmark data as output data.

[0492] In the step of generating landmark data (S35C), coordinate values ​​for feature points on the body including the affected area can be obtained from modeling data using the third model (M3).

[0493] In the step (40C) of generating a work path of the robot arm, a work path can be generated using the alignment result and landmark data.

[0494] In one embodiment, in the step (40C) of generating a work path of the robot arm, the work path of the robot arm (100) may be generated using the alignment result and landmark data. For example, in the step (40C) of generating a work path of the robot arm, the alignment result may be corrected using information on the position and posture of the affected area (W) acquired using the landmark data. Alternatively, the information on the position and posture of the affected area (W) and the alignment result may be considered together to establish a surgical plan, and a corresponding work path may be acquired.

[0495] In another embodiment, in the step (40C) of generating the work path of the robot arm, the work path generated from the alignment results may be corrected using landmark data. For example, in the step (40C) of generating the work path of the robot arm, the work path may be corrected using information on the position and posture of the affected area (W) obtained using landmark data.

[0496] FIG. 29 is a drawing illustrating a surgical robot control method according to another embodiment of the present invention.

[0497] Since the surgical robot control method according to another embodiment of the present invention differs only in some steps compared to the surgical robot control method according to one embodiment of the present invention, the following description will be omitted and the differences will be described. Referring to FIG. 28, the surgical robot control method according to another embodiment of the present invention may further include a step (S35D) of measuring a separation distance with respect to the affected area.

[0498] In the step (S35D) of measuring the spacing between the body structures constituting the affected area (W), the spacing between the body structures constituting the affected area (W) can be measured. In the step (S35D) of measuring the spacing between the body structures constituting the affected area (W), the spacing between the body structures constituting the affected area (W) can be measured using a spacing measurement sensor (GM).

[0499] In one embodiment, in the step (S35D) of measuring the spacing between the affected area, the spacing between the first bone (B1) and the second bone (B2) constituting the affected area (W) can be measured. For example, in the step (S35D) of measuring the spacing between the affected area, the spacing between the first bone (B1) and the second bone (B2) can be measured by applying a predetermined pressure between the first bone (B1) and the second bone (B2). Thus, in the step (S35D) of measuring the spacing between the affected area, information can be obtained that can quantitatively analyze the physical response characteristics of the first bone (B1) and the second bone (B2) or the behavior according to the pressure between the joints.

[0500] In the step (S35D) of measuring the spacing between the affected area, the spacing between multiple points on the affected area (W) can be obtained. For example, in the step (S35D) of measuring the spacing between the affected area, the spacing between two points on the affected area (W) can be obtained using the first sensor unit (GMa) and the second sensor unit (GMb).

[0501] In one embodiment, in the step (S35D) of measuring the spacing between the first and second sensors (GMa) arranged on the rotation path of the second bone (B2), a first spacing between the first sensor (GMa) and the second sensor (GMb) can be obtained by using the first sensor (GMa) and the second sensor (GMb) arranged on the rotation path of the second bone (B2).

[0502] That is, in the step (S35D) of measuring the spacing for the affected area, the first spacing (S1) and the second spacing (S2) can be obtained according to the rotation of the second bone (B2) with respect to the first bone (B1). Thus, in the step (S35D) of measuring the spacing for the affected area, various information for establishing a surgical plan or generating a corresponding work path can be obtained.

[0503] In the step (S40D) of generating the work path of the robot arm, the work path of the robot arm (100) can be generated using the matching result and the gap distance. For example, in the step (S40D) of generating the work path of the robot arm, the gap distance can be used to correct the matching result, or the surgical plan and the corresponding work path can be obtained by considering both the gap distance and the matching result. Alternatively, in the step (S40D) of generating the work path of the robot arm, the gap distance can be used to correct the work path generated from the matching result.

[0504] Figure 30 is a drawing showing an enlarged view of part A of Figure 1.

[0505] Referring to FIG. 30, a lower extremity assist device (600) according to one embodiment of the present invention may include a connecting portion (610), a lower extremity assist link (620), and a lower extremity assist joint (630).

[0506] The connecting portion (610) can be connected to the operating table (OT). For example, the connecting portion (610) can be inserted into a connecting hole (H) that extends in one direction on the operating table (OT). The connecting portion (610) can be inserted into the connecting hole (H) and move along the direction in which the connecting hole (H) extends.

[0507] The connecting portion (610) can be connected to the lower limb auxiliary link (620). The connecting portion (610) can be inserted into the connecting hole (H) of the operating table (OT) and connected to the lower limb auxiliary link (620). That is, the connecting portion (610) can move the lower limb auxiliary link (620) along the connecting hole (H) of the operating table (OT). Thus, the connecting portion (610) can change the position at which the lower limb auxiliary link (620) supports the patient (P).

[0508] The lower extremity auxiliary link (620) may be connected to the connecting portion (610) or to the lower extremity auxiliary joint (630). The lower extremity auxiliary link (620) may contact and support the patient (P). For example, the lower extremity auxiliary link (620) may support the lower extremity of the patient (P) by transmitting a load transmitted from the patient (P) to the operating table (OT) through the connecting portion (610).

[0509] The lower extremity auxiliary joint (630) may be positioned at one end of the lower extremity auxiliary link (620). For example, the lower extremity auxiliary joint (630) may be positioned between a pair of lower extremity auxiliary links (620). The lower extremity auxiliary joint (630) may rotate to change the position and posture of the pair of lower extremity auxiliary links (620). Thus, the lower extremity auxiliary joint (630) may change the posture of the patient (P) supported by the lower extremity auxiliary link (620).

[0510] In one embodiment, the lower extremity assist device (600) may be electrically connected to the control unit (500). The lower extremity assist device (600) may change its position or posture by receiving a control signal from the control unit (500). For example, the connection unit (610) may change its position at which it is inserted into the connection hole (H) by receiving a control signal from the control unit (500). Alternatively, the lower extremity assist joint (630) may change the posture of a pair of connected lower extremity assist links (620) by receiving a control signal from the control unit (500).

[0511] Figure 31 is a drawing that briefly illustrates the process of performing surgery using a surgical robot system.

[0512] Figure 31(a) is a diagram illustrating the steps for establishing a preoperative surgical plan. In this step, the surgical plan can be established using 3D modeling data for the affected area (W).

[0513] That is, during the preoperative surgical planning stage, surgical planning information can be generated, including information about the surgical procedure, such as the incision location, incision range, and tissue removal order of the affected area (W). For example, the surgical planning information can be used in the process of generating a robotic arm's working path.

[0514] Figure 31(b) is a diagram illustrating a step of measuring a return area. In the step of measuring a return area, the return area (W) is photographed to obtain image information about the return area (W), and scan data, which is three-dimensional information about the return area (W), can be obtained.

[0515] In the step of measuring the affected area, image information including the affected area (W) can be obtained using a photographing unit (200). In the step of measuring the affected area, the accuracy of the image information can be improved by using a photographing unit (200) that uses light of a preset wavelength band.

[0516] In the step of measuring the affected area, multiple image information about the affected area (W) can be acquired. For example, in the step of measuring the affected area, the photographing unit (200) can be positioned to have various postures and positions to acquire multiple image information about the affected area (W).

[0517] In the step of measuring the refund, the quality of image information can be improved by adjusting the timing of the emission light (LE) emitted from the photographing unit (200) and the second light (L2) emitted from the lighting unit (300), or by adjusting the position and posture of the photographing unit (200).

[0518] In the step of measuring the return, scan data for the return (W) can be obtained from image information. In the step of measuring the return, the accuracy of the scan data can be improved by correcting the scan data using a return marker (WM).

[0519] In the step of measuring the affected area, the distance between the affected areas (S) can be measured. For example, in the step of measuring the affected area, the distance between the bodies forming the affected area (S) can be measured using a distance measurement sensor (GM).

[0520] In the step of measuring the affected area, landmark data can be generated based on modeling data. For example, in the step of measuring the affected area, coordinate values ​​for characteristic points on the body can be defined as landmark data based on the modeling data.

[0521] In the step of measuring the affected area, cartilage data can be generated to correct the scan data. For example, in the step of measuring the affected area, information on the thickness of the cartilage (C) in the affected area (W) can be acquired to obtain cartilage data, and by applying the cartilage data to the scan data, 3D information on the bone (B) from which the cartilage (C) is excluded can be obtained.

[0522] Figure 31(c) is a diagram illustrating a step of matching modeling data and scan data. In one embodiment, the step of matching modeling data and scan data may use a pre-trained learning model to match the modeling data and scan data to generate a matching result. In another embodiment, the step of matching modeling data and scan data may use a 6D pose estimation technique or an ICP algorithm to match the modeling data and scan data to generate a matching result.

[0523] Figure 31(d) illustrates the steps for evaluating a 3D model. This step can determine the accuracy of scan data or alignment results generated from image information. This step can determine the accuracy of information acquired for surgical procedures, ensuring patient safety and improving the success rate of the surgery.

[0524] Figure 31(e) is a diagram illustrating the steps involved in establishing a surgical plan during surgery. During this step, the surgical plan can be modified or newly created based on situations that arise during the surgery. For example, during this step, image information regarding the affected area (W) can be repeatedly acquired at preset intervals and the matching results updated.

[0525] Figures 31(f) and 31(g) are diagrams illustrating steps for generating and verifying a robot arm's work path for a wound. In the step of generating and verifying a robot arm's work path for a wound, the matching results can be used to generate the robot arm's work path and verify its appropriateness. For example, in the step of generating and verifying a robot arm's work path for a wound, the robot arm's position for drilling can be set, and a work path for moving the robot arm to that position can be generated.

[0526] In the step of creating and verifying the working path of the robot arm for the affected area, the working path of the robot arm (100) for performing the surgery can be created by matching the position and posture of the actual affected area (W) with the surgical plan information.

[0527] In one embodiment, the step of generating and verifying the work path of the robot arm for the affected area may generate the work path by considering landmark data and the spacing interval. The step of generating and verifying the work path of the robot arm for the affected area may generate the work path using the landmark data and the spacing interval together with the alignment result, or the work path generated based on the alignment result may be corrected using the landmark data and the spacing interval.

[0528] In another embodiment, the step of generating and verifying the robot arm's work path for the affected area may generate the robot arm's work path based on the alignment results and landmark information. The step of generating and verifying the robot arm's work path for the affected area may generate the robot arm's work path by considering both the alignment results and landmark information, or the work path generated based on the alignment results may be corrected based on the landmark information.

[0529] Figure 31(h) is a drawing illustrating a step of setting up a robot arm for surgery. For example, in the step of setting up the robot arm, a guide or the like for guiding the direction of movement of the end effector (EF) may be mounted on one side of the robot arm (100) to enable the end effector (EF) to move precisely during surgery.

[0530] Fig. 31(i) is a diagram illustrating a step for controlling the operation of a robot arm. In the step for controlling the operation of the robot arm, the robot arm (100) can be operated according to a work path. For example, in the step for controlling the operation of the robot arm, the robot arm (100) can be controlled to bring the end effector (EF) into contact with the affected area (W), or the end effector (EF) can be placed in a position adjacent to the affected area (W) and then the surgeon (ST) can manually control the robot arm (100) to bring the end effector (EF) into contact with the affected area (W).

[0531] While the present invention has been described with reference to the embodiments illustrated in the drawings, these are merely exemplary, and those skilled in the art will appreciate that various modifications and variations of the embodiments are possible. Therefore, the true scope of technical protection of the present invention should be determined by the technical spirit of the appended claims.

[0532] According to embodiments of the present invention, a surgical robot system and a surgical robot system control method can be applied to a robot system that obtains information about a wound and performs surgery on the wound.

Claims

1. A robot arm having at least one joint and capable of accessing a target area using an end effector mounted on one end; A photographing unit for acquiring image information including the above-mentioned refund; and A control unit for generating a working path of the robot arm based on the image information; The above filming department A light emitting unit that irradiates the first light of the first wavelength band to the affected area; and A surgical robot system, comprising a light receiving unit that collects reflected light reflected from the affected area and obtains image information about the affected area.

2. In paragraph 1, A surgical robot system further comprising a lighting unit that irradiates a second light of a second wavelength band different from the first wavelength band to the affected area.

3. In paragraph 2, A surgical robot system, wherein the first minimum wavelength value and the first maximum wavelength value of the first wavelength band have values ​​greater than the second minimum wavelength value and the second maximum wavelength value of the second wavelength band, respectively.

4. In paragraph 1, The above filming department A surgical robot system further comprising a filter unit that filters the reflected light incident on the light receiving unit into light of a third wavelength band.

5. In paragraph 4, A surgical robot system in which the filter unit filters light emitted from the light emitting unit into light of a third wavelength band.

6. In paragraph 4, The surgical robot system, wherein the third wavelength band corresponds to the first wavelength band.

7. In paragraph 2, The above filming department It further includes a filter unit that filters the reflected light incident on the light receiving unit into light of a third wavelength band; A surgical robot system, wherein the third minimum wavelength value of the third wavelength band has a value greater than the second maximum wavelength value of the second wavelength band.

8. In paragraph 1, The above control unit Obtain modeling data, which is a three-dimensional model of the above-mentioned area, from the input unit, and Based on the above image information, scan data, which is three-dimensional information about the affected area, is obtained, Matching the above modeling data and the above scan data to generate a matching result for the above-mentioned area, A surgical robot system that generates the work path using the above matching results.

9. In paragraph 8, The above control unit Creating at least one first reference point on the modeling data, and obtaining a second reference point on the scan data corresponding to the first reference point, A surgical robot system that matches the first reference point and the second reference point to generate a matching result for the affected area.

10. In paragraph 1, The above-mentioned photographing unit repeatedly acquires the image information according to a preset first cycle, A surgical robot system in which the control unit corrects the work path using image information acquired for each of the first cycles.

11. In paragraph 10, The above photographing unit obtains the image information including the affected area marker reflecting the movement of the affected area, A surgical robot system, wherein the control unit corrects the work path based on the image information including the return marker according to the first cycle.

12. In paragraph 11, The above control unit Obtain modeling data, which is a three-dimensional model of the above-mentioned area, from an external photographing device, and Based on the image information at the first point in time, first scan data for the affected area corresponding to the first point in time and first affected area marker data for the affected area marker are acquired, Obtaining second marker data corresponding to the second time point based on the image information at the second time point, A surgical robot system that acquires second scan data for the affected area corresponding to a second time point using the first scan data, the first affected area marker data, and the second affected area marker data.

13. In paragraph 10, It further includes a lighting unit that irradiates the second light of a second wavelength band different from the first wavelength band to the affected area according to a preset second cycle; The above control unit A surgical robot system that controls at least one of the photographing unit and the lighting unit so that the lighting unit is turned off during the time that the photographing unit acquires the image information.

14. In paragraph 8, The above control unit A surgical robot system that generates the alignment result based on the modeling data and the scan data using the first model that has been learned.

15. In paragraph 14, The above first model is The learning modeling data corresponding to the above modeling data and the learning scan data corresponding to the above scan data are used as input data, A surgical robot system that is learned by using the learning matching result generated by matching the above modeling data and the above scan data as output data.

16. In paragraph 8, The above control unit A surgical robot system that obtains a medical image including the affected area from the input unit, extracts a region of interest from the medical image, and obtains the modeling data.

17. In paragraph 16, The above control unit A surgical robot system that obtains modeling data from the medical image using a second model that has been previously learned.

18. In paragraph 8, The above control unit Based on the above modeling data, landmark data for the feature points on the above-mentioned area are acquired, A surgical robot system that generates the work path based on the landmark data and the alignment result.

19. In paragraph 18, The above control unit A surgical robot system that obtains landmark data from the modeling data using a learned third model.

20. A robotic arm having at least one joint; An end effector mounted on one end of the above robot arm; and It includes a photographing unit placed between the robot arm and the end effector; The above filming department A light emitting unit that irradiates a first light of a first wavelength band to the affected area; and A surgical robot system, comprising a light receiving unit that collects reflected light reflected from the affected area and obtains image information about the affected area.

21. In paragraph 20, A surgical robot system further comprising a lighting unit that irradiates the affected area with a second light of a second wavelength band different from the first wavelength band.

22. In paragraph 21, A surgical robot system, wherein the first minimum wavelength value and the first maximum wavelength value of the first wavelength band have values ​​greater than the second minimum wavelength value and the second maximum wavelength value of the second wavelength band, respectively.

23. In paragraph 20, The above filming department A surgical robot system further comprising a filter unit that filters the reflected light incident on the light receiving unit into light of a third wavelength band.

24. A step of obtaining modeling data, which is a three-dimensional model of a refund, from an input unit; A step of obtaining scan data, which is three-dimensional information about the affected area, based on image information about the affected area; A step of matching the modeling data and the scan data to generate a matching result for the affected area; and A surgical robot system control method, comprising: a step of generating a working path of a robot arm that can access the affected area using an end effector mounted on one end using the above alignment result; 25. In paragraph 24, In the step of generating the above matching result, A surgical robot system control method for generating the matching result based on the modeling data and the scan data using the first model that has been learned.

26. In paragraph 24, Further comprising a step of obtaining landmark data for feature points on the refund based on the modeling data; The steps to create the above work path are A surgical robot system control method for generating the work path based on the landmark data and the alignment result.

27. In paragraph 24, The step of generating the above matching result is A step of creating at least one first reference point on the above modeling data; A step of acquiring a second reference point on the scan data corresponding to the first reference point; and A surgical robot system control method, comprising: a step of matching the first reference point and the second reference point to generate a matching result for the affected area.

28. In paragraph 24, The step of acquiring the above scan data is A step of obtaining first scan data for the affected area corresponding to the first point in time based on the image information at the first point in time and first affected area marker data for the affected area marker reflecting the movement of the affected area; A step of obtaining second marker data corresponding to the second point in time based on the image information of the second point in time; and A surgical robot system control method, comprising: a step of obtaining second scan data for the affected area corresponding to the second time point using the first scan data, the first affected area marker data, and the second affected area marker data.

Images