Related systems and methods for tissue recognition in surgical robots
By designing a tissue recognition system in the surgical robot system, using image acquisition and processor to identify the in vivo tissue categories and provide tactile feedback, the problem of the lack of tactile feedback in the surgical robot system is solved, and the precision and reality of surgical operations are improved.
Patent Information
- Application Number
- CN202410838254.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-26
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2044-06-26
AI Technical Summary
The lack of tactile feedback in the existing surgical robot system makes it difficult for surgeons to accurately identify the hardness and texture of tissues in the body, affecting the fineness and reality of the surgical operation, and increasing the difficulty of the operation.
A tissue recognition system was designed to collect image data of tissues in the body through the image acquisition device at the end of the surgical instrument, use processors and tissue recognition models to extract image features, activate activation modules and build relationship diagrams, identify tissue categories, and use tactile sensation. The feedback system provides real feedback.
Accurate identification and tactile feedback of the types of tissues in the body are achieved, improving the precision and reality of the surgical operation, and reducing the difficulty of the operation.
Smart Images

Figure CN118383874B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of medical devices, and in particular relates to a system and method for tissue recognition by a surgical robot. Background Art
[0002] The current surgical robot system lacks tactile feedback from the surgeon’s hand during the operation, and can only rely on the endoscope system to estimate what human tissue the surgical instrument is currently encountering. It is sometimes difficult to accurately identify the hardness and texture of tissues through naked eye observation of endoscopic images, which can easily cause excessive or insufficient force during the operation, damaging tissues or sutures. At the same time, it is also impossible to perceive subtle changes during the operation, affecting the precision of the operation. In addition, due to the lack of tactile feedback, it is difficult for surgeons to get a real feel for the operation, increasing the difficulty of the operation. Summary of the invention
[0003] In order to solve the problem that it is difficult to distinguish tissue types during the existing surgical cutting and suturing process, the present invention proposes the following technical solutions:
[0004] A first aspect of the present invention provides a tissue identification system, comprising: a surgical instrument, including an image acquisition device at the end, wherein the image acquisition device is used to acquire image data of in vivo tissue, and the image data includes multiple groups of partition data; a processor, storing a tissue identification model, the tissue identification model includes a recognition module and multiple activation modules, the recognition module is connected to each of the activation modules, and each of the activation modules is associated with different groups of partition data respectively; the processor is configured to: extract image features of each of the partition data, and based on the image features, activate the associated activation module, and determine the activation duration; the identification module generates a relationship diagram of each of the activation modules relative to the activation duration, and based on the relationship diagram, identifies at least one tissue category of the in vivo tissue.
[0005] Preferably, the activation module includes an activation unit and multiple inhibition units; the input end of each of the inhibition units is connected to the first activation unit of a group of other activation modules, and the output end of each of the inhibition units is connected to the second activation unit of the same activation module; the processor is configured to: based on the image features, perform an excitation operation on the activation process of the activation unit in the associated activation module; the inhibition unit performs an inhibition operation on the activation process of the second activation unit in response to the completion of the activation of the same group of activation units connected to the input end.
[0006] Preferably, the processor is configured as follows: within a preset range of the activation sequence, if all activation units in the same group have completed activation, an inhibition operation is performed on the activation process of the second activation unit through an inhibition unit connected to the activation units in the same group; wherein the preset activation sequence is sorted according to the length of the activation duration.
[0007] Preferably, the multiple inhibition units are divided into multiple groups, wherein each group of inhibition units represents a different scene category; the processor is configured to: in response to the completion of activation of each of the activation units one by one, dynamically select an inhibition unit group of at least one scene category; and use the dynamically selected inhibition unit group to perform an inhibition operation on the activation of the second activation unit, wherein the inhibition operation is used to extend the activation duration.
[0008] Preferably, the processor is further configured to: determine the extension duration of the activation duration for each inhibition operation according to a preset weight of each inhibition unit.
[0009] Preferably, a standard waveform is provided in the activation module, and the standard waveform is in a standard time scale; the processor is configured to: based on the image features, control the potential of the activation module to increase along with the standard waveform according to the exposure time scale of the image acquisition device; when the potential reaches the peak of the standard waveform, record the current time under the standard time scale to obtain the activation duration of each activation module.
[0010] Preferably, the standard waveform diagram includes a plurality of segments, each of the segments includes a rising segment and a reset segment, and the peak is located at the connection between the rising segment and the reset segment in each of the segments; the processor is further configured to: control the potential of the activation module to increase with the rising segment, and control the activation module to reset the potential and extend the activation time in the reset segment; in each of the segments, superimpose the extended activation time on the activation time when the potential reaches the peak; and determine the final activation time based on the superimposed time corresponding to each of the segments.
[0011] Preferably, the processor is further configured to: determine the potential growth curve of each of the partitioned data under the exposure time scale according to the image features and a preset conversion relationship; based on the potential growth curve, control the potential of the activation module to increase along with the standard waveform; wherein the increasing speed of the potential of the activation module is related to the growth speed of the potential growth curve.
[0012] Preferably, the processor is further configured to: obtain a preset reference relationship graph; calculate the similarity between the reference relationship graph and the relationship graph; and determine at least one tissue category of the in vivo tissue according to the similarity.
[0013] Preferably, the processor is also configured to: divide the relationship graph into multiple local segments according to multiple preset duration intervals of activation duration; match the local similarity between each of the local segments and the reference relationship graph, and obtain the similarity between the reference relationship graph and the relationship graph based on each of the similarities.
[0014] A second aspect of the present invention provides a training method for a tissue recognition model, wherein the tissue recognition model is applied to a tissue recognition system as described above, and the tissue recognition model also includes an optimization module, and the training method includes: obtaining a training data set, wherein the training data set includes training images and annotation data, and the annotation data is used to annotate the real tissue categories in the training images; according to a preset connection probability, in each of the activation modules, the input end of the inhibition unit is randomly connected to the activation units of other activation modules, and the initial weight of each of the inhibition units is set; the training data set is input into the tissue recognition model, and at least one tissue estimation category in the training image is identified by the tissue recognition model; according to the annotation data and the at least one tissue estimation category, the activation units of other activation modules randomly connected to the input end of each inhibition unit and the initial weight of each inhibition unit are adjusted by the optimization module; the adjustment is stopped until the tissue recognition model meets the preset convergence condition, and the first activation unit of a group of other activation modules finally connected to the input end of each inhibition unit and the preset weight of each inhibition unit when performing the inhibition operation are determined.
[0015] A third aspect of the present invention provides a tactile feedback system, which includes a main operating console and a slave operating device, and a processor connected to the main operating console and the slave operating device respectively; the slave operating device includes a surgical instrument connected at the proximal end, which is used to collect image data of in vivo tissue when driving an image acquisition device set at the end of the surgical instrument, and the image data includes multiple groups of partition data; the processor stores a tissue recognition model, and the tissue recognition model includes a recognition module and a plurality of activation modules, the recognition module is connected to each of the activation modules, and each of the activation modules is associated with different groups of partition data respectively; the processor is configured to: obtain image data from the slave operating device; extract image features of each of the partition data, and based on the image features, activate the associated activation module, and determine the activation time; the recognition module generates a relationship diagram of each of the activation modules relative to the activation time, and based on the relationship diagram, identifies at least one tissue category of the in vivo tissue; based on the at least one tissue category, perform tactile feedback on the main operating console.
[0016] The beneficial effects of the present invention are: the image data of the tissue is collected through the end of the surgical instrument, the image data is divided into partition data of multiple local areas, each partition data is converted into an activation signal of the activation module, the activation module is activated, and a relationship diagram is constructed with the activation duration to represent the relative strength of the characteristics of each partition data, which is used to identify one or more tissue types of the tissue in the body. The tactile feedback can be completed directly using the endoscope system to enhance the realism of the surgical operation. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] Figure 1 is a schematic diagram of a usage scenario of a surgical robot in an embodiment of the present invention;
[0018] Figure 2 is a structural schematic diagram of a surgical instrument in an embodiment of the present invention;
[0019] Figure 3 is a structural schematic diagram of the operation of multiple surgical instruments in an embodiment of the present invention;
[0020] Figure 4 is another structural schematic diagram of the operation of multiple surgical instruments in an embodiment of the present invention;
[0021] Figure 5 is a structural diagram of a tissue identification system in an embodiment of the present invention;
[0022] Figure 6 is another structural schematic diagram of the tissue identification system in an embodiment of the present invention;
[0023] Figure 7 is a structural schematic diagram of a tissue identification model in an embodiment of the present invention;
[0024] Figure 8 is another structural schematic diagram of the tissue identification model in an embodiment of the present invention;
[0025] Fig. 9 is a first control schematic diagram of a tissue identification model in an embodiment of the present invention;
[0026] Fig.10 is a second control schematic diagram of the tissue identification model in an embodiment of the present invention;
[0027] Fig.11 is a third control schematic diagram of the tissue identification model in an embodiment of the present invention;
[0028] Fig.12 is a fourth control schematic diagram of the tissue identification model in an embodiment of the present invention;
[0029] Fig.13 is the fifth control schematic diagram of the tissue identification model in the embodiment of the present invention;
[0030] Fig.14 It is a flowchart of a training method for a tissue recognition model in an embodiment of the present invention. DETAILED DESCRIPTION
[0031] In order to facilitate the understanding of the present invention, the present invention is described in more detail below in conjunction with the accompanying drawings and specific embodiments. Preferred embodiments of the present invention are provided in the accompanying drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described in this specification. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present invention more thorough and comprehensive.
[0032] It should be noted that, unless otherwise defined, all technical and scientific terms used in this specification have the same meaning as those understood by a person skilled in the art of the present invention. The terms used in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. For example, the term "plurality" includes two or more.
[0033] To facilitate understanding, the surgical robot is first explained below.
[0034] like Figure 1 , Figure 2 As shown, the surgical robot includes a main operation console 1 and a slave operation device 2. The main operation console 1 is used to send control commands to the slave operation device 2 according to the doctor's operation to control the slave operation device 2, and is also used to display the image obtained from the slave operation device 2. The slave operation device 2 is used to respond to the control command sent by the main operation console 1 and perform corresponding operations, and the slave operation device 2 is also used to obtain images in the body.
[0035] The slave operation device 2 includes a mechanical arm 21, a power mechanism 22 disposed on the mechanical arm 21, a surgical instrument 100 disposed on the power mechanism 22, and a sleeve 24 sleeved with the long axis 100 of the surgical instrument 100. The mechanical arm 21 is used to adjust the position of the surgical instrument 100; the power mechanism 22 is used to drive the surgical instrument 100 to perform corresponding operations, and the end effector 111 of the surgical instrument 100 is used to extend into the body, and perform surgical operations and / or obtain in-body images through the end instrument located at the distal end.
[0036] Specifically, Figure 3 , Figure 4 As shown, the long axis 110 of the surgical instrument 100 passes through the sleeve 23, and the end effector 111 thereof extends out of the sleeve 23 and is driven by the power mechanism 22 to perform operations. Figure 3 In the figure, the area where the long axis 110 of the surgical instrument 100 is located inside the sleeve 23 is a rigid area; Figure 4In the embodiment, the area where the long axis 110 of the surgical instrument 100 is located inside the sleeve 23 is a flexible area, and the sleeve bends along with the flexible area. The sleeve 23 may also be omitted.
[0037] In order to provide a satisfactory sterile environment during surgical robot surgery, it is necessary to isolate sterile instruments and sterile instruments. Generally, the robotic arm 21 and power mechanism 22 of the operating device 2 are sterile, and the surgical instrument 100 needs to be sterile. A sterile coupling device needs to be configured between the sterile power mechanism 22 and the sterile surgical instrument 100 to isolate the sterile power mechanism 22 and the sterile surgical instrument 100.
[0038] In addition, for the surgical instrument 170, the end effector is set as an image acquisition device to obtain in vivo images / image data for observing in vivo scenes. Without adding additional hardware, the image data obtained by the surgical instrument can also be used to further identify tissue types, and then determine the texture and hardness of the tissue, and feedback to the main hand end or display prompts.
[0039] The above describes the surgical robot in the application scenario of the tissue recognition system. The following describes the first embodiment of the tissue recognition system in the present invention, as shown below:
[0040] like Figure 5 As shown, the tissue recognition system includes: a surgical instrument 510, including an image acquisition device 520 at the end, wherein the image acquisition device 520 is used to acquire image data M of the body tissue, and the image data M includes multiple groups of partition data m; a processor, storing a tissue recognition model 530, the tissue recognition model includes a recognition module 531 and multiple activation modules 532, the recognition module 531 is connected to each of the activation modules 532, and each of the activation modules 532 is respectively associated with different groups of partition data m;
[0041] The processor is configured to: extract image features of each of the partition data m, and based on the image features, activate the associated activation module 532 and determine the activation duration; the identification module 531 generates a relationship graph of each of the activation modules 532 relative to the activation duration, and based on the relationship graph, identify at least one tissue category of the in vivo tissue.
[0042] In addition, the tissue identification system may also include a cannula 540, which is placed in the patient's body through an incision 550 on the patient's body surface; the surgical instrument 510 enters the patient's body through the internal channel of the cannula 540, and the image acquisition device 520 at the end is extended to a position where the internal tissue can be observed.
[0043] In this embodiment, for in vivo tissues such as a target tissue H (other surrounding tissues are not shown), each partition data may include only the image of the target tissue H, or only the image of other surrounding tissues, or include part of the target tissue H and part of other surrounding tissues at the same time.
[0044] In this embodiment, the image features of each partition data may specifically include: color, light intensity, contrast, color difference, texture and other features, and multiple image features preset activation algorithms jointly affect the activation of the activation module. For example, y=(a1, a2, ..., ai)f(x), where y represents the degree of activation of the activation module, f(x) represents the activation function, and (a1, a2, ..., ai) represents the influence coefficient of i image features.
[0045] In this embodiment, the image features of each partition data are different. When the activation module is activated by the activation algorithm, the activation time of each activation module is different by adjusting the influence coefficient, so as to distinguish the type of each partition data. For example, for each partition data including only the target tissue H, or each partition data including only other surrounding tissues, the activation time of the activation module is basically the same; while for the partition data including both the target tissue H and other surrounding tissues, the activation time of the activation module is between the activation time of the first two.
[0046] Specifically, the activation process of the activation module refers to: when the activation module receives a voltage / current signal, a potential is generated, and when the potential threshold is reached, an activation signal is output. Therefore, the aforementioned (a1, a2, ..., ai) f (x) refers to the voltage / current input, y refers to the potential value, and the activation duration refers to the duration from when the activation module receives the voltage / current input to when it outputs the activation signal. That is, the partition data of different tissue categories have different image features, which makes the voltage / current input size different, resulting in different activation durations for the activation module to reach the potential threshold and output the activation signal.
[0047] For example, Figure 6 As shown, based on all the partition data of row i and column j, each activation module 610 arranged in sequence is associated, where the association can be in column order or row order, where i, j∈1,2,... The ijth partition data activates the ijth activation module and records the activation duration. A relationship graph 630 is generated in the identification module 620, and index values 1, 2,..., k are assigned as ordinates in the order of the 11th activation module to the ijth activation module; the activation duration is represented by unit time ts as the horizontal coordinate; thereby generating a relationship graph 630 of the activation module relative to the activation duration.
[0048] The visualization state of in vivo tissues is usually known, including tissues such as fat, muscle, lesions, and cortex, as well as tissues that have been treated by ultrasonic scalpels, electrosurgery, argon-helium scalpels, water jets, laser scalpels, etc., and their image features, such as color, light intensity, contrast, degree of color difference, texture, etc., are all different. At the same time, the hierarchy of each tissue type is basically fixed, so for the tissue categories contained in the collected image data, the relative positions between various tissue categories can be further used to assist in the identification of tissues.
[0049] Specifically, the image data may be an image representation of the collected in vivo tissue, and the partition data may be each image partition in the image; the visualization state of each image partition may be determined by the activation duration; at the same time, an activation module associated with each image partition is assigned an index value according to the division order of each image partition in the image, and the relative position between each image partition may be determined according to the index value size of each activation module.
[0050] The tissue identification system of the present invention is generally described above. The structure of the activation module is specifically described below, and a second embodiment of the tissue identification system of the present invention is provided, which is specifically as follows:
[0051] like Figure 7 As shown, in the tissue identification system, the activation module 700 includes an activation unit 710 and a plurality of inhibition units 720, and each activation module is composed of the same units; the input end of each inhibition unit is connected to the first activation unit of a group of other activation modules, such as the input end of the inhibition unit 730 is connected to the first activation unit 710, the first activation unit 740 and the first activation unit 750 of a group of other activation modules; the output end of each inhibition unit 720 is connected to the second activation unit 710 of the same activation module;
[0052] The first activation unit and the second activation unit are named based on the different connections with the input and output ends of the inhibition unit; relatively speaking, when one activation unit is the second activation unit, other activation units are the first activation units if they are connected to the input end of the inhibition unit.
[0053] The processor is configured to: perform an excitation operation on the activation process of the activation unit in the associated activation module based on the image features; and the inhibition unit performs an inhibition operation on the activation process of the second activation unit in response to the completion of the activation of the same group of activation units connected to the input end.
[0054] In this embodiment, image features such as color, light intensity, contrast, degree of color difference, texture, etc. are used in an activation algorithm such as: y=(a1, a2, ..., ai)f(x) to calculate the y value, and perform an excitation operation on the activation unit according to the y value; for example, an activation potential is preset for each activation unit, the y value is a voltage / current value, and the excitation operation is performed with the dynamic input of the y value, and the activation unit is activated when the preset activation potential is reached.
[0055] For example, for the influence coefficient a1, it is dynamically adjusted by the three dimensions of the red component R, green component G, and blue component B of the color in the image feature, a1=q1*R+ q2*G+ q3*B, for example: a1=0.1*R+0.3*G+0.6B; the light intensity, contrast, and color difference are dynamically adjusted in direct proportion to the influence coefficients a2, a3, and a4; similarly, the influence coefficients of other image features can be set. The activation function f(x) can be a positive growth function, and multiple coefficients affect the growth rate of the positive growth function. Ultimately, the activation algorithm y=(a1, a2, ..., ai)f(x) increases the voltage / current value at different growth rates according to the image features, activates the preset activation potential of each activation unit, completes the excitation operation, and distinguishes different local tissue types according to different excitation durations.
[0056] In this embodiment, the suppression operation of the suppression unit is used to denoise the tissue representation of the graphic data, so as to improve the recognition of tissue categories when there is noise in the in vivo tissue represented by the graphic data, and enhance the robustness of the recovery of the in vivo tissue's own characteristics when there is occlusion of the in vivo tissue.
[0057] For example, Figure 7 As shown, when the activation units 710, 740 and 750 of the same group are activated, the suppression unit 730 suppresses the activation of the activation unit 760; when the activation units 710 and 760 of the same group are activated, the suppression unit 770 suppresses the activation of the activation unit 740, and the suppression unit 780 suppresses the activation of the activation unit 750. The activation duration is extended by suppressing the activation.
[0058] In addition, activation unit 710, activation unit 710, activation unit 710, activation unit 710, activation unit 710 are activated under the excitation and inhibition of image features and inhibition units, and send signals to recognition module 790, determine the activation duration according to the signal reception time interval of each activation unit, and receive a relationship diagram between each activation unit and the activation duration.
[0059] In this embodiment, within the preset range of the activation sequence, if the activation units in the same group have all been activated, an inhibition operation is performed on the activation process of the second activation unit through an inhibition unit connected to the activation units in the same group; wherein the preset activation sequence is sorted according to the length of the activation duration.
[0060] Specifically, sorting by the length of the activation duration can be expressed as: each activation unit is sorted in the order of the first activation completed, the second activation completed, ..., the nth activation completed. The preset range of the activation sequence is the first activation unit to complete the activation, to the i-th activation unit, i∈1, 2, ..., n. When the activation units in the same group are the first to the i-th activation units that have completed the activation, their commonly connected inhibition units will be activated to perform an inhibition operation on the second activation unit. Preferably, the preset range of the activation sequence is 1-5, that is, the first to the fifth activation units that have completed the activation, if they are in the same group, the commonly connected inhibition units are activated and the inhibition operation is performed.
[0061] For example, Figure 7 As shown, if activation unit 710 is the first activation completed, activation unit 740 is the i-1th activation completed, and activation unit 750 is the i+1th activation completed, then inhibition unit 730 will not be activated, and the inhibition operation will not be performed on the activation of activation unit 760; if activation unit 710 is the first activation completed, and activation unit 760 is the i-1th activation completed, then inhibition unit 770 and inhibition unit 780 are activated, and inhibition operations are performed on the activation of activation unit 740 and activation unit 750, respectively.
[0062] In this embodiment, the multiple inhibition units are divided into multiple groups, wherein each group of inhibition units represents a different scene category; the processor is configured to: in response to the completion of activation of each of the activation units one by one, dynamically select an inhibition unit group of at least one scene category; and use the dynamically selected inhibition unit group to perform an inhibition operation on the activation of the second activation unit, wherein the inhibition operation is used to extend the activation duration.
[0063] In this embodiment, each group of inhibition units under each scene category is pre-trained to indicate the scene where different types of tissues appear in the same image data. For example, the image data may represent one or more of the tissues such as fat, muscle, lesion, and cortex, or may represent at least one tissue after surgical treatment by ultrasonic scalpel, electric scalpel, argon-helium scalpel, water scalpel, laser scalpel, etc., and a combination of the above-mentioned at least one normal tissue.
[0064] Specifically, for each scene category, a group of inhibition units is constructed to perform inhibition operations on activation units. For example, in each activation module, the number of inhibition units in each group is 5. If the tissue recognition network contains 81 activation modules, the number of activation units in a group is 405. If there are 100 scene categories, the total number of inhibition units is 40500. Unselected inhibition unit groups will not be activated to perform inhibition operations.
[0065] Specifically, as each activation unit is activated one by one, each group of inhibition units presets an activation duration interval. When the activated activation unit is in a certain activation duration interval, at least one group of inhibition units including the activation duration interval is selected; the at least one group of inhibition units is applied to perform the inhibition operation. The inhibition unit group applied later needs to be selected from the inhibition unit group applied earlier, and so on, until all activation units are activated. Exemplarily, the corresponding activation duration interval is set based on each of the aforementioned organizations.
[0066] In another embodiment, each group of inhibition units will be activated without dynamic selection, and inhibition operations are performed on the activation units respectively, and multiple groups of activation durations output by the activation units under the inhibition operations of different groups of inhibition units are recorded.
[0067] For example, Figure 8 One of the activation modules shown includes at least suppression unit group 810, suppression unit group 820, suppression unit group 830, and activation unit 840; according to the suppression operation of suppression unit group 810 on activation unit 840, an activation duration 850 is output; according to the suppression operation of suppression unit group 820 on activation unit 840, an activation duration 860 is output; according to the suppression operation of suppression unit group 830 on activation unit 840, an activation duration 870 is output.
[0068] Multiple relationship graphs generated based on the activation durations of different groups are merged, and the multiple relationship graphs are merged into one relationship graph using methods such as mean processing, maximum processing, and minimum processing.
[0069] In another embodiment, the five inhibition units in each activation module are activated one by one according to the activation sequence, and are respectively connected to the first activation units of other activation modules in different ways, each of which corresponds to a group of inhibition units.
[0070] The above specifically describes the activation module structure of the tissue identification system in the present invention. The following specifically describes the activation process of the activation module and provides a third embodiment of the tissue identification system in the present invention, which is specifically as follows:
[0071] like Fig. 9In the example, the activation module 910 is provided with a standard waveform 911, and the standard waveform 911 is in a standard time scale ts; the processor is configured to: based on the image feature 912, according to the exposure time scale T of the image acquisition device, control the potential of the activation module 910 to increase along with the standard waveform 911; when the potential reaches the peak 913 of the standard waveform, record the current time under the standard time scale ts to obtain the activation duration of the activation module 910, such as the position of the activation module identification point 921 in the relationship diagram 920.
[0072] Exemplarily, the standard waveform may be a gamma waveform, which simulates the feedback of brain electrical nerves to visual input.
[0073] In this embodiment, according to the image feature 912, according to the preset conversion relationship, the potential growth curve 914 of each partition data under the exposure time scale is determined; based on the potential growth curve 914, the potential of the activation module 910 is controlled to increase with the standard waveform 911. The preset conversion relationship can be the setting of (a1, a2, ..., ai) in the aforementioned activation algorithm f=(a1, a2, ..., ai)f(x). The ordinate of the potential growth curve 914 refers to the increased potential amount V.
[0074] Among them, the standard time scale ts and the exposure time scale T are independent of each other and will not affect each other. When the activation module 910 starts to increase the potential along with the standard waveform 911, the duration starts to be calculated in the standard time scale ts; when the potential increases to the peak 913, the activation duration of the potential increase is determined. As for different activation modules starting to increase the potential at different time points under the exposure time scale T, it does not affect the activation duration determined under the standard time scale ts.
[0075] Specifically, the increasing speed of the potential of the activation module is related to the growth speed of the potential growth curve. Exemplarily, for image feature 1010, image feature 1020, and image feature 1030, after a preset conversion relationship, potential growth curves 1011, 1021, and 1031 with potential growth speeds from low to high are obtained respectively; at t1 under the exposure time scale T, activation module 1012, activation module 1022, and activation module 1032 respectively increase to the positions of potential 1013, potential 1023, and potential 1033 of the standard waveform; potential 1013 is the lowest, potential 1023 is relatively high, and potential 1033 is the highest and has reached the peak.
[0076] Further, if activation module 1012, activation module 1022, and activation module 1032 are activated respectively, it can be seen in relationship graph 1040 that activation duration ts of activation module 1012 is the longest, activation module 1022 is the second, and activation duration ts of activation module 1032 is the shortest. Activation module 1012, activation module 1022, and activation module 1032 can be assigned index values 1, 2, and 3 respectively to generate relationship graph 1040, 1, 2, 3∈k.
[0077] In this embodiment, Fig.11 As shown, for the potential growth curve 1110 of a certain image feature, when the activation module 1120 is activated along with the standard waveform 1120, the standard waveform 1121 includes multiple segments 1122, each of which includes a rising segment 1123 and a reset segment 1124, and the peak is at the connection between the rising segment 1123 and the reset segment 1124 in each of the segments 1122.
[0078] The processor is also configured to: control the potential of the activation module 1120 to increase with the rising section 1123, and control the activation module 1120 to reset the potential and extend the activation time in the reset section 1124; in each of the segments 1122, add the extended activation time to the activation time when the potential reaches the peak; and determine the final activation time according to the superimposed time corresponding to each of the segments 1121.
[0079] Specifically, the activation of the activation module 1120 is divided into multiple cycles, each cycle is shown in the relationship diagram 1130, including an activation period 1131 and a suppression period 1132:
[0080] A. In the activation period 1131, when the interval 1111 of the potential growth curve 1110 is input into the activation module, the potential of the activation module 1120 is controlled to grow to a peak along with the rising section 1123;
[0081] B. In the inhibition period 1132, since the potential of the reset segment 1124 drops to the reset potential, when the activation module is input in the interval 1112 of the potential growth curve 1110, the potential of the activation module 1120 also drops to the reset potential, so it has no effect on the activation time of the activation module 1120.
[0082] Preferably, each activation period 1131 and each inhibition period 1132 can be set to 10ts, that is, 10 unit durations, each unit duration can be set to 5ms, and each activation period 1131 and each inhibition period 1132 lasts for 50ms.
[0083] In addition, the control activation module 1120 extends the activation time in the reset section 1124, such as Fig.12As shown, during the activation of the suppression unit in the activation module 1120, the activation module 1120 continuously generates a constant voltage high potential, forms a pulse signal graph 1210 under multiple cycles of activation, and performs a suppression operation based on the pulse signal graph 1210:
[0084] A. In the activation period 1131, although the suppression unit outputs a high level 1211, the activation module 1120 at this time is based on the potential growth curve, and the potential increases along with the rising section 1123; therefore, the high level 1211 output by the suppression unit at this time will not have an inhibitory effect on the activation of the activation module 1120.
[0085] B. In the inhibition period 1132, the high level 1212 output by the inhibition unit replaces the level reset of the reset section 1124, inhibiting the activation of the activation module 1120. As the high level 1212 lasts in the inhibition period 1132, the activation module 1120 is controlled to extend the activation duration.
[0086] In addition, the extension duration of each inhibition operation on the activation duration is determined according to the preset weight of each inhibition unit. The preset weight and the extension duration may be in direct proportion, and the larger the weight, the longer the extension duration; or the extension duration may be determined according to the weight in other preset functional relationships. That is, as the preset weight increases, the activation time of the control inhibition unit is longer, and the activation module 1120 continues to generate a constant voltage high potential for a longer time, and the activation duration is longer.
[0087] In another embodiment, if Fig.13 As shown, the activation module includes an inhibition unit 1310 and an activation unit 1320; the excitation operation performed by the activation unit 1320 and the inhibition operation performed by the inhibition unit 1310 are both performed in the aforementioned activation period, and only the potential reset is performed in the inhibition period; when the inhibition unit 1310 is activated, the inhibition operation is triggered, and at the beginning of the activation period, the potential of the activation unit 1320 will be suppressed to maintain it at the original potential level, and the potential level will not increase with the preset standard waveform, that is, the potential level of the interval 1311-1321 is maintained until the inhibition operation of the inhibition unit 1310 is completed, and the excitation operation of the activation unit 1320 is started at position 1321, and the potential level increases with the preset standard waveform.
[0088] In this embodiment, in addition to extending the start excitation operation time of the activation unit 1320 by the suppression unit 1310, the peak duration of the activation unit 1320 can also be extended to extend the activation duration. The activation unit 1320 starts to perform the excitation operation at position 1331, so that the potential reaches the peak position 1341; then, the potential input of the suppression unit 1310 is received at the peak position 1341, and the peak is maintained according to the duration of the potential input, until the suppression unit 1310 stops the potential input, and the peak falls back at position 1342.
[0089] In one embodiment, a preset reference relationship graph is obtained; the similarity between the reference relationship graph and the relationship graph is calculated; and at least one tissue category of the in vivo tissue is determined based on the similarity.
[0090] In this embodiment, at least one corresponding reference relationship graph is stored under different scene categories. For the relationship graph determined by the inhibition units of multiple groups of different scene categories, the similarity is calculated for the reference relationship graph and the relationship graph under the same scene category respectively; then the reference relationship graph with the largest similarity is taken, and according to the organizational category set for each activation duration interval, it is set as the organizational category of the activation duration interval corresponding to the relationship graph; each index value determines the organizational category accordingly and is mapped to the partition data, and each partition data is each image partition in the image representation.
[0091] Furthermore, adjacent regions are connected and their boundaries are defined, and the tissue category is marked for each connected region in the overall image data. A bounding box may also be set for each connected region, and the bounding box is marked with the tissue category.
[0092] In one embodiment, the relationship graph is divided into multiple local segments according to multiple preset duration intervals of the activation duration; the local similarity between each of the local segments and the reference relationship graph is matched, and based on each of the similarities, the similarity between the reference relationship graph and the relationship graph is obtained.
[0093] In this embodiment, the local segments divided by each preset time interval are compared respectively, which means that for each tissue category in the image data, you may not care about its relative position, but only care about the relative activation time between each tissue category and its own activation time. The similarity here is expressed as: in each preset time interval, the activation time of each activation module in the reference relationship diagram and the activation time of each activation module in the relationship diagram are calculated. Specifically, the difference between the two can be determined by one of the mean difference and variance, or further combined with covariance.
[0094] In another embodiment, multiple curves are fitted to local segments of each preset time interval, and an overall curve is fitted to the reference relationship graph; the relative positions of the multiple curves are not considered, and matching is performed in the overall curve, and the matching of each curve on the overall curve does not completely overlap, and the overlap ratio of each curve and the overall curve is determined based on the matching results to obtain the similarity.
[0095] Furthermore, since the image data basically contains a fixed part of the surgical instrument, when calculating the similarity, the part of the surgical instrument in the relationship diagram can be first matched with the reference relationship diagram, and then the similarity between other parts of the body tissue in the relationship diagram and the reference relationship diagram can be further calculated.
[0096] The tissue recognition system of the present invention is described in detail above. The following is a detailed description of the training process of the tissue recognition model. Fig.14 , provides an embodiment of the training method of the tissue recognition model in the present invention, which is specifically as follows:
[0097] S141, obtaining a training data set, wherein the training data set includes a training image and annotation data, and the annotation data is used to annotate the real tissue category in the training image;
[0098] In this embodiment, the training image includes a real endoscopic image acquired by an image acquisition device, and may also include a virtual endoscopic image acquired by a virtual endoscope in a three-dimensional tissue model; the annotation data includes annotations of real tissue categories in the training image, specifically including the boundaries between each part of the tissue in the training image and the tissue category marked in each boundary area. The annotation data may also specifically include the bounding box of each part of the tissue in the training image and the tissue category marked in each bounding box.
[0099] Specifically, the training data set may include one or more sets of training data; each set of training data corresponds to a scene category of an application scenario, and the same application scenario contains the same one or more tissue categories; for example, one application scenario contains only fat, and another application scenario contains fat and fascia. The size and relative position of different tissue category areas in the same application scenario can be further marked or not.
[0100] In addition, when the training data set contains multiple groups of training data, after learning each group of training data, the next group of training data is learned, and so on.
[0101] S142, randomly connecting the input end of the inhibition unit with the activation units of other activation modules in each activation module according to a preset connection probability, and setting an initial weight of each inhibition unit;
[0102] In this embodiment, in a training cycle, the input end of each inhibition unit is randomly connected to the activation units of other activation modules according to a preset connection probability. For example, the preset connection probability is 0.2, that is, each inhibition unit has a 20% probability of being randomly connected to the activation units of each other activation module. Statistically, each inhibition unit is expected to be connected to 20% of the other activation units.
[0103] In addition, if the activation unit in the same inhibition module is connected to multiple inhibition units, each inhibition unit is randomly connected to other activation units according to the preset connection probability, which means that one activation unit can receive inhibition operations from multiple inhibition units at the same time, increasing the depth of the tissue recognition network, improving the feature fitting ability, making it easier to learn the weight of each inhibition unit, improving the robustness of the network, and reducing the impact of noise on the recognition results.
[0104] In this embodiment, the initial weight of each inhibition unit is set to determine the extension duration of the inhibition operation on the activation duration. As the training cycle increases, the initial weight can be set according to the trend of increasing weight in the early stage, such as setting the initial weight to 0.1 and the extension duration to 1ms; in the later stage, it can be dynamically adjusted in a cross-increase manner until the tissue recognition network converges.
[0105] S143, inputting the training data set into the tissue recognition model, and identifying at least one tissue estimation category in the training image through the tissue recognition model;
[0106] In this embodiment, based on other activated units randomly connected to the inhibition unit in each training cycle and the set initial weights, at least the tissue estimation category of the training images in a set of training data is estimated.
[0107] Each random connection method between the inhibitory unit and other activation units corresponds to an application scenario. It is estimated that after a set of training data is learned, the other activation units and initial weights randomly connected to each inhibitory unit can be reset, and then the next set of training data can be learned. In order to improve the computational efficiency of model application, each set of training data corresponds to a set of inhibitory units, and the connection between each set of inhibitory units and other activation units remains fixed after learning.
[0108] S144, adjusting, by the optimization module, activation units of other activation modules randomly connected to the input end of each of the inhibition units and an initial weight of each of the inhibition units according to the labeled data and the at least one tissue estimation category;
[0109] In this embodiment, in each training cycle, the loss between the estimated category of the identified tissue and the labeled data is calculated; according to back propagation / gradient descent, the initial weight of each inhibition unit is adjusted according to the loss, so that the relationship graph between the activation module and the activation duration can better represent the feature combination of the corresponding application scenario and is more accurate in identifying the tissue category.
[0110] Specifically, the training image contains multiple partitions, and the tissue category of each partition includes two cases: 1) the partition belongs to a specific tissue category, 2) the partition belongs to the boundary of two specific tissue categories; the latter sets the proportion of two specific tissue categories, and calculates the intersection of union (IOU) between the tissue estimation category and the same tissue category in the annotation data. When the IOU is lower than the threshold, the first activation unit randomly connected to the inhibition unit and the initial weight are adjusted through the optimization module. The preset threshold can be 0.7.
[0111] In another embodiment, in the optimization module, a learning strategy such as STDP (Spike Timing dependent plasticity) can also be used to adjust the initial weight of each inhibitory unit, wherein the potential growth line between the activation unit and the image feature belongs to the excitatory synapse, transmitting presynaptic pulses, and the number of activation units is the number of excitatory synapses; the line between the inhibition unit and the activation unit in the same activation module belongs to the inhibitory synapse, and the activation unit is controlled to transmit postsynaptic pulses through the inhibitory operation, and the number of inhibition units is the number of inhibitory synapses.
[0112] Specifically, the weight of the corresponding inhibitory unit is adjusted according to the time difference between the presynaptic pulse emission and the postsynaptic pulse emission; in each training cycle, among the other randomly connected activation units, the degree of influence of each other activation unit on the inhibitory unit is continuously adjusted, as well as the initial weight.
[0113] Among them, during the adjustment process, the influence of some other activation units on the inhibition unit is attenuated to zero; so that among the other randomly connected activation units, another part of other activation units are retained to maintain a connection relationship with the inhibition unit, and the inhibition unit depends on the activation of the other part of other activation units in the application stage to perform the inhibition operation.
[0114] S145. Stop adjusting until the tissue recognition model meets the preset convergence condition, determine the first activation unit of a group of other activation modules that are finally connected to the input end of each of the inhibition units, and the preset weight of each of the inhibition units when performing the inhibition operation.
[0115] In this embodiment, when the tissue recognition model is trained by loss calculation and back propagation / gradient descent, when the loss reaches a threshold or reaches a preset number of training times, it is determined that the convergence condition is met. When the tissue recognition model is trained by the STDP learning strategy, it is determined that the convergence condition is met when the preset number of training times is reached.
[0116] The tissue recognition system of the present invention is described above. The application scenario of the tissue recognition system of the present invention is described below, and an embodiment of the tactile feedback system of the present invention is provided, which is specifically as follows:
[0117] When the tissue recognition system of the present invention is applied to tactile feedback, the tactile feedback system includes a master operation console and a slave operation device, and a processor connected to the master operation console and the slave operation device respectively; the slave operation device includes a surgical instrument connected at the proximal end, which is used to collect image data of in vivo tissue when driving an image acquisition device set at the end of the surgical instrument, and the image data includes multiple groups of partition data; the processor stores a tissue recognition model, and the tissue recognition model includes a recognition module and multiple activation modules, the recognition module is connected to each of the activation modules, and each of the activation modules is respectively associated with different groups of partition data;
[0118] The processor is configured to: acquire image data from the slave operating device; extract image features of each of the partition data, and based on the image features, activate the associated activation module and determine the activation duration; the identification module generates a relationship graph of each of the activation modules relative to the activation duration, and based on the relationship graph, identifies at least one tissue category of the in vivo tissue; and provide tactile feedback to the main operating console based on the at least one tissue category.
[0119] In this embodiment, the texture and hardness of the tissue are determined according to the tissue type, and a prompt can be displayed on the display screen according to the texture and hardness of the tissue. In addition, when operating surgical instruments such as surgical scissors, suture needles, and sutures to operate on the target tissue, the texture and hardness of the tissue are converted into mechanical parameters and act on the main hand end, providing the doctor with a real surgical touch at the main operating table.
[0120] It should be noted that other sorting schemes that can be easily thought of by those skilled in the art within the technical scope disclosed in the present invention should also be within the protection scope of the present invention and will not be described in detail here.
[0121] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example for illustration. In actual applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the mobile terminal can be divided into different functional units or modules to complete all or part of the functions described above. The functional modules in the embodiment can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the modules in the above-mentioned mobile terminal can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.
[0122] The embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.
[0123] An embodiment of the present application provides a computer program product. When the computer program product runs on a mobile terminal, the mobile terminal can implement the steps in the above-mentioned method embodiments when executing the computer program product.
[0124] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.
[0125] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.
[0126] As used in the specification of this application and the appended claims, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting" depending on the context. Similarly, the phrase "if it is determined" or "if the described condition or event is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce the described condition or event is detected" or "in response to detecting the described condition or event" depending on the context.
[0127] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.
[0128] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.
[0129] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.
[0130] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0131] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.
[0132] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0133] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electric carrier signals and telecommunication signals.
[0134] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention.
Claims
1. A tissue identification system, It is characterized in that The tissue identification system comprises: A surgical instrument, comprising an image acquisition device at the distal end, wherein the image acquisition device is used to acquire image data of in vivo tissue, the image data comprising multiple sets of zone data; A processor storing a tissue recognition model, wherein the tissue recognition model includes a recognition module and a plurality of activation modules, wherein the recognition module is connected to each of the activation modules, and each of the activation modules is respectively associated with different groups of partition data; The processor is configured to: Extracting image features of each partition data, and activating the associated activation module based on the image features, and determining the activation duration; The identification module generates a relationship graph of each of the activation modules relative to the activation duration, and identifies at least one tissue category of the in vivo tissue based on the relationship graph.
2. The tissue identification system according to claim 1, It is characterized in that The activation module includes an activation unit and a plurality of inhibition units; the input end of each of the inhibition units is connected to a first activation unit of a group of other activation modules, and the output end of each of the inhibition units is connected to a second activation unit of the same activation module; The processor is configured to: Based on the image features, performing an excitation operation on an activation process of an activation unit in an associated activation module; The suppression unit performs a suppression operation on an activation process of the second activation unit in response to completion of activation of the first activation unit.
3. The tissue identification system according to claim 2, It is characterized in that The processor is configured to: Within the preset range of the activation sequence, if the first activation units are all activated, a suppression operation is performed on the activation process of the second activation unit through a suppression unit connected to the first activation unit; The preset activation timings are sorted according to the length of the activation durations.
4. The tissue identification system according to claim 2, It is characterized in that The plurality of inhibition units are divided into a plurality of groups, wherein each group of inhibition units represents a different scene category; The processor is configured to: In response to each of the activation units being activated one by one, dynamically selecting a group of inhibition units of at least one scene category; Using the dynamically selected inhibition unit group, an inhibition operation on the activation of the second activation unit is performed, wherein the inhibition operation is used to extend the activation duration.
5. The tissue identification system according to claim 4, It is characterized in that The processor is further configured to: The extension duration of the activation duration by each suppression operation is determined according to the preset weight of each suppression unit.
6. The tissue identification system according to claim 1, It is characterized in that The activation module is provided with a standard waveform diagram, and the standard waveform diagram is in a standard time scale; The processor is configured to: Based on the image features and in accordance with the exposure time scale of the image acquisition device, controlling the potential of the activation module to increase along with the standard waveform diagram; When the potential reaches the peak of the standard waveform, the current time under the standard time scale is recorded to obtain the activation duration of each activation module.
7. The tissue identification system according to claim 6, It is characterized in that The standard waveform diagram includes a plurality of segments, each of which includes a rising segment and a reset segment, and the wave peak is located at the connection between the rising segment and the reset segment in each segment; The processor is further configured to: Controlling the potential of the activation module to increase gradually along the rising section, and controlling the activation module to reset the potential and extend the activation time in the reset section; In each of the segments, the extended activation duration is superimposed on the activation duration when the potential reaches the peak; The final activation duration is determined according to the superimposed duration corresponding to each of the segments.
8. The tissue identification system according to claim 6, It is characterized in that The processor is further configured to: According to the image features and a preset conversion relationship, determining a potential growth curve of each partition data at the exposure time scale; Based on the potential growth curve, controlling the potential of the activation module to increase gradually along with the standard waveform diagram; The increasing speed of the potential of the activation module is related to the increasing speed of the potential growth curve.
9. The tissue identification system according to claim 1, It is characterized in that The processor is further configured to: Obtaining a preset reference relationship diagram; Calculating the similarity between the reference relationship graph and the relationship graph; At least one tissue category of the in vivo tissue is determined according to the similarity.
10. The tissue identification system according to claim 9, It is characterized in that The processor is further configured to: Dividing the relationship graph into a plurality of local segments according to a plurality of preset duration intervals of the activation duration; The local similarities between each of the local segments and the reference relationship graph are matched, and the similarity between the reference relationship graph and the relationship graph is obtained according to each of the similarities.
11. A method for training a tissue recognition model, wherein the tissue recognition model is applied to a tissue recognition system as claimed in any one of claims 1 to 10, It is characterized in that The tissue recognition model also includes an optimization module, and the training method includes: Acquire a training data set, wherein the training data set includes training images and annotation data, and the annotation data is used to annotate real tissue categories in the training images; According to a preset connection probability, in each of the activation modules, the input end of the inhibition unit is randomly connected to the activation units of other activation modules, and the initial weight of each of the inhibition units is set; Inputting the training data set into the tissue recognition model, and identifying at least one tissue estimation category in the training image by the tissue recognition model; According to the labeled data and the at least one tissue estimation category, adjusting, by the optimization module, activation units of other activation modules randomly connected to an input end of each of the inhibition units, and an initial weight of each of the inhibition units; The adjustment is stopped until the tissue recognition model meets the preset convergence condition, and the first activation unit of a group of other activation modules finally connected to the input end of each inhibition unit and the preset weight of each inhibition unit when performing the inhibition operation are determined.
12. A tactile feedback system, It is characterized in that The tactile feedback system includes a master operating console and a slave operating device, and a processor connected to the master operating console and the slave operating device respectively; The slave operation device includes a surgical instrument connected at the proximal end, which is used to collect image data of in vivo tissue when driving an image acquisition device provided at the distal end of the surgical instrument, wherein the image data includes multiple sets of zone data; The processor stores a tissue recognition model, the tissue recognition model includes a recognition module and a plurality of activation modules, the recognition module is connected to each of the activation modules, and each of the activation modules is respectively associated with different groups of partition data; The processor is configured to: Acquiring image data from the slave operating device; Extracting image features of each partition data, and activating the associated activation module based on the image features, and determining the activation duration; The identification module generates a relationship diagram of each of the activation modules relative to the activation duration, and identifies at least one tissue category of the in vivo tissue based on the relationship diagram; Based on the at least one tissue category, tactile feedback is provided to the main operating console.
