Flexible instrument detection methods, devices, surgical systems, equipment and media

Abstract

The present invention provides a detection method, device, surgical system, equipment and medium for a flexible instrument, the method comprising: obtaining shape data of the flexible instrument detected by the shape sensor; obtaining designated information of a designated part of the flexible instrument detected by a designated sensor, wherein the designated information can at least characterize the position of the designated part; and determining the target position of the flexible instrument in a real physiological channel based on the designated information and the shape data.

Description

Technical Field

[0001] The present invention relates to the medical field, and in particular to a detection method, device, surgical system, equipment and medium for flexible instruments. Background Art

[0002] In medical activities, the flexible device can be inserted into a physiological channel (such as the trachea) and moved along the physiological channel to the lesion area, and after reaching the lesion area, treatment, detection and other activities are carried out. During the movement of the flexible device, the flexible device needs to be positioned.

[0003] In the existing related technologies, sensors can be configured in flexible devices to collect the motion trajectory of the sensors, and then the motion trajectory is aligned with the complete drawing of the physiological channel to locate the position of the flexible device. However, in this process, it is difficult to accurately and effectively collect continuous motion trajectories, which leads to poor positioning accuracy. Summary of the invention

[0004] The present invention provides a detection method, device, surgical system, equipment and medium for a flexible instrument to solve the problem of poor positioning accuracy.

[0005] According to a first aspect of the present invention, a detection method of a flexible instrument is provided, wherein the flexible instrument is located in a real physiological channel, and the flexible instrument is provided with a shape sensor and a designated sensor at a designated position;

[0006] The method comprises:

[0007] Acquiring shape data of the flexible device detected by the shape sensor;

[0008] Acquire designated information of a designated part of the flexible device detected by a designated sensor, wherein the designated information can at least represent the position of the designated part;

[0009] Based on the designated information and the shape data, a target position of the flexible instrument in a real physiological channel is determined.

[0010] Optionally, determining a target position of the flexible device in a real physiological channel based on the specified information and the shape data includes:

[0011] Based on the specified information, the shape data is modified to obtain modified shape data;

[0012] Based on the corrected shape data, the target position of the flexible instrument in the real physiological channel is determined.

[0013] Optionally, the shape sensor is used to detect curvature information of multiple parts of the flexible instrument, and the shape data includes: positions of multiple instrument position points constructed based on the curvature information of the multiple parts in a first coordinate system based on the shape sensor; the shape of the flexible instrument matches the distribution shape of the multiple instrument position points;

[0014] The designated sensor is a magnetic sensor, and the magnetic sensor is arranged in the magnetic field generated by the magnetic field generator. The designated information includes: the position of one or more designated position points in a second coordinate system based on the magnetic field generator;

[0015] Based on the specified information, the shape data is corrected to obtain the corrected shape data, including:

[0016] Projecting the designated position point and / or the instrument position point to a target coordinate system to obtain a first position point corresponding to the instrument position point and a second position point corresponding to the designated position point; both the first position point and the second position point are in the target coordinate system;

[0017] Among all the first position points, the first position points within the specified range determined based on the second position are screened out, and the positions of the first position points that are not screened out and the second position points in the target coordinate system are determined as the corrected shape data.

[0018] Optionally, determining a target position of the flexible device in a real physiological channel based on the corrected shape data includes:

[0019] The target position is determined by comparing the corrected shape data with the shape of a pre-established virtual physiological channel.

[0020] Optionally, determining the target position by comparing the corrected shape data with the shape of a pre-established virtual physiological channel includes:

[0021] Determining the shape of the navigation path in the virtual physiological channel; the navigation path is: the path of the flexible instrument moving to the lesion area after entering the real physiological channel;

[0022] By comparing the modified shape data with the shape of the navigation path, matching a target path segment in the navigation path, the shape of the target path segment being the same as or similar to the shape represented by the modified shape data;

[0023] The target position is determined based on the position of the target path segment in the virtual physiological channel.

[0024] Optionally, determining the shape of the navigation path in the virtual physiological channel includes:

[0025] In the center line of the virtual physiological channel, a curve segment covered by the navigation path is determined, and the shape of the curve segment is used as the shape of the navigation path.

[0026] Optionally, the flexible device detection method further includes:

[0027] Based on the target position, a virtual position of a virtual flexible instrument for simulating the flexible instrument in the virtual physiological channel is determined.

[0028] Optionally, the specified information is specified posture information of the specified part;

[0029] The detection method further comprises:

[0030] Based on the designated posture information, a virtual posture of a specific part of the virtual flexible instrument in the virtual physiological channel is determined, and the specific part is used to simulate the designated part.

[0031] Optionally, the detection method further includes:

[0032] The virtual physiological channel is displayed by using a display device, and the virtual flexible instrument is displayed at the virtual position in the virtual physiological channel, and the specific part is displayed as the virtual posture.

[0033] Optionally, the flexible device detection method further includes:

[0034] Planning a navigation path in the virtual physiological channel, wherein the navigation path is a path for the flexible instrument to move to the lesion area after entering the real physiological channel;

[0035] When the virtual physiological channel is displayed using a display device, the navigation path is displayed on the virtual physiological channel.

[0036] Optionally, the designated location is the end of the flexible instrument.

[0037] Optionally, the shape sensor is a fiber optic sensor provided on the flexible device.

[0038] Optionally, the designated sensor is a magnetic sensor provided on the flexible device.

[0039] According to a second aspect of the present invention, there is provided a detection device for a flexible device, comprising:

[0040] A shape acquisition module, used to acquire shape data of the flexible device detected by the shape sensor;

[0041] A designated acquisition module, used to acquire designated information of a designated part of the flexible device detected by a designated sensor, wherein the designated information can at least represent the position of the designated part;

[0042] A position determination module is used to determine the target position of the flexible instrument in the real physiological channel based on the specified information and the shape data.

[0043] According to a third aspect of the present invention, there is provided a surgical system, comprising a flexible instrument, a data processing unit, a shape sensor and a designated sensor.

[0044] The data processing unit is used to execute the detection method of the flexible device involved in the first aspect and its optional solutions.

[0045] According to a fourth aspect of the present invention, there is provided an electronic device, comprising a processor and a memory.

[0046] The memory is used to store codes;

[0047] The processor is used to execute the code in the memory to implement the flexible device detection method involved in the first aspect and its optional solutions.

[0048] According to a fifth aspect of the present invention, there is provided a storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for detecting a flexible device involved in the first aspect and its optional solutions.

[0049] In the detection method, device, surgical system, equipment and medium of the flexible instrument provided by the present invention, the shape data of the flexible instrument can be obtained, and the positioning of the flexible instrument can be realized based on the shape of the instrument. Since the shape data of the flexible instrument does not need to be continuously tracked, compared with the solution that requires continuous tracking to collect the motion trajectory, the present invention can avoid the problem of inaccurate positioning due to the inability to continuously track, which helps to ensure the accuracy of positioning. At the same time, most of the prior art only focuses on the position of the flexible instrument, while the present invention determines the designated information of the designated part of the flexible instrument through the detection results of the designated sensor, so that based on the designated information and shape data, the positioning accuracy of the flexible instrument can be effectively improved, and the influence of the error of the shape detection of the designated part on the positioning accuracy can be reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0051] Figure 1 is a schematic diagram of the structure of a surgical system in an exemplary embodiment of the present invention;

[0052] Figure 2 is a schematic diagram of the structure of a surgical system in another exemplary embodiment of the present invention;

[0053] Figure 3 is a flow chart of a method for detecting a flexible device in an exemplary embodiment of the present invention;

[0054] Figure 4 is a schematic diagram of a process for determining a target position in an exemplary embodiment of the present invention;

[0055] Figure 5 is a schematic diagram of a process for determining corrected shape data in an exemplary embodiment of the present invention;

[0056] Figure 6 is a schematic diagram of a process of determining a target position by shape comparison in an exemplary embodiment of the present invention;

[0057] Figure 7 is a flow chart of a method for detecting a flexible device in another exemplary embodiment of the present invention;

[0058] Figure 8 is a schematic diagram of a process of planning and displaying a navigation path in an exemplary embodiment of the present invention;

[0059] Fig. 9 is a schematic diagram of a display interface of a virtual trachea in an exemplary embodiment of the present invention;

[0060] Fig.10 is a schematic diagram of a program module of a detection device for a flexible device in an exemplary embodiment of the present invention;

[0061] Fig.11 is a schematic diagram of a program module of a detection device for a flexible device in another exemplary embodiment of the present invention;

[0062] Fig.12 FIG. 1 is a schematic diagram of the structure of an electronic device in an exemplary embodiment of the present invention. DETAILED DESCRIPTION

[0063] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0064] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0065] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0066] Please refer to Figure 1 An embodiment of the present invention provides a surgical system, including a flexible instrument 101 and a data processing unit 104 .

[0067] The flexible device 101 can be understood as any device that can achieve flexible bending. Furthermore, the flexible device 101 can be a device that can extend into a physiological channel (such as an airway) and move along the physiological channel. Based on the flexible device, it can be used for treatment or inspection. Regardless of the purpose and how the flexible device 101 achieves flexible bending, it does not deviate from the scope of the embodiments of the present invention.

[0068] The flexible instrument 101 may be loaded with corresponding components based on the different purposes to be achieved. For example, the flexible instrument 101 may be used as an endoscope (e.g., a bronchoscope), in which case the flexible instrument 101 may be equipped with an image acquisition unit; for another example, the flexible instrument 101 may also be used to form an instrument channel, in which case surgical instruments and inspection instruments may enter the physiological channel through the instrument channel to complete the corresponding treatment and inspection. In addition, the flexible instrument 101 may be provided with a control unit for controlling the movement of the flexible instrument 101 (e.g., bending, moving, etc.).

[0069] The data processing unit 104 may be understood as any device or combination of devices capable of implementing data processing, which may be used to execute the flexible device detection method provided in the embodiment of the present invention.

[0070] To facilitate the implementation of the flexible instrument detection method provided in the embodiment of the present invention, the surgical system in the embodiment of the present invention further includes a shape sensor 102 and a designated sensor 103 .

[0071] The shape sensor 102 may be understood as a sensor that can be used to detect the shape of a flexible instrument (part or all of an instrument segment). Specifically, the shape sensor is used to detect curvature information of multiple parts of the flexible instrument. Correspondingly, the data processing unit (or the shape sensor itself) may form shape data based on the curvature information. The shape data includes: positions of multiple instrument position points constructed based on the curvature information of the multiple parts in a first coordinate system based on the shape sensor; the shape of the flexible instrument matches the distribution shape of the multiple instrument position points; the first coordinate system may also be understood as a coordinate system used in the shape sensor or the data processing unit to describe instrument point positions.

[0072] exist Figure 1 In the illustrated embodiment, the shape sensor 102 may be disposed on the flexible device 101, and the shape sensor 102 may be bent along with the flexible device 101. In an embodiment not shown, the shape sensor may not be disposed on the flexible device. For example, the shape sensor may be a development detection device, and the development detection device may be used to realize the development of the flexible device 101 in the body, thereby extracting position points and / or lines based on the development results to characterize the shape of the device.

[0073] The designated sensor 103 may be understood as a sensor for detecting the position (or posture) of a designated part of the flexible device; the detected position (or posture) may be, for example, any data in a second coordinate system that describes the position (or posture) of the designated part (e.g., yaw angle, roll angle, pitch angle, etc.).

[0074] The designated part may refer to any one or more parts of the flexible device;

[0075] The second coordinate system may be understood as a coordinate system used to describe the position (or posture) of a specified part in a specified sensor.

[0076] On this basis, the selection of the designated part can be based on the importance of each part in the flexible device, for example, a part with a higher importance can be selected. It can also be based on the accuracy of the shape of each part when the shape sensor detects the shape of the device (which can also be regarded as the accuracy of the device position points of each part). For example, a part with relatively low accuracy can be selected as the designated part to make up for the lack of accuracy.

[0077] exist Figure 1In the embodiment shown, the designated sensor 103 may be provided on the flexible device 101, for example, on a designated part. At this time, as the position and posture of the designated part change, the position and posture of the designated sensor 103 will also change accordingly. At this time, the position and posture measured by the designated sensor itself are the position and posture of the designated part. In an embodiment not shown, the designated sensor may not be provided on the flexible device.

[0078] Figure 2 The data processing unit 204, the shape sensor 202, the designated sensor 203 and the flexible device 201 in the embodiment shown in the figure can be referred to as Figure 1 The data processing unit 104 , the shape sensor 102 , the designated sensor 103 and the flexible device 101 in the illustrated embodiment are understood, and the same or similar contents are not repeated here.

[0079] exist Figure 2 In the illustrated embodiment, the shape sensor 202 may be an optical fiber sensor, and the designated sensor 203 may be a magnetic field sensor, which may be used in conjunction with the magnetic field generator 205. In other examples, the designated sensor 203 may also be implemented using an inertial sensor or the like.

[0080] For the fiber optic sensor, an optical fiber may be provided therein, and then a plurality of device position points may be obtained by analyzing the light signal propagating in the optical fiber, thereby using the plurality of device position points to depict the bending shape of the optical fiber. Since the fiber optic sensor can bend along with the flexible device 201, the plurality of device position points can also depict the bending shape of the flexible device 201 (i.e., the device shape).

[0081] Among them, since the fiber optic sensor relies on the propagation of light, and the distance that light propagates to the end of the fiber optic sensor is long, correspondingly, the accuracy of the shape and instrument position point at the end (or near the end) is poor. Therefore, it is more necessary to combine with a designated sensor to make up for the defect of poor detection accuracy. At the same time, the end of the flexible instrument can reflect the depth of the flexible instrument into the physiological channel and how the end of the flexible instrument changes under controlled operation. Therefore, the importance of the end is relatively high; if the endoscope is taken as an example, the end can also reflect where the image is collected by the image acquisition unit and in what direction it is collected. It can be seen that based on the above reasons, selecting the end of the flexible instrument as the designated part can provide sufficient and effective basis for the control of the flexible instrument, and can also make up for the accuracy defects of the fiber optic sensor.

[0082] For the magnetic field sensor, when using magnetism for positioning and navigation, it is necessary to use the magnetic field generator 205 to generate a magnetic field environment of a certain intensity. The positioning principle of the magnetic field sensor 203 is to feedback the current location and posture of the magnetic field sensor 203 based on the intensity and direction of the magnetic field emitted by the magnetic field generator 205. Furthermore, the second coordinate system can be, for example, a coordinate system based on the magnetic field generator.

[0083] In a specific example, the magnetic field sensor installed at the end of the flexible instrument can obtain the current position and posture of the end of the flexible instrument, for example, its yaw angle, roll angle, pitch angle (i.e., specified posture information) and three coordinate positions (i.e., specified position points) in a Cartesian coordinate system (i.e., a second coordinate system).

[0084] Please refer to Figure 3 , an embodiment of the present invention provides a method for detecting a flexible device.

[0085] In one embodiment, the flexible device is located in a real physiological channel, and the flexible device is provided with a shape sensor and a designated sensor at a designated position. Optionally, the real physiological channel may be a bronchus or the like.

[0086] like Figure 3 As shown, the method includes:

[0087] S301: Acquire shape data of the flexible device detected by the shape sensor;

[0088] The shape sensor and shape data can be found in Figure 1 , Figure 2 The relevant descriptions in the illustrated embodiments are understood;

[0089] S302: Acquire designated information of a designated part of the flexible device detected by a designated sensor;

[0090] For the designated parts and designated information, please refer to Figure 1 , Figure 2 The relevant description in the embodiment shown is understood; the specified information can at least represent the position of the specified part;

[0091] S303: Determine a target position of the flexible device in a real physiological channel based on the designated information and the shape data.

[0092] The target position can be fed back to the user through visual, auditory, tactile and other perceptual means. For example, a virtual physiological channel can be used to simulate a real physiological channel, and a virtual flexible instrument can be used to simulate the flexible instrument, so as to show the user the position of the virtual flexible instrument in the virtual physiological channel.

[0093] In the above scheme, the shape data of the flexible instrument can be obtained, and the positioning of the flexible instrument can be achieved based on the shape of the instrument (i.e., the shape represented by the shape data). Since the shape data of the flexible instrument does not need to be continuously tracked, compared with the scheme that requires continuous tracking to collect the motion trajectory, the present invention can avoid the problem of inaccurate positioning due to the inability to continuously track, which helps to ensure the accuracy of positioning. At the same time, most of the prior art only focuses on the position of the flexible instrument, while the present invention determines the designated information of the designated part of the flexible instrument through the detection results of the designated sensor, so that based on the designated information and shape data, the positioning accuracy of the flexible instrument can be effectively improved, and the influence of the error of the shape detection of the designated part on the positioning accuracy can be reduced.

[0094] In one implementation, please refer to Figure 4 The process of determining the target position of the flexible device in the real physiological channel based on the specified information and the shape data may include:

[0095] S401: Based on the specified information, the shape data is corrected to obtain corrected shape data;

[0096] S402: Determine the target position of the flexible instrument in the real physiological channel based on the corrected shape data.

[0097] In an example of step S402, the target position may be determined by comparing the corrected shape data with the shape of a pre-established virtual physiological channel.

[0098] The virtual physiological channel can be any virtual object used to simulate a real physiological channel in a virtual coordinate system. In a specific example, the virtual physiological channel can be constructed based on CT data of a target human body. Then, for different human bodies, a virtual physiological channel (such as a virtual trachea, a virtual bronchial tree, etc.) that accurately reflects the structure of its physiological channel can be formed. In some other examples, the virtual physiological channel can also adopt a standardized virtual physiological channel.

[0099] A virtual flexible instrument may also be provided in the virtual physiological channel; the virtual flexible instrument may be any virtual object used to simulate a flexible instrument in a virtual coordinate system, for example, a virtual object formed by complex modeling, or a long strip of a simple component in a three-dimensional shape;

[0100] Among them, the correction of shape data can be understood as: since the designated information can reflect the position of the designated part in space, the shape data can reflect the shape of the flexible device in space, and then, the position of the designated part will impose certain constraints on the shape of the designated part of the flexible device. In the above steps S401 and S402, it is precisely by using this constraint that the error of the detected shape at the designated part is compensated. It can be seen that through the above correction process, it can help to ensure that the corrected shape data can more accurately depict the true shape of the flexible device and improve the accuracy of the positioning of the flexible device.

[0101] In one embodiment, the shape sensor is used to detect curvature information of multiple parts of the flexible instrument, and the shape data includes: the positions of multiple instrument position points constructed based on the curvature information of the multiple parts in a first coordinate system based on the shape sensor; the shape of the flexible instrument matches the distribution shape of the multiple instrument position points; the designated sensor is a magnetic sensor, and the magnetic sensor is arranged in the magnetic field generated by the magnetic field generator, and the designated information includes: the positions of one or more designated position points in a second coordinate system based on the magnetic field generator;

[0102] For the corresponding Figure 5 The process of correcting the shape data based on the specified information to obtain the corrected shape data may include:

[0103] S501: Projecting the designated position point and / or the instrument position point to a target coordinate system to obtain a first position point corresponding to the instrument position point and a second position point corresponding to the designated position point;

[0104] S502: Among all the first position points, first position points within a specified range determined based on the second position are screened out, and positions of the first position points that are not screened out and the second position points in the target coordinate system are determined as the corrected shape data.

[0105] in:

[0106] If the target coordinate system is the first coordinate system, the first position point is the instrument position point, and the second position point is the position point after the designation position point is projected;

[0107] If the target coordinate system is the second coordinate system, the first position point is the position point after the instrument position point is projected, and the second position point is the designated position point;

[0108] Specifically, since the relative position of the shape sensor and the designated sensor is known, the relative position and posture relationship (such as a conversion matrix) between the first coordinate system and the second coordinate system can be obtained by pre-calibration, and then, in step S501, the corresponding projection can be implemented based on the calibrated relative position and posture relationship;

[0109] If the target coordinate system is a virtual coordinate system belonging to a virtual physiological channel used to simulate a real physiological channel, the first position point is the position point after the instrument position point is projected, and the second position point is the position point after the designated position point is projected; the virtual coordinate system can be understood as a coordinate system used to construct a virtual physiological channel;

[0110] Specifically, the relative posture relationship of the first coordinate system with respect to the virtual coordinate system can be calibrated before or after the flexible device enters the real physiological channel, and / or the relative posture relationship of the second coordinate system with respect to the virtual coordinate system can be calibrated; then, in step S501, the corresponding projection can be realized based on the calibrated relative posture relationship.

[0111] The specified range in step S502 may be, for example, a spherical space range of a specified radius with the second position as the center, or may be, for example, a long strip space range with the second position as the center. Through step S502, the first position point near the second position point may be replaced by the second position point, thereby reflecting the detection result of the specified sensor on the specified part.

[0112] In addition, if there is no first position point within the nearby specified range, or there is only a first position point that overlaps with the second position point, the implementation result of step S502 may be similar to that before the implementation.

[0113] In one embodiment, if the target coordinate system is a virtual coordinate system, the distribution shapes of the first position points and the second position points that have not been screened out can be directly used as the corrected shape data, so as to achieve comparison with the pre-established virtual physiological channel shape, thereby achieving positioning. In another embodiment, if the target coordinate system is not a virtual coordinate system (for example, the first coordinate system or the second coordinate system), the first position points and the second position points that have not been screened out can also be projected to the virtual coordinate system, and the distribution shapes of the projected position points can be used as the shape represented by the shape data, so as to achieve comparison with the virtual physiological channel shape; in other embodiments, even if the target coordinate system is not a virtual coordinate system, it can be compared with the virtual physiological channel shape without being projected to the virtual coordinate system.

[0114] In the above scheme, the distribution shapes of the first position points and the second position points that have not been screened out can comprehensively reflect the detection results of the shape sensor on the positions of various parts in the flexible instrument, and the position detection results of the designated sensor on the designated parts in the flexible instrument, thereby realizing the integration of the position detection results. It can also play a role in compensating for the accuracy and realize the complementarity between different sensors. For example, when the shape sensor adopts an optical fiber sensor, the position detection accuracy of certain parts of the flexible instrument (such as the end) is poor. At this time, the detection results of the designated sensor (reflected as the designated position points) can compensate for this poor accuracy. Then, the determined shape data can more accurately reflect the shape of the flexible instrument. On this basis, the accuracy of the positioning of the flexible instrument can be improved.

[0115] In one implementation, please refer to Figure 6 , determining the target position by comparing the corrected shape data with the shape of a pre-established virtual physiological channel may include:

[0116] S601: Determine the shape of the navigation path in the virtual physiological channel;

[0117] S602: Matching a target path segment in the navigation path by comparing the corrected shape data with the shape of the navigation path;

[0118] The shape of the target path segment is the same as or similar to the shape represented by the corrected shape data;

[0119] S603: Determine the target position based on the position of the target path segment in the virtual physiological channel.

[0120] The navigation path in step S601 can be any path pre-planned in the virtual physiological channel, and can be specifically used to simulate the path of the flexible instrument moving to the lesion area after entering the real physiological channel; the process of determining its shape can be, for example, the process of determining a plurality of path position points or a curve segment formed by a plurality of path position points; correspondingly, in an example of step S602, N consecutive path position points can be taken multiple times from a plurality of path position points, and the distance between each position point in the N position points of the shape data (such as the first position point and the second position point that are not screened out, or the position point projected to the virtual coordinate system) and the corresponding path position point in the continuous N path position points is calculated, and then based on this, a path segment with the most matching (same or most similar) shape is calculated as the target path segment. The above is only an example, and any method in the field that can achieve matching between point sets and curves can be applied here as an optional solution.

[0121] In an example of step S601, it may include: determining the curve segment covered by the navigation path in the center line of the virtual physiological channel, and using the shape of the curve segment as the shape of the navigation path. Furthermore, the position point in the curve segment can be understood as the path position point used for comparison in step S603.

[0122] In one example of step S603, the target position may be represented by one or more path position points in the target path segment. In other examples, the target position may be represented by extracting feature position points (or other feature information) based on the target path segment.

[0123] In the above scheme, since the flexible device is usually controlled to move along a pre-planned navigation path, by comparing the shape represented by the shape data with the shape of the navigation path, the amount of data can be effectively saved and the processing efficiency can be improved without affecting the detection result.

[0124] Figure 7 Steps S701, S702, and S703 in the embodiment shown are the same or similar to Figure 3 The steps S301, S302, and S303 in the illustrated embodiment are repeated and will not be described in detail here.

[0125] exist Figure 7 In the illustrated embodiment, the specified information is the specified position and posture information of the specified part;

[0126] The detection method also includes:

[0127] S704: Based on the target position, determining a virtual position of a virtual flexible instrument for simulating the flexible instrument in the virtual physiological channel;

[0128] The virtual position may be understood as any position that can describe the position of the virtual flexible device in the virtual flexible device, and may be specifically represented by a position point in the virtual flexible device in the virtual coordinate system.

[0129] In a specific example, if the target position is represented by one or more positions in the virtual physiological channel, for example, using Figure 6 The target path segment mentioned in the embodiment is characterized by one or more path position points and feature position points, then:

[0130] In one example, the path position point and the feature position point can be directly used to represent the virtual position. For example, the path position point (or feature position point) can be the two ends of the target path segment. Correspondingly, the virtual position can be represented by the two end positions of the virtual flexible instrument. Then, the two end positions of the target path segment can be directly used as the two end positions of the virtual flexible instrument to represent the virtual position.

[0131] In another example, the virtual position can also be calculated in combination with the length, shape, etc. of the virtual flexible instrument. For example, the path position point (or characteristic position point) can be one end of the target path segment. Correspondingly, the virtual position can be represented by the positions of the two ends of the virtual flexible instrument. In this case, the path position point (or characteristic position point) can be used as one end of the virtual flexible instrument, and the position of the other end of the virtual flexible instrument in the virtual physiological channel can be calculated based on the length of the virtual flexible instrument and other information, thereby obtaining the positions of the two ends of the virtual flexible instrument, and representing the virtual position accordingly.

[0132] In other examples, the number of position points representing the target position and the number of position points representing the virtual position may not be limited to 1 or 2, but may be 3 or more.

[0133] Regardless of the method used, sufficient basis can be provided for positioning and displaying the virtual flexible instrument; S705: based on the specified posture information, determining the virtual posture of the specific part of the virtual flexible instrument in the virtual physiological channel;

[0134] The specific part is used to simulate the designated part. The virtual posture of the specific part can reflect the posture of the designated part in the real physiological channel;

[0135] In an example of step S705, the designated posture information can be projected into the virtual coordinate system of the virtual physiological channel (for example, the projection can be realized based on the conversion relationship between the first coordinate system and the virtual coordinate system), and the information describing the posture can be extracted, and the virtual posture can be determined accordingly, wherein, if the designated part refers to the end of the flexible instrument, the virtual posture can also be understood as the orientation of the end of the flexible instrument. In one example, the orientation can be indicated by the posture of the end of the virtual flexible instrument relative to the virtual coordinate system.

[0136] In other examples, the specified posture information may also be corrected and adjusted before being projected into the virtual coordinate system;

[0137] During detection, most of the existing technologies only focus on the position of the flexible instrument. The above scheme determines the virtual posture of the specified part of the flexible instrument through the detection results of the specified sensor, which can provide more diverse and rich information for the execution (or review, etc.) of the operation. For example, through the determination of the virtual posture, it is convenient for the user to timely and accurately understand the impact of the manipulation of the flexible instrument on the specified part, so as to better complete the operation of the flexible instrument;

[0138] S706: Displaying the virtual physiological channel by using a display device, displaying the virtual flexible instrument at the target position in the virtual physiological channel, and displaying the specific part as the virtual posture;

[0139] In some examples, only the target position may be displayed without displaying the target posture.

[0140] The display device may be a two-dimensional display device or a three-dimensional display device. As long as the presented result can be perceived by human eyes, it does not depart from the scope of the embodiments of the present invention.

[0141] In the above scheme, the intuitive presentation of virtual position and virtual posture is achieved, which provides a sufficient and accurate basis for the user's control, and facilitates the next step of surgery or the advancement of flexible instruments (for example, advancement along the direction of the navigation path).

[0142] In one implementation, please refer to Figure 8 , the detection method also includes:

[0143] S801: planning a navigation path in the virtual physiological channel;

[0144] S802: When displaying the virtual physiological channel using a display device, display the navigation path on the virtual physiological channel.

[0145] The navigation path is: the path of the flexible instrument moving to the lesion area after entering the real physiological channel;

[0146] In one example, if a lung bronchial tree model (i.e., a virtual trachea) is used as a virtual physiological channel, the CT data of pre-operative medical images of the target human body (e.g., a patient) can be obtained, and the lung bronchi can be extracted and reconstructed using computer vision and image processing methods to obtain a three-dimensional model of the patient's lung bronchial tree (i.e., a virtual physiological channel). The doctor can then mark the location of the lesion (which can reflect the lesion area) therein, and the computer uses the reconstructed model to calculate the accessible bronchial path as a navigation path. On this basis, in order to achieve shape matching, the corresponding center line of the bronchial path can also be extracted.

[0147] by Fig. 9 As shown in the figure, for example, a center line 902 of a planned path can be extracted from a virtual trachea 900, the center line 902 starting from a starting point 903 and ending at a lesion area near a lesion point 901, wherein a virtual flexible instrument 904 is also illustrated. It can be seen that when displaying the virtual trachea 900, the virtual flexible instrument 904 and the navigation path (for example, the center line 902 thereof is displayed in a specified color) can be displayed simultaneously. In addition, the end posture of the virtual flexible instrument 904 can be displayed as a virtual posture.

[0148] The virtual trachea 901 may be a three-dimensional virtual trachea. When presenting, if the display device is a two-dimensional image display device, a virtual camera source may be used to frame the virtual trachea (including the virtual flexible instrument) to obtain a two-dimensional image, thereby displaying the two-dimensional image of the virtual trachea on the display device. In addition, at least one of a lesion point, a virtual skeleton, a virtual blood vessel, etc. may also be displayed in the virtual trachea.

[0149] Among them, by displaying the planned path, the positioning of the flexible instrument can be achieved, and the planned path can be automatically planned and displayed during the movement, so that the flexible instrument can be tracked and positioned relative to the planned path, simplifying the doctor's identification task (the doctor does not need to plan the path by himself, nor does he need to use other means to track and observe whether the flexible instrument moves along the planned path), speeding up the operation efficiency, and smoothly completing the entire operation process.

[0150] Please refer to Fig.10 The embodiment of the present invention further provides a flexible device detection device 1000, comprising:

[0151] A shape acquisition module 1001 is used to acquire shape data of the flexible device detected by the shape sensor;

[0152] A designated acquisition module 1002 is used to acquire designated information of a designated part of the flexible device detected by a designated sensor, wherein the designated information can at least represent the position of the designated part;

[0153] The position determination module 1003 is used to determine the target position of the flexible device in the real physiological channel based on the specified information and the shape data.

[0154] Optionally, the location determination module 1003 is specifically configured to:

[0155] Based on the specified information, the shape data is modified to obtain modified shape data;

[0156] Based on the corrected shape data, the target position of the flexible instrument in the real physiological channel is determined.

[0157] Optionally, the location determination module 1003 is specifically configured to:

[0158] Projecting the designated position point and / or the instrument position point to a target coordinate system to obtain a first position point corresponding to the instrument position point and a second position point corresponding to the designated position point; both the first position point and the second position point are in the target coordinate system;

[0159] Among all the first position points, the first position points within the specified range determined based on the second position are screened out, and the positions of the first position points that are not screened out and the second position points in the target coordinate system are determined as the corrected shape data.

[0160] Optionally, the location determination module 1003 is specifically configured to:

[0161] The target position is determined by comparing the corrected shape data with the shape of a pre-established virtual physiological channel.

[0162] Optionally, the location determination module 1003 is specifically configured to:

[0163] Determining the shape of the navigation path in the virtual physiological channel; the navigation path is: the path of the flexible instrument moving to the lesion area after entering the real physiological channel;

[0164] By comparing the modified shape data with the shape of the navigation path, matching a target path segment in the navigation path, the shape of the target path segment being the same as or similar to the shape represented by the modified shape data;

[0165] The target position is determined based on the position of the target path segment in the virtual physiological channel.

[0166] Optionally, the location determination module 1003 is specifically configured to:

[0167] In the center line of the virtual physiological channel, a curve segment covered by the navigation path is determined, and the shape of the curve segment is used as the shape of the navigation path.

[0168] Please refer to Fig.11 , wherein the shape acquisition module 1101, the specified acquisition module 1102, the position determination module 1103 and Fig.10 The shape acquisition module 1001, the specified acquisition module 1002, and the position determination module 1003 in the illustrated embodiment are the same or similar, and the repeated contents will not be repeated here.

[0169] In one embodiment, the flexible device detection device 1100 further includes:

[0170] A virtual position determination module 1104, configured to determine a virtual position of a virtual flexible instrument for simulating the flexible instrument in the virtual physiological channel based on the target position;

[0171] A virtual posture determination module 1105, for determining a virtual posture of a specific part of the virtual flexible instrument in the virtual physiological channel based on the specified posture information, the specific part being used to simulate the specified part;

[0172] The display module 1106 is used to display the virtual physiological channel by using a display device, and to display the virtual flexible instrument at the target position in the virtual physiological channel, and to display the specific part as the target posture.

[0173] In one embodiment, the flexible device detection device 1100 further includes:

[0174] A path planning module 1107 is used to plan a navigation path in the virtual physiological channel, wherein the navigation path is a path for the flexible instrument to move to the lesion area after entering the real physiological channel;

[0175] The display module 1106 is further configured to display the navigation path on the virtual physiological channel when the virtual physiological channel is displayed using a display device.

[0176] Please refer to Fig.12 , an electronic device 1200 is provided, including:

[0177] Processor 1201; and

[0178] Memory 1202, used to store executable instructions of the processor;

[0179] The processor 1201 is configured to execute the above-mentioned method by executing the executable instructions.

[0180] The processor 1201 can communicate with the memory 1202 via the bus 1203 .

[0181] An embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the above-mentioned method is implemented.

[0182] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0183] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A detection device for a flexible device, characterized in that: The flexible device is located in a real physiological channel, the flexible device is provided with a shape sensor, and a designated sensor is provided at a designated position; the shape sensor is different from the designated sensor; The device comprises: A shape acquisition module, used to acquire shape data of the flexible device detected by the shape sensor; A designated acquisition module, used to acquire designated information of a designated part of the flexible device detected by a designated sensor, wherein the designated information can at least represent the position of the designated part; A position determination module is used to determine the target position of the flexible instrument in the real physiological channel based on the specified information and the shape data.

2. The detection device for flexible equipment according to claim 1, characterized in that: Determining a target position of the flexible device in a real physiological channel based on the specified information and the shape data includes: Based on the specified information, the shape data is modified to obtain modified shape data; Based on the corrected shape data, the target position of the flexible instrument in the real physiological channel is determined.

3. The detection device for flexible equipment according to claim 2, characterized in that: The shape sensor is used to detect curvature information of multiple parts of the flexible instrument, and the shape data includes: positions of multiple instrument position points constructed based on the curvature information of the multiple parts in a first coordinate system based on the shape sensor; the shape of the flexible instrument matches the distribution shape of the multiple instrument position points; The designated sensor is a magnetic sensor, and the magnetic sensor is arranged in the magnetic field generated by the magnetic field generator. The designated information includes: the position of one or more designated position points in a second coordinate system based on the magnetic field generator; Based on the specified information, the shape data is corrected to obtain the corrected shape data, including: Projecting the designated position point and / or the instrument position point to a target coordinate system to obtain a first position point corresponding to the instrument position point and a second position point corresponding to the designated position point; both the first position point and the second position point are in the target coordinate system; Among all the first position points, the first position points within the specified range determined based on the second position are screened out, and the positions of the first position points that are not screened out and the second position points in the target coordinate system are determined as the corrected shape data.

4. The detection device for flexible equipment according to claim 2, characterized in that: Determining a target position of the flexible device in a real physiological channel based on the corrected shape data includes: The target position is determined by comparing the corrected shape data with the shape of a pre-established virtual physiological channel.

5. The detection device for flexible equipment according to claim 4, characterized in that: Determining the target position by comparing the corrected shape data with the shape of a pre-established virtual physiological channel includes: Determining the shape of the navigation path in the virtual physiological channel; the navigation path is: the path of the flexible instrument moving to the lesion area after entering the real physiological channel; By comparing the modified shape data with the shape of the navigation path, matching a target path segment in the navigation path, the shape of the target path segment being the same as or similar to the shape represented by the modified shape data; The target position is determined based on the position of the target path segment in the virtual physiological channel.

6. The detection device for flexible equipment according to claim 5, characterized in that: Determining the shape of the navigation path in the virtual physiological channel includes: In the center line of the virtual physiological channel, a curve segment covered by the navigation path is determined, and the shape of the curve segment is used as the shape of the navigation path.

7. The detection device for a flexible device according to claim 4, characterized in that: Also includes: The virtual position determination module is used to determine the virtual position of a virtual flexible instrument used to simulate the flexible instrument in the virtual physiological channel based on the target position.

8. The detection device for flexible equipment according to claim 7, characterized in that: The designated information is designated position information of the designated part; The detection device further comprises: The virtual posture determination module is used to determine the virtual posture of a specific part of the virtual flexible instrument in the virtual physiological channel based on the specified posture information, and the specific part is used to simulate the specified part.

9. The detection device for a flexible device according to claim 8, characterized in that: The detection device further comprises: The display module is used to display the virtual physiological channel by using a display device, and to display the virtual flexible instrument at the virtual position in the virtual physiological channel, and to display the specific part as the virtual posture.

10. The detection device for flexible equipment according to claim 9, characterized in that: Also includes: A path planning module, used for planning a navigation path in the virtual physiological channel, wherein the navigation path is a path for the flexible instrument to move to the lesion area after entering the real physiological channel; The display module is further configured to display the navigation path on the virtual physiological channel when the virtual physiological channel is displayed using a display device.

11. The detection device for a flexible device according to any one of claims 1, 2, 4 to 10, characterized in that: The designated part is the end of the flexible device; The shape sensor is an optical fiber sensor provided on the flexible device; The designated sensor is a magnetic sensor provided on the flexible device.

12. A surgical system, characterized in that: Including flexible devices, data processing units, shape sensors and specified sensors, The data processing unit is used to execute the function of the detection device for a flexible device according to any one of claims 1 to 11.

13. An electronic device, characterized in that: Including processor and memory, The memory is used to store codes; The processor is used to execute the code in the memory to implement the function of the detection device for a flexible device according to any one of claims 1 to 11.

14. A storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the function of the detection device for a flexible device according to any one of claims 1 to 11 is realized.

Images