Anti-collision early warning method, system and equipment for nasal endoscopic surgical instrument

Abstract

The invention belongs to the technical field of surgical operation assistance, and particularly relates to an anti-collision early warning method, system and equipment for a nasal endoscopic surgical instrument, and the method comprises the steps: obtaining an endoscopic image and a general intranasal modeling; the entering length and the lens direction of the nasal endoscope are detected in real time, and the position of the tail end of the nasal endoscope in the general intranasal modeling is determined according to the entering length and the lens direction of the nasal endoscope; marking information is waited and detected, in general intranasal modeling, the tail end of the nasal endoscope serves as a projection point, the real-time lens direction serves as the projection direction, the inner wall of the nasal cavity serves as a projection plane, the marking shape in the marking information is projected, and the actual position of the high-risk part is obtained by projecting the marking shape on the inner wall of the nasal cavity; updating the position of the high-risk part to the actual position of the high-risk part; in general intranasal modeling, a first visual early warning or a second visual early warning is selected based on whether the location of the high risk site is updated. The problem that a nasal endoscope and matched surgical instruments frequently collide with key areas of a human body can be solved.

Description

Technical Field

[0001] This application belongs to the field of surgical assistance technology, and in particular relates to methods, systems and equipment for collision avoidance warning of nasal endoscopic surgical instruments. Background Technology

[0002] A nasal endoscope is a minimally invasive medical examination instrument. It is an optical device that can perform detailed examinations of the nasal cavity. With the help of matching surgical instruments, the surgery can be made more precise and can reach areas that traditional surgery cannot reach.

[0003] There are many critical danger zones inside the human nasal cavity. If these critical danger zones are accidentally bumped or damaged by surgical instruments (usually caused by a frontal impact from the endoscope or a head-mounted surgical instrument), it can lead to various postoperative complications or even catastrophic consequences. Critical danger zones include, but are not limited to: the internal carotid artery, optic nerve, skull base, anterior / posterior ethmoidal artery, sphenopalatine artery, orbital wall, pituitary gland, and cavernous sinus. In current intranasal examinations, doctors often rely on experience to avoid critical danger zones. However, due to factors such as blind spots, lack of tactile feedback, and distraction, there is a problem of frequent collisions between the nasal endoscope and its instruments and critical danger zones during surgery. Therefore, there is an urgent need for a collision prevention and warning method for nasal endoscopic surgical instruments to assist medical staff in using nasal endoscopes. Summary of the Invention

[0004] This application provides a method, system, and device for collision avoidance warning of nasal endoscopic surgical instruments, which can solve the problem of frequent collisions between the nasal endoscope and its associated surgical instruments and critical danger areas during existing nasal examination surgeries.

[0005] In a first aspect, embodiments of this application provide a method for anti-collision warning of nasal endoscopic surgical instruments, applied to an anti-collision warning device for nasal endoscopic surgical instruments, the method comprising: Acquire endoscopic images and general intranasal modeling; wherein, the general intranasal modeling records the locations of multiple high-risk sites; The insertion length and lens direction of the nasal endoscope are detected in real time, and the position of the endoscope tip in the general intranasal modeling is determined based on the insertion length and lens direction of the nasal endoscope. The system waits for and detects the marking information. In the general intranasal modeling, the end of the nasal endoscope is used as the projection point, the real-time lens direction is used as the projection direction, and the nasal cavity wall is used as the projection surface. The marking shape in the marking information is projected onto the nasal cavity wall to obtain the actual location of the high-risk area. The location of the high-risk area is then updated to its actual location. Here, the marking information refers to the actual location of the high-risk area marked by medical personnel or artificial intelligence in the endoscopic image. The marking shape is a closed contour, and the actual location of the high-risk area is inside the projected closed contour. In the general intranasal modeling, a first visual warning or a second visual warning is selected based on whether the location of the high-risk site has been updated. The first visual warning is used to alert the name and direction of the high-risk site in the endoscopic image for high-risk sites whose locations have not been updated. The second visual warning is used to alert the name, direction, and distance of the high-risk site in the updated endoscopic image for high-risk sites whose locations have been updated.

[0006] The technical solutions described in this application embodiment have at least the following technical effects: The anti-collision warning method for nasal endoscopic surgical instruments provided in this application firstly acquires endoscopic images and a general nasal intramural model. This step involves acquiring endoscopic images collected by the nasal endoscope and also acquiring a general nasal intramural model to roughly reflect the locations of multiple high-risk sites. Secondly, the insertion length and lens direction of the nasal endoscope are detected in real time, and the position of the nasal endoscope tip in the general nasal intramural model is determined based on the insertion length and lens direction. This step detects the length of the nasal endoscope entering the patient's nasal cavity and the direction of nasal endoscope torsion, and then determines the position of the nasal endoscope tip in the general nasal intramural model based on the insertion length and lens direction. Subsequently, the marked information is waited for and detected. In the general intranasal modeling, the shape of the marked information is projected onto the nasal cavity wall using the endoscope tip as the projection point, the real-time lens direction as the projection direction, and the nasal cavity wall as the projection surface. This projection yields the actual location of the high-risk area. The location of the high-risk area is then updated to reflect its actual position. In this step, the marked information determined by medical personnel or AI is received, and the location of the high-risk area is updated in real time based on this information. This approach takes into account individual errors from different patients and improves the accuracy of high-risk area location in general intranasal modeling. Finally, in the general intranasal modeling, a first-visual warning or a second-visual warning is selected based on whether the location of the high-risk area has been updated. This step provides warnings for all high-risk areas. For high-risk areas without updated locations, only their name and direction are indicated. For high-risk areas with updated locations, their name, direction, and distance are indicated. This eliminates the need for doctors to rely solely on experience to avoid high-risk areas, thus resolving the problem of frequent collisions between the nasal endoscope and related surgical instruments and critical danger areas during intranasal examinations and surgeries. This method can solve the problem of frequent collisions between the nasal endoscope and its associated surgical instruments and critical danger areas during existing intranasal examination surgeries.

[0007] Secondly, embodiments of this application provide a collision avoidance warning system for nasal endoscopic surgical instruments, applied to a collision avoidance warning device for nasal endoscopic surgical instruments. The system includes: An acquisition unit is used to acquire endoscopic images and general intranasal modeling; wherein, the general intranasal modeling records the locations of multiple high-risk sites; The detection unit is used to detect the insertion length and lens direction of the nasal endoscope in real time, and determine the position of the end of the nasal endoscope in the general intranasal modeling based on the insertion length and lens direction of the nasal endoscope. A marking unit is used to wait for and detect marking information, and in the general intranasal modeling, using the end of the nasal endoscope as the projection point, the real-time lens direction as the projection direction, and the nasal cavity wall as the projection surface, projects the marking shape in the marking information onto the nasal cavity wall to obtain the actual location of the high-risk area, and then updates the location of the high-risk area to its actual location; wherein, the marking information refers to the actual location of the high-risk area marked by medical personnel or artificial intelligence in the endoscopic image; the marking shape is a closed contour, and the actual location of the high-risk area is inside the projection of the closed contour; The warning unit is used to select a first visual warning or a second visual warning based on whether the location of the high-risk site has been updated in the general intranasal modeling. The first visual warning is used to alert the name and direction of the high-risk site in the endoscopic image for the high-risk site whose location has not been updated. The second visual warning is used to alert the name, direction and distance of the high-risk site in the endoscopic image after the location has been updated for the high-risk site after the location has been updated.

[0008] Thirdly, embodiments of this application provide a collision avoidance warning device for nasal endoscopic surgical instruments, including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the method described in any of the first aspects above.

[0009] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in any of the first aspects above.

[0010] Fifthly, embodiments of this application provide a computer program product that, when running on a nasal endoscopic surgical instrument anti-collision warning device, causes the nasal endoscopic surgical instrument anti-collision warning device to execute the nasal endoscopic surgical instrument anti-collision warning method described in any of the first aspects above.

[0011] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a schematic flowchart of a collision avoidance warning method for nasal endoscopic surgical instruments provided in an embodiment of this application; Figure 2 This is a schematic diagram of the nasal endoscope tube and multiple sets of photosensitive sensors in step S210 of the anti-collision warning method for nasal endoscopic surgical instruments provided in an embodiment of this application. Figure 3 This is a schematic diagram of the anti-collision warning system for nasal endoscopic surgical instruments provided in the embodiments of this application; Figure 4 This is a schematic diagram of the anti-collision warning device for nasal endoscopic surgical instruments provided in the embodiments of this application. Detailed Implementation

[0014] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0015] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0016] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0017] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0018] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0019] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0020] In related technologies, there are many critical danger zones inside the human nasal cavity. If these critical danger zones are accidentally bumped or damaged by surgical instruments, it may lead to various postoperative complications or even catastrophic consequences. Critical danger zones include, but are not limited to: the internal carotid artery, optic nerve, skull base bone, anterior / posterior ethmoidal artery, sphenopalatine artery, orbital wall, pituitary gland, and cavernous sinus. In current intranasal examination surgeries, doctors often avoid critical danger zones based on experience. However, due to factors such as blind spots, lack of tactile feedback, and distraction, there is a problem of frequent collisions between the nasal endoscope and its associated surgical instruments and critical danger zones during surgery. Therefore, there is an urgent need for a collision prevention and warning method for nasal endoscopic surgical instruments to assist medical staff in using nasal endoscopes.

[0021] To address the aforementioned issues, this application provides a method for collision avoidance warning of nasal endoscopic surgical instruments. The method first involves acquiring endoscopic images and a general nasal intramural model. This step involves acquiring endoscopic images collected by the nasal endoscope and also acquiring a general nasal intramural model to roughly reflect the locations of multiple high-risk sites. Second, the insertion length and lens direction of the nasal endoscope are detected in real time, and the position of the nasal endoscope tip in the general nasal intramural model is determined based on the insertion length and lens direction. This step involves detecting the length of the nasal endoscope entering the patient's nasal cavity and the direction of nasal endoscope distortion, and then determining the position of the nasal endoscope tip in the general nasal intramural model based on the insertion length and lens direction. Subsequently, the marked information is waited for and detected. In the general intranasal modeling, the shape of the marked information is projected onto the nasal cavity wall using the endoscope tip as the projection point, the real-time lens direction as the projection direction, and the nasal cavity wall as the projection surface. This projection yields the actual location of the high-risk area. The location of the high-risk area is then updated to reflect its actual position. In this step, the marked information determined by medical personnel or AI is received, and the location of the high-risk area is updated in real time based on this information. This approach takes into account individual errors from different patients and improves the accuracy of high-risk area location in general intranasal modeling. Finally, in the general intranasal modeling, a first-visual warning or a second-visual warning is selected based on whether the location of the high-risk area has been updated. This step provides warnings for all high-risk areas. For high-risk areas without updated locations, only their name and direction are indicated. For high-risk areas with updated locations, their name, direction, and distance are indicated. This eliminates the need for doctors to rely solely on experience to avoid high-risk areas, thus resolving the problem of frequent collisions between the nasal endoscope and related surgical instruments and critical danger areas during intranasal examinations and surgeries. This method can solve the problem of frequent collisions between the nasal endoscope and its associated surgical instruments and critical danger areas during existing intranasal examination surgeries.

[0022] The anti-collision warning method for nasal endoscopic surgical instruments provided in this application embodiment can be applied to an anti-collision warning device for nasal endoscopic surgical instruments. In this case, the anti-collision warning device for nasal endoscopic surgical instruments is the executing subject of the anti-collision warning method for nasal endoscopic surgical instruments provided in this application embodiment. This application embodiment does not impose any restrictions on the specific type of anti-collision warning device for nasal endoscopic surgical instruments.

[0023] For example, a collision avoidance warning device for nasal endoscopic surgical instruments may include an acquisition device, a data acquisition device, and a control device. The acquisition device and data acquisition device are communicatively connected to the control device. The acquisition device is also communicatively connected to the nasal endoscope. The acquisition device may be a wired communication module or a wireless communication module, used to acquire endoscopic images, the lens orientation of the nasal endoscope, and marking information. The data acquisition device may include photosensitive sensors distributed on the outer surface of the endoscopic tubing, used to detect the insertion length of the nasal endoscope in real time. It may also include a touch screen module, which is integrated with the display screen of the nasal endoscope, used to acquire marking information. It may also include a gyroscope or gravimeter, located at the end of the nasal endoscope, used to determine the reference orientation.

[0024] The control device can be a microcontroller, microprocessor, mobile phone, tablet computer, laptop computer, netbook, desktop computer, computer, laptop computer, etc.

[0025] To better understand the anti-collision warning method for nasal endoscopic surgical instruments provided in this application, the specific implementation process of the anti-collision warning method for nasal endoscopic surgical instruments provided in this application will be described below by way of example.

[0026] Figure 1 This illustration shows a schematic flowchart of a collision avoidance warning method for nasal endoscopic surgical instruments provided in an embodiment of this application. The collision avoidance warning method for nasal endoscopic surgical instruments includes: S100 acquires endoscopic images and general intranasal modeling. The general intranasal modeling records the locations of multiple high-risk sites.

[0027] It is understandable that the acquisition device obtains real-time endoscopic images from the nasal endoscope. The endoscopic images refer to the images acquired by the nasal endoscope. General nasal modeling can be a model of the nasal cavity of multiple people at different age stages. It can be a model obtained by three-dimensional reconstruction of a human body or human specimen. The model includes the locations of multiple high-risk sites, including but not limited to: internal carotid artery, optic nerve, skull base bone, anterior / posterior ethmoidal artery, sphenopalatine artery, orbital wall, pituitary gland, cavernous sinus, etc. After determining the patient's age, a model of the nasal cavity of the corresponding age stage is obtained.

[0028] The S200 detects the insertion length and lens direction of the nasal endoscope in real time, and determines the position of the endoscope tip in general intranasal modeling based on the insertion length and lens direction.

[0029] It is understandable that the insertion length of the nasal endoscope is detected in real time by a detection device. This detection device could be a photosensitive sensor distributed across the outer surface of the endoscope tube. Because the nasal cavity is in a dark environment, while the external environment is illuminated, the insertion length of the nasal endoscope can be determined based on the feedback signal from the photosensitive sensor. The feedback signal from the photosensitive sensor indicates the junction between the dark and illuminated environments, and the distance from this junction to the lens is the insertion length of the nasal endoscope. The lens orientation of the nasal endoscope is detected in real time by an acquisition device. The nasal endoscope has a built-in lens orientation adjustment function, so the lens orientation of the nasal endoscope can be directly read from within the nasal endoscope using the acquisition device.

[0030] After detecting the insertion length of the nasal endoscope, it is compared with the previous insertion length to obtain the insertion length difference. Based on the previous position of the nasal endoscope tip, the insertion length difference is moved. Then, according to the detected lens direction of the nasal endoscope, the lens direction of the nasal endoscope tip in the general nasal modeling is adjusted to determine the position of the nasal endoscope tip in the general nasal modeling.

[0031] In one possible implementation, in step S200, multiple sets of photosensitive sensors are provided on the outer surface of the nasal endoscope's tubing. The same set of photosensitive sensors is evenly distributed across the same cross-section of the nasal endoscope tubing, and the distance between the multiple sets of photosensitive sensors is the same. Real-time detection of the nasal endoscope's insertion length includes: S210 receives real-time light sensing data from multiple photosensitive sensors.

[0032] As can be understood, a nasal endoscope is inserted through the nostril. The inside of the nostril and nasal cavity is a dark environment, while the environment outside the nostril is a bright environment. Therefore, by installing multiple sets of photosensitive sensors on the nasal endoscope's tubing, the boundary between the inside and outside of the nasal cavity is determined, thereby determining the insertion length of the nasal endoscope. For example... Figure 2 The nasal endoscope tube is shown in the image. Figure 2 The center dot is a photosensitive sensor.

[0033] The system receives multiple sets of detection data from photosensitive sensors in groups. The detection data of the photosensitive sensors is the light intensity measured by the photosensitive sensors. Therefore, the real-time light sensing data includes n sets of data, where n is the number of photosensitive sensor groups. Each set of data is the light intensity data measured in real time by a set of photosensitive sensors.

[0034] The same set of photosensitive sensors being evenly distributed on the same cross section of the nasal endoscope tube means that multiple photosensitive sensors of the same set are distributed in the shape of regular polygons on the circular cross section of the nasal endoscope tube. That is, when the number of photosensitive sensors in a set is m, the m photosensitive sensors are distributed in the shape of regular m-gons on the circular cross section of the nasal endoscope tube.

[0035] S220 calculates the average light intensity of each group of photosensitive sensors based on real-time light sensing data, and obtains an average light intensity array.

[0036] It can be understood that the average value of each of the n sets of data in the real-time light sensing data is calculated to obtain n average values. These n average values ​​are arranged in the order of the n sets of photosensitive sensors to form an average light intensity array a[n]. That is, the i-th element a[i] in the average light intensity array a[n] is the average light intensity of the photosensitive sensor sorted into the i-th set.

[0037] S230, find the two adjacent elements with the largest difference in the average light intensity array, and obtain the insertion length of the nasal endoscope based on the number of digits of the two adjacent elements.

[0038] It is understandable that we can subtract every two adjacent elements of the average light intensity array a[n] to get b[n-1], that is, b[i]=a[i+1]-a[i]. Then find the largest element in b[n-1] and set it as b[x]. Then the two adjacent elements with the largest difference in the average light intensity array are a[x+1] and a[x]. The number of digits of the two adjacent elements is x and x+1, which means that the boundary between the inside and outside of the nasal cavity is likely to be between the x and x+1 sets of photosensitive sensors. That is, the entry length of the nasal endoscope is between x×X1 and (x+1)×X1, where X1 is the straight-line distance between the two sets of photosensitive sensors. Therefore, we can consider the entry length of the nasal endoscope to be (x+0.5)×X1.

[0039] This setup allows for quick and real-time acquisition of the nasal endoscope's insertion length, which is beneficial for low-latency synchronous synchronization of the nasal endoscope's tip position within general nasal intramodeling.

[0040] In one possible implementation, in step S200, the position of the nasal endoscope tip in the general intranasal modeling is determined based on the insertion length and lens orientation of the nasal endoscope, including: S240: After the nasal endoscope enters the nasal cavity, the nasal endoscope entry length and lens direction are acquired at a fixed frequency, and the changes in entry length and lens direction are calculated each time.

[0041] It is understandable that the presence of the nasal endoscope in the nasal cavity can be determined by multiple sets of photosensitive sensors in steps S210 to S230, or by a button operated by a medical staff member (this button is pressed when the nasal endoscope has just entered the nasal cavity). After the nasal endoscope enters the nasal cavity, the insertion length and lens direction are acquired at a fixed frequency. The current acquisition of the insertion length and lens direction is subtracted from the previous acquisition to calculate the change in insertion length and lens direction.

[0042] This setup ensures that this step is performed immediately after the nasal endoscope enters the nasal cavity, guaranteeing that the starting point is precisely at the nostril entrance when this step is first executed.

[0043] S250, using the change in length and the change in lens direction as displacement vectors, adds the displacement vector to the previously obtained position of the nasal endoscope tip to obtain the current position of the nasal endoscope tip in the general nasal modeling.

[0044] It can be understood that by taking the change in entry length as the modulus and the change in lens direction as the vector direction, the displacement vector can be obtained. Then, add a displacement vector to the previously obtained position of the nasal endoscope tip. This gives the current position of the nasal endoscope tip in the general nasal model. The previous position of the nasal endoscope tip is traced back to the position of the nasal endoscope tip before that, and so on, until it is traced back to the starting point of the nostril entrance.

[0045] It is important to note that the frequency at which the nasal endoscope insertion length and lens direction are obtained should not be too low; it should be greater than 30Hz to ensure the correct displacement vector is obtained. The process satisfies the idea of ​​differential calculus.

[0046] This setup first ensures that the starting point is the nasal inlet, and then, using the concept of differential calculus, treats the displacement of the endoscope tip as a series of small displacement vectors. Then through a series of displacement vectors The sum of these values ​​determines the position of the nasal endoscope tip in the general intranasal modeling.

[0047] The S300 system waits for and detects the marked information. In general intranasal modeling, using the endoscope tip as the projection point, the real-time lens direction as the projection direction, and the nasal cavity wall as the projection surface, it projects the marked shape from the marked information onto the nasal cavity wall to obtain the actual location of the high-risk area. The location of the high-risk area is then updated to reflect its actual position. Here, the marked information refers to the actual location of the high-risk area marked by medical personnel or artificial intelligence in the endoscopic image; the marked shape is a closed contour, and the actual location of the high-risk area is inside the projected closed contour.

[0048] It's understandable that medical staff can use the touchscreen module to mark the actual location of a high-risk area in an endoscopic image. This marking can be done by selecting a portion of the endoscopic image. In the simulated projection operation within the general nasal intranasal modeling, the marked shape from the marking information is first read as the projection target. Then, the end of the nasal endoscope is used as the projection point, the real-time lens direction as the projection direction, and the nasal cavity wall as the projection surface, thus forming three targets: a point, a line, and a surface. The projection target (marked shape) is projected from the projection point along the projection direction onto the projection surface. The projection on this surface is the actual location of the high-risk area. At this point, the actual location of the high-risk area is determined in the general nasal intranasal modeling. The original high-risk area location is then deleted to update the current location. Since the marked shape is a closed contour, its projection is also a closed contour; the interior of the projected contour on the projection surface is the actual location of the high-risk area.

[0049] Alternatively, an image recognition machine model can be trained to replace medical staff in labeling. The training process can follow the conventional training process for image recognition models.

[0050] This setup effectively takes into account the individual errors of different patients, quickly determines the actual location of high-risk sites, and improves the location accuracy of high-risk sites in general intranasal modeling.

[0051] S400, in general intranasal modeling, selects either a first visual warning or a second visual warning based on whether the location of a high-risk site has been updated; the first visual warning is used to indicate the name and direction of the high-risk site in the endoscopic image, and is used for high-risk sites whose locations have not been updated; the second visual warning is used to indicate the name, direction, and distance of the updated high-risk site in the endoscopic image, and is used for high-risk sites whose locations have been updated.

[0052] Understandably, the endoscopic image should display the names and approximate directions of all high-risk areas. This can be achieved by displaying an indicator around the central crosshair of the endoscopic image. This indicator can rotate 360° and can show the direction of the high-risk area, indicating its projection on the endoscopic image relative to the central crosshair. It can also display the name of the high-risk area – this is the first visual warning. If the location of the high-risk area has been updated, the specific distance can be displayed around the indicator – this is the second visual warning.

[0053] This setup enables early warning of high-risk areas, eliminating the need for medical staff to memorize the types and locations of high-risk areas and reducing the chance of collisions during surgery.

[0054] Optionally, the method also includes: S510, when issuing a first visual warning or a second visual warning, displays an indicator around the central crosshair of the endoscopic image. The indicator moves in a circle around the central crosshair, pointing towards the projection of the high-risk area onto the endoscopic image.

[0055] It is understandable that when conducting first or second visual warnings, an indicator is used to point out the direction of high-risk areas on the endoscopic image. The indicator is a directional marker that moves in a circle around the central crosshair of the endoscopic image, always pointing to the projection of the high-risk area on the endoscopic image. This allows medical staff to simultaneously notice the direction of the high-risk area when operating the nasal endoscope, thereby paying attention to the corresponding direction in advance and reducing the chance of collisions during the operation.

[0056] The S520 dynamically adjusts the size of the indicator marker based on the distance between the endoscope tip and the high-risk area, and determines the color of the indicator marker based on the background color on the endoscopic image. The name of the high-risk area to which the indicator marker is pointed is also marked next to the indicator marker.

[0057] It is understandable that the size of the indicator is variable and negatively correlated with the distance between the endoscope tip and the high-risk area; that is, the farther the endoscope tip is from the high-risk area, the smaller the indicator, and vice versa. Considering that the image color may vary greatly under different locations and viewing angles, the color of the indicator can also be dynamically adjusted so that the indicator is always relative to its background color. The relative color can be determined by the relative position on the 12-color wheel. In addition, to indicate the name of the high-risk area, the name of the high-risk area is marked next to the indicator.

[0058] This setup specifically implements the indication function of first-visual warning or second-visual warning, and indicates relevant information in a more natural and obvious way.

[0059] Optionally, the method may further include the following steps before obtaining a general intranasal model: S530, prepare universal intranasal modeling for different bone ages and genders.

[0060] It is understandable that the nasal cavity locations differ between patients of different bone ages and sexes. Therefore, it is necessary to prepare different universal intranasal models for patients of different bone ages and sexes. A universal intranasal model can be obtained by performing 3D reconstruction of a human body or human specimen. The model records the locations of multiple high-risk sites, including but not limited to: the internal carotid artery, optic nerve, skull base, anterior / posterior ethmoidal artery, sphenopalatine artery, medial orbital wall, pituitary gland, and cavernous sinus. Furthermore, by selecting human bodies or human specimens of different bone ages and sexes as targets for 3D reconstruction, universal intranasal models for different bone ages and sexes can be obtained.

[0061] S540 obtains the patient's bone age and gender, and matches the corresponding general intranasal model based on the patient's bone age and gender.

[0062] It is understandable that the patient's bone age and gender are obtained, and then the corresponding universal intranasal model is matched in a database that records all universal intranasal models.

[0063] This setup can greatly improve the compatibility between general intranasal modeling and the patient's body.

[0064] Optionally, a distance measuring device is provided at the end of the nasal endoscope, and a plane to be measured is provided at the head of the surgical instrument accompanying the nasal endoscope. The distance between the plane to be measured and the tip of the surgical instrument is a fixed first length, and the distance measuring device can detect the distance relative to the plane to be measured; the method further includes: S550: When it is detected that the nasal endoscope is equipped with matching surgical instruments, the distance measuring device is controlled to measure the distance. The measured distance is added to the first length to obtain the extension length of the surgical instruments.

[0065] It is understandable that the ranging device can be a laser rangefinder, and the plane to be measured can be a mirror of any material. When the ranging device detects the reflected laser, it means that the nasal endoscope is equipped with matching surgical instruments. Then the laser rangefinder detects the distance X2 between itself and the plane to be measured. X2 is then added to the distance from the plane to be measured to the top of the surgical instrument (i.e., the first length) to obtain the extension length of the surgical instrument. The extension length refers to the length by which the surgical instrument extends from the end of the nasal endoscope. In other words, the extension length refers to how much the installed matching surgical instruments have been extended.

[0066] S560, when calculating the distance between the tip of the nasal endoscope and the high-risk site, subtract the extension length from the distance between the two.

[0067] It is understandable that, in order to prevent the extended surgical instruments from colliding with high-risk areas, the distance between the tip of the nasal endoscope and the high-risk area is calculated by subtracting the extension length from the distance between them. This is equivalent to extending the tip of the nasal endoscope by an additional length.

[0068] This setup allows for adaptive changes in the position of the nasal endoscope's tip after the accompanying surgical instruments are installed, preventing the instruments from impacting high-risk areas.

[0069] Optionally, the method also includes: S570: A visual warning for a high-risk area is only displayed when the distance between the tip of the nasal endoscope and the high-risk area is less than a safe distance threshold; otherwise, the visual warning is hidden. The high-risk area can be any high-risk area, and the visual warning is either a first visual warning or a second visual warning.

[0070] It is understandable that a visual warning of the high-risk area will only be displayed on the endoscope's display terminal when the tip of the nasal endoscope approaches within a safe distance threshold of the target high-risk area; otherwise, the visual warning of the target high-risk area will be hidden. That is, only when the distance between the tip of the nasal endoscope and the target high-risk area is within a certain range will there be an indicator pointing to the target high-risk area on the display terminal; the indicator will be hidden in other distance situations.

[0071] This setup prevents the endoscope's display terminal from having too many indicators or cluttered information, allowing medical staff to focus on the most important indicators immediately.

[0072] Optionally, the method also includes: S580 sets different safety distance thresholds for different high-risk areas; among them, the safety distance threshold for high-risk areas is positively correlated with the criticality of the high-risk area.

[0073] It is understandable that different high-risk sites have different degrees of criticality. For example, the internal carotid artery is one of the main arteries supplying blood to the brain. If it ruptures during surgery, the mortality rate is extremely high. On the other hand, pituitary damage can affect the secretion of various hormones and cause endocrine disorders. The two are obviously different in terms of criticality. Therefore, it is necessary to set different safe distance thresholds for different high-risk sites. The safe distance threshold for high-risk sites is positively correlated with the criticality of the high-risk site.

[0074] For example, the order of criticality of high-risk sites could be: internal carotid artery > optic nerve > skull base bone > cavernous sinus > anterior / posterior ethmoidal artery > sphenopalatine artery > medial orbital wall > pituitary gland.

[0075] This setup takes into account the criticality of different high-risk areas, and therefore sets different safety distance thresholds, which can improve the collision avoidance priority of high-critical high-risk areas.

[0076] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0077] Corresponding to the anti-collision warning method for nasal endoscopic surgical instruments described in the above embodiments, this application also provides an anti-collision warning system for nasal endoscopic surgical instruments, wherein each unit of the system can realize each step of the anti-collision warning method for nasal endoscopic surgical instruments. Figure 3 The diagram shows a structural block diagram of the anti-collision warning system for nasal endoscopic surgical instruments provided in the embodiments of this application. For ease of explanation, only the parts related to the embodiments of this application are shown.

[0078] Reference Figure 3 The system includes: An acquisition unit is used to acquire endoscopic images and general intranasal modeling; wherein, the general intranasal modeling records the locations of multiple high-risk sites; The detection unit is used to detect the insertion length and lens direction of the nasal endoscope in real time, and determine the position of the end of the nasal endoscope in the general intranasal modeling based on the insertion length and lens direction of the nasal endoscope. A marking unit is used to wait for and detect marking information, and in the general intranasal modeling, using the end of the nasal endoscope as the projection point, the real-time lens direction as the projection direction, and the nasal cavity wall as the projection surface, projects the marking shape in the marking information onto the nasal cavity wall to obtain the actual location of the high-risk area, and then updates the location of the high-risk area to its actual location; wherein, the marking information refers to the actual location of the high-risk area marked by medical personnel or artificial intelligence in the endoscopic image; the marking shape is a closed contour, and the actual location of the high-risk area is inside the projection of the closed contour; The warning unit is used to select a first visual warning or a second visual warning based on whether the location of the high-risk site has been updated in the general intranasal modeling. The first visual warning is used to alert the name and direction of the high-risk site in the endoscopic image for the high-risk site whose location has not been updated. The second visual warning is used to alert the name, direction and distance of the high-risk site in the endoscopic image after the location has been updated for the high-risk site after the location has been updated.

[0079] It should be noted that the information interaction and execution process between the above-mentioned units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, which will not be repeated here.

[0080] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units is used as an example. In practical applications, the above functions can be assigned to different functional units as needed, that is, the internal structure of the device can be divided into different functional units to complete all or part of the functions described above. The functional units in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0081] This application also provides a collision avoidance warning device for nasal endoscopic surgical instruments. Figure 4 This is a schematic diagram of the anti-collision warning device for nasal endoscopic surgical instruments provided in one embodiment of this application. Figure 4 As shown, the control device 5 of the anti-collision warning device for nasal endoscopic surgical instruments in this embodiment includes: at least one processor 50 ( Figure 4 Only one is shown in the image), at least one memory 51 ( Figure 4 (Only one is shown in the image) and a computer program 52 stored in the at least one memory 51 and executable on the at least one processor 50. When the processor 50 executes the computer program 52, it causes the control device 5 of the nasal endoscopic surgical instrument anti-collision warning device to perform the steps in any of the above embodiments of the nasal endoscopic surgical instrument anti-collision warning method, or causes the control device 5 of the nasal endoscopic surgical instrument anti-collision warning device to perform the functions of each unit in the above embodiments of the device.

[0082] Exemplarily, the computer program 52 may be divided into one or more units, which are stored in the memory 51 and executed by the processor 50 to complete this application. The one or more units may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program 52 in the control device 5 of the nasal endoscopic surgical instrument anti-collision warning device.

[0083] The control device 5 of the nasal endoscopic surgical instrument anti-collision warning device can be a microcontroller, microprocessor, mobile phone, tablet computer, wearable device, vehicle-mounted device, laptop computer, ultra-mobile personal computer (UMPC), netbook, personal digital assistant (PDA), desktop computer, smart screen, smart TV, or handheld device with wireless communication function. The control device 5 of the nasal endoscopic surgical instrument anti-collision warning device may include, but is not limited to, a processor 50 and a memory 51. Those skilled in the art will understand that... Figure 4 This is merely an example of the control device 5 for the anti-collision warning device of nasal endoscopic surgical instruments, and does not constitute a limitation on the control device 5 for the anti-collision warning device of nasal endoscopic surgical instruments. It may include more or fewer components than shown, or combine certain components, or different components, such as input / output devices, network access devices, buses, etc.

[0084] The processor 50 can be a Central Processing Unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0085] In some embodiments, the memory 51 may be an internal storage unit of the control device 5 of the endoscopic surgical instrument collision avoidance warning device, such as a hard disk or memory of the control device 5. In other embodiments, the memory 51 may be an external storage device of the control device 5, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the control device 5. Further, the memory 51 may include both internal and external storage units of the control device 5. The memory 51 is used to store the operating system, application programs, bootloader, data, and other programs, such as the program code of the computer program. The memory 51 can also be used to temporarily store data that has been output or will be output.

[0086] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps in any of the above method embodiments.

[0087] This application provides a computer program product that, when run on a nasal endoscopic surgical instrument anti-collision warning device, enables the nasal endoscopic surgical instrument anti-collision warning device to implement the steps in any of the above-described method embodiments.

[0088] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying the computer program code to the anti-collision warning device for nasal endoscopic surgical instruments, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks.

[0089] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0090] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0091] In the embodiments provided in this application, it should be understood that the disclosed endoscopic surgical instrument collision prevention warning method, endoscopic surgical instrument collision prevention warning system, and endoscopic surgical instrument collision prevention warning device can be implemented in other ways. For example, the embodiments of the endoscopic surgical instrument collision prevention warning method, endoscopic surgical instrument collision prevention warning system, and endoscopic surgical instrument collision prevention warning device described above are merely illustrative. For instance, the division of units is merely a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0092] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0093] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for collision prevention and early warning of nasal endoscopic surgical instruments, characterized in that, A collision avoidance warning device for nasal endoscopic surgical instruments, the method comprising: Acquire endoscopic images and general intranasal modeling; wherein, the general intranasal modeling records the locations of multiple high-risk sites; The insertion length and lens direction of the nasal endoscope are detected in real time, and the position of the endoscope tip in the general intranasal modeling is determined based on the insertion length and lens direction of the nasal endoscope. The system waits for and detects the marking information. In the general intranasal modeling, the end of the nasal endoscope is used as the projection point, the real-time lens direction is used as the projection direction, and the nasal cavity wall is used as the projection surface. The marking shape in the marking information is projected onto the nasal cavity wall to obtain the actual location of the high-risk area. The location of the high-risk area is then updated to its actual location. Here, the marking information refers to the actual location of the high-risk area marked by medical personnel or artificial intelligence in the endoscopic image. The marking shape is a closed contour, and the actual location of the high-risk area is inside the projected closed contour. In the general intranasal modeling, a first visual warning or a second visual warning is selected based on whether the location of the high-risk site has been updated. The first visual warning is used to alert the name and direction of the high-risk site in the endoscopic image for high-risk sites whose locations have not been updated. The second visual warning is used to alert the name, direction, and distance of the high-risk site in the updated endoscopic image for high-risk sites whose locations have been updated.

2. The anti-collision warning method for nasal endoscopic surgical instruments as described in claim 1, characterized in that, Multiple sets of photosensitive sensors are provided on the outer surface of the nasal endoscope tube. The same set of photosensitive sensors are evenly distributed on the same cross section of the nasal endoscope tube, and the distance between the multiple sets of photosensitive sensors is the same. The real-time detection of the insertion length of the nasal endoscope includes: Receives real-time light sensing data from multiple photosensitive sensors; The average light intensity of each group of photosensitive sensors is calculated based on the real-time light sensing data to obtain an average light intensity array. Find the two adjacent elements with the largest difference in the average light intensity array, and obtain the insertion length of the nasal endoscope based on the number of digits of the two adjacent elements.

3. The anti-collision warning method for nasal endoscopic surgical instruments as described in claim 1, characterized in that, Determining the position of the nasal endoscope tip in the universal intranasal modeling based on the insertion length and lens direction of the nasal endoscope includes: After the nasal endoscope enters the nasal cavity, the insertion length and lens direction of the nasal endoscope are acquired at a fixed frequency, and the change in insertion length and lens direction are calculated each time. Using the change in entry length and the change in lens direction as displacement vectors, the displacement vectors are added to the previously obtained position of the nasal endoscope tip to obtain the current position of the nasal endoscope tip in the general intranasal modeling.

4. The anti-collision warning method for nasal endoscopic surgical instruments as described in claim 1, characterized in that, The method further includes: When the first visual warning or the second visual warning is issued, an indicator is displayed around the central crosshair of the endoscopic image; wherein, the indicator moves in a circle around the central crosshair and points to the projection of the high-risk area on the endoscopic image; Dynamically adjust the size of the indicating marker according to the distance between the end of the nasal endoscope and the high-risk site, and determine the color of the indicating marker according to the background color on the endoscopic image. The name of the high-risk site pointed to is also marked beside the indicating marker.

5. The anti-collision warning method for nasal endoscopic surgical instruments as described in claim 1, characterized in that, Before obtaining the general nasal internal model, the method further includes: Prepare the general nasal internal models of different genders with different bone ages; Obtain the bone age and gender of the patient, and match the corresponding general nasal internal model according to the bone age and gender of the patient.

6. The anti-collision warning method for nasal endoscopic surgical instruments as described in claim 1, characterized in that, A distance measuring device is provided at the end of the nasal endoscope, and a measurement plane is provided at the head of the supporting surgical instrument of the nasal endoscope. The distance between the measurement plane and the top end of the supporting surgical instrument is a fixed first length, and the distance measuring device is used to detect the distance between it and the measurement plane; the method further includes: When it is detected that the nasal endoscope is equipped with a supporting surgical instrument, control the distance measuring device to measure the distance, and add the first length to the measured distance to obtain the extension length of the surgical instrument; When calculating the distance between the end of the nasal endoscope and the high-risk site, subtract the extension length from the distance between the two.

7. The anti-collision warning method for nasal endoscopic surgical instruments as described in claim 1, characterized in that, The method further includes: When the distance between the end of the nasal endoscope and the target high-risk site is less than the safety distance threshold, display the visual warning of the target high-risk site; otherwise, hide the visual warning of the target high-risk site; wherein, the target high-risk site is any one of the high-risk sites, and the visual warning is the first visual warning or the second visual warning.

8. The anti-collision warning method for nasal endoscopic surgical instruments as described in claim 7, characterized in that, The method further includes: Set different safety distance thresholds for different high-risk sites; wherein, the safety distance threshold of the high-risk site is positively correlated with the criticality of the high-risk site.

9. A collision avoidance warning system for nasal endoscopic surgical instruments, characterized in that, Applied to a collision prevention warning device for nasal endoscopic surgical instruments, the system includes: An acquisition unit, configured to acquire an endoscopic image and a general nasal internal model; wherein, the positions of multiple high-risk sites are recorded in the general nasal internal model; A detection unit, configured to detect the insertion length and the lens direction of the nasal endoscope in real time, and determine the position of the end of the nasal endoscope in the general nasal internal model according to the insertion length and the lens direction of the nasal endoscope; A marking unit, configured to wait for and detect marking information, and in the general nasal internal model, project the marking shape in the marking information with the end of the nasal endoscope as the projection point, the real-time lens direction as the projection direction, and the nasal cavity inner wall as the projection plane. The projection on the nasal cavity inner wall obtains the actual position of the high-risk site, and then update the position of the high-risk site to the actual position of the high-risk site; wherein, the marking information refers to the actual position of the high-risk site marked by medical staff or artificial intelligence in the endoscopic image; the marking shape is a closed contour, and the actual position of the high-risk site is the inside of the contour of the projected closed contour. The warning unit is used to select a first visual warning or a second visual warning based on whether the location of the high-risk site has been updated in the general intranasal modeling. The first visual warning is used to alert the name and direction of the high-risk site in the endoscopic image for the high-risk site whose location has not been updated. The second visual warning is used to alert the name, direction and distance of the high-risk site in the endoscopic image after the location has been updated for the high-risk site after the location has been updated.

10. A collision avoidance warning device for nasal endoscopic surgical instruments, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 8.

Images