An endoscope-based imaging display method and a computer readable storage medium
By automatically identifying lesion areas and adjusting the display range through endoscopic imaging, the problem of lesion areas being difficult to observe in the display screen is solved, improving the convenience and efficiency of observation and surgical operations.
Patent Information
- Application Number
- CN202410002101.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-01-02
- Publication Date
- 2026-11-10
- Estimated Expiration
- 2044-01-02
AI Technical Summary
During endoscopic imaging, the location of the lesion area in the display screen is not conducive to observation, which increases the difficulty of observation and surgical operation, and requires a high level of skill from the operator.
The endoscopic imaging system automatically identifies the outline of the lesion area and adjusts the display range based on the center or outline of the lesion area, prioritizing the display of the lesion area. It also optimizes the display center and size by combining the projection range of the surgical instruments to facilitate observation and operation.
It reduces the difficulty of adjusting the imaging screen when manually operating the endoscope, improves the observation efficiency of the lesion area and the convenience of surgical operation, and reduces the skill requirements of the operator.
Smart Images

Figure CN117618127B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of endoscopic image processing technology, and more specifically to an endoscope-based imaging display method and a computer-readable storage medium. Background Technology
[0002] An endoscope is a commonly used medical device consisting of a beam guide structure and an imaging component. It enters the body through natural openings or small incisions, providing external imaging for the examination and surgical treatment of organs or tissues. Clinically, the commonly used laparoscope is a rigid endoscope, which uses a series of cylindrical optical lenses to transmit images to an eyepiece, and then performs imaging through an independent imaging component.
[0003] Common imaging components include camera-based and ultrasound-based imaging components. During endoscopy, the imaging component and beam structure transmit the image to a display device for real-time viewing. However, in some cases, the display position of the area requiring close observation is not conducive to observation, creating observational obstacles for the observer and operational difficulties for the surgical operator. Summary of the Invention
[0004] The technical problem to be solved by this application is to provide an endoscope-based imaging display method and a computer-readable storage medium, which has the characteristics of enabling better observation of lesion areas or surgical operations on lesion areas, and reducing the difficulty of adjusting and controlling the imaging screen when manually operating the endoscope.
[0005] In a first aspect, one embodiment provides an endoscope-based imaging display method, comprising: Obtain endoscopic images; Based on the endoscopic imaging, the presence of lesion areas is identified. If so, the edges of the lesion areas are identified to obtain the outline of the lesion areas. The endoscopic imaging is displayed based on the outline of the lesion area.
[0006] In one embodiment, the method of displaying endoscopic images based on the contour range of the lesion region includes: The endoscope image is displayed with the center point of the outline of the lesion area as the display center.
[0007] In one embodiment, displaying endoscopic images with the center point of the outline of the lesion region as the display center includes: The endoscope image is displayed with the center of the circumcircle of the outline of the lesion area as the display center.
[0008] In one embodiment, the method of displaying endoscopic images based on the contour range of the lesion region includes: The outline of the lesion region is obtained as the first range; The projection range of the surgical instruments at the tip of the endoscope is used as the second range; The endoscope image is displayed with the center point of the third range that surrounds the first and second ranges as the display center.
[0009] In one embodiment, the third range includes a circular range inscribed in the circumcircle of the first range and the circumcircle of the second range, and the center of the circular range lies on a straight line determined by the center of the circumcircle of the first range and the center of the circumcircle of the second range.
[0010] In one embodiment, the method of displaying endoscopic images based on the contour range of the lesion region includes: The outline of the lesion area is greater than or equal to the window boundary of the display window.
[0011] In one embodiment, the step of identifying the presence of a lesion region based on the endoscopic imaging, and if so, identifying the edge of the lesion region to obtain the contour of the lesion region, includes: Based on the endoscopic imaging, it is determined whether there is only one lesion area. If so, the edge of the only lesion area is identified to obtain the outline of the lesion area.
[0012] In one embodiment, the step of identifying the presence of a lesion region based on the endoscopic imaging, and if so, identifying the edge of the lesion region to obtain the contour of the lesion region, includes: Based on the endoscopic imaging, identify whether there are two or more lesion areas. If so, obtain the edges of the two or more lesion areas respectively to obtain the outlines of the two or more lesion areas. Based on the outlines of the two or more lesion areas, take the lesion area with the largest outline range as the main lesion area. Identify the lesion area with the largest lesion area range as the main lesion area, and identify the edges of the main lesion area to obtain the outline of the main lesion area. The method of displaying endoscopic images based on the contour range of the lesion region includes: The endoscopic imaging is displayed based on the outline of the main lesion area.
[0013] In one embodiment, the step of identifying the presence of a lesion region based on endoscopic imaging, and if so, identifying the edge of the lesion region to obtain its outline, further includes: Identify the relative positional relationship between other lesion areas and the main lesion area; The method of displaying endoscopic images based on the outline of the main lesion region includes: Mark the corresponding position at the edge of the basic display area.
[0014] In a second aspect, one embodiment provides a computer-readable storage medium storing a program that can be loaded by a processor and executed by any of the endoscope-based imaging and display methods described in the above embodiments.
[0015] The beneficial effects of this invention are: Because the lesion area can be automatically captured based on the endoscope's imaging and the basic display range in the display screen can be adjusted based on the lesion area, doctors and other observers or surgeons can better observe or perform surgical operations on the lesion area, reducing the difficulty of adjusting and controlling the imaging screen when manually operating the endoscope, and reducing the requirements for the operator's skill level. Attached Figure Description
[0016] Figure 1 This is a schematic diagram of a lesion area displayed in a prior art display screen according to an embodiment of this application; Figure 2 This is a schematic flowchart of an endoscope-based imaging display method according to an embodiment of this application; Figure 3 This is one embodiment of the present application based on Figure 1 The illustrated embodiment is a schematic diagram of the search display center; Figure 4 This is one embodiment of the present application based on Figure 1 The diagram shown illustrates the adjusted display result of the embodiment. Figure 5 This application Figure 2 A schematic diagram of the method flow of one embodiment of step S30; Figure 6 This is a schematic diagram of a lesion area displayed in a prior art display screen with reference to surgical instrument projection, according to an embodiment of this application; Figure 7 This is one embodiment of the present application based on Figure 6 The illustrated embodiment is a schematic diagram of the search display center; Figure 8 This is one embodiment of the present application. Figure 6The diagram shown illustrates the adjusted display result of the embodiment. Figure 9 This application Figure 2 A schematic diagram of the method flow of one embodiment of step S20; Figure 10 This is a schematic diagram of a prior art display screen showing multiple lesion areas according to an embodiment of this application; Figure 11 This is one embodiment of the present application based on Figure 10 The illustrated embodiment is a schematic diagram of the display result after adjustment. Detailed Implementation
[0017] The present invention will now be described in further detail with reference to specific embodiments and accompanying drawings. Similar elements in different embodiments are referred to by associated similar element reference numerals. In the following embodiments, many details are described to facilitate a better understanding of this application. However, those skilled in the art will readily recognize that some features may be omitted in different situations, or may be replaced by other elements, materials, or methods. In some cases, certain operations related to this application are not shown or described in the specification. This is to avoid obscuring the core parts of this application with excessive description. For those skilled in the art, detailed description of these related operations is not necessary; they can fully understand the related operations based on the description in the specification and general technical knowledge in the art.
[0018] Furthermore, the features, operations, or characteristics described in the specification can be combined in any suitable manner to form various embodiments. At the same time, the steps or actions in the method description can be rearranged or adjusted in a manner obvious to those skilled in the art. Therefore, the various orders in the specification and drawings are only for the clear description of a particular embodiment and do not imply a necessary order, unless otherwise stated that a particular order must be followed.
[0019] The serial numbers assigned to components in this document, such as "first" and "second," are used only to distinguish the described objects and have no sequential or technical meaning. The terms "connection" and "linkage" used in this application, unless otherwise specified, include both direct and indirect connections (linkages).
[0020] To facilitate the explanation of the inventive concept of this application, the following is a brief description of the endoscope and its imaging and display technologies.
[0021] Endoscopic imaging primarily includes camera-based imaging and ultrasound-based imaging; therefore, the endoscope tip is equipped with a camera or ultrasound component. During endoscopic surgery, the endoscope tip is inserted into the body through natural openings or a small incision. The endoscopic image is then transmitted via a beam guide and displayed on a display device.
[0022] As the imaging image is displayed, if a lesion area is present in the image, the lesion area can be seen on the displayed image. However, please refer to... Figure 1 In some cases, the lesion area is displayed on the edge of the display window, while the lesion area is an area that needs to be observed in detail or is the area to be operated on. Manually operating the endoscope to adjust the image is difficult and requires a high level of skill from the operator.
[0023] In view of this, one embodiment of this application provides an endoscope-based imaging display method. Based on this display method, the lesion area of the endoscope image can be automatically captured, and the basic display range in the display screen can be adjusted based on the lesion area. This allows doctors or other observers or surgeons to better observe or perform surgical operations on the lesion area, reducing the difficulty of adjusting and controlling the imaging screen when manually operating the endoscope, as well as the skill level of the operator.
[0024] Please refer to Figure 2 The endoscopic-based imaging display method specifically includes: Step S10: Obtain endoscopic images.
[0025] The endoscopic imaging includes the imaging obtained by the endoscopic imaging component. In some embodiments, the imaging component can be a camera capturing component, an ultrasonic detection component for ultrasonic imaging, or a radar scanning component for radar scanning imaging, etc.
[0026] Step S20: Based on endoscopic imaging, identify whether there is a lesion area. If so, identify the edge of the lesion area to obtain the outline of the lesion area.
[0027] Based on the acquired endoscopic images, the presence of lesion areas is automatically identified. If so, the edges of the lesion areas are automatically acquired to obtain the contour of the lesion areas. In one embodiment, lesion areas are automatically identified based on a lesion area detection method, which can be a deep learning-based lesion area detection neural network.
[0028] In one embodiment, endoscopic imaging is used to identify whether there is only one lesion area. If so (see reference...), Figure 1 If only one lesion area is shown, then the edge of the only existing lesion area is identified to obtain the outline of the lesion area.
[0029] Step S30: Display the endoscopic image based on the outline of the lesion area.
[0030] In one embodiment, the endoscopic image is displayed with the center point of the contour range of the lesion area as the display center. Please refer to... Figure 3 Specifically, in one embodiment, a circumcircle of the lesion region is defined, and the endoscope image is displayed with the center of this circumcircle as the display center. Please refer to [reference needed]. Figure 4 As the final display result of one embodiment, in order to display the lesion area more clearly, the outline of the lesion area is made to be greater than or equal to the window boundary of the display window. The first preset boundary distance d1 can be set based on actual needs.
[0031] In one embodiment, the endoscope image is displayed proportionally with the center point of the outline of the lesion area as the display center, that is, only the display position is adjusted, and the display size is not adjusted.
[0032] In one embodiment, the center point of the lesion region's outline is used as the display center, and the minimum distance between the outline of the lesion region and the window boundary of the display window is used as the first preset boundary distance d1 to display the endoscope image. That is, both the display position and the display size are adjusted; please refer to [reference needed]. Figure 4 The display results are set with the minimum distance between the outline of the lesion area and the window boundary of the display window as the first preset boundary distance d1. The image is magnified to adjust the size of the displayed image so as to better observe the lesion area.
[0033] In one embodiment, please refer to Figure 5 The outline of the lesion area serves as the basis for the endoscopic imaging, including: Step S301: Obtain the outline range of the lesion area as the first range.
[0034] Step S302: Obtain the surgical instrument projection range at the tip of the endoscope as the second range.
[0035] Surgical instruments at the tip of the endoscope include biopsy forceps, stone retrieval baskets, etc. Different instruments can be selected depending on the situation. When operating the surgical instruments at the tip of the endoscope on the lesion area, in addition to focusing on observing the lesion area, it is also necessary to focus on observing the movement of the surgical instruments. In this embodiment, please refer to... Figure 6 The second range is obtained by taking the projection range of the surgical instruments at the tip of the endoscope (since the projection of the surgical instruments does not actually exist in the display screen, it is represented by a dashed line in the figure) to facilitate the observation of the movement of the surgical instruments.
[0036] Step S303: Display the endoscope image with the center point of the third range that surrounds the first and second ranges as the display center.
[0037] The center point of the third range, which includes the first range containing the lesion area and the second range containing the projection range of the surgical instruments at the tip of the endoscope, is used as the display center to display the image of the endoscope. This makes it easy to observe the lesion area, the movement of the surgical instruments, and the process of the surgical instruments acting on the lesion area.
[0038] Those skilled in the art will understand that as the surgical instruments move closer to the lesion area, the relative positions of the first and second extents change, and the third extent will also change.
[0039] In one embodiment, please refer to Figure 7 The third range includes a circular range inscribed in the circumcircle of the first range and the circumcircle of the second range, and the center of this circular range lies on a straight line defined by the centers of the circumcircles of the first and second ranges. In one embodiment, since the projection of the selected surgical instrument is circular, the circumcircle of the circular projection range is the circular outline of the circular projection range.
[0040] In one embodiment, the endoscope image is displayed with the center point of a third region surrounding the first and second regions as the display center. Please refer to... Figure 8 This is the final display result of one embodiment. To more clearly display the lesion area and surgical instruments, the outline of the lesion area is made to be greater than or equal to a first preset boundary distance d1 from the window boundary of the display window. This first preset boundary distance d1 can be set based on actual needs. In one embodiment, please refer to... Figure 8 To display surgical instruments more clearly, the outline of the projected contour of the surgical instrument is made to be greater than or equal to a second preset boundary distance d2 between the outline of the projection of the surgical instrument and the window boundary of the display window. This second preset boundary distance d2 can be set based on actual needs. In one embodiment, a first preset boundary distance d1 is equal to the second preset boundary distance d2.
[0041] In one embodiment, the endoscope image is displayed proportionally with the center point of the third range as the display center, that is, only the display position is adjusted, and the display size is not adjusted.
[0042] In one embodiment, the center point of the third range is used as the display center, the minimum distance between the outline of the lesion area and the window boundary of the display window is used as the first preset boundary distance d1, and the minimum distance between the outline of the surgical instrument projection and the window boundary of the display window is used as the second preset boundary distance d2, to display the endoscope image. That is, both the display position and the display size are adjusted. Please refer to [reference needed]. Figure 8 The display results are set with the minimum distance between the outline of the lesion area and the window boundary of the display window as the first preset boundary distance d1, and the minimum distance between the outline of the surgical instrument projection and the window boundary of the display window as the second preset boundary distance d2. The displayed image is magnified to adjust the size of the displayed image so as to better observe the lesion area.
[0043] In one embodiment, please refer to Figure 9 Step S20 includes: Step S201: Based on endoscopic imaging, identify whether there are two or more lesion areas. If so, obtain the edges of the two or more lesion areas respectively to obtain the outlines of the two or more lesion areas respectively.
[0044] The system automatically identifies lesion areas in the acquired endoscopic images. If there are two or more lesion areas, please refer to [the relevant documentation]. Figure 10 If there are three lesion areas, the edges of the three lesion areas are obtained respectively to obtain the outlines of the three lesion areas.
[0045] Step S202: Based on the contours of the two or more lesion regions, the lesion region with the largest contour range is identified as the main lesion region.
[0046] Please refer to Figure 10 Based on the obtained contours of the three lesion regions, the size relationship of the contour ranges of the three lesion regions can be obtained. In one embodiment, the size relationship can be obtained by comparing the area of the calculated contour ranges of the three lesion regions, or it can be obtained by other existing technical methods.
[0047] Step S203: Identify the edge of the main lesion area to obtain the outline of the main lesion area.
[0048] Step S30 includes: displaying the endoscopic image based on the outline of the main lesion area.
[0049] Based on this implementation plan, when there are more than two lesion areas in endoscopic imaging, priority should be given to observing and / or treating the largest lesion area, which is beneficial for observing and analyzing the most important lesion or cause.
[0050] In one embodiment, when there are two or more lesion regions in the endoscopic imaging, step S20 further includes: identifying the relative positional relationship between the other lesion regions and the main lesion region. In step S30, displaying the endoscopic image based on the outline range of the main lesion region includes: marking at corresponding positions at the edges of the basic display range.
[0051] In one embodiment, please refer to Figure 10 The largest lesion area in the center is the main lesion area. The relative positions of the other two lesion areas to the main lesion area are as follows: one is located to the upper left of the main lesion area, and the other is located to the lower left of the main lesion area. When displaying the endoscope image based on the outline of the main lesion area, corresponding positions are marked at the edges of the basic display area. Regarding the marking method, in one embodiment, brightness markers can be used at the corresponding positions, or simply markers such as "∆" can be used. In one embodiment, please refer to... Figure 11 The lesions can be sorted based on their size, and a sorting number can be marked in the corresponding position. Since the size of the upper left lesion area is larger than that of the lower left lesion area, the upper left lesion area is represented by number ①, and the lower left lesion area is represented by number ②. The corresponding numbers are then displayed in the appropriate positions.
[0052] This marking method serves two purposes: firstly, it prevents observers or operators from overlooking or forgetting other lesion areas due to their absence in the displayed image; secondly, it guides further operational directions and / or control methods. At this point, the operator can obtain the main display interface of other lesion areas by changing the angle of the imaging component or switching the imaging mode.
[0053] The endoscope-based imaging display method described above automatically captures the lesion area of the endoscope image based on the endoscope imaging, and adjusts the basic display range in the display screen based on the lesion area. This allows doctors or other observers or surgeons to better observe or perform surgical operations on the lesion area, reducing the difficulty of adjusting and controlling the imaging screen when manually operating the endoscope, and also reducing the requirements for the operator's skill level.
[0054] One embodiment of this application provides a computer-readable storage medium storing a program, the stored program including methods that can be loaded by a processor and processed in any of the above embodiments.
[0055] Those skilled in the art will understand that all or part of the functions of the various methods in the above embodiments can be implemented by hardware or by computer programs. When all or part of the functions in the above embodiments are implemented by computer programs, the program can be stored in a computer-readable storage medium, which may include: read-only memory, random access memory, disk, optical disk, hard disk, etc., and the program is executed by a computer to achieve the above functions. For example, the program can be stored in the memory of a device, and when the program in the memory is executed by the processor, all or part of the above functions can be achieved. In addition, when all or part of the functions in the above embodiments are implemented by computer programs, the program can also be stored in a server, another computer, disk, optical disk, flash drive, or external hard drive, etc., and can be downloaded or copied to the memory of a local device, or the system of the local device can be updated. When the program in the memory is executed by the processor, all or part of the functions in the above embodiments can be achieved.
[0056] The above examples illustrate the present invention only to aid in understanding it and are not intended to limit the scope of the invention. Those skilled in the art can make various simple deductions, modifications, or substitutions based on the principles of this invention.
Claims
1. An imaging display method based on an endoscope, characterized in that, include: Obtain endoscopic images; Based on the endoscopic imaging, the presence of lesion areas is identified. If so, the edges of the lesion areas are identified to obtain the outline of the lesion areas. The endoscopic image is displayed based on the outline of the lesion area. The method of displaying endoscopic imaging based on the outline range of the lesion region includes: the distance between the outline of the lesion region and the window boundary of the display window is greater than or equal to a preset boundary distance.
2. The endoscope-based imaging and display method as described in claim 1, characterized in that, The method of displaying endoscopic images based on the contour range of the lesion region includes: The endoscope image is displayed with the center point of the outline of the lesion area as the display center.
3. The endoscope-based imaging and display method as described in claim 2, characterized in that, The method of displaying endoscopic images with the center point of the outline of the lesion region as the display center includes: The endoscope image is displayed with the center of the circumcircle of the outline of the lesion area as the display center.
4. The endoscope-based imaging and display method as described in claim 1, characterized in that, The method of displaying endoscopic images based on the contour range of the lesion region includes: The outline of the lesion region is obtained as the first range; The projection range of the surgical instruments at the tip of the endoscope is used as the second range; The endoscope image is displayed with the center point of the third range that surrounds the first and second ranges as the display center.
5. The endoscope-based imaging and display method as described in claim 4, characterized in that, The third range includes a circular range inscribed in the circumcircle of the first range and the circumcircle of the second range, and the center of the circular range lies on a straight line determined by the center of the circumcircle of the first range and the center of the circumcircle of the second range.
6. The endoscope-based imaging and display method as described in any one of claims 1 to 5, characterized in that, The method of identifying the presence of a lesion region based on endoscopic imaging, and if so, identifying the edge of the lesion region to obtain its outline, includes: Based on the endoscopic imaging, it is determined whether there is only one lesion area. If so, the edge of the only lesion area is identified to obtain the outline of the lesion area.
7. The endoscope-based imaging and display method as described in any one of claims 1 to 5, characterized in that, The method of identifying the presence of a lesion region based on endoscopic imaging, and if so, identifying the edge of the lesion region to obtain its outline, includes: Based on the endoscopic imaging, identify whether there are two or more lesion areas. If so, obtain the edges of the two or more lesion areas respectively to obtain the outlines of the two or more lesion areas. Based on the outlines of the two or more lesion areas, take the lesion area with the largest outline range as the main lesion area. Identify the lesion area with the largest lesion area range as the main lesion area, and identify the edges of the main lesion area to obtain the outline of the main lesion area. The method of displaying endoscopic images based on the contour range of the lesion region includes: The endoscopic imaging is displayed based on the outline of the main lesion area.
8. The endoscope-based imaging and display method as described in claim 7, characterized in that, The method of identifying the presence of a lesion region based on endoscopic imaging, and if so, identifying the edge of the lesion region to obtain its outline, further includes: Identify the relative positional relationship between other lesion areas and the main lesion area; The method of displaying endoscopic images based on the outline of the main lesion region includes: Mark the corresponding position at the edge of the basic display area.
9. A computer-readable storage medium, characterized in that, The medium stores a program that can be loaded by a processor and executed as described in any one of claims 1 to 8.
Citation Information
Patent Citations
Surgical field tracking and adjusting method and surgical robot system
CN114948209A
Program, information processing method, and information processing device
CN116583215A
Fluorescent endoscope equipment
JP2004024656A
Systems and methods for masking endoscopic images
US20230209008A1
Diagnosis method and ultrasound information display system therefor
WO2002041751A2
