Endoscope image display method, device and equipment and computer readable storage medium
By displaying target images on an endoscopic monitor and combining them with a lesion atlas database, the problem of young doctors' inability to accurately determine the nature of lesions has been solved, enabling rapid and accurate lesion diagnosis and improving the homogenization of medical quality.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SHENZHEN CONCEMED MEDICAL TECHNOLOGY CO LTD
- Filing Date
- 2025-12-12
- Publication Date
- 2026-04-10
AI Technical Summary
When displaying images, existing medical endoscope equipment makes it difficult for young or inexperienced doctors to quickly and accurately determine the nature of lesions, leading to large differences in diagnosis and affecting the homogenization of medical quality.
While displaying the target image on the endoscope monitor, the target lesion atlas is identified and displayed using a lesion atlas database through feature extraction and similarity calculation, assisting doctors in diagnosis.
It improved the speed and accuracy of doctors' judgment on the nature of lesions, reduced diagnostic discrepancies, and enhanced the homogenization of medical quality.
Smart Images
Figure CN121817763A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of endoscopy technology, and in particular to an endoscopy image display method, apparatus, device, and computer-readable storage medium. Background Technology
[0002] Medical endoscopes are essential equipment for doctors to detect lesions inside a patient's body. Endoscopes can capture images of lesions in real time and display them to the doctor.
[0003] Existing standard medical endoscope equipment only displays the images captured by the endoscope on the monitor. Doctors need to rely on memory or experience to judge the nature of the lesions shown in the images. However, for young or inexperienced doctors, it is difficult to quickly and accurately judge the nature of the lesions shown in the images, resulting in large diagnostic differences between different doctors in the endoscope images, leading to a decline in the homogenization of overall medical quality. Summary of the Invention
[0004] In view of this, the purpose of this application is to overcome the shortcomings of the prior art and provide an endoscopic image display method, the method comprising: Acquire the target image captured by the endoscope and display the target image in the first display area of the monitor corresponding to the endoscope; Based on the target image, the target lesion atlas is determined in a preset lesion atlas database; The target lesion atlas is displayed in the second display area of the monitor, so that the doctor can make a lesion diagnosis on the target image by combining the target lesion atlas.
[0005] In one embodiment, the lesion atlases in the preset lesion atlas database are stored according to imaging mode and tissue type; the step of determining the target lesion atlas in the preset lesion atlas database based on the target image includes: Identify the imaging pattern, tissue type, and lesion type corresponding to the target image; Based on the imaging mode, the tissue type, and the lesion type, a target atlas set is determined in a preset lesion atlas database; Based on the target image, target lesion atlases are determined in the target atlas set.
[0006] In one embodiment, the step of comparing the target image with each lesion atlas in the target atlas set to determine the overall similarity between the target image and each lesion atlas includes: The target image and each lesion map in the target map set are respectively input into a preset feature extraction network to obtain a first feature map corresponding to the target image and a second feature map corresponding to each lesion map. Based on the first feature map and each of the second feature maps, the overall similarity between the target image and each of the lesion atlases is determined respectively; Based on the overall similarity, the target lesion atlas is determined in the target atlas set.
[0007] In one embodiment, the step of determining the overall similarity between the target image and each of the lesion atlases based on the first feature map and each of the second feature maps includes: The first feature map and each of the second feature maps are normalized in the channel dimension to obtain the first normalized feature map and the second normalized feature map. Based on the preset spatial displacement value, a correlation body is constructed between the first normalized feature map and each of the second normalized feature maps; Based on each of the aforementioned correlation bodies, the overall similarity between the target image and each of the aforementioned lesion atlases is determined.
[0008] In one embodiment, the step of constructing a correlation body between the first normalized feature map and each of the second normalized feature maps based on a preset spatial displacement value includes: For each spatial location in the first normalized feature map, a target matching region corresponding to the spatial location is determined in each of the second normalized feature maps based on the spatial location and a preset spatial displacement value. Calculate the local similarity between the spatial location and the target matching region; Based on the local similarity between each spatial location and the target matching region in each of the second normalized feature maps, a correlation body is constructed between the first normalized feature map and each of the second normalized feature maps.
[0009] In one embodiment, the step of determining the target lesion atlas in the target atlas set based on the overall similarity includes: Based on the overall similarity between the target image and each lesion atlas, all lesion atlases in the target atlas set are sorted in descending order of overall similarity to obtain a lesion atlas similarity sequence. The first preset number of lesion atlases in the lesion atlas similarity sequence are identified as target lesion atlases.
[0010] In one embodiment, the step of displaying the target lesion atlas in the second display area of the display includes: In response to a map display command, the target lesion map is displayed in the second display area of the display; or When a suspected lesion is identified in the target image, the atlas of the target lesion is displayed in the second display area of the display.
[0011] This application also provides an endoscope image display device, the endoscope image display device comprising: The first display module is used to acquire the target image collected by the endoscope and display the target image in the first display area of the display corresponding to the endoscope. The determination module is used to determine the target lesion atlas in a preset lesion atlas database based on the target image; The second display module is used to display the target lesion atlas in the second display area of the display, so that the doctor can make a lesion diagnosis on the target image by combining the target lesion atlas.
[0012] This application also provides an endoscope device, which includes a processor and a memory, the memory storing a computer program, and the processor executing the computer program to implement the above-described endoscopic image display method.
[0013] This application also provides a computer-readable storage medium storing a computer program that, when run on a processor, executes the above-described endoscopic image display method.
[0014] The embodiments of this application have the following beneficial effects: This application embodiment acquires a target image obtained by an endoscope and displays the target image in a first display area of a monitor corresponding to the endoscope; based on the target image, a target lesion atlas is determined in a preset lesion atlas database; the target lesion atlas is displayed in a second display area of the monitor, so that the doctor can complete the lesion diagnosis of the target image by combining the target lesion atlas. By displaying the target image obtained by the endoscope and the target lesion atlas corresponding to the target image on the monitor corresponding to the endoscope, the doctor can quickly and accurately determine the nature of the lesion shown in the target image, reduce the diagnostic differences between different doctors, and improve the homogenization of the overall medical quality. Attached Figure Description
[0015] To more clearly illustrate the technical solutions of this application, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of this application and therefore should not be considered as a limitation on the scope of protection of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.
[0016] Figure 1 A schematic flowchart of the first embodiment of the endoscopic image display method provided in this application; Figure 2 A schematic diagram of the structure of the medical endoscope device provided in this application; Figure 3 This is a schematic diagram of the lesion atlas provided in this application; Figure 4 A schematic diagram of the display of the target image and target lesion atlas provided in this application; Figure 5 A flowchart illustrating a second embodiment of the endoscopic image display method provided in this application; Figure 6 A schematic diagram of the storage structure of the lesion atlas database provided in this application; Figure 7 A schematic flowchart of the third embodiment of the endoscopic image display method provided in this application; Figure 8 A schematic flowchart of the fourth embodiment of the endoscopic image display method provided in this application; Figure 9 A flowchart illustrating the fifth embodiment of the endoscopic image display method provided in this application; Figure 10 This is a schematic diagram of the endoscopic image display device provided in this application. Detailed Implementation
[0017] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.
[0018] The components of the embodiments of this application described and illustrated in the accompanying drawings can be arranged and designed in a variety of different configurations. Therefore, the following detailed description of the embodiments of this application provided in the drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.
[0019] In the following, the terms “comprising,” “having,” and their cognates, which may be used in various embodiments of this application, are intended only to indicate a particular feature, number, step, operation, element, component, or combination thereof, and should not be construed as excluding, firstly, the presence of one or more other features, numbers, steps, operations, elements, components, or combinations thereof, or adding the possibility of one or more features, numbers, steps, operations, elements, components, or combinations thereof.
[0020] Furthermore, the terms "first," "second," and "third" are used only to distinguish descriptions and should not be interpreted as indicating or implying relative importance.
[0021] Unless otherwise specified, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which the various embodiments of this application pertain. Terms (such as those defined in commonly used dictionaries) shall be interpreted as having the same meaning as in their contextual meaning in the relevant technical field and shall not be construed as having an idealized or overly formal meaning, unless clearly defined in the various embodiments of this application.
[0022] It is understood that the method of this application is applied to an endoscopic image display system, which can be mounted in a medical endoscope device. For ease of description, the following embodiments use an endoscopic image display system as the execution subject for illustration.
[0023] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.
[0024] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating a first embodiment of the endoscopic image display method provided in this application, the method comprising: Step S101: Acquire the target image captured by the endoscope and display the target image in the first display area of the monitor corresponding to the endoscope.
[0025] In this embodiment, the endoscopic image display system acquires the target image captured by the endoscope during operation and displays the target image in the first display area of the monitor corresponding to the endoscope. Specifically, as shown... Figure 2 As shown, the medical endoscope device provided in this application includes an endoscope 10, an endoscope processor 20, and a display 30. The endoscope image display system operates in the endoscope processor 20. During operation, the endoscope 10 acquires a target image and transmits it to the endoscope processor 20. The endoscope processor 20 transmits the target image to the display 30, and the first display area of the display 30 displays the target image.
[0026] Step S102: Based on the target image, determine the target lesion atlas in a preset lesion atlas database.
[0027] In this embodiment, after acquiring the target image, the endoscopic image display system determines the target lesion atlas from a preset lesion atlas database. The preset lesion atlas database is stored in the memory of the endoscope processor. It should be noted that the lesion atlas includes lesion images, lesion names, lesion properties, lesion definitions, and diagnostic techniques, etc.; for example, taking the esophageal ectopic sebaceous gland atlas as an example: the lesion image is as follows... Figure 3 As shown. Lesion nature: benign disease. Lesion definition: Ectopic sebaceous glands occasionally found in organs of ectoderm origin such as the lips, oral cavity, and salivary glands; when they appear in the esophagus of endoderm origin, they are called esophageal ectopic sebaceous glands. Diagnostic techniques: (a) Ectopic sebaceous glands are mostly multiple yellow flat lesions; under white light, yellowish-white flat lesions can be seen in the esophagus; (b) Under white light, close-range endoscopic observation shows chrysanthemum-like lesions clustered together with a white protrusion in the center; the lesion boundaries are indistinct, and single lesions are relatively rare.
[0028] Step S103: The target lesion atlas is displayed in the second display area of the display so that the doctor can make a lesion diagnosis on the target image by combining the target lesion atlas.
[0029] In this embodiment, after the endoscopic image display system determines the target lesion atlas, it displays the target lesion atlas in the second display area of the monitor, allowing the doctor to complete the lesion diagnosis of the target image by combining the target lesion atlas. For example, as shown... Figure 4 As shown, Figure 4 This diagram illustrates the display of a target image and a target lesion atlas on a monitor. The camera system display area (i.e., the first display area) displays the target image, and the atlas display area (i.e., the second display area) displays the target lesion atlas. Furthermore, the monitor can display the lesion name, lesion nature, lesion definition, and diagnostic techniques corresponding to the target lesion atlas in other locations within the display area.
[0030] The endoscopic image display system of this embodiment acquires a target image captured by an endoscope and displays the target image in a first display area of a monitor corresponding to the endoscope. Based on the target image, a target lesion atlas is determined from a preset lesion atlas database. The target lesion atlas is then displayed in a second display area of the monitor, enabling doctors to diagnose the lesions in the target image by combining the target lesion atlas with the lesion atlas. By displaying the target image acquired by the endoscope and the corresponding target lesion atlas in the monitor corresponding to the endoscope, doctors can quickly and accurately determine the nature of the lesions displayed in the target image, reducing diagnostic discrepancies between different doctors and improving the homogenization of overall medical quality.
[0031] Please refer to Figure 5 , Figure 5 This is a flowchart illustrating a second embodiment of the endoscopic image display method provided in this application. The difference between the second embodiment and the first embodiment is that the lesion atlases in the preset lesion atlas database are stored according to imaging mode and tissue type; the step of determining the target lesion atlas in the preset lesion atlas database based on the target image includes: Step S201: Identify the imaging mode, tissue type, and lesion type corresponding to the target image.
[0032] In this embodiment, the endoscopic image display system identifies the imaging mode, tissue type, and lesion type corresponding to the target image. It should be noted that the endoscopic image display system can identify the target image based on a pre-trained recognition model to determine the corresponding imaging mode, tissue type, and lesion type; alternatively, the endoscopic image display system can use the imaging mode, tissue type, and lesion type input by the doctor as the corresponding imaging mode, tissue type, and lesion type for the target image.
[0033] Step S202: Determine the target atlas set in the preset lesion atlas database according to the imaging mode, the tissue type and the lesion type.
[0034] In this embodiment, the endoscopic image display system determines the target atlas set from a preset lesion atlas database based on the imaging mode, tissue type, and lesion type. It should be noted that the lesion atlases in the preset lesion atlas database are stored in groups according to imaging mode, tissue type, and lesion type; for example, such as... Figure 6 As shown, the preset lesion atlas database first groups the lesion atlases according to imaging mode (white light imaging mode, special light imaging mode 1, special light imaging mode 2), then groups them according to tissue type (esophagus, antrum, gastric angle, etc.), and finally groups them according to lesion type. For example, if the endoscopic image display system determines that the imaging mode corresponding to the target image is white light imaging mode, the tissue type is esophagus, and the lesion type is glycogen acanthosis, then it directly determines the glycogen acanthosis atlas set under the esophagus group of white light imaging mode from the preset lesion atlas database.
[0035] Understandably, the pre-set lesion atlas database may contain tens of thousands of lesion atlases. By using a three-level index to group and store them, the number of lesion atlases that need to be matched each time may only be a few dozen or even fewer, thus speeding up the entire matching process.
[0036] Step S203: Based on the target image, determine the target lesion atlas in the target atlas set.
[0037] In this embodiment, the endoscopic image display system compares the target image with each lesion atlas in the target atlas set to determine the target lesion atlas.
[0038] In this embodiment, the lesion atlases in the preset lesion atlas database are grouped and stored according to imaging mode, tissue type, and lesion type. The endoscopic image display system can quickly determine the corresponding target atlas set from the preset lesion atlas database based on the imaging mode, tissue type, and lesion type corresponding to the target image. Then, based on the target image, the target lesion atlas is determined from the target atlas set. This effectively reduces the number of lesion atlases to be compared and improves the efficiency of determining the target lesion atlas.
[0039] Please refer to Figure 7 , Figure 7 This is a flowchart illustrating a third embodiment of the endoscopic image display method provided in this application. The difference between the third embodiment and the first to second embodiments is that the step of comparing the target image with each lesion atlas in the target atlas set to determine the overall similarity between the target image and each lesion atlas includes: Step S301: Input the target image and each lesion map in the target map set into a preset feature extraction network to obtain a first feature map corresponding to the target image and a second feature map corresponding to each lesion map.
[0040] In this embodiment, after determining the target atlas set, the endoscopic image display system inputs the target image and each lesion atlas in the target atlas set into a preset feature extraction network to obtain a first feature map corresponding to the target image and a second feature map corresponding to each lesion atlas. The feature extraction network is a deep convolutional neural network (CNN).
[0041] Step S302: Based on the first feature map and each of the second feature maps, determine the overall similarity between the target image and each of the lesion atlases.
[0042] In this embodiment, for each second feature map, the endoscopic image display system performs similarity calculation based on the first feature map and the second feature map, thereby determining the overall similarity between the target image and the lesion atlas corresponding to the second feature map.
[0043] Step S303: Based on the overall similarity, determine the target lesion atlas in the target atlas set.
[0044] In this embodiment, the endoscopic image display system selects the lesion atlas with the highest overall similarity or an overall similarity greater than a preset threshold from the target atlas set based on the overall similarity between each lesion atlas in the target atlas set and the target image. It is understood that there can be one or more target lesion atlases. When there are multiple target lesion atlases, the endoscopic image display system prioritizes displaying the target lesion atlas with the highest overall similarity in the second display area of the monitor. Furthermore, the physician can control the second display area to sequentially display multiple target lesion atlases, allowing the physician to determine the most suitable target lesion atlas for lesion diagnosis of the target image.
[0045] In one embodiment, the step of determining the target lesion atlas in the target atlas set based on the overall similarity includes: Step S3031: Based on the overall similarity between the target image and each lesion atlas, sort all the lesion atlases in the target atlas set in descending order of overall similarity to obtain a lesion atlas similarity sequence.
[0046] Step S3032: The first preset number of lesion atlases in the lesion atlas similarity sequence are determined as target lesion atlases.
[0047] In this embodiment, after determining the overall similarity between the target image and each lesion atlas, the endoscopic image display system sorts all lesion atlases in the target atlas set in descending order of overall similarity, obtaining a lesion atlas similarity sequence. The top predetermined number of lesion atlases in the lesion atlas similarity sequence are then selected as target lesion atlases. It is understood that this embodiment selects multiple lesion atlases with high overall similarity as target lesion atlases to avoid potential selection errors that might result from overly absolute selection of lesion atlases with high overall similarity as target lesion atlases. This increases the tolerance for errors in target lesion atlas selection, allowing the physician to determine the most suitable target lesion atlas from a small number of target lesion atlases, thus improving accuracy.
[0048] The endoscopic image display system of this embodiment can quickly determine the target lesion atlas from the target atlas set by calculating the overall similarity between the first feature map corresponding to the target image and the second feature map corresponding to each lesion atlas. This improves the efficiency of determining the target lesion atlas and thus helps to improve the efficiency of lesion diagnosis of the target image by combining the target lesion atlas.
[0049] Please refer to Figure 8 , Figure 8This is a flowchart illustrating a fourth embodiment of the endoscopic image display method provided in this application. The difference between the fourth embodiment and the first to third embodiments is that the step of determining the overall similarity between the target image and each lesion atlas based on the first feature map and each of the second feature maps includes: Step S401: Normalize the first feature map and each of the second feature maps in the channel dimension to obtain the first normalized feature map and the second normalized feature map.
[0050] In this embodiment, after obtaining the first feature map corresponding to the target image and the second feature map corresponding to each lesion atlas, the endoscopic image display system performs normalization processing on the first feature map and each second feature map in the channel dimension to obtain the first normalized feature map and the second normalized feature map.
[0051] Step S402: Based on the preset spatial displacement value, construct the correlation body between the first normalized feature map and each of the second normalized feature maps.
[0052] In this embodiment, for each spatial location in the first normalized feature map, the endoscopic image display system determines the target matching region in each second normalized feature map based on a preset spatial displacement value and spatial location. It then calculates the local similarity between the spatial location and the target matching region. Finally, based on the local similarity between all spatial locations in the first normalized feature map and the corresponding target matching region in the second normalized feature map, it determines the correlation body between the first and second normalized feature maps. The above steps are performed for each second normalized feature map to obtain the correlation body between the first and second normalized feature maps.
[0053] In one embodiment, the step of constructing a correlation body between the first normalized feature map and each of the second normalized feature maps based on a preset spatial displacement value includes: Step S4021: For each spatial location in the first normalized feature map, determine the target matching region corresponding to the spatial location in each second normalized feature map based on the spatial location and a preset spatial displacement value.
[0054] In this embodiment, for each spatial location in the first normalized feature map, the endoscopic image display system determines the target matching region corresponding to the spatial location in each second normalized feature map based on the spatial location and a preset spatial displacement value. Here, the spatial location refers to the position coordinates of each pixel in the first normalized feature map, and the preset spatial displacement value is the offset relative to the spatial location, used by the second normalized feature map to determine the target matching region. The spatial displacement value can be set according to actual conditions and is not limited here.
[0055] For example, the preset spatial displacement value is 4, that is, it can move from -4 to +4 in the horizontal direction (a total of 9 values), and it can also move from -4 to +4 in the vertical direction (a total of 9 values). That is, for each spatial position (i, j) in the first normalized feature map, the spatial position (i, j) is located in the second normalized feature map, and the corresponding target matching area is the 9×9 adjacent area centered on (i, j).
[0056] Step S4022: Calculate the local similarity between the spatial location and the target matching region.
[0057] In this embodiment, the endoscopic image display system calculates the local similarity between each spatial location in the first normalized feature map and the corresponding target matching region in the second normalized feature map.
[0058] Step S4023: Based on the local similarity between each spatial location and the target matching region in each of the second normalized feature maps, construct a correlation body between the first normalized feature map and each of the second normalized feature maps.
[0059] In this embodiment, for each second normalized feature map, the endoscopic image display system constructs a correlation body between the first normalized feature map and the second normalized feature map based on the local similarity between each spatial location in the first normalized feature map and the target matching region in the second normalized feature map. Specifically, the formula for calculating the constructed correlation body is:
[0060] in, For the correlation matrix, c is the number of feature map channels. This is the first normalized feature map. This is the second normalized feature map, where p is the spatial location and d is the target matching region.
[0061] Step S403: Based on each of the correlation bodies, determine the overall similarity between the target image and each of the lesion atlases.
[0062] In this embodiment, after determining the correlation volume between the first normalized feature map and each second normalized feature map, the endoscopic image display system determines the overall similarity between the target image and each lesion atlas based on the corresponding correlation volume. The calculation formula is as follows:
[0063] Where S represents the overall similarity. and The spatial dimensions of the feature map. The preset spatial displacement value, For the related body The result is obtained by applying an activation function (such as Leaky ReLU), where h and w are the spatial dimensions of the correlated volume, and d is the local similarity.
[0064] The endoscopic image display system of this embodiment normalizes the first feature map and each second feature map in the channel dimension to obtain a first normalized feature map and a second normalized feature map. Based on preset spatial displacement values, a correlation body is constructed between the first normalized feature map and each second normalized feature map. Based on each correlation body, the overall similarity between the target image and each lesion atlas is determined. By introducing a displacement search mechanism of spatial displacement values when calculating the overall similarity between the target image and each lesion atlas, the area for matching spatial positions is expanded, thereby avoiding deviations in similarity judgment due to slight misalignments. This improves the accuracy of calculating the overall similarity between the target image and each lesion atlas, thus contributing to the accuracy of the determined target lesion atlas.
[0065] Please refer to Figure 9 , Figure 9 This is a flowchart illustrating the fifth embodiment of the endoscopic image display method provided in this application. The difference between the fifth embodiment and the first to fourth embodiments is that the step of displaying the target lesion atlas in the second display area of the display includes: Step S501: In response to the atlas display command, the atlas of the target lesion is displayed in the second display area of the display.
[0066] Step S502: When a suspected lesion is identified in the target image, the target lesion atlas is displayed in the second display area of the display.
[0067] In this embodiment, the endoscopic image display system has multiple trigger modes for displaying the target lesion atlas.
[0068] In one embodiment, the triggering mode is a manual switching mode. In this mode, the doctor needs to manually trigger the function of displaying the target lesion atlas using the endoscope remote control button, generating an atlas display command. The endoscopic image display system responds to the atlas display command and displays the target lesion atlas in the second display area of the monitor. If there are many related atlases, the atlas matching module built into the host will automatically find the N atlases with the highest matching degree for manual cyclic switching. N can be configured according to user preferences, generally set in the range of 4 to 10.
[0069] In one embodiment, the triggering mode is either an automatic playback mode or a one-click matching mode. In automatic playback mode, when the endoscopic image display system identifies a suspected lesion in the target image, the atlas display area automatically plays an atlas of the target lesion that matches or is similar to the identified location and lesion. For example, in an esophageal examination, the atlas display area plays an esophageal diagnostic atlas. In one-click matching mode, when the endoscopic image display system identifies a suspected lesion in the target image, after triggering the matching button, the display automatically plays an atlas of the target lesion that matches or is similar to the identified location and lesion. Preferably, when displaying the target image and the target lesion atlas simultaneously, the target image remains unchanged, i.e., the target image is frozen, facilitating the doctor's comparison and observation of static images.
[0070] The endoscopic image display system of this embodiment displays the target lesion atlas in response to an atlas display command in the second display area of the display. When a suspected lesion is identified in the target image, the target lesion atlas is displayed in the second display area of the display. By setting multiple different trigger modes for displaying the target lesion atlas, doctors can select different trigger modes according to their own needs, improving the doctor's user experience.
[0071] refer to Figure 10 , Figure 10 This is a schematic diagram of the endoscopic image display device provided in this application. The endoscopic image display device includes: The first display module 10 is used to acquire the target image collected by the endoscope and display the target image in the first display area of the display corresponding to the endoscope.
[0072] The determination module 20 is used to determine the target lesion atlas in a preset lesion atlas database based on the target image.
[0073] The second display module 30 is used to display the target lesion atlas in the second display area of the display, so that the doctor can make a lesion diagnosis on the target image by combining the target lesion atlas.
[0074] It is understood that the endoscopic image display device of this embodiment corresponds to the endoscopic image display method of the above embodiment, and the options in the above embodiment are also applicable to this embodiment, so they will not be described again here.
[0075] This application also provides an endoscope device, exemplary of which includes a processor and a memory, wherein the memory stores a computer program, and the processor executes the computer program to cause the endoscope device to perform the above-described endoscopic image display method.
[0076] The processor can be an integrated circuit chip with signal processing capabilities. The processor can be a general-purpose processor, including at least one of a Central Processing Unit (CPU), Graphics Processing Unit (GPU), Network Processor (NP), Digital Signal Processor (DSP), Application-Specific Integrated Circuit (ASIC), Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The general-purpose processor can be a microprocessor or any conventional processor, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application.
[0077] The memory can be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc. The memory is used to store computer programs, and the processor can execute the computer programs accordingly after receiving execution instructions.
[0078] This application also provides a computer storage medium for storing the computer program used in the aforementioned endoscopic device. The computer storage medium can be a readable storage medium, a non-volatile storage medium, or a volatile storage medium. For example, the computer storage medium may include, but is not limited to, various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.
[0079] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that, in alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.
[0080] In addition, the functional modules or units in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.
[0081] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an endoscope device (which may be a smartphone, personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.
[0082] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.
Claims
1. A method for displaying endoscopic images, characterized in that, The method includes: Acquire the target image captured by the endoscope and display the target image in the first display area of the monitor corresponding to the endoscope; Based on the target image, the target lesion atlas is determined in a preset lesion atlas database; The target lesion atlas is displayed in the second display area of the monitor, so that the doctor can make a lesion diagnosis on the target image by combining the target lesion atlas.
2. The endoscopic image display method according to claim 1, characterized in that, The lesion atlases in the preset lesion atlas database are stored according to imaging mode and tissue type; the step of determining the target lesion atlas in the preset lesion atlas database based on the target image includes: Identify the imaging pattern, tissue type, and lesion type corresponding to the target image; Based on the imaging mode, the tissue type, and the lesion type, a target atlas set is determined in a preset lesion atlas database; Based on the target image, target lesion atlases are determined in the target atlas set.
3. The endoscopic image display method according to claim 2, characterized in that, The step of comparing the target image with each lesion atlas in the target atlas set to determine the overall similarity between the target image and each lesion atlas includes: The target image and each lesion map in the target map set are respectively input into a preset feature extraction network to obtain a first feature map corresponding to the target image and a second feature map corresponding to each lesion map. Based on the first feature map and each of the second feature maps, the overall similarity between the target image and each of the lesion atlases is determined respectively; Based on the overall similarity, the target lesion atlas is determined in the target atlas set.
4. The endoscopic image display method according to claim 3, characterized in that, The step of determining the overall similarity between the target image and each of the lesion atlases based on the first feature map and each of the second feature maps includes: The first feature map and each of the second feature maps are normalized in the channel dimension to obtain the first normalized feature map and the second normalized feature map. Based on the preset spatial displacement value, a correlation body is constructed between the first normalized feature map and each of the second normalized feature maps; Based on each of the aforementioned correlation bodies, the overall similarity between the target image and each of the aforementioned lesion atlases is determined.
5. The endoscopic image display method according to claim 4, characterized in that, The step of constructing a correlation body between the first normalized feature map and each of the second normalized feature maps based on preset spatial displacement values includes: For each spatial location in the first normalized feature map, a target matching region corresponding to the spatial location is determined in each of the second normalized feature maps based on the spatial location and a preset spatial displacement value. Calculate the local similarity between the spatial location and the target matching region; Based on the local similarity between each spatial location and the target matching region in each of the second normalized feature maps, a correlation body is constructed between the first normalized feature map and each of the second normalized feature maps.
6. The endoscopic image display method according to claim 3, characterized in that, The step of determining the target lesion atlas in the target atlas set based on the overall similarity includes: Based on the overall similarity between the target image and each lesion atlas, all lesion atlases in the target atlas set are sorted in descending order of overall similarity to obtain a lesion atlas similarity sequence. The first preset number of lesion atlases in the lesion atlas similarity sequence are identified as target lesion atlases.
7. The endoscopic image display method according to any one of claims 1-6, characterized in that, The step of displaying the target lesion atlas in the second display area of the display includes: In response to a map display command, the target lesion map is displayed in the second display area of the display; or When a suspected lesion is identified in the target image, the atlas of the target lesion is displayed in the second display area of the display.
8. An endoscopic image display device, characterized in that, The endoscopic image display device includes: The first display module is used to acquire the target image collected by the endoscope and display the target image in the first display area of the display corresponding to the endoscope. The determination module is used to determine the target lesion atlas in a preset lesion atlas database based on the target image; The second display module is used to display the target lesion atlas in the second display area of the display, so that the doctor can make a lesion diagnosis on the target image by combining the target lesion atlas.
9. An endoscopic device, characterized in that, The endoscopic device includes a processor and a memory, the memory storing a computer program, and the processor executing the computer program to implement the endoscopic image display method according to any one of claims 1-7.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when run on a processor, executes the endoscopic image display method according to any one of claims 1-7.