Endoscope camera system and custom color fluorescence fusion display method thereof
Through the custom color fluorescence fusion display method, the problem of insufficient distinction between fluorescence images and white light images in the endoscopic system is solved, which improves diagnostic accuracy and is suitable for clinical applications.
Patent Information
- Application Number
- CN202411078005.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-06
- Publication Date
- 2024-11-15
AI Technical Summary
In the fusion display of fluorescence images and white light images, the existing endoscopic system is insufficient in the distinction, making it difficult to distinguish the boundaries of the tissue hierarchy, affecting the accuracy of diagnosis.
Provide a custom color fluorescence fusion display method. By acquiring fluorescence images and visible light images, obtaining fluorescence intensity distribution information, and converting and fusion processing of fluorescence images based on the mapping range parameters and weight parameters input by the user to generate more suitable clinical conditions. The endoscopic image applied.
It improves the fusion effect of fluorescence images and visible light images, enhances the visualization of lesions, and improves the accuracy and reliability of endoscopic diagnosis.
Smart Images

Figure CN118948185A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of medical equipment technology, and in particular to an endoscopic camera system and a customized color fluorescence fusion display method thereof. Background Art
[0002] Endoscopic systems can observe tissues inside organisms and have been increasingly used in the medical field. Endoscopic systems are usually capable of ordinary light imaging, that is, imaging the inside of an organism under ordinary light. However, ordinary light images have certain limitations. For example, some lesions are difficult to identify in ordinary light images. In this regard, fluorescent endoscopic imaging has gradually become popular in minimally invasive surgery. In endoscopic examinations, specific tissue sites (such as blood vessels, lymph nodes, and tumors) are marked with the help of fluorescent dyes or tracers. Endoscopic fluorescent imaging has greatly facilitated the inspection and precise positioning of lesions during surgical operations.
[0003] Normally, the fluorescence image needs to be fused with the white light background image to obtain information such as the position range of the lesion in the field of view of the white light laparoscope. The doctor can judge the corresponding tissue structure by the change of fluorescence intensity in the fluorescence image. However, after the fluorescence image and the white light image are fused and displayed, it will affect the doctor's judgment of the fluorescence intensity, and then affect the doctor's diagnosis of the lesion. Since the human eye has a good ability to distinguish colors, different fluorescence intensities are displayed in different colors in the prior art, so that the difference in fluorescence intensity can be intuitively reflected through the difference in color. However, on the one hand, the human eye's preference for color is subjective, and the current color conversion scheme is not suitable for everyone. On the other hand, the existing fusion display method has poor clinical adaptability, and in clinical applications, there are often problems such as insufficient differentiation and difficulty in distinguishing the boundaries of tissue levels. Summary of the invention
[0004] The following is a summary of the subject matter described in detail in this application. This summary is not intended to limit the scope of the claims.
[0005] The embodiment of the present application provides an endoscopic camera system and a customized color fluorescence fusion display method thereof, which can make the display effect of the endoscopic image after the fusion of the fluorescence image and the visible light image better and more suitable for actual clinical application scenarios.
[0006] On the one hand, the embodiment of the present application provides a customized color fluorescence fusion display method, which is applied to an endoscope camera host, including:
[0007] Acquire fluorescent images and visible light images taken by an endoscope camera;
[0008] Acquiring fluorescence intensity distribution information of the fluorescence image from the fluorescence image;
[0009] Get the fluorescence intensity mapping range parameters input by the user;
[0010] Determining a corresponding mapping range of the fluorescence intensity distribution according to the fluorescence intensity mapping range parameter, and obtaining a target mapping relationship between the fluorescence intensity distribution and the color distribution;
[0011] Converting the fluorescence image into a color fluorescence image according to the fluorescence intensity distribution information and the target mapping relationship;
[0012] fusing the color fluorescence image and the visible light image to obtain an endoscopic image;
[0013] The endoscopic image is displayed on a display.
[0014] On the other hand, the embodiment of the present application also provides a customized color fluorescence fusion display method, which is applied to an endoscope camera host, comprising:
[0015] Acquire fluorescent images and visible light images taken by an endoscope camera;
[0016] Acquiring fluorescence intensity distribution information of the fluorescence image from the fluorescence image;
[0017] Obtaining a target mapping relationship between fluorescence intensity distribution and color distribution;
[0018] Converting the fluorescence image into a color fluorescence image according to the fluorescence intensity distribution information and the target mapping relationship;
[0019] Obtaining a user-defined weight parameter input by a user, wherein the user-defined weight parameter includes a first fusion curve with different fusion coefficients corresponding to different fluorescence intensities, or a second fusion curve with different fusion coefficients corresponding to different visible light intensities;
[0020] Performing fusion processing on the color fluorescence image and the visible light image according to the custom weight parameter to obtain an endoscopic image;
[0021] The endoscopic image is displayed on a display.
[0022] On the other hand, the embodiment of the present application also provides a customized color fluorescence fusion display method, which is applied to an endoscope camera host, comprising:
[0023] Acquire fluorescent images and visible light images taken by an endoscope camera;
[0024] Acquiring fluorescence intensity distribution information of the fluorescence image from the fluorescence image;
[0025] Acquire color conversion parameters input by a user, and determine a target mapping relationship between fluorescence intensity distribution and color distribution according to the color conversion parameters;
[0026] Converting the fluorescence image into a color fluorescence image according to the fluorescence intensity distribution information and the target mapping relationship;
[0027] Obtaining a custom weight parameter input by a user, and fusing the color fluorescence image and the visible light image according to the custom weight parameter to obtain an endoscopic image;
[0028] The endoscopic image is displayed on a display.
[0029] On the other hand, an embodiment of the present application further provides an endoscope camera system, comprising:
[0030] Endoscope;
[0031] An endoscope camera host is connected to the endoscope, and is used to obtain the fluorescent image and visible light image taken by the endoscope. The endoscope camera host is also used to execute the customized color fluorescence fusion display method mentioned in any of the above embodiments.
[0032] On the other hand, an embodiment of the present application further provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it implements the customized color fluorescence fusion display method mentioned in any of the above embodiments.
[0033] On the other hand, an embodiment of the present application further provides a computer-readable storage medium, which stores a computer program and is applied to an endoscope camera host. The computer program is executed by a processor to implement a custom color fluorescence fusion display method as mentioned in any of the above embodiments.
[0034] On the other hand, the embodiment of the present application further provides a computer program product, which includes a computer program, and the computer program is stored in a computer-readable storage medium. The processor of the computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the computer device implements the custom color fluorescence fusion display method mentioned in any of the above embodiments.
[0035] The embodiments of the present application include at least the following beneficial effects:
[0036] On the one hand, a custom color fluorescence fusion display method provided by an embodiment of the present application obtains a fluorescence image and a visible light image taken by an endoscope camera, and then obtains the fluorescence intensity distribution information of the fluorescence image from the fluorescence image, obtains the fluorescence intensity mapping range parameter input by the user, and then determines the corresponding mapping range of the fluorescence intensity distribution according to the fluorescence intensity mapping range parameter, obtains the target mapping relationship between the fluorescence intensity distribution and the color distribution, and converts the fluorescence image into a color fluorescence image according to the fluorescence intensity distribution information and the target mapping relationship, and then fuses the color fluorescence image and the visible light image to obtain an endoscopic image, and displays the endoscopic image through a display. In an embodiment of the present application, the doctor can customize the mapping range of the fluorescence intensity distribution by inputting the fluorescence intensity mapping range parameter, and obtain the mapping relationship between the set fluorescence intensity distribution and the color distribution. For scenes with strong fluorescence intensity, the mapping range of the fluorescence intensity distribution is determined by selecting a larger fluorescence intensity mapping range parameter, so as to improve the color dynamic range of the image and ensure that high-brightness fluorescence signals can be displayed delicately without overexposure. For scenes with weak fluorescence intensity, the mapping range of the fluorescence intensity distribution is determined by selecting a smaller fluorescence intensity mapping range parameter to enhance the color distinction of weak fluorescence signals, so as to make the display effect of the endoscopic image after the fusion of the fluorescence image and the visible light image better, thereby improving the accuracy and reliability of endoscopic diagnosis, and being more suitable for actual clinical application scenarios.
[0037] On the other hand, a custom color fluorescence fusion display method provided in an embodiment of the present application obtains a fluorescence image and a visible light image taken by an endoscope camera, then obtains the fluorescence intensity distribution information of the fluorescence image from the fluorescence image, obtains the target mapping relationship between the fluorescence intensity distribution and the color distribution, and then converts the fluorescence image into a color fluorescence image according to the fluorescence intensity distribution information and the target mapping relationship, and obtains a custom weight parameter input by the user, wherein the custom weight parameter includes a first fusion curve with different fusion coefficients corresponding to different fluorescence intensities, or a second fusion curve with different fusion coefficients corresponding to different visible light intensities, and fuses the color fluorescence image and the visible light image according to the custom weight parameter to obtain an endoscopic image, and finally displays the endoscopic image through a display. In the embodiment of the present application, the doctor can select the fusion curve adjustment with different fusion coefficients by inputting the custom weight parameter, thereby adjusting the layers of the color fluorescence image and the visible light image according to the different fusion curves, which can make the display effect of the endoscopic image after the fusion of the fluorescence image and the visible light image better, thereby improving the accuracy and reliability of endoscopic diagnosis, and being more suitable for actual clinical application scenarios.
[0038] On the other hand, a custom color fluorescence fusion display method provided in an embodiment of the present application obtains a fluorescence image and a visible light image captured by an endoscope camera, then obtains the fluorescence intensity distribution information of the fluorescence image from the fluorescence image, and obtains the color conversion parameters input by the user, determines the target mapping relationship between the fluorescence intensity distribution and the color distribution according to the color conversion parameters, and then converts the fluorescence image into a color fluorescence image according to the fluorescence intensity distribution information and the target mapping relationship, and then obtains the custom weight parameters input by the user, and fuses the color fluorescence image and the visible light image according to the custom weight parameters to obtain an endoscopic image, and finally displays the endoscopic image through a display. In the embodiment of the present application, the doctor can adjust the mapping relationship between the fluorescence intensity distribution and the color distribution of the fluorescence image by inputting the color conversion parameters, and selects the fusion curve adjustment with different fusion coefficients by inputting the custom weight parameters, so as to adjust the layers of the color fluorescence image and the visible light image according to different fusion curves, so as to make the display effect of the endoscopic image after the fusion of the fluorescence image and the visible light image better, thereby improving the accuracy and reliability of endoscopic diagnosis, and being more suitable for actual clinical application scenarios.
[0039] On the other hand, an endoscopic camera system provided by an embodiment of the present application obtains a fluorescent image and a visible light image taken by an endoscopic camera, and then obtains fluorescence intensity distribution information of the fluorescent image from the fluorescent image, obtains a fluorescence intensity mapping range parameter input by a user, and then determines a corresponding mapping range of the fluorescence intensity distribution according to the fluorescence intensity mapping range parameter, obtains a target mapping relationship between the fluorescence intensity distribution and the color distribution, and converts the fluorescent image into a color fluorescent image according to the fluorescence intensity distribution information and the target mapping relationship, and then fuses the color fluorescent image and the visible light image to obtain an endoscopic image, and displays the endoscopic image on a display. In an embodiment of the present application, the doctor can customize the mapping range of the fluorescence intensity distribution by inputting the fluorescence intensity mapping range parameter, and obtain the mapping relationship between the set fluorescence intensity distribution and the color distribution. For scenes with strong fluorescence intensity, the mapping range of the fluorescence intensity distribution is determined by selecting a larger fluorescence intensity mapping range parameter, so as to improve the color dynamic range of the image and ensure that high-brightness fluorescence signals can be displayed delicately without overexposure. For scenes with weak fluorescence intensity, the mapping range of the fluorescence intensity distribution is determined by selecting a smaller fluorescence intensity mapping range parameter to enhance the color distinction of weak fluorescence signals, so as to make the display effect of the endoscopic image after the fusion of the fluorescence image and the visible light image better, thereby improving the accuracy and reliability of endoscopic diagnosis, and being more suitable for actual clinical application scenarios.
[0040] Other features and advantages of the present application will be set forth in the following description, and in part will become apparent from the description, or may be understood by practicing the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] The accompanying drawings are used to provide further understanding of the technical solution of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the technical solution of the present application and do not constitute a limitation on the technical solution of the present application.
[0042] Figure 1 is a schematic structural diagram of an endoscope camera system provided in an embodiment of the present application;
[0043] Figure 2 It is a flow chart of a customized color fluorescence fusion display method provided in an embodiment of the present application;
[0044] Figure 3 This is a mapping parameter setting interface view provided in an embodiment of the present application;
[0045] Figure 4 This is a mapping parameter setting interface view provided in an embodiment of the present application;
[0046] Figure 5 This is a mapping parameter setting interface view provided in an embodiment of the present application;
[0047] Figure 6 This is a fusion weight adjustment interface view provided in an embodiment of the present application;
[0048] Figure 7 This is a fusion weight adjustment interface view provided in an embodiment of the present application;
[0049] Figure 8 It is a flow chart of a customized color fluorescence fusion display method provided in an embodiment of the present application;
[0050] Fig. 9 It is a flow chart of a customized color fluorescence fusion display method provided in an embodiment of the present application;
[0051] Fig.10 It is a structural block diagram of an endoscope camera host provided by an embodiment of the application;
[0052] Fig.11 It is a partial structural block diagram of a terminal provided in an embodiment of the application;
[0053] Fig.12 It is a partial structural block diagram of a server provided in an application embodiment. DETAILED DESCRIPTION
[0054] The present application is further described below in conjunction with the accompanying drawings and specific embodiments. The described embodiments should not be regarded as limiting the present application, and all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present application.
[0055] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it can be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.
[0056] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.
[0057] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate to describe the embodiments of the present application, for example, it can be implemented in an order other than those illustrated or described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0058] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.
[0059] It should be understood that in the description of the embodiments of the present application, the meaning of multiple (or multiple items) is more than two, greater than, less than, exceed, etc. are understood to not include the number, and above, below, within, etc. are understood to include the number.
[0060] Endoscopic systems can observe tissues inside organisms and have been increasingly used in the medical field. Endoscopic systems are usually capable of ordinary light imaging, that is, imaging the inside of an organism under ordinary light. However, ordinary light images have certain limitations. For example, some lesions are difficult to identify in ordinary light images. In this regard, fluorescent endoscopic imaging has gradually become popular in minimally invasive surgery. In endoscopic examinations, specific tissue sites (such as blood vessels, lymph nodes, and tumors) are marked with the help of fluorescent dyes or tracers. Endoscopic fluorescent imaging has greatly facilitated the inspection and precise positioning of lesions during surgical operations.
[0061] Normally, the fluorescence image needs to be fused with the background image of visible light to obtain information such as the position range of the lesion in the field of view of the visible light laparoscope. The doctor can judge the corresponding tissue structure by the change of fluorescence intensity in the fluorescence image. However, after the fluorescence image and the visible light image are fused and displayed, it will affect the doctor's judgment of the fluorescence intensity, and then affect the doctor's diagnosis of the lesion. Since the human eye has a good ability to distinguish colors, different fluorescence intensities are displayed in different colors in the prior art, so that the difference in fluorescence intensity can be intuitively reflected through the difference in color. However, on the one hand, the human eye's preference for color is subjective, and the current color conversion scheme is not suitable for everyone. On the other hand, the existing fusion display method has poor clinical adaptability, and in clinical applications, there are often problems such as insufficient differentiation and difficulty in distinguishing the boundaries of tissue levels.
[0062] Reference Figure 1 As shown, Figure 1 The schematic structural diagram of an endoscope camera system provided in an embodiment of the present application, the endoscope camera system 100 includes components such as an endoscope 110, a display 120, and an endoscope camera host 130. The endoscope includes an operating portion 111 and an insertion portion 112. The operating portion 111 and the insertion portion 112 can be an integrated structure or a detachable structure. The insertion portion 112 is used to insert into a part to be observed 160. The endoscope 110 also includes at least one image sensor, which can be arranged at the front end of the insertion portion 112. In addition, the endoscope camera system 100 also includes a camera host 131, a light source host 132, a communication cable 140, and a light guide 150.
[0063] The insertion part 112 includes a mirror tube, an illumination light path and an image sensor. The front end of the mirror tube is inserted into the part to be observed 160. The mirror tube can be a hard tube or a soft tube. The illumination light path is connected to the light guide 150 correspondingly, and is used to irradiate the light source to the part to be observed 160. Specifically, the image sensor may include a first image sensor and a second image sensor, which are used to convert the optical signal to obtain an electrical signal, that is, to obtain an image signal. It should be understood that different optical signals will be generated for different light sources. For example, when a visible light source irradiates the part to be observed 160, the part to be observed 160 can directly reflect the visible light to obtain an optical signal of visible light. When a special light source irradiates the part to be observed 160, an optical signal excited by the special light can be obtained. The optical axes of the first image sensor and the second image sensor can be arranged in parallel or at a certain angle. The left image signal and the right image signal collected by the first image sensor and the second image sensor can correspond to the stereoscopic image observed by the left and right eyes of a person, thereby simulating the binocular stereoscopic vision of the human eye. Further, the image sensor includes but is not limited to a CCD sensor, a CMOS sensor, etc.
[0064] The operating part 111 is connected to the camera host 131 through the communication cable 140, and can be used to obtain the optical signal of the observation part 160 by the insertion part 112, convert it into an image signal, and transmit the image signal to the camera host 131 through the communication cable 140 for processing. Preliminary signal processing can be performed in the operating part 111, such as amplification, filtering and other processing. In some embodiments, the operating part 111 can also send the image signal to the camera host 131 by wireless transmission. The operating part 111 can also include a focusing component, which can be used to shape the optical path of the optical signal, thereby adjusting the imaging focal length of the operating part 111. The operating part 111 can also include a handle, on which a plurality of buttons are arranged, and the user can call the function corresponding to the button by pressing the button.
[0065] In some embodiments, an image signal processing module and an artificial intelligence processing module are provided in the camera host 131. The image signal processing module and the artificial intelligence processing module obtain the image signal transmitted by the operating unit 111, and after processing the image signal, output an endoscopic image of the observed part 160, including a visible light image or a special light image, such as a visible light endoscopic image, a fluorescent endoscopic image, a narrow-band light endoscopic image, and a multi-spectral endoscopic image.
[0066] The display 120 is connected to the camera host 131 through a communication cable 140, and is used to display the endoscopic image output by the camera host 131. Specifically, the display 120 may include a liquid crystal display, an LED display, an OLED display, and a touch display for human-computer interaction, etc., or may be a display on an electronic device such as a mobile phone, a tablet computer, or a personal computer connected to the camera host 131.
[0067] One end of the light guide 150 is connected to the light source interface of the light source host 132, and the other end is connected to the light source interface of the endoscope 110. The light guide 150 can be used to provide a light source to the part to be observed 160 through the endoscope 110. The light source may include a visible light source and a special light source. Specifically, the visible light source is an LED light source, which can provide a plurality of monochromatic lights of different wavelength ranges, such as blue light, green light, red light, etc. In some embodiments, the visible light source may also be a combination of a plurality of monochromatic lights, or a wide-spectrum visible light source; the special light source may be a laser light source corresponding to a fluorescent agent, such as a near-infrared light or other laser. In some embodiments, a fluorescent agent may be injected into the part to be observed 160 before the endoscope camera system 100 is used, and the fluorescent agent may generate fluorescence after absorbing the laser light generated by the laser light source. It should be understood that the part to be observed 160 may be a human body.
[0068] It should be noted that those skilled in the art should understand that Figure 1 The above is only an example of the endoscopic camera system 100 and does not constitute a limitation of the endoscopic camera system 100. The endoscopic camera system 100 may include more or fewer components than those shown in the figure, or a combination of certain components, or different components. For example, the endoscopic camera system 100 may also include a dilator, a smoke control device, input and output devices, a network access device, etc.
[0069] Reference Figure 2 As shown, Figure 2 A flowchart of a customized color fluorescence fusion display method provided in an embodiment of the present application, which can be referred to as Figure 1 The described endoscopic imaging system implementation specifically includes but is not limited to the following steps S100 to S700:
[0070] Step S100, obtaining a fluorescent image and a visible light image taken by an endoscope camera.
[0071] In this step, when the endoscope is located near the target tissue, the user can obtain an endoscopic image by controlling the endoscopic camera system to photograph the target tissue. Specifically, by converting the acquired optical signal of the target tissue into an electrical signal, an endoscopic image signal is obtained, and the image signal is further processed to obtain an endoscopic image of the target tissue. Different optical signals are generated when different light sources are used to illuminate the part to be observed. For example, when a visible light source is used to illuminate the part to be observed, the part to be observed can directly reflect the visible light to obtain an optical signal of visible light, and a visible light image can be generated based on the optical signal of visible light. When collecting fluorescent image signals, the light source provides a narrow-band laser to excite the fluorescent agent to produce fluorescence. Before using the endoscopic camera system to image the part to be observed, a fluorescent agent is injected into the part to be observed. During the imaging process, special light illumination for exciting the fluorescent agent can be provided to the part to be observed, so that the fluorescent agent is excited to produce fluorescence.
[0072] Specifically, the target tissue is the anatomical structure or tissue of the part to be observed in the human body, and the part to be observed includes but is not limited to the blood circulation system, lymphatic system and tumor tissue. Among them, the visible light image can be a color RGB image or a grayscale image. The fluorescence image can be a grayscale image or a monochrome image, or it can be a color fluorescence image obtained according to the mapping relationship between signal intensity and color value. It should be noted that the fluorescence image can present a high contrast between the diseased tissue such as tumor cells and the normal tissue image, so that the doctor can observe the size, contour and other information of the tumor to achieve the purpose of precise treatment.
[0073] Step S200, obtaining fluorescence intensity distribution information of the fluorescence image from the fluorescence image.
[0074] In this step, the endoscope camera host obtains the fluorescence intensity distribution information of the fluorescence image from the fluorescence image. It can be understood that in endoscopic fluorescence imaging, the fluorescence intensity distribution information refers to the spatial distribution of the fluorescence signal intensity in the image, reflecting the difference in fluorescence intensity of the fluorescent substance in different positions or regions in the target tissue. In one embodiment, the fluorescence intensity distribution can be determined by measuring the fluorescence intensity of each pixel in the image, thereby obtaining the fluorescence intensity distribution information.
[0075] Step S300, obtaining the fluorescence intensity mapping range parameters input by the user.
[0076] In this step, the user can input the fluorescence intensity mapping range parameters to the endoscope camera host through the input device of the endoscope camera system, and then the endoscope camera host obtains the fluorescence intensity mapping range parameters input by the user for processing. It should be understood that the fluorescence intensity mapping range parameters can be a set of parameters set when defining or limiting the fluorescence intensity mapping range, which defines the dynamic range of the fluorescence signal intensity in the fluorescence image, that is, the interval from the darkest (no fluorescence or background noise) to the brightest (highest fluorescence intensity). Different fluorescent reagents and imaging conditions may cause a wide range of changes in fluorescence intensity, depending on the specific endoscopic application and the doctor's needs for surgery, so it is necessary to optimize the display of the final endoscopic image by adjusting the mapping range. In some embodiments, a display device in the endoscopic imaging system can be used as an input device, such as a touch display, so that the user can input the fluorescence intensity mapping range parameters and can intuitively see the operations performed on the endoscopic image. In some embodiments, the input device can be an input device with physical buttons in the endoscopic imaging system, including a keyboard, a mouse, a scroll wheel, a trackball, a multi-function knob, etc., or it can be a mobile portable input device, such as a mobile phone with a touch screen, a tablet computer or a personal computer, etc., or a combination of one or more of them.
[0077] Step S400, determining a corresponding mapping range of the fluorescence intensity distribution according to the fluorescence intensity mapping range parameter, and obtaining a target mapping relationship between the fluorescence intensity distribution and the color distribution;
[0078] In this step, the endoscope camera host determines the mapping range of the corresponding fluorescence intensity distribution according to the fluorescence intensity mapping range parameter, and obtains the target mapping relationship between the fluorescence intensity distribution and the color distribution. Specifically, the fluorescence intensity distribution reflects the fluorescence intensity at different wavelengths, and the corresponding color distribution refers to the range of color values in a specific color space. The mapping relationship between the fluorescence intensity distribution and the color distribution shows that fluorescence of different wavelengths corresponds to different colors, and the fluorescence intensity of each color is different. Therefore, the mapping range of the corresponding fluorescence intensity distribution can be determined according to the fluorescence intensity mapping range parameter. The user can select the appropriate fluorescence intensity mapping range parameter and map it to the original color distribution. For example, a larger fluorescence intensity mapping range parameter can be selected in a scene with a strong fluorescence intensity to display more details of the target tissue (including darker and brighter areas), or a smaller fluorescence intensity mapping range parameter can be selected in a scene with a weak fluorescence intensity to highlight the fluorescence signal of a specific intensity to adapt to the current diagnostic or surgical imaging conditions, thereby obtaining a target mapping relationship to ensure that the fluorescence signal in the endoscopic image can be presented in a clearer and more accurate manner, which is convenient for subsequent analysis by doctors.
[0079] It should be noted that the mapping relationship can specifically be a mapping relationship table or a mapping relationship function. In one embodiment, if the fluorescence intensity of a certain wavelength is high and the wavelength corresponds to red, the area can be represented by a brighter red in the endoscopic image. In another embodiment, if the fluorescence intensity of a certain wavelength is low and the wavelength corresponds to purple, the area can be represented by a darker purple in the mapping diagram. In this way, the color distribution and intensity changes of the fluorescent substance at different wavelengths can be intuitively displayed.
[0080] Step S500 , converting the fluorescence image into a color fluorescence image according to the fluorescence intensity distribution information and the target mapping relationship.
[0081] In this step, the endoscope camera host converts the fluorescent image into a color fluorescent image according to the fluorescence intensity distribution information and the target mapping relationship. Specifically, it converts fluorescent signals of different intensities into pixel values of different colors, so that they are intuitively represented in the color fluorescent image. For example, the fluorescence signal pixel position can be determined according to the fluorescence intensity distribution information, and then the corresponding color is assigned to each pixel with a fluorescent signal according to the target mapping relationship, that is, according to the different fluorescence intensities, and finally the mapped color value is redistributed to each pixel of the original fluorescent image to generate a color fluorescent image for the doctor to observe and analyze.
[0082] Step S600: fusing the color fluorescence image and the visible light image to obtain an endoscopic image.
[0083] In this step, after the endoscope camera host obtains the visible light image and the color fluorescence image, a fused image can be generated based on the visible light image and the color fluorescence image to obtain an endoscopic image. Specifically, the visible light image and the color fluorescence image can be fused to obtain a fused image, which contains both the information of the visible light image and the information of the color fluorescence image. Among them, the fused image can be generated by superimposing the layers of the visible light image and the color fluorescence image. For example, the color fluorescence image can be processed to have a certain transparency, and then the color fluorescence image is superimposed on the visible light image to form a fused image. Alternatively, each pixel on each fused image can be synthesized according to the pixel value of each pixel on the visible light image and the pixel value of the corresponding pixel in the color fluorescence image. Specifically, certain weights can be assigned to the visible light image and the color fluorescence image respectively, and the sum of the weights is 1. After generating the visible light image and the color fluorescence image, the pixel values of each pixel of the two are weighted and summed to obtain the pixel value of each pixel of the fused image. By changing the size of the weight, the proportion of the visible light image information and the color fluorescence image information in the information reflected by the fused image can be adjusted. It is understandable that endoscopic images of different image types may include different types of fused images. For example, a monochrome fused image can be obtained by fusing a monochrome fluorescence image (including but not limited to a green fluorescence image) with a visible light image; a color fused image can be obtained by fusing a color fluorescence image with a visible light image.
[0084] Step S700: displaying the endoscopic image on a display.
[0085] In this step, the endoscope camera host displays the endoscopic image obtained after fusion processing through a display, which includes but is not limited to a liquid crystal display, an LED display or an OLED display, and can also be a display on an electronic device such as a mobile phone, a tablet computer or a personal computer.
[0086] The customized color fluorescence fusion display method provided in the embodiment of the present application obtains a fluorescence image and a visible light image taken by an endoscope camera, and then obtains the fluorescence intensity distribution information of the fluorescence image from the fluorescence image, obtains the fluorescence intensity mapping range parameter input by the user, and then determines the corresponding mapping range of the fluorescence intensity distribution according to the fluorescence intensity mapping range parameter, obtains the target mapping relationship between the fluorescence intensity distribution and the color distribution, and converts the fluorescence image into a color fluorescence image according to the fluorescence intensity distribution information and the target mapping relationship, and then fuses the color fluorescence image and the visible light image to obtain an endoscopic image, and displays the endoscopic image through a display. In an embodiment of the present application, the doctor can customize the mapping range of the fluorescence intensity distribution by inputting the fluorescence intensity mapping range parameter, and obtain the mapping relationship between the set fluorescence intensity distribution and the color distribution. For scenes with strong fluorescence intensity, the mapping range of the fluorescence intensity distribution is determined by selecting a larger fluorescence intensity mapping range parameter, so as to improve the color dynamic range of the image and ensure that high-brightness fluorescence signals can be displayed delicately without overexposure. For scenes with weak fluorescence intensity, the mapping range of the fluorescence intensity distribution is determined by selecting a smaller fluorescence intensity mapping range parameter to enhance the color distinction of weak fluorescence signals, so as to make the display effect of the endoscopic image after the fusion of the fluorescence image and the visible light image better, thereby improving the accuracy and reliability of endoscopic diagnosis, and being more suitable for actual clinical application scenarios.
[0087] In one embodiment, the step S300 of obtaining the fluorescence intensity mapping range parameter input by the user specifically includes the following steps S310 to S320:
[0088] Step S310: Displaying a mapping parameter setting interface via a display and / or a display screen of an endoscope camera host.
[0089] In this step, the mapping parameter setting interface can be displayed through a separate display, or through the display screen of the endoscope camera host, or both. It should be noted that the display can be independent and directly connected to the endoscope camera host to display the mapping parameter setting interface. The endoscope camera host usually also includes a human-computer interaction interface, such as a touch screen, buttons or knobs, for controlling the functions and parameters of the endoscope. In one embodiment, the mapping parameter setting interface can also be displayed directly on the display screen of the endoscope camera host without the need for an additional display.
[0090] Step S320: Obtain the user's operation on the mapping parameter setting interface to obtain the fluorescence intensity mapping range parameter.
[0091] In this step, the user's operation on the mapping parameter setting interface may be an input operation of the user inputting the fluorescence intensity mapping range parameter on the mapping parameter setting interface, and then the endoscope camera host responds to the user's input operation to obtain and get the fluorescence intensity mapping range parameter.
[0092] In one embodiment, referring to Figure 3 to Figure 4 As shown, the mapping parameter setting interface includes a fluorescence intensity range setting control, and the endoscope camera host obtains the user's operation on the mapping parameter setting interface to obtain the fluorescence intensity mapping range parameter, including obtaining the user's operation on the fluorescence intensity range setting control in the mapping parameter setting interface to obtain the fluorescence intensity mapping range parameter. In this step, the user can click the fluorescence intensity range setting control in the mapping parameter setting interface to enter the fluorescence intensity mapping range parameter.
[0093] In one embodiment, the fluorescence intensity range setting control may be a stepless adjustment control for continuously setting an upper limit setting value and / or a lower limit setting value of a mapping range of the fluorescence intensity distribution. Figure 3 As shown, the user can adjust the upper limit setting value or the lower limit setting value of the fluorescence intensity mapping range parameter by clicking the sliding stepless adjustment control. In another embodiment, the fluorescence intensity range setting control can also be a gear adjustment control, such as Figure 4 As shown, each gear corresponds to a different fluorescence intensity mapping range, such as gear 1, gear 2, gear 3, etc. The user can click the sliding gear adjustment control to select different gears to determine the fluorescence intensity mapping range. It should be noted that the fluorescence intensity mapping range parameters obtained by the above two adjustment methods are different. The upper limit setting value or the lower limit setting value of the fluorescence intensity mapping range parameter obtained by adjusting the stepless adjustment control is a continuous value expressed as a percentage, and the fluorescence intensity mapping range parameter obtained by adjustment is highly accurate; while when adjusting through the gear adjustment control, since the gear is fixed, only the preset gears can be selected during adjustment, and the obtained fluorescence intensity mapping range parameter is discrete, and the operation is simple during adjustment.
[0094] In one embodiment, the above step S300 further includes:
[0095] Step S330, obtaining the fluorescence intensity mapping range parameters input by the user through the buttons on the endoscope camera handle.
[0096] In this step, the endoscope camera is equipped with a handle, and the user can also input the fluorescence intensity mapping range parameters by operating the buttons on the handle, for example, specifying a button (preset or custom button function) on the handle of the endoscope as the button corresponding to the input fluorescence intensity mapping range parameters, such as a stepless adjustment button and a gear adjustment button. During the operation, the doctor presses the stepless adjustment button to directly execute the continuous setting of the upper limit setting value and / or lower limit setting value of the mapping range of the fluorescence intensity distribution mentioned in the above embodiment, or presses the gear adjustment button to select the fluorescence intensity mapping range parameters of different gears.
[0097] In one embodiment, in the above steps S100 to S800, the hue distribution ranges of the color distribution in the target mapping relationship corresponding to the mapping ranges of different fluorescence intensity distributions are the same or different. It is understandable that after the endoscope host obtains different fluorescence intensity mapping range parameters input by the user and determines the corresponding fluorescence intensity distribution mapping range, the mapping ranges of different fluorescence intensity distributions can be mapped to the same hue distribution range of a color distribution, or the mapping ranges of different fluorescence intensity distributions can be mapped to different hue distribution ranges of a color distribution.
[0098] In another embodiment, in the above steps S100 to S800, in the target mapping relationship, the smaller the mapping range of the fluorescence intensity distribution, the smaller the range of the color distribution. It can be understood that in the target mapping relationship between the fluorescence intensity distribution and the color distribution obtained by the endoscope host, the smaller the mapping range of the corresponding fluorescence intensity distribution determined according to the fluorescence intensity mapping range parameter, the smaller the range of the color distribution. For example, when the user faces a scene with a small change in fluorescence intensity, the range of the fluorescence intensity distribution is determined by selecting an appropriate fluorescence intensity mapping range parameter. At this time, the range of the color distribution in the target mapping relationship obtained can automatically adapt to the range of the fluorescence intensity distribution selected by the user, thereby better displaying the intensity change of fluorescence.
[0099] In one embodiment, a customized color fluorescence fusion display method of the embodiment of the present application further includes the following steps:
[0100] Get the color setting parameters entered by the user.
[0101] Further, the step S400 determines the mapping range of the corresponding fluorescence intensity distribution according to the fluorescence intensity mapping range parameter to obtain the target mapping relationship between the fluorescence intensity distribution and the color distribution, including determining the mapping range of the corresponding fluorescence intensity distribution according to the fluorescence intensity mapping range parameter, and determining the hue distribution range of the corresponding color distribution according to the color setting parameter, and obtaining the target mapping relationship according to the mapping range of the fluorescence intensity distribution and the hue distribution range. In this step, the endoscope camera host can first obtain the fluorescence intensity mapping range parameter input by the user, and then obtain the color setting parameter, and then determine the mapping range of the corresponding fluorescence intensity distribution according to the fluorescence intensity mapping range parameter, and also determine the hue distribution range of the corresponding color distribution according to the color setting parameter, so as to obtain the target mapping relationship according to the mapping range of the fluorescence intensity distribution and the hue distribution range. It should be noted that the hue distribution range of the color distribution in the target mapping relationship can be a default hue distribution range corresponding to the original fluorescence intensity distribution, for example, purple, blue, cyan, green, yellow, and red, which are arranged in order from short to long wavelengths based on the visible light spectrum, are mapped in sequence from weak to strong original fluorescence intensity distribution. The user can set the default hue distribution range by entering color setting parameters, for example, the hue distribution range is limited to the four colors of purple, blue, cyan, and green, and the hue distribution ranges of different color orders can be set according to the fluorescence intensity distribution. Taking the fluorescence intensity distribution from weak to strong as an example, the area with weaker fluorescence intensity is mapped to purple, the area with stronger fluorescence intensity is mapped to green, and the fluorescence intensity in the middle is mapped to blue and cyan in sequence, or the color order can be reversed, and the area with weaker fluorescence intensity is mapped to green, the area with stronger fluorescence intensity is mapped to purple, and the fluorescence intensity in the middle is mapped to cyan and blue in sequence according to the fluorescence intensity distribution from weak to strong, so as to obtain the target mapping relationship.
[0102] In another embodiment, the endoscope camera host obtains the fluorescence intensity mapping range parameters input by the user, and obtains the color setting parameters input by the user, including displaying a mapping parameter setting interface through a display and / or a display screen of the endoscope camera host, wherein the mapping parameter setting interface includes a fluorescence intensity range setting control and a color setting control, obtaining the fluorescence intensity mapping range parameters by the user's operation on the fluorescence intensity range setting control, and obtaining the color setting parameters by the user's operation on the color setting control. In this step, if Figure 5As shown, the user can input the fluorescence intensity mapping range parameter through the fluorescence intensity range setting control on the mapping parameter setting interface, and input the color setting parameter through the color setting control, so as to adjust the mapping relationship with different parameters and improve the display effect of the endoscope image. In one embodiment, the user can click on the sliding color setting control to input the color setting parameter, and the color bar can also be used to display the hue distribution range in the mapping parameter setting interface, so that the user can observe the adjustment made to the hue distribution range in real time. In another embodiment, the principle and implementation method of inputting the fluorescence intensity mapping range parameter can be referred to the above-mentioned step S310 to step S320 embodiment, which will not be repeated here.
[0103] In one embodiment, the fluorescence intensity mapping range parameter of any of the above embodiments includes a lower limit setting value and / or an upper limit setting value of the fluorescence intensity mapping range, and the mapping range of the corresponding fluorescence intensity distribution is determined according to the fluorescence intensity mapping range parameter to obtain a target mapping relationship between the fluorescence intensity distribution and the color distribution, including determining the mapping range of the corresponding fluorescence intensity distribution according to the fluorescence intensity mapping range parameter, so that the lower limit setting value of the fluorescence intensity mapping range corresponds to the starting hue value of the hue distribution range of the color distribution, and / or the upper limit setting value of the fluorescence intensity mapping range corresponds to the ending hue value of the hue distribution range of the color distribution. In this step, determining the mapping range of the corresponding fluorescence intensity distribution according to the fluorescence intensity mapping range parameter can specifically be to map the lower limit setting value of the fluorescence intensity mapping range to the starting hue value of the hue distribution range of the color distribution, or to map the upper limit setting value to the ending hue value of the hue distribution range of the color distribution, or to set the lower limit setting value and the upper limit setting value at the same time, and map them to the starting hue value and the ending hue value respectively. Specifically, the starting hue value may be a color ranked higher in the hue distribution range, corresponding to the starting point of the hue distribution range, and the ending hue value may be a color ranked lower in the hue distribution range, corresponding to the ending point of the hue distribution range.
[0104] In one embodiment, the fluorescence intensity mapping range parameter of any of the above embodiments includes a lower limit setting value and / or an upper limit setting value of the fluorescence intensity mapping range. The user can set the upper limit value and the lower limit value of the mapping range of the fluorescence intensity distribution differently according to actual needs, and the custom color fluorescence fusion display method of the present application also includes: obtaining the current mapping relationship between the fluorescence intensity distribution and the color distribution. It should be noted that the current mapping relationship between the fluorescence intensity distribution and the color distribution may be the current default mapping relationship, which means that the original fluorescence intensity distribution is mapped to the original color distribution. Since this mapping relationship may not be the optimal mapping relationship, the mapping range of the fluorescence intensity distribution in the current mapping relationship can be adjusted according to the fluorescence intensity mapping range parameter, and re-mapped with the color distribution after increasing or reducing the mapping range of the fluorescence intensity distribution to improve the display effect of the endoscopic image.
[0105] Further, in any of the above-mentioned steps, determining the corresponding mapping range of the fluorescence intensity distribution according to the fluorescence intensity mapping range parameter to obtain the target mapping relationship between the fluorescence intensity distribution and the color distribution includes at least one of the following steps:
[0106] Expand the upper limit value of the mapping range of the fluorescence intensity distribution in the current mapping relationship to the upper limit setting value.
[0107] In this step, the endoscope camera host can expand the upper limit value of the mapping range of the fluorescence intensity distribution in the current mapping relationship to the upper limit setting value, and adjust the mapping range of the fluorescence intensity distribution in the current mapping relationship to keep the lower limit value unchanged and only expand the upper limit value of the mapping range.
[0108] Alternatively, the upper limit value of the mapping range of the fluorescence intensity distribution in the current mapping relationship is reduced to the upper limit setting value.
[0109] In this step, the endoscope camera host can also reduce the upper limit value of the mapping range of the fluorescence intensity distribution in the current mapping relationship to the upper limit setting value. The mapping range of the fluorescence intensity distribution in the current mapping relationship can keep the lower limit value unchanged and only reduce the upper limit value of the mapping range.
[0110] Alternatively, the lower limit value of the mapping range of the fluorescence intensity distribution in the current mapping relationship is reduced to the lower limit setting value.
[0111] In this step, the endoscope camera host can also reduce the lower limit value of the mapping range of the fluorescence intensity distribution in the current mapping relationship to the lower limit setting value. The mapping range of the fluorescence intensity distribution in the current mapping relationship can keep the upper limit value unchanged and only reduce the lower limit value of the mapping range.
[0112] Alternatively, the lower limit value of the mapping range of the fluorescence intensity distribution in the current mapping relationship is increased to the lower limit setting value.
[0113] In this step, the endoscope camera host can also increase the lower limit value of the mapping range of the fluorescence intensity distribution in the current mapping relationship to the lower limit setting value. The mapping range of the fluorescence intensity distribution in the current mapping relationship can keep the upper limit value unchanged and only increase the lower limit value of the mapping range.
[0114] Alternatively, the mapping range of the fluorescence intensity distribution in the current mapping relationship is expanded to the interval between the upper limit setting value and the lower limit setting value.
[0115] In this step, the endoscope camera host may also expand the mapping range of the fluorescence intensity distribution in the current mapping relationship to the interval between the upper limit setting value and the lower limit setting value, that is, expand the upper and lower limit values of the mapping range at the same time.
[0116] Alternatively, the mapping range of the fluorescence intensity distribution in the current mapping relationship is narrowed to a range between an upper limit setting value and a lower limit setting value.
[0117] In this step, the endoscope camera host may also reduce the mapping range of the fluorescence intensity distribution in the current mapping relationship to the interval between the upper limit setting value and the lower limit setting value, that is, reduce the upper and lower limit values of the mapping range at the same time.
[0118] It is understandable that in any of the target mapping relationships mentioned in the above embodiments, the mapping relationship between the fluorescence intensity distribution and the color distribution is a linear mapping relationship. In one embodiment, the mapping relationship between the fluorescence intensity distribution and the color distribution may also be a nonlinear mapping relationship.
[0119] In one embodiment, the step S100 of acquiring the fluorescent image and the visible light image captured by the endoscope camera includes at least one of the following steps S110 to S130:
[0120] Step S110, obtaining the user's image adjustment parameters and the fluorescent image and the original visible light image taken by the endoscope camera, and processing the original visible light image according to the image adjustment parameters to obtain a visible light image.
[0121] In this step, the user can input image adjustment parameters to pre-process and adjust the original visible light image separately. The endoscope camera host obtains the user's image adjustment parameters as well as the fluorescent image and original visible light image taken by the endoscope camera, and then adjusts the original visible light image according to the image adjustment parameters to obtain a visible light image.
[0122] Step S120, obtaining the user's image adjustment parameters and the original fluorescence image and visible light image captured by the endoscope camera, and processing the original fluorescence image according to the image adjustment parameters to obtain a fluorescence image.
[0123] In this step, the user can input image adjustment parameters to pre-process and adjust the original fluorescence image separately. The endoscope camera host obtains the user's image adjustment parameters and the fluorescence image and original visible light image taken by the endoscope camera, and then adjusts the original fluorescence image according to the image adjustment parameters to obtain the fluorescence image.
[0124] Step S130, obtaining the user's image adjustment parameters and the original fluorescence image and the original visible light image taken by the endoscope camera, and processing the original fluorescence image and the original visible light image according to the image adjustment parameters to obtain the fluorescence image and the visible light image.
[0125] In this step, the user can input image adjustment parameters to pre-process and adjust the original fluorescence image and the original visible light image respectively. The endoscope camera host obtains the user's image adjustment parameters and the fluorescence image and the original visible light image taken by the endoscope camera, and then adjusts the original fluorescence image and the original visible light image according to the image adjustment parameters to obtain the fluorescence image and the visible light image.
[0126] In one embodiment, the image adjustment parameters in the above steps S110 to S130 include at least one of a custom contrast, a custom saturation, and a custom color style. The endoscope camera host can obtain one or more image adjustment parameters of the user and the original fluorescence image or the original visible light image, and then process the original fluorescence image or the original visible light image, for example, separately adjust the contrast, saturation and color style of the original fluorescence image to obtain a processed fluorescence image, or separately adjust the contrast, saturation and color style of the original visible light image to obtain a processed visible light image.
[0127] In one embodiment, any one of the above embodiments fuses the color fluorescence image and the visible light image to obtain an endoscopic image, including at least one of the following steps S610 to S620:
[0128] Step S610: fusing the color fluorescence image and the visible light image with preset weight parameters to obtain an endoscopic image.
[0129] In this step, after the endoscope camera host obtains the color fluorescence image and the visible light image, the color fluorescence image and the visible light image can be fused according to the preset weight parameters to finally obtain the endoscope image. Specifically, certain weights can be assigned to the visible light image and the color fluorescence image respectively, and the sum of the weights is 1. After obtaining the visible light image and the color fluorescence image, the pixel value of each pixel point of the two is weighted and summed to obtain the pixel value of each pixel point of the fused image, and then the endoscope image is generated by calculation.
[0130] Step S620, obtaining the custom weight parameter input by the user, and fusing the color fluorescence image and the visible light image according to the custom weight parameter to obtain an endoscopic image.
[0131] In this step, the endoscope camera host can also obtain the user-defined weight parameters input by the user, and then fuse the color fluorescence image and the visible light image according to the user-defined weight parameters to finally obtain the endoscope image. The specific fusion process embodiment can be found in the above step S600, which will not be repeated here.
[0132] In one embodiment, the custom weight parameter or preset weight parameter in the above steps S610 to S620 can be a first fusion curve or a second fusion curve. Specifically, the first fusion curve represents different fusion coefficients corresponding to different fluorescence intensities, and the second fusion curve represents different fusion coefficients corresponding to different visible light intensities in the visible light image. It should be understood that the fusion coefficient is used to adjust the transparency of the fluorescence image or the visible light image. The endoscope camera host can allocate weights to the color fluorescence image and the visible light image based on the first fusion curve and the second fusion curve, for example, according to the fluorescence intensity of each pixel in the color fluorescence image, determine a suitable fusion coefficient, that is, determine it as the fusion weight of the color fluorescence image. Furthermore, it is also possible to determine a suitable fusion coefficient according to the visible light intensity of each pixel in the visible light image, that is, the second fusion coefficient, that is, determine it as the fusion weight of the visible light image, so as to adjust the level of the color fluorescence image and the visible light image according to different fusion curves, and improve the display effect of the fused endoscopic image.
[0133] In one embodiment, in the first fusion curve, the fusion coefficient of the first segmented curve whose fluorescence intensity is less than the first intensity value increases as the fluorescence intensity increases, and the fusion coefficient of the second segmented curve whose fluorescence intensity is greater than the first intensity value decreases as the fluorescence intensity increases. In another embodiment, in the second fusion curve, the fusion coefficient of the third segmented curve whose visible light intensity is less than the second intensity value increases as the visible light intensity increases, and the fusion coefficient of the fourth segmented curve whose light intensity is greater than the second intensity value decreases as the visible light intensity increases.
[0134] In one embodiment, the first segmented curve and the second segmented curve mentioned in the above embodiment are linear curves. In another embodiment, the third segmented curve and the fourth segmented curve mentioned in the above embodiment are linear curves.
[0135] In one embodiment, obtaining the user-defined weight parameter input by the user in the above step S620 includes at least one of the following steps:
[0136] Step S621: obtaining a first intensity value input by a user, and determining a first fusion curve according to the first intensity value.
[0137] In this step, the user can input a first intensity value by connecting to the input device of the endoscopic camera system, and then the endoscopic camera host obtains the first intensity value and determines the first fusion curve according to the first intensity value. The user can customize and adjust the first fusion curve according to the current diagnostic or surgical imaging conditions to obtain an accurate endoscopic image.
[0138] Step S622: Acquire a second intensity value input by the user, and determine a second fusion curve according to the second intensity value.
[0139] In this step, the user can input a second intensity value by connecting to the input device of the endoscopic camera system, and then the endoscopic camera host obtains the second intensity value and determines the second fusion curve according to the second intensity value. The user can customize the second fusion curve according to the current diagnostic or surgical imaging conditions to obtain an accurate endoscopic image.
[0140] In one embodiment, obtaining the user-defined weight parameter input by the user in the above step S620 includes at least one of the following steps:
[0141] The custom weight parameters input by the user are obtained through the buttons on the endoscope camera handle.
[0142] In this step, the endoscope camera is equipped with a handle, and the user can also input custom weight parameters by operating the buttons on the handle. For example, a certain button (preset or custom button function) is designated on the handle of the endoscope as the button corresponding to the input of the custom weight parameter. The endoscope camera host responds to the button and obtains the custom weight parameter input by the user.
[0143] The fusion weight adjustment interface is displayed through the display screen of the display and / or the endoscope camera host, and the user's operation on the fusion weight adjustment interface is obtained to obtain the user-defined weight parameters input by the user.
[0144] In this step, the fusion weight adjustment interface can be displayed through a separate display, or through the display screen of the endoscope camera host, or both. It should be noted that the display can be independent and directly connected to the endoscope camera host to display the mapping parameter setting interface. The endoscope camera host usually also includes a human-computer interaction interface, such as a touch screen, buttons or knobs, for controlling the functions and parameters of the endoscope. In one embodiment, the fusion weight adjustment interface can also be displayed directly on the display screen of the endoscope camera host without the need for an additional display. It can be understood that the user's operation on the fusion weight adjustment interface can be an input operation of the user entering a custom weight parameter on the fusion weight adjustment interface, and then the endoscope camera host responds to the user's input operation to obtain and obtain the custom weight parameter.
[0145] In one embodiment, referring to Figure 6 to Figure 7 As shown, the fusion weight adjustment interface includes a custom weight parameter input control, and the endoscope camera host obtains the user's operation on the fusion weight adjustment interface to obtain the custom weight parameter, including obtaining the user's operation on the custom weight parameter input control in the fusion weight adjustment interface to obtain the custom weight parameter. In this step, the user can click on the custom weight parameter input control in the fusion weight adjustment interface to enter the custom weight parameter. In one embodiment, Figure 6 As shown, the endoscope camera host obtains the first intensity value input by the user, determines the first fusion curve according to the first intensity value, and can display the first fusion curve in the fusion weight adjustment interface. In another embodiment, as Figure 7 As shown, the endoscope camera host can also obtain a second intensity value input by the user, determine a second fusion curve according to the second intensity value, and correspondingly display the second fusion curve on the fusion weight adjustment interface.
[0146] Reference Figure 8 As shown, Figure 8 A flowchart of a customized color fluorescence fusion display method provided in an embodiment of the present application, which can be referred to as Figure 1 The described endoscopic imaging system implementation specifically includes but is not limited to the following steps S800 to S1400:
[0147] Step S800, obtaining a fluorescent image and a visible light image taken by an endoscope camera.
[0148] In this step, details may refer to any embodiment of the above step S100, which will not be described again here.
[0149] Step S900: acquiring fluorescence intensity distribution information of the fluorescence image from the fluorescence image.
[0150] In this step, details may refer to any embodiment of the above step S200, which will not be repeated here.
[0151] Step S1000, obtaining a target mapping relationship between fluorescence intensity distribution and color distribution.
[0152] In this step, details may refer to any one of the embodiments of the above step S400, which will not be described again here.
[0153] Step S1100 , converting the fluorescence image into a color fluorescence image according to the fluorescence intensity distribution information and the target mapping relationship.
[0154] In this step, details may refer to any embodiment of the above step S500, which will not be repeated here.
[0155] Step S1200, obtaining a user-defined weight parameter input by a user, wherein the user-defined weight parameter includes a first fusion curve with different fusion coefficients corresponding to different fluorescence intensities, or a second fusion curve with different fusion coefficients corresponding to different visible light intensities.
[0156] In this step, the endoscope camera host can also obtain the user-defined weight parameters input by the user. Specifically, the first fusion curve represents different fusion coefficients corresponding to different fluorescence intensities, and the second fusion curve represents different fusion coefficients corresponding to different visible light intensities in the visible light image. It should be understood that the fusion coefficient is used to adjust the transparency of the fluorescent image or the visible light image.
[0157] Step S1300 , fusing the color fluorescence image and the visible light image according to the custom weight parameters to obtain an endoscopic image.
[0158] In this step, after obtaining the custom weight parameters input by the user, the endoscope camera host fuses the color fluorescence image and the visible light image according to the custom weight parameters, and finally obtains the endoscopic image. Specifically, the endoscope camera host can assign weights to the color fluorescence image and the visible light image based on the first fusion curve and the second fusion curve, and the sum of the weights is 1. For example, a suitable fusion coefficient is determined according to the fluorescence intensity of each pixel in the color fluorescence image, that is, it is determined as the fusion weight of the color fluorescence image. Furthermore, a suitable fusion coefficient, that is, the second fusion coefficient, can also be determined according to the visible light intensity of each pixel in the visible light image, that is, it is determined as the fusion weight of the visible light image. After obtaining the visible light image and the color fluorescence image, the pixel values of each pixel of the two are weighted and summed to obtain the pixel value of each pixel of the fused image, and then the endoscopic image is generated by calculation, so as to adjust the levels of the color fluorescence image and the visible light image according to different fusion curves, and improve the display effect of the fused endoscopic image.
[0159] Step S1400: displaying the endoscopic image on a display.
[0160] In this step, details may refer to any embodiment of the above step S700, which will not be repeated here.
[0161] The custom color fluorescence fusion display method provided in the embodiment of the present application obtains the fluorescence image and the visible light image taken by the endoscope camera, then obtains the fluorescence intensity distribution information of the fluorescence image from the fluorescence image, obtains the target mapping relationship between the fluorescence intensity distribution and the color distribution, and then converts the fluorescence image into a color fluorescence image according to the fluorescence intensity distribution information and the target mapping relationship, and obtains the custom weight parameter input by the user, wherein the custom weight parameter includes a first fusion curve with different fusion coefficients corresponding to different fluorescence intensities, or a second fusion curve with different fusion coefficients corresponding to different visible light intensities, and fuses the color fluorescence image and the visible light image according to the custom weight parameter to obtain an endoscopic image, and finally displays the endoscopic image through a display. In the embodiment of the present application, the doctor can select the fusion curve adjustment with different fusion coefficients by inputting the custom weight parameter, thereby adjusting the layers of the color fluorescence image and the visible light image according to the different fusion curves, which can make the display effect of the endoscopic image after the fusion of the fluorescence image and the visible light image better, thereby improving the accuracy and reliability of endoscopic diagnosis, and being more suitable for actual clinical application scenarios.
[0162] In one embodiment, the fusion coefficients in the above steps S800 to S1400 are used to adjust the transparency of the fluorescent image or the visible light image.
[0163] In one embodiment, in the first fusion curve, the fusion coefficient of the first segmented curve whose fluorescence intensity is less than the first intensity value increases as the fluorescence intensity increases, and the fusion coefficient of the second segmented curve whose fluorescence intensity is greater than the first intensity value decreases as the fluorescence intensity increases. In another embodiment, in the second fusion curve, the fusion coefficient of the third segmented curve whose visible light intensity is less than the second intensity value increases as the visible light intensity increases, and the fusion coefficient of the fourth segmented curve whose light intensity is greater than the second intensity value decreases as the visible light intensity increases.
[0164] In one embodiment, the first segmented curve and the second segmented curve mentioned in the above embodiment are linear curves, or the third segmented curve and the fourth segmented curve are linear curves.
[0165] In one embodiment, obtaining the user-defined weight parameter input by the user in the above step S1200 includes at least one of the following steps:
[0166] A first intensity value input by a user is acquired, and a first fusion curve is determined according to the first intensity value.
[0167] In this step, details may refer to any one of the embodiments of the above step S621, which will not be repeated here.
[0168] A second intensity value input by the user is obtained, and a second fusion curve is determined according to the second intensity value.
[0169] In this step, details may refer to any one of the embodiments of the above step S622, which will not be repeated here.
[0170] In one embodiment, obtaining the custom weight parameter input by the user in any of the above embodiments includes at least one of the following steps:
[0171] Obtaining the custom weight parameters input by the user through the buttons on the endoscope camera handle;
[0172] In this step, the endoscope camera is equipped with a handle, and the user can also input custom weight parameters by operating the buttons on the handle. For example, a certain button (preset or custom button function) is designated on the handle of the endoscope as the button corresponding to the input of the custom weight parameter. The endoscope camera host responds to the button and obtains the custom weight parameter input by the user.
[0173] The fusion weight adjustment interface is displayed through the display screen of the display and / or the endoscope camera host, and the user's operation on the fusion weight adjustment interface is obtained to obtain the user-defined weight parameters input by the user.
[0174] In this step, the fusion weight adjustment interface can be displayed through a separate display, or through the display screen of the endoscope camera host, or both. It should be noted that the display can be independent and directly connected to the endoscope camera host to display the mapping parameter setting interface. The endoscope camera host usually also includes a human-computer interaction interface, such as a touch screen, buttons or knobs, for controlling the functions and parameters of the endoscope. In one embodiment, the fusion weight adjustment interface can also be displayed directly on the display screen of the endoscope camera host without the need for an additional display. It can be understood that the user's operation on the fusion weight adjustment interface can be an input operation of the user entering a custom weight parameter on the fusion weight adjustment interface, and then the endoscope camera host responds to the user's input operation to obtain and obtain the custom weight parameter.
[0175] In one embodiment, referring to Figure 6 to Figure 7 As shown, the fusion weight adjustment interface includes a custom weight parameter input control, and the endoscope camera host obtains the user's operation on the fusion weight adjustment interface to obtain the custom weight parameter, including obtaining the user's operation on the custom weight parameter input control in the fusion weight adjustment interface to obtain the custom weight parameter. In this step, the user can click on the custom weight parameter input control in the fusion weight adjustment interface to enter the custom weight parameter. In one embodiment, Figure 6 As shown, the endoscope camera host obtains the first intensity value input by the user, determines the first fusion curve according to the first intensity value, and can display the first fusion curve in the fusion weight adjustment interface. In another embodiment, as Figure 7 As shown, the endoscope camera host can also obtain a second intensity value input by the user, determine a second fusion curve according to the second intensity value, and correspondingly display the second fusion curve on the fusion weight adjustment interface.
[0176] In one embodiment, obtaining the target mapping relationship between the fluorescence intensity distribution and the color distribution in step S1000 of any of the above embodiments includes the following steps S1010 to S1030:
[0177] Step S1010, obtaining the fluorescence intensity mapping range parameters input by the user.
[0178] In this step, details may refer to any one of the embodiments of the above step S300, which will not be described again here.
[0179] Step S1020, obtaining the current mapping relationship between the fluorescence intensity distribution and the color distribution.
[0180] In this step, the endoscope host obtains the current mapping relationship between the fluorescence intensity distribution and the color distribution. The fluorescence intensity distribution reflects the fluorescence intensity at different wavelengths, and the corresponding color distribution refers to the range of color values in a specific color space. The mapping relationship between the fluorescence intensity distribution and the color distribution shows that fluorescence of different wavelengths corresponds to different colors, and the fluorescence intensity of each color is different. The mapping relationship can be a mapping relationship table or a mapping relationship function. It should be noted that the mapping relationship can be a mapping relationship table or a mapping relationship function. In one embodiment, if the fluorescence intensity of a certain wavelength is high and the wavelength corresponds to red, the area can be represented by a brighter red in the endoscopic image. In another embodiment, if the fluorescence intensity of a certain wavelength is low and the wavelength corresponds to purple, the area can be represented by a darker purple in the mapping diagram. In this way, the color distribution and intensity changes of the fluorescent substance at different wavelengths can be intuitively displayed.
[0181] Step S1030 , adjusting the mapping range of the fluorescence intensity distribution in the current mapping relationship according to the fluorescence intensity mapping range parameter to obtain a target mapping relationship.
[0182] In this step, the endoscope host adjusts the mapping range of the fluorescence intensity distribution in the current mapping relationship according to the fluorescence intensity mapping range parameter to obtain the target mapping relationship. Specifically, the current mapping relationship between the fluorescence intensity distribution and the color distribution may be the current default mapping relationship, which means that the original fluorescence intensity distribution is mapped to the color distribution. Since this mapping relationship may not be the optimal mapping relationship, the mapping range of the fluorescence intensity distribution in the current mapping relationship may be adjusted according to the fluorescence intensity mapping range parameter, and remapped to the original color distribution after increasing or reducing the range. For example, the mapping range can be expanded to display more details of the target tissue (including darker and brighter areas), or the range can be reduced to highlight the fluorescence signal of a specific intensity to adapt to the current diagnostic or surgical imaging conditions, thereby obtaining the target mapping relationship and ensuring that the fluorescence signal in the endoscopic image can be presented in a clearer and more accurate manner to facilitate the doctor's subsequent analysis.
[0183] In one embodiment, the step S1010 of obtaining the fluorescence intensity mapping range parameter input by the user comprises the following steps:
[0184] Step S1011: displaying a mapping parameter setting interface through a display and / or a display screen of an endoscope camera host, wherein the mapping parameter setting interface includes a fluorescence intensity range setting control.
[0185] Step S1012: Obtain the user's operation on the fluorescence intensity range setting control in the mapping parameter setting interface to obtain the fluorescence intensity mapping range parameter.
[0186] In one embodiment, the fluorescence intensity range setting control may be a stepless adjustment control for continuously setting the upper limit setting value and / or lower limit setting value of the mapping range of the fluorescence intensity distribution. In another embodiment, the fluorescence intensity range setting control may also be a gear adjustment control, and each gear corresponds to a different mapping range.
[0187] It can be understood that the detailed implementation methods and achieved effects of the above-mentioned steps S1011 to S1012 and their related sub-steps have been described in detail in other embodiments of the present application and will not be repeated here.
[0188] In one embodiment, the fluorescence intensity mapping range parameter of any of the above embodiments includes a lower limit setting value and / or an upper limit setting value of the fluorescence intensity mapping range, and adjusting the mapping range of the fluorescence intensity distribution in the current mapping relationship according to the fluorescence intensity mapping range parameter to obtain a target mapping relationship includes adjusting the mapping range of the fluorescence intensity distribution according to the fluorescence intensity mapping range parameter so that the lower limit setting value of the fluorescence intensity mapping range corresponds to the starting hue value of the color distribution, and / or the upper limit setting value of the fluorescence intensity mapping range corresponds to the ending hue value of the color distribution. In this step, adjusting the mapping range of the fluorescence intensity distribution according to the fluorescence intensity mapping range parameter may specifically be mapping the upper limit setting value of the fluorescence intensity mapping range to the ending hue value of the color distribution, or mapping the lower limit setting value to the starting hue value of the color distribution, or may simultaneously adjust the upper limit setting value and the lower limit setting value of the mapping range of the fluorescence intensity distribution, and respectively map the upper limit setting value and the lower limit setting value to the ending hue value and the starting hue value. Specifically, the starting hue value may be a color ranked higher in the hue distribution range, corresponding to the starting point of the hue distribution range, and the ending hue value may be a color ranked lower in the hue distribution range, corresponding to the ending point of the hue distribution range.
[0189] In one embodiment, the step S800 of acquiring the fluorescent image and the visible light image captured by the endoscope camera includes at least one of the following steps S810 to S830:
[0190] Step S810, obtaining the user's image adjustment parameters and the fluorescent image and original visible light image captured by the endoscope camera, and processing the original visible light image according to the image adjustment parameters to obtain a visible light image.
[0191] Step S820, obtaining the user's image adjustment parameters and the original fluorescence image and visible light image taken by the endoscope camera, and processing the original fluorescence image according to the image adjustment parameters to obtain a fluorescence image.
[0192] Step S830, obtaining the user's image adjustment parameters and the original fluorescence image and the original visible light image taken by the endoscope camera, and processing the original fluorescence image and the original visible light image according to the image adjustment parameters to obtain the fluorescence image and the visible light image.
[0193] It can be understood that the detailed implementation methods and achieved effects of the above-mentioned steps S810 to S830 and their related sub-steps have been described in detail in other embodiments of the present application. The features of other steps in the embodiments of the present application, such as steps S110 to S130 and their related sub-steps, can be introduced into steps S810 to S830 and will not be repeated here.
[0194] In one embodiment, the image adjustment parameters in the above steps S810 to S830 include at least one of a custom contrast, a custom saturation, and a custom color style. The endoscope camera host can obtain one or more image adjustment parameters of the user and the original fluorescence image or the original visible light image, and then process the original fluorescence image or the original visible light image, for example, separately adjust the contrast and saturation of the original fluorescence image and adjust the color style of the image to obtain a processed fluorescence image, or separately adjust the contrast and saturation of the original visible light image and adjust the color style of the image to obtain a processed visible light image.
[0195] Reference Fig. 9 As shown, Fig. 9 A flowchart of a customized color fluorescence fusion display method provided in an embodiment of the present application, which can be referred to as Figure 1 The described endoscopic imaging system implementation specifically includes but is not limited to the following steps S1500 to S2000:
[0196] Step S1500, obtaining the fluorescent image and visible light image taken by the endoscope camera.
[0197] In this step, details may refer to any embodiment of the above step S100, which will not be described again here.
[0198] Step S1600, obtaining fluorescence intensity distribution information of the fluorescence image from the fluorescence image.
[0199] In this step, details may refer to any embodiment of the above step S200, which will not be repeated here.
[0200] Step S1700: obtaining color conversion parameters input by a user, and determining a target mapping relationship between fluorescence intensity distribution and color distribution according to the color conversion parameters.
[0201] In this step, the endoscope camera host obtains the color conversion parameters input by the user, and determines the target mapping relationship between the fluorescence intensity distribution and the color distribution according to the color conversion parameters.
[0202] Step S1800 , converting the fluorescence image into a color fluorescence image according to the fluorescence intensity distribution information and the target mapping relationship.
[0203] In this step, details may refer to any embodiment of the above step S500, which will not be repeated here.
[0204] Step S1900, obtaining the custom weight parameters input by the user, and fusing the color fluorescence image and the visible light image according to the custom weight parameters to obtain an endoscopic image.
[0205] In this step, details may refer to any one of the embodiments of the above-mentioned steps S1200 to S1300, which will not be repeated here.
[0206] Step S2000, displaying the endoscopic image on a display.
[0207] In this step, details may refer to any embodiment of the above step S700, which will not be repeated here.
[0208] The custom color fluorescence fusion display method provided in the embodiment of the present application obtains the fluorescence image and the visible light image taken by the endoscope camera, then obtains the fluorescence intensity distribution information of the fluorescence image from the fluorescence image, and obtains the color conversion parameters input by the user, determines the target mapping relationship between the fluorescence intensity distribution and the color distribution according to the color conversion parameters, and then converts the fluorescence image into a color fluorescence image according to the fluorescence intensity distribution information and the target mapping relationship, and then obtains the custom weight parameters input by the user, and fuses the color fluorescence image and the visible light image according to the custom weight parameters to obtain the endoscopic image, and finally displays the endoscopic image through the display. In the embodiment of the present application, the doctor can adjust the mapping relationship between the fluorescence intensity distribution and the color distribution of the fluorescence image by inputting the color conversion parameters, and select the fusion curve adjustment with different fusion coefficients by inputting the custom weight parameters, so as to adjust the layers of the color fluorescence image and the visible light image according to different fusion curves, so as to make the display effect of the endoscopic image after the fusion of the fluorescence image and the visible light image better, thereby improving the accuracy and reliability of endoscopic diagnosis, and being more suitable for actual clinical application scenarios.
[0209] In one embodiment, the above step S1700 specifically includes the following steps:
[0210] Step S1710, obtaining the fluorescence intensity mapping range parameters input by the user.
[0211] Step S1720, obtaining the current mapping relationship between the fluorescence intensity distribution and the color distribution.
[0212] Step S1730, adjusting the mapping range of the fluorescence intensity distribution in the current mapping relationship according to the fluorescence intensity mapping range parameter to obtain a target mapping relationship.
[0213] In one embodiment, the step S1710 of obtaining the fluorescence intensity mapping range parameter input by the user includes:
[0214] Step S1711, displaying a mapping parameter setting interface through a display and / or a display screen of an endoscope camera host.
[0215] Step S1712: Obtain the user's operation on the mapping parameter setting interface to obtain the fluorescence intensity mapping range parameter.
[0216] It can be understood that the detailed implementation methods and achieved effects of the above-mentioned steps S1710 to S1730 and their related sub-steps have been described in detail in other embodiments of the present application. The features of other steps in the embodiments of the present application, such as steps S300 to S400, steps S310 to S320 and their related sub-steps, can be introduced into steps S1710 to S1730, and steps S1711 to S1712, and will not be repeated here.
[0217] In one embodiment, the custom weight parameter of any of the above embodiments may be a first fusion curve or a first fusion curve. Specifically, the first fusion curve represents different fusion coefficients corresponding to different fluorescence intensities, and the second fusion curve represents different fusion coefficients corresponding to different visible light intensities in the visible light image. It should be understood that the fusion coefficient is used to adjust the transparency of the fluorescence image or the visible light image.
[0218] In one embodiment, obtaining the custom weight parameter input by the user in any of the above embodiments includes at least one of the following steps:
[0219] Obtaining the custom weight parameters input by the user through the buttons on the endoscope camera handle;
[0220] In this step, the endoscope camera is equipped with a handle, and the user can also input custom weight parameters by operating the buttons on the handle. For example, a certain button (preset or custom button function) is designated on the handle of the endoscope as the button corresponding to the input of the custom weight parameter. The endoscope camera host responds to the button and obtains the custom weight parameter input by the user.
[0221] The fusion weight adjustment interface is displayed through the display screen of the display and / or the endoscope camera host, and the user's operation on the fusion weight adjustment interface is obtained to obtain the user-defined weight parameters input by the user.
[0222] In this step, the fusion weight adjustment interface can be displayed through a separate display, or through the display screen of the endoscope camera host, or both are displayed at the same time. It should be noted that the display can be independent and directly connected to the endoscope camera host to display the mapping parameter setting interface. The endoscope camera host usually also includes a human-computer interaction interface, such as a touch screen, buttons or knobs, for controlling the functions and parameters of the endoscope. In one embodiment, the fusion weight adjustment interface can also be displayed directly on the display screen of the endoscope camera host without the need for an additional display. It is understandable that the user's operation on the fusion weight adjustment interface can be an input operation of the user entering a custom weight parameter on the fusion weight adjustment interface, and then the endoscope camera host responds to the user's input operation to obtain and obtain the custom weight parameter. It is understandable that the detailed implementation method of the above steps to obtain the custom weight parameter input by the user and the effect achieved have been described in detail in other embodiments of the present application, and will not be repeated here.
[0223] It is to be understood that, although the steps in the above-mentioned various flow charts are shown in sequence according to the indication of the arrows, these steps are not necessarily performed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in the present embodiment, the execution of these steps does not have a strict order restriction, and these steps can be performed in other orders. Moreover, at least a portion of the steps in the above-mentioned flow charts can include a plurality of steps or a plurality of stages, and these steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of the steps or stages in other steps or other steps.
[0224] An embodiment of the present application provides an endoscopic camera system, which includes an endoscope and an endoscopic camera host. The endoscopic camera host is connected to the endoscope, and is used to obtain fluorescent images and visible light images taken by the endoscope. The endoscopic camera host is also used to execute the custom color fluorescence fusion display method mentioned in any of the above embodiments.
[0225] Reference Fig.10 , Fig.10 The structural block diagram of an endoscope camera host provided in an embodiment of the present application, the endoscope camera host 1100 includes components such as a housing 1110, a communication interface 1120, a display interface 1130, and a processor 1140. Those skilled in the art can understand that Fig.10 The structure of the endoscope camera host 1100 shown in the figure does not constitute a limitation on the endoscope camera host 1100, and may include more or less components than those shown in the figure, or combine certain components, or different components.
[0226] The communication interface 1120 is disposed on the housing 1110 and can be used to connect to the endoscope via wired or wireless communication to obtain image signals from the endoscope;
[0227] The display interface 1130 is disposed on the housing 1110 and can be used to communicate with a display device and output an endoscopic image;
[0228] The processor 1140 acquires the endoscope data through the communication interface 1120 and performs image processing, and then outputs the endoscope image to the display through the display interface 1130 for display.
[0229] In this embodiment, the processor 1140 included in the endoscope camera host 1100 can execute the customized color fluorescence fusion display method of the previous embodiment.
[0230] The electronic device for executing the above-mentioned customized color fluorescence fusion display method provided in the embodiment of the present application may be a terminal 1200, referring to Fig.11 , Fig.11 The terminal 1200 is a partial structural block diagram of the embodiment of the present application. The terminal 1200 includes: a camera assembly 1210, a memory 1220, an input unit 1230, a display unit 1240, a sensor 1250, an audio circuit 1260, a wireless fidelity (Wi re less fidelity, WiFi for short) module 1270, a processor 1280, and a power supply 1290. Those skilled in the art can understand that Fig.11 The structure of the terminal 1200 shown in the figure does not constitute a limitation on the terminal 1200, and the terminal 1200 may include more or less components than those shown in the figure, or combine certain components, or arrange the components differently.
[0231] The camera assembly 1210 can be used to collect images or videos. Optionally, the camera assembly 1210 includes a front camera and a rear camera. Usually, the front camera is set on the front panel of the terminal, and the rear camera is set on the back of the terminal. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth of field camera, a wide-angle camera, and a telephoto camera, so as to realize the fusion of the main camera and the depth of field camera to realize the background blur function, the fusion of the main camera and the wide-angle camera to realize panoramic shooting and VR (Virtua l Reality) shooting function or other fusion shooting functions.
[0232] The memory 1220 may be used to store software programs and modules. The processor 1280 executes various functional applications and data processing of the terminal by running the software programs and modules stored in the memory 1220 .
[0233] The input unit 1230 may be used to receive input digital or character information and generate key signal input related to the terminal's settings and function control. Specifically, the input unit 1230 may include a touch panel 1231 and other input devices 1232 .
[0234] The display unit 1240 may be used to display input information or provided information and various menus of the terminal. The display unit 1240 may include a display panel 1241 .
[0235] The audio circuit 1260 , the speaker 1261 , and the microphone 1262 may provide an audio interface.
[0236] The power source 1290 may be alternating current, direct current, a disposable battery, or a rechargeable battery.
[0237] The number of sensors 1250 may be one or more, and the one or more sensors 1250 include but are not limited to: acceleration sensors, gyroscope sensors, pressure sensors, optical sensors, etc. Among them:
[0238] The acceleration sensor can detect the magnitude of acceleration on the three coordinate axes of the coordinate system established by the terminal. For example, the acceleration sensor can be used to detect the components of gravity acceleration on the three coordinate axes. The processor 1280 can control the display unit 1240 to display the user interface in a horizontal view or a vertical view according to the gravity acceleration signal collected by the acceleration sensor. The acceleration sensor can also be used for collecting game or user motion data.
[0239] The gyroscope sensor can detect the body direction and rotation angle of the terminal, and the gyroscope sensor can cooperate with the acceleration sensor to collect the user's 3D actions on the terminal. The processor 1280 can implement the following functions based on the data collected by the gyroscope sensor: motion sensing (such as changing the UI according to the user's tilt operation), image stabilization during shooting, game control, and inertial navigation.
[0240] The pressure sensor can be set in the side frame of the terminal and / or the lower layer of the display unit 1240. When the pressure sensor is set in the side frame of the terminal, the user's holding signal of the terminal can be detected, and the processor 1280 performs left and right hand recognition or shortcut operation according to the holding signal collected by the pressure sensor. When the pressure sensor is set in the lower layer of the display unit 1240, the processor 1280 controls the operability controls on the UI interface according to the user's pressure operation on the display unit 1240. The operability control includes at least one of a button control, a scroll bar control, an icon control, and a menu control.
[0241] The optical sensor is used to collect the ambient light intensity. In one embodiment, the processor 1280 can control the display brightness of the display unit 1240 according to the ambient light intensity collected by the optical sensor. Specifically, when the ambient light intensity is high, the display brightness of the display unit 1240 is increased; when the ambient light intensity is low, the display brightness of the display unit 1240 is decreased. In another embodiment, the processor 1280 can also dynamically adjust the shooting parameters of the camera assembly 1210 according to the ambient light intensity collected by the optical sensor.
[0242] In this embodiment, the processor 1280 included in the terminal 1200 can execute the custom color fluorescence fusion display method of the previous embodiment. It should be understood that the terminal 1200 can be a mobile phone, a computer, an intelligent voice interaction device, a smart home appliance, a car terminal, etc., but is not limited thereto.
[0243] The electronic device for executing the above-mentioned customized color fluorescence fusion display method provided in the embodiment of the present application can also be a server 1300, referring to Fig.12 , Fig.12 This is a partial structural block diagram of a server 1300 provided in an embodiment of the present application. The server 1300 may have relatively large differences due to different configurations or performances, and may include one or more processors 1310 (Centra l Processing Un its, referred to as CPU), one or more memories 1320, and storage media 1330 (for example, one or more mass storage devices). Among them, the memory 1320 and the storage medium 1330 may be temporary storage or permanent storage. The program stored in the storage medium 1330 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the server 1300. Furthermore, the processor 1310 may be configured to communicate with the storage medium 1330 to execute a series of instruction operations in the storage medium 1330 on the server 1300.
[0244] The server 1300 may further include one or more power supplies (not shown), one or more wired or wireless network interfaces 1340 , one or more input / output interfaces 1350 , and the like.
[0245] In this embodiment, the processor 1310 in the server 1300 can be used to execute the custom color fluorescent fusion display method of the previous embodiment. It should be understood that the server 1300 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content De l ivery Network, content distribution network), and big data and artificial intelligence platforms. In addition, the server 1300 can also be a node server in a blockchain network.
[0246] The terminal 1200 and the server 1300 may be connected directly or indirectly via wired or wireless communication, which is not limited in the embodiments of the present application.
[0247] An embodiment of the present application provides a computer-readable storage medium, which stores a computer program and is applied to an endoscope camera host. When the computer program is executed by a processor, it implements the customized color fluorescence fusion display method mentioned in any of the above embodiments.
[0248] An embodiment of the present application provides a computer program product, which includes a computer program, wherein computer instructions are stored in a computer-readable storage medium, a processor of a computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the computer device executes the customized color fluorescence fusion display method mentioned in any one of the above embodiments.
[0249] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.
[0250] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0251] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0252] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store program codes.
[0253] It should also be understood that the various implementations provided in the embodiments of the present application can be combined arbitrarily to achieve different technical effects.
[0254] The above is a specific description of the preferred implementation of the present application, but the present application is not limited to the above implementation mode. Technical personnel familiar with the field can also make various equivalent modifications or substitutions under the shared conditions without violating the spirit of the present application. These equivalent modifications or substitutions are all included in the scope defined by the claims of the present application.
Claims
1. A custom color fluorescence fusion display method, applied to an endoscope camera host, characterized in that: The method comprises: Acquire fluorescent images and visible light images taken by an endoscope camera; Acquiring fluorescence intensity distribution information of the fluorescence image from the fluorescence image; Get the fluorescence intensity mapping range parameters input by the user; Determining a corresponding mapping range of the fluorescence intensity distribution according to the fluorescence intensity mapping range parameter, and obtaining a target mapping relationship between the fluorescence intensity distribution and the color distribution; Converting the fluorescence image into a color fluorescence image according to the fluorescence intensity distribution information and the target mapping relationship; fusing the color fluorescence image and the visible light image to obtain an endoscopic image; The endoscopic image is displayed on a display.
2. The customized color fluorescence fusion display method according to claim 1, characterized in that: The step of obtaining the fluorescence intensity mapping range parameter input by the user includes: Displaying a mapping parameter setting interface via a display and / or a display screen of the endoscope camera host; The user's operation on the mapping parameter setting interface is obtained to obtain the fluorescence intensity mapping range parameter.
3. The customized color fluorescence fusion display method according to claim 2, characterized in that: The mapping parameter setting interface includes a fluorescence intensity range setting control, and the obtaining of the fluorescence intensity mapping range parameter by the user's operation on the mapping parameter setting interface includes: Acquiring a user's operation on the fluorescence intensity range setting control in the mapping parameter setting interface to obtain a fluorescence intensity mapping range parameter; The fluorescence intensity range setting control is a stepless adjustment control for continuously setting an upper limit setting value and / or a lower limit setting value of a mapping range of the fluorescence intensity distribution, or the fluorescence intensity range setting control is a gear adjustment control, each gear corresponding to a different mapping range.
4. The customized color fluorescence fusion display method according to claim 1, characterized in that: The step of obtaining the fluorescence intensity mapping range parameter input by the user includes: The fluorescence intensity mapping range parameters input by the user are obtained through the buttons on the endoscope camera handle.
5. The customized color fluorescence fusion display method according to claim 1, characterized in that: The hue distribution ranges of the color distribution in the target mapping relationship corresponding to the mapping ranges of different fluorescence intensity distributions are the same or different; Alternatively, in the target mapping relationship, the smaller the mapping range of the fluorescence intensity distribution is, the smaller the hue distribution range of the color distribution is.
6. The customized color fluorescence fusion display method according to claim 1, characterized in that: Also includes: Get the color setting parameters entered by the user; The step of determining the corresponding mapping range of the fluorescence intensity distribution according to the fluorescence intensity mapping range parameter to obtain a target mapping relationship between the fluorescence intensity distribution and the color distribution includes: The mapping range of the corresponding fluorescence intensity distribution is determined according to the fluorescence intensity mapping range parameter, and the hue distribution range of the corresponding color distribution is determined according to the color setting parameter, and the target mapping relationship is obtained according to the mapping range of the fluorescence intensity distribution and the hue distribution range.
7. The customized color fluorescence fusion display method according to claim 6, characterized in that: Also includes: The step of obtaining the fluorescence intensity mapping range parameter input by the user and obtaining the color setting parameter input by the user includes: Displaying a mapping parameter setting interface through a display and / or a display screen of the endoscope camera host, wherein the mapping parameter setting interface includes a fluorescence intensity range setting control and a color setting control; Acquiring a user's operation on the fluorescence intensity range setting control to obtain a fluorescence intensity mapping range parameter; The user's operation on the color setting control is obtained to obtain color setting parameters.
8. The customized color fluorescence fusion display method according to any one of claims 1 to 7, characterized in that: The fluorescence intensity mapping range parameter includes a lower limit setting value and / or an upper limit setting value of the fluorescence intensity mapping range; determining the corresponding mapping range of the fluorescence intensity distribution according to the fluorescence intensity mapping range parameter to obtain the target mapping relationship between the fluorescence intensity distribution and the color distribution includes: Determine the corresponding mapping range of the fluorescence intensity distribution according to the fluorescence intensity mapping range parameters, so that the lower limit setting value of the fluorescence intensity mapping range corresponds to the starting hue value of the hue distribution range of the color distribution, and / or the upper limit setting value of the fluorescence intensity mapping range corresponds to the ending hue value of the hue distribution range of the color distribution.
9. The customized color fluorescence fusion display method according to any one of claims 1 to 7, characterized in that: The fluorescence intensity mapping range parameter includes an upper limit setting value and / or a lower limit setting value of the fluorescence intensity mapping range; the method further includes: Obtain the current mapping relationship between fluorescence intensity distribution and color distribution; Determining the corresponding mapping range of the fluorescence intensity distribution according to the fluorescence intensity mapping range parameter to obtain the target mapping relationship between the fluorescence intensity distribution and the color distribution includes one of the following: Expanding the upper limit value of the mapping range of the fluorescence intensity distribution in the current mapping relationship to the upper limit setting value; Alternatively, the upper limit value of the mapping range of the fluorescence intensity distribution in the current mapping relationship is reduced to the upper limit setting value; Alternatively, the lower limit value of the mapping range of the fluorescence intensity distribution in the current mapping relationship is reduced to the lower limit setting value; Alternatively, the lower limit value of the mapping range of the fluorescence intensity distribution in the current mapping relationship is increased to the lower limit setting value; Alternatively, the mapping range of the fluorescence intensity distribution in the current mapping relationship is expanded to an interval between the upper limit setting value and the lower limit setting value; Alternatively, the mapping range of the fluorescence intensity distribution in the current mapping relationship is narrowed to a range between the upper limit setting value and the lower limit setting value.
10. The customized color fluorescence fusion display method according to any one of claims 1 to 7, characterized in that: In the target mapping relationship, the mapping relationship between the fluorescence intensity distribution and the color distribution is a linear mapping relationship.
11. The customized color fluorescence fusion display method according to any one of claims 1 to 7, characterized in that: The step of obtaining the fluorescent image and the visible light image captured by the endoscope camera includes one of the following: Acquire the user's image adjustment parameters and the fluorescent image and the original visible light image taken by the endoscope camera, and process the original visible light image according to the image adjustment parameters to obtain the visible light image; Alternatively, the user's image adjustment parameters and the original fluorescent image and visible light image taken by the endoscope camera are obtained, and the original fluorescent image is processed according to the image adjustment parameters to obtain the fluorescent image; Alternatively, the user's image adjustment parameters and the original fluorescent image and the original visible light image taken by the endoscope camera are obtained, and the original fluorescent image and the original visible light image are processed according to the image adjustment parameters to obtain the fluorescent image and the visible light image respectively; The image adjustment parameters include at least one of the following: Customizable contrast; Custom saturation; Custom color style.
12. The customized color fluorescence fusion display method according to any one of claims 1 to 7, characterized in that: The fusing the color fluorescence image and the visible light image to obtain an endoscopic image includes one of the following: fusing the color fluorescence image and the visible light image with a preset weight parameter to obtain an endoscopic image; or, The user-defined weight parameters input by the user are obtained, and the color fluorescence image and the visible light image are fused according to the user-defined weight parameters to obtain an endoscopic image.
13. The customized color fluorescence fusion display method according to claim 12, characterized in that: The custom weight parameter or the preset weight parameter is one of the following: a first fusion curve, wherein the first fusion curve represents different fusion coefficients corresponding to different fluorescence intensities; or, a second fusion curve, wherein the second fusion curve represents different fusion coefficients corresponding to different visible light intensities in the visible light image; The fusion coefficient is used to adjust the transparency of the fluorescent image or the visible light image.
14. The customized color fluorescence fusion display method according to claim 13, characterized in that: In the first fusion curve, the fusion coefficient of the first segmented curve whose fluorescence intensity is less than the first intensity value increases as the fluorescence intensity increases, and the fusion coefficient of the second segmented curve whose fluorescence intensity is greater than the first intensity value decreases as the fluorescence intensity increases; Alternatively, in the second fusion curve, the fusion coefficient of the third segmented curve whose visible light intensity is less than the second intensity value increases as the visible light intensity increases, and the fusion coefficient of the fourth segmented curve whose light intensity is greater than the second intensity value decreases as the visible light intensity increases.
15. The customized color fluorescence fusion display method according to claim 14, characterized in that: The first segmented curve and the second segmented curve are linear curves, or the third segmented curve and the fourth segmented curve are linear curves.
16. The customized color fluorescence fusion display method according to claim 15, characterized in that: The step of obtaining the custom weight parameter input by the user includes one of the following: Acquire a first intensity value input by a user, and determine the first fusion curve according to the first intensity value; Alternatively, a second intensity value input by a user is obtained, and the second fusion curve is determined according to the second intensity value.
17. The customized color fluorescence fusion display method according to claim 12, characterized in that: The step of obtaining the custom weight parameter input by the user includes: Obtaining a custom weight parameter input by a user through a button on the endoscope camera handle; Alternatively, a fusion weight adjustment interface is displayed on a display and / or a display screen of the endoscope camera host, and user operations on the fusion weight adjustment interface are obtained to obtain user-defined weight parameters input by the user.
18. A custom color fluorescence fusion display method, applied to an endoscope camera host, characterized in that: The method comprises: Acquire fluorescent images and visible light images taken by an endoscope camera; Acquiring fluorescence intensity distribution information of the fluorescence image from the fluorescence image; Obtaining a target mapping relationship between fluorescence intensity distribution and color distribution; Converting the fluorescence image into a color fluorescence image according to the fluorescence intensity distribution information and the target mapping relationship; Obtaining a user-defined weight parameter input by a user, wherein the user-defined weight parameter includes a first fusion curve with different fusion coefficients corresponding to different fluorescence intensities, or a second fusion curve with different fusion coefficients corresponding to different visible light intensities; Performing fusion processing on the color fluorescence image and the visible light image according to the custom weight parameter to obtain an endoscopic image; The endoscopic image is displayed on a display.
19. A custom color fluorescence fusion display method, applied to an endoscope camera host, characterized in that: The method comprises: Acquire fluorescent images and visible light images taken by an endoscope camera; Acquiring fluorescence intensity distribution information of the fluorescence image from the fluorescence image; Acquire color conversion parameters input by a user, and determine a target mapping relationship between fluorescence intensity distribution and color distribution according to the color conversion parameters; Converting the fluorescence image into a color fluorescence image according to the fluorescence intensity distribution information and the target mapping relationship; Obtaining a custom weight parameter input by a user, and fusing the color fluorescence image and the visible light image according to the custom weight parameter to obtain an endoscopic image; The endoscopic image is displayed on a display.
20. An endoscope camera system, characterized in that: include: Endoscope; An endoscope camera host is connected to the endoscope, and is used to obtain the fluorescent image and visible light image taken by the endoscope. The endoscope camera host is also used to execute the customized color fluorescence fusion display method described in any one of claims 1 to 19.
