Image processing method, device, electronic device and storage medium

Abstract

The present application discloses an image processing method, device, electronic device and storage medium, and relates to the field of image processing technology. The method includes: obtaining a first RGB image formed by the reflection of the outgoing light of the endoscope illumination light source on the patient's tissue and blood vessels, and the illumination light source includes a narrow-band amber light source and a narrow-band red light source; decomposing the R channel of the first RGB image to obtain a first component image with narrow-band red light as the main response component and a second component image with narrow-band amber light as the main response component; based on the spectral mapping matrix, mapping the first component image and the second component image to obtain a second RGB image; based on the color adjustment matrix, color enhancing the second RGB image to obtain a target RGB image, and the target RGB image is used for observing and processing tissue and blood vessels. In this way, it is possible to achieve separate processing of the individual response images of the narrow-band light source to achieve the effect of enhancing the vascular tissue.

Description

Technical Field

[0001] The present application relates to the field of image recognition, and in particular to an image processing method, device, electronic device and storage medium. Background Art

[0002] With the vigorous development of endoscopic applications, the demand for high-quality endoscopic imaging is also increasing. In endoscopic application scenarios, doctors usually use white light imaging that is consistent with human vision for routine operations. When further exploration of lesions is required, narrow-band light imaging is used.

[0003] For example, when exploring the deep tissue and blood vessels of the human body, since the hemoglobin in this part has a strong absorption characteristic for the narrow-band light source with a central wavelength of about 600nm, an amber light source and a red light source are usually used as the illumination light source for narrow-band light imaging to improve the recognizability of the deep tissue and blood vessels. Among them, when using an amber light source and a red light source as the illumination light source for narrow-band light imaging, the two light sources usually emit light at the same time, and after being reflected by the tissue and blood vessels in the human body, a fused image is obtained, and the tissue and blood vessels are observed and analyzed based on the fused image.

[0004] However, since the fused image is obtained by the joint action of two light sources in the same frame, the amber light source and the red light source are not differentiated and processed, and thus the specific information generated by the two light sources on tissues and blood vessels cannot be directly reflected, thereby weakening the enhancement effect on the lesion area.

[0005] Therefore, there is an urgent need for an image processing technology for processing the fused image, so as to individually process the specific information generated by a light source of a specific wavelength band on tissue blood vessels, thereby achieving the purpose of enhancing narrow-band imaging for specific lesions or locations. Summary of the invention

[0006] The embodiments of the present application provide an image processing method, device, electronic device and storage medium, which are used to solve the problem that the existing technical solutions cannot directly reflect the specific information generated by two light sources on tissue blood vessels, thereby weakening the enhancement effect on the lesion area.

[0007] On the one hand, an embodiment of the present application provides an image processing method, which includes: obtaining a first RGB image formed by reflection of the output light of an endoscope illumination light source on the patient's tissue and blood vessels, the illumination light source including a narrow-band amber light source and a narrow-band red light source; decomposing the R channel of the first RGB image to obtain a first component image with narrow-band red light as the main response component and a second component image with narrow-band amber light as the main response component; mapping the first component image and the second component image based on a spectral mapping matrix to obtain a second RGB image; and color enhancing the second RGB image based on a color adjustment matrix to obtain a target RGB image, which is used to observe and process the tissue and blood vessels.

[0008] In a possible embodiment, the R channel of the first RGB image is decomposed to obtain a first component image with narrow-band red light as a main response component and a second component image with narrow-band amber light as a main response component, including: obtaining a first spectral reflectance of tissue blood vessels in a first light band corresponding to the narrow-band red light, and a first calibrated response value of the corresponding imaging signal; and, obtaining a second spectral reflectance of tissue blood vessels in a second light band corresponding to the narrow-band amber light, and a second calibrated response value of the corresponding imaging signal; based on a ratio relationship between the first spectral reflectance and the first calibrated response value, decomposing the R channel data of the first RGB image to obtain a first component image; and, based on a ratio relationship between the second spectral reflectance and the second calibrated response value, decomposing the R channel data to obtain a second component image.

[0009] In a possible embodiment, decomposing the R channel of the first RGB image to obtain a first component image with narrow-band red light as a main response component and a second component image with narrow-band amber light as a main response component includes: inputting the first RGB image into a trained neural network model; wherein the training data set of the neural network model includes a plurality of training data pairs, the training data pairs include first training data and second training data, the first training data is R channel data corresponding to a narrow-band amber light source of one brightness level and a narrow-band red light source of one brightness level, and the second training data is R channel data corresponding to a narrow-band amber light source of one brightness level, or R channel data corresponding to a narrow-band red light source of one brightness level; decomposing the R channel data of the first RGB image based on the neural network model to obtain the first component image and the second component image.

[0010] In a possible embodiment, based on the spectral mapping matrix, mapping processing is performed on the first component image and the second component image to obtain the second RGB image, including: determining the type of light source included in the illumination light source; based on the type of light source, determining the mapping parameters of the spectral mapping matrix; wherein the mapping parameters at least satisfy the first constraint condition and the second constraint condition, the first constraint condition is used to distinguish the difference between deep blood and shallow blood in tissue blood vessels, and the second constraint condition is used to prevent the second RGB image from having color cast due to B channel data deviation; based on the mapping parameters, mapping processing is performed on the first component image and the second component image to obtain the second RGB image.

[0011] In a possible embodiment, mapping parameters of a spectral mapping matrix are determined based on the type of light source, including: determining a first row vector, a second row vector, a third row vector and mapping parameters in each row vector contained in the spectral mapping matrix based on the type of light source; wherein the product of a column vector formed by the first component image and the second component image with the first row vector and the second row vector, respectively, satisfies a first constraint; and the product of a column vector with the first row vector and the third row vector, respectively, satisfies a second constraint.

[0012] In a possible embodiment, based on the type of light source, the first row vector, the second row vector, the third row vector and the mapping parameters in each row vector contained in the spectral mapping matrix are determined, including: if the light source type consists of an amber light source and a red light source, then the spectral mapping matrix is ​​determined to consist of the first row vector, the second row vector and the third row vector; wherein each row vector contains two mapping parameters; then, the column vector is composed of the first component image and the second component image.

[0013] In a possible embodiment, based on the type of light source, the first row vector, the second row vector, the third row vector and the mapping parameters in each row vector contained in the spectral mapping matrix are determined, including: if the light source type consists of an amber light source, a red light source and a green light source, then the spectral mapping matrix is ​​determined to consist of a first row vector, a second row vector and a third row vector; wherein each row vector contains three mapping parameters, and the third column mapping parameter of the first row vector and the second row vector is 0; then, the column vector is composed of the first component image, the second component image and the G channel data in the first RGB image.

[0014] In a possible embodiment, based on the type of light source, the first row vector, the second row vector, the third row vector and the mapping parameters in each row vector contained in the spectral mapping matrix are determined, including: if the type of light source is composed of at least one of a blue-violet light source and a blue light source, an amber light source, and a red light source, then the spectral mapping matrix is ​​determined to be composed of a first row vector, a second row vector and a third row vector; wherein each row vector contains three mapping parameters, and the third column mapping parameter of the first row vector and the second row vector is 0; then, the column vector is composed of the first component image, the second component image and the B channel data in the first RGB image.

[0015] In a possible embodiment, based on the type of light source, the first row vector, the second row vector, the third row vector and the mapping parameters in each row vector contained in the spectral mapping matrix are determined, including: if the type of light source is composed of at least one of a blue-violet light source and a blue light source, an amber light source, a red light source, and a green light source, then the spectral mapping matrix is ​​determined to be composed of a first row vector, a second row vector and a third row vector; wherein each row vector contains four mapping parameters, and the third column mapping parameter and the fourth column mapping parameter of the first row vector and the second row vector are 0; then, the column vector is composed of the first component image, the second component image and the G channel data and the B channel data in the first RGB image.

[0016] In a possible embodiment, based on the color adjustment matrix, color enhancement is performed on the second RGB image to obtain a target RGB image, including: determining the color adjustment matrix, wherein the color adjustment matrix is ​​composed of three row vectors, and each row vector includes three color adjustment parameters; multiplying the color adjustment matrix by the column vector corresponding to the second RGB image to obtain the target RGB image, wherein the column vector corresponding to the second RGB image is composed of R channel data, G channel data, and B channel data.

[0017] On the one hand, an embodiment of the present application provides an image processing device, which includes: an acquisition module, used to acquire a first RGB image formed by reflection of the output light of an endoscope illumination light source on the patient's tissue and blood vessels, the illumination light source including a narrow-band amber light source and a narrow-band red light source; a decomposition module, used to decompose the R channel of the first RGB image to obtain a first component image with narrow-band red light as the main response component and a second component image with narrow-band amber light as the main response component; a mapping module, used to map the first component image and the second component image based on a spectral mapping matrix to obtain a second RGB image; an enhancement module, used to perform color enhancement on the second RGB image based on a color adjustment matrix to obtain a target RGB image, and the target RGB image is used for observing and processing tissue and blood vessels.

[0018] In a possible embodiment, the central wavelength of the narrow-band amber light source is greater than or equal to 585 nm and less than or equal to 615 nm; the central wavelength of the narrow-band red light source is greater than or equal to 620 nm and less than or equal to 640 nm.

[0019] In a possible embodiment, the decomposition module is used to: obtain a first spectral reflectance of tissue blood vessels in a first light band corresponding to narrow-band red light, and a first calibrated response value of a corresponding imaging signal; and obtain a second spectral reflectance of tissue blood vessels in a second light band corresponding to narrow-band amber light, and a second calibrated response value of the corresponding imaging signal; based on a ratio relationship between the first spectral reflectance and the first calibrated response value, decompose the R channel data of the first RGB image to obtain a first component image; and based on a ratio relationship between the second spectral reflectance and the second calibrated response value, decompose the R channel data to obtain a second component image.

[0020] In a possible embodiment, a decomposition module is used to: input the first RGB image into a trained neural network model; wherein the training data set of the neural network model includes multiple training data pairs, the training data pairs include first training data and second training data, the first training data is R channel data corresponding to a narrow-band amber light source of one brightness level and a narrow-band red light source of one brightness level, and the second training data is R channel data corresponding to a narrow-band amber light source of one brightness level, or R channel data corresponding to a narrow-band red light source of one brightness level; based on the neural network model, the R channel data of the first RGB image is decomposed to obtain a first component image and a second component image.

[0021] In a possible embodiment, the mapping module is used to: determine the type of light source included in the illumination light source; based on the type of light source, determine the mapping parameters of the spectral mapping matrix; wherein the mapping parameters at least satisfy a first constraint and a second constraint, the first constraint is used to distinguish between deep blood and shallow blood in tissue blood vessels, and the second constraint is used to prevent the second RGB image from being colored due to B channel data deviation; based on the mapping parameters, the first component image and the second component image are mapped to obtain a second RGB image.

[0022] In a possible embodiment, the mapping module is used to: determine the first row vector, the second row vector, the third row vector and the mapping parameters in each row vector contained in the spectral mapping matrix based on the type of light source; wherein the product of the column vector composed of the first component image and the second component image with the first row vector and the second row vector, respectively, satisfies the first constraint condition; and the product of the column vector with the first row vector and the third row vector, respectively, satisfies the second constraint condition.

[0023] In a possible embodiment, the mapping module is used to: if the light source type is an amber light source and a red light source, determine that the spectral mapping matrix is ​​composed of a first row vector, a second row vector and a third row vector; wherein each row vector contains two mapping parameters; then, the column vector is composed of a first component image and a second component image.

[0024] In a possible embodiment, the mapping module is used to: if the light source types consist of an amber light source, a red light source, and a green light source, then determine that the spectral mapping matrix consists of a first row vector, a second row vector, and a third row vector; wherein each row vector contains three mapping parameters, and the third column mapping parameter of the first row vector and the second row vector is 0; then, the column vector is composed of the first component image, the second component image, and the G channel data in the first RGB image.

[0025] In a possible embodiment, the mapping module is used to: if the light source type consists of at least one of a blue-violet light source and a blue light source, an amber light source, and a red light source, then determine that the spectral mapping matrix consists of a first row vector, a second row vector, and a third row vector; wherein each row vector contains three mapping parameters, and the third column mapping parameter of the first row vector and the second row vector is 0; then, the column vector is composed of the first component image, the second component image, and the B channel data in the first RGB image.

[0026] In a possible embodiment, the mapping module is used to: if the light source type consists of at least one of a blue-violet light source and a blue light source, an amber light source, a red light source, and a green light source, then determine that the spectral mapping matrix consists of a first row vector, a second row vector, and a third row vector; wherein each row vector contains four mapping parameters, and the third column mapping parameter and the fourth column mapping parameter of the first row vector and the second row vector are 0; then, the column vector is composed of the first component image, the second component image, and the G channel data and the B channel data in the first RGB image.

[0027] In a possible embodiment, the enhancement module is used to: determine a color adjustment matrix, wherein the color adjustment matrix is ​​composed of three row vectors, each row vector contains three color adjustment parameters; multiply the color adjustment matrix by the column vector corresponding to the second RGB image to obtain a target RGB image, wherein the column vector corresponding to the second RGB image is composed of R channel data, G channel data and B channel data.

[0028] On the one hand, an embodiment of the present application provides an electronic device, which includes a processor and a memory, wherein the memory stores program code, and when the program code is executed by the processor, the processor executes any one of the above-mentioned image processing methods.

[0029] On the one hand, an embodiment of the present application provides a computer-readable storage medium, which includes a program code. When the storage medium is run on an electronic device, the program code is used to enable the electronic device to execute any of the above-mentioned image processing methods.

[0030] On the one hand, an embodiment of the present application provides an endoscope, which includes a lighting unit, an imaging unit and a processing unit. The lighting unit is used to provide an illumination light source. After the outgoing light of the illumination light source is reflected by the patient's tissue and blood vessels, a first RGB image is generated through the imaging unit. The processing unit is used to perform any of the above-mentioned image processing methods on the first RGB image.

[0031] The beneficial effects of this application are as follows:

[0032] (1) The R channel of the first RGB image is decomposed into two component images that mainly respond to different narrow-band light sources. This decomposition method can realize the separate processing of imaging information corresponding to different narrow-band light sources, and thus can more accurately extract image information related to blood vessels.

[0033] (2) The component images obtained by decomposition are mapped using the spectral mapping matrix to generate a second RGB image, which can effectively adjust the spectral characteristics of the image and enhance the display effect of specific wavelength signals, thereby improving the visualization quality of blood vessels.

[0034] (3) The second RGB image is enhanced by the color adjustment matrix to obtain the target RGB image, which can significantly improve the color balance and visual effect of the image and make the details and color characteristics of tissue blood vessels more clearly visible.

[0035] (4) Through a series of image processing steps, the contrast, clarity and color accuracy are comprehensively improved. This comprehensive image quality improvement effect can better meet the needs of clinical diagnosis for high-quality images in the field of endoscopic image processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0037] Figure 1 This is a flowchart of an implementation of an image processing method in an embodiment of the present application.

[0038] Figure 2 This is a flowchart of an implementation of a spectral mapping method in an embodiment of the present application.

[0039] Figure 3 Schematic diagram of the structure of an image processing device in an embodiment of the present application.

[0040] Figure 4 A schematic diagram of the hardware structure of an electronic device in an embodiment of the present application.

[0041] Figure 5 This is a schematic diagram of the structure of an endoscope in an embodiment of the present application. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical scheme and advantages of the present application clearer, the technical scheme in the embodiment of the present application will be clearly and completely described below in conjunction with the drawings in the embodiment of the present application. Obviously, the described embodiment is only a part of the embodiment of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application. In the absence of conflict, the embodiments in the present application and the features in the embodiments can be arbitrarily combined with each other. In addition, although the logical order is shown in the flow chart, in some cases, the steps shown or described can be performed in an order different from that here.

[0043] The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the invention described herein can be implemented in sequences other than those illustrated or described herein.

[0044] The following is a brief introduction to the design concept of the embodiment of the present application:

[0045] When using an endoscope to explore the deep tissue and blood vessels of the human body, since the hemoglobin in this part has a strong absorption characteristic for a narrow-band light source with a central wavelength of about 600nm, an amber light source and a red light source are usually used as illumination light sources for narrow-band light imaging, so as to improve the recognizability of deep tissue and blood vessels. Among them, when using an amber light source and a red light source as illumination light sources for narrow-band light imaging, the two light sources usually emit light at the same time, and after being reflected by the tissue and blood vessels in the human body, a fused image is obtained, and the tissue and blood vessels are observed and analyzed based on the fused image. However, since the fused image is obtained by the joint action of the two light sources in the same frame, the amber light source and the red light source are not distinguished and processed, and thus the specific information generated by the two light sources on the tissue and blood vessels respectively cannot be directly reflected, thereby weakening the enhancement effect on the lesion area. Therefore, there is an urgent need for an image processing technology for processing the fused image to realize the separate processing of the specific information generated by the light source of a specific wavelength band on the tissue and blood vessels, so as to achieve the purpose of enhancing narrow-band imaging for specific lesions or parts.

[0046] In view of this, an embodiment of the present application provides an image processing method, device, electronic device and storage medium, wherein the method includes: obtaining a first RGB image formed by reflection of the output light of an endoscope illumination light source on the patient's tissue blood vessels, the illumination light source including a narrow-band amber light source and a narrow-band red light source; decomposing the R channel of the first RGB image to obtain a first component image with narrow-band red light as the main response component and a second component image with narrow-band amber light as the main response component; based on a spectral mapping matrix, mapping the first component image and the second component image to obtain a second RGB image; based on a color adjustment matrix, color enhancing the second RGB image to obtain a target RGB image, and the target RGB image is used for observing and processing tissue blood vessels.

[0047] In this way, by decomposing the R channel of the first RGB image into two component images that mainly respond to different narrow-band light sources, the imaging information corresponding to different narrow-band light sources can be processed separately, and the image information related to blood vessels can be extracted more accurately. Secondly, the component images obtained by decomposition are mapped using the spectral mapping matrix to generate a second RGB image, which can effectively adjust the spectral characteristics of the image and enhance the display effect of specific wavelength signals, thereby improving the visualization quality of blood vessels. In addition, the second RGB image is color enhanced by the color adjustment matrix to obtain the target RGB image, which can significantly improve the color balance and visual effect of the image, making the details and color characteristics of tissue vessels more clearly visible.

[0048] This series of image processing steps fully considers the correlation between narrow-band amber light and narrow-band red light, and their spectral absorption characteristics for hemoglobin, and proposes a solution to accurately adjust the differentiation of deep bleeding points. By decomposing the original R channel into two independent components corresponding to narrow-band amber light and narrow-band red light at the image processing end, and according to the relationship between the two, a solution is given under different light source illumination combinations to determine the deep bleeding point area and color adjustment to enhance the differentiation, which systematically improves the recognition accuracy and contrast differentiation of deep bleeding points. A comprehensive improvement in contrast, clarity and color accuracy is achieved. This comprehensive image quality improvement effect can better meet the needs of clinical diagnosis for high-quality images in the field of endoscopic image processing.

[0049] refer to Figure 1 , is a flowchart of an implementation of an image processing method provided in an embodiment of the present application, and the specific implementation process of the method is as follows:

[0050] S101, acquiring a first RGB image formed by reflection of the output light of the endoscope illumination light source on the patient's tissue blood vessels;

[0051] In an embodiment of the present application, the endoscope includes a lighting unit, an imaging unit and a processing unit.

[0052] Among them, the lighting unit is used to provide an illumination light source for the endoscope. The illumination light source can be specifically composed of a blue-violet light source, a blue light source, a green light source, an amber light source, and a red light source. These five LED light sources can synthesize white light or any other combination of multi-spectrum mixed light. The light source illumination combination used in the embodiment of the present application contains at least two types of narrow-band amber light sources and narrow-band red light sources. Other illumination light sources are not limited. Among them, the central wavelength of the narrow-band amber light source is greater than or equal to 585nm and less than or equal to 615nm; the central wavelength of the narrow-band red light source is greater than or equal to 620nm and less than or equal to 640nm.

[0053] After the emitted light of the illumination light source is reflected by the patient's tissue blood vessels, the first RGB image is generated by the imaging unit, wherein the blue channel is recorded as , the green channel is recorded as , the red channel is recorded as .

[0054] The processing unit is used to perform image processing on the first RGB image. The specific processing method can refer to the image processing method in the embodiment of the present application. However, it does not mean that the image processing method in the embodiment of the present application is only applicable to the processing unit of the endoscope. It can also be a server or other processor with image processing function, without specific limitation.

[0055] After acquiring the first RGB image formed by the reflection of the outgoing light of the endoscope illumination light source on the patient's tissue blood vessels, the image processing method of the embodiment of the present application is introduced below using the processing unit of the endoscope as an application scenario.

[0056] S102, decomposing the R channel of the first RGB image to obtain a first component image with narrow-band red light as a main response component and a second component image with narrow-band amber light as a main response component.

[0057] After the first RGB image is acquired, in order to realize separate processing of imaging signals corresponding to the narrow-band amber light source and the narrow-band red light source, the first RGB image needs to be decomposed.

[0058] Since the narrow-band amber light source and the narrow-band red light source are both light sources with wavelengths in the red spectrum, the imaging signals formed are mainly concentrated in the R channel. Deep blood vessels and bleeding points have a relatively stronger absorption capacity for narrow-band amber light, while narrow-band red light provides more Therefore, in order to further improve the differentiation of tissue and blood vessels, we first need to Decomposition is performed to obtain a first component image with narrow-band red light as the main response component and a second component image with narrow-band amber light as the main response component. Specific decomposition methods include: spectral reconstruction technology method and deep learning method.

[0059] (1) Spectral reconstruction technology.

[0060] In a possible embodiment, the R channel of the first RGB image is decomposed to obtain a first component image with narrow-band red light as a main response component and a second component image with narrow-band amber light as a main response component, including:

[0061] Acquire a first spectral reflectance of tissue blood vessels in a first light band corresponding to narrow-band red light, and a first calibrated response value of a corresponding imaging signal; and acquire a second spectral reflectance of tissue blood vessels in a second light band corresponding to narrow-band amber light, and a second calibrated response value of a corresponding imaging signal; based on a ratio relationship between the first spectral reflectance and the first calibrated response value, decompose R channel data of a first RGB image to obtain a first component image; and based on a ratio relationship between the second spectral reflectance and the second calibrated response value, decompose R channel data to obtain a second component image.

[0062] For example, when decomposing the first RGB image into the first component image season ,in, is the first RGB image, is a single-component estimation matrix related to the spectral response characteristics of the endoscope system and can be expressed as: , is the spectral reflectance of the patient's tissue blood vessels in the light band corresponding to the narrow-band red light, is the calibrated response value of the tissue blood vessel in the endoscope system under this optical band and its corresponding optical power. The main ingredients come from ,therefore Can be simplified to One of them is to obtain and The method is to construct a target scene, measure the spectral reflectance of tissue blood vessels offline by a spectrophotometer, and traverse the optical power levels of the corresponding light source at the same time, and record the corresponding calibration response value of the endoscope system under each level.

[0063] Similarly, the second image component can be obtained .

[0064] In addition, the estimated matrix The spectral reconstruction techniques used include but are not limited to R matrix method, principal component analysis method, Wiener estimation method, comparative measurement method, etc.

[0065] (2) Deep learning methods.

[0066] In a possible embodiment, the R channel of the first RGB image is decomposed to obtain a first component image with narrow-band red light as a main response component and a second component image with narrow-band amber light as a main response component, including:

[0067] The first step is to train the neural network model. The specific training methods include:

[0068] First, define and construct a training data set. In the target scene, the illumination light sources are narrow-band amber light source and narrow-band red light source. The adjustable levels of the two light sources are traversed respectively, and the corresponding R channel data under multiple level combinations are collected as the first training data. Each first training data is the R channel data corresponding to a narrow-band amber light source of a brightness level and a narrow-band red light source of a brightness level, recorded as , which represents the synthetic R channel image corresponding to the brightness level of the i-th narrow-band amber light source and the brightness level of the j-th narrow-band red light source.

[0069] Then, a narrow-band amber light source is used as the illumination light source, and the adjustable levels of the light source are traversed, and the R channel data corresponding to each level is collected as the second training data. The second training data is the R channel data corresponding to the narrow-band amber light source of one brightness level, which is recorded as , represents the R channel image corresponding to the brightness level of the i-th narrow-band amber light source. Similarly, using a narrow-band red light source as the illumination light source, traverse the adjustable levels of the light source, and collect the corresponding R channel data at each level as the second training data. At this time, the second training data is the R channel data corresponding to a brightness level of the narrow-band red light source, recorded as , represents the R channel image corresponding to the brightness level of the j-th narrow-band red light source.

[0070] Finally, and ,as well as and As training data pairs, they are organized into training data sets. Using deep learning methods, the training data sets are input into the neural network model for training. The goal is to transform each All are mapped to and , until the training results converge.

[0071] In the second step, the first RGB image is input into the trained neural network model, and the R channel data of the first RGB image is decomposed based on the neural network model to obtain a first component image and a second component image.

[0072] By the above method, the first RGB image is decomposed into two component images mainly responding to different narrow-band light sources. This decomposition method can realize separate processing of imaging information corresponding to different narrow-band light sources, and thus can more accurately extract image information related to blood vessels.

[0073] S103, mapping the first component image and the second component image based on the spectral mapping matrix to obtain a second RGB image;

[0074] In the embodiment of the present application, after obtaining the first component image and the second component image, the first component image and the second component image are further mapped to obtain a second RGB image. The mapping process is implemented based on the spectral mapping matrix. The specific implementation method is as follows: Figure 2 As shown, the following steps are included:

[0075] S1031, determining the types of light sources included in the lighting light source.

[0076] In the embodiments of the present application, the types of light sources include the following combinations:

[0077] Combination 1: amber light source and red light source;

[0078] Combination 2: amber light source, red light source and green light source;

[0079] Combination 3: at least one of a violet light source and a blue light source, an amber light source, and a red light source;

[0080] Combination 4: at least one of a bluish-violet light source and a blue light source, an amber light source, a red light source, and a green light source.

[0081] S1032: Determine mapping parameters of the spectral mapping matrix based on the type of light source.

[0082] The mapping parameters at least satisfy the first constraint and the second constraint. The first constraint is used to distinguish the difference between deep blood and shallow blood in tissue blood vessels, and the second constraint is used to prevent the second RGB image from leaking color due to B channel data deviation. Specifically:

[0083] Based on the type of light source, determine the first row vector, the second row vector, the third row vector and the mapping parameters in each row vector contained in the spectral mapping matrix; wherein the product of the column vector composed of the first component image and the second component image with the first row vector and the second row vector, respectively, satisfies the first constraint condition; and the product of the column vector with the first row vector and the third row vector, respectively, satisfies the second constraint condition.

[0084] Specifically, the mapping parameters need to satisfy the following constraints: the ratio between the first component image and the second component image is greater than or equal to the first threshold, and the first threshold is greater than 1; the first product between the column vector formed by the first component image and the second component image and the first row vector, and the second product between the column vector and the second row vector, satisfy the first constraint: the ratio between the first product and the second product is greater than or equal to the second threshold, and the second threshold is greater than 1; the first product, and the third product between the column vector and the third row vector, satisfy the second constraint: the ratio between the first product and the third product is greater than or equal to the third threshold, and the third threshold is greater than 1; wherein, in the first row vector, the ratio between the product between the first column parameter and the first threshold and the first sum of the second column parameter, and in the second row vector, the ratio between the product between the first column parameter and the first threshold and the second sum of the second column parameter, is greater than or equal to the second threshold.

[0085] Through the above constraints, it is possible to determine whether the current tissue blood vessel is a deep bleeding point or a deep blood vessel.

[0086] In one embodiment, if the light source type is combination 1: amber light source and red light source, it is determined that the spectral mapping matrix is ​​composed of the first row vector, the second row vector and the third row vector; wherein each row vector contains two mapping parameters; then, the column vector is composed of the first component image and the second component image. At this time, the spectral mapping process can be expressed by the following formula:

[0087] (1)

[0088] in, is the spectral mapping matrix, which consists of six mapping parameters, namely ; are respectively a first component image and a second component image; They are respectively the R, G, and B channel data of the second RGB image.

[0089] Under this condition, the corresponding constraint formula is:

[0090] (2)

[0091] In inequality group (1), , , They are the first threshold, the second threshold and the third threshold respectively. Among them, between the first threshold and the second threshold, the following constraint formula must be satisfied:

[0092] (3)

[0093] In one embodiment, if the light source type is combination 2: amber light source, red light source and green light source, it is determined that the spectral mapping matrix is ​​composed of a first row vector, a second row vector and a third row vector; wherein each row vector contains three mapping parameters, and the third column mapping parameter of the first row vector and the second row vector is 0; then, the column vector is composed of the first component image, the second component image and the G channel data in the first RGB image.

[0094] At this point, the spectral mapping process can be expressed by the following formula:

[0095] (4)

[0096] In formula (4), G is the G channel data of the first RGB image, is the spectral mapping matrix, including , , , , , , seven mapping parameters; , are respectively a first component image and a second component image; , , They are respectively the R, G, and B channel data of the second RGB image.

[0097] Under this condition, the corresponding constraint formula is:

[0098] (5)

[0099] In the inequality group (5), between the first threshold and the second threshold, constraint formula (3) must also be satisfied.

[0100] In one embodiment, if the light source type is combination 3: at least one of a blue-violet light source and a blue light source, an amber light source, and a red light source, then the spectral mapping matrix is ​​determined to be composed of a first row vector, a second row vector, and a third row vector; wherein each row vector contains three mapping parameters, and the third column mapping parameter of the first row vector and the second row vector is 0; then, the column vector is composed of the first component image, the second component image, and the B channel data in the first RGB image. At this time, the spectral mapping process can be expressed by the following formula:

[0101] (6)

[0102] In formula (6), B is the B channel data of the first RGB image, is the spectral mapping matrix, including , , , , , , seven mapping parameters; , are respectively a first component image and a second component image; , , They are respectively the R, G, and B channel data of the second RGB image.

[0103] Under this condition, the corresponding constraint formula is:

[0104] (7)

[0105] In inequality group (7), the first threshold and the second threshold must also satisfy constraint formula (3).

[0106] In one embodiment, if the light source type is combination 4: at least one of a violet light source and a blue light source, an amber light source, a red light source, and a green light source, then the spectral mapping matrix is ​​determined to be composed of a first row vector, a second row vector, and a third row vector; wherein each row vector contains four mapping parameters, and the third column mapping parameter and the fourth column mapping parameter of the first row vector and the second row vector are 0; then, the column vector is composed of the first component image, the second component image, and the G channel data and the B channel data in the first RGB image. At this time, the spectral mapping process can be expressed by the following formula:

[0107] (8)

[0108] In formula (8), G and B are the G and B channel data of the first RGB image, respectively. is the spectral mapping matrix, including , , , , , , , Eight mapping parameters; , are respectively a first component image and a second component image; , , They are respectively the R, G, and B channel data of the second RGB image.

[0109] Under this condition, the corresponding constraint formula is:

[0110] (9)

[0111] In the inequality group (9), the first threshold and the second threshold must also satisfy the constraint formula (3).

[0112] S1033: Based on the mapping parameters, perform mapping processing on the first component image and the second component image to obtain a second RGB image.

[0113] Through the above method, it is possible to decompose the first RGB image into the first component image and the second component image, and to distinguish whether the current tissue blood vessel is a deep bleeding point or a deep blood vessel.

[0114] S104, based on the color adjustment matrix, color enhancement is performed on the second RGB image to obtain a target RGB image, and the target RGB image is used for observing and processing tissue blood vessels.

[0115] In an embodiment of the present application, after obtaining the second RGB image, the second RGB image is further color enhanced based on a color adjustment matrix to obtain a target RGB image, including: determining the color adjustment matrix, wherein the color adjustment matrix is ​​composed of three row vectors, and each row vector contains three color adjustment parameters; multiplying the color adjustment matrix by the column vector corresponding to the second RGB image to obtain the target RGB image, wherein the column vector corresponding to the second RGB image is composed of R channel data, G channel data, and B channel data.

[0116] The specific implementation process can be expressed by the following formula:

[0117] (10)

[0118] In formula (10), M is the color adjustment matrix, , , , , , , , , are the color adjustment parameters corresponding to the color adjustment matrix respectively; , , are the R, G, and B channel parameters corresponding to the target RGB image. Appropriate increase, Appropriate reduction can further enhance the contrast between deep bleeding points and superficial areas.

[0119] The above-mentioned image processing method can realize separate processing of imaging information corresponding to different narrow-band light sources by decomposing the R channel of the first RGB image into two component images that mainly respond to different narrow-band light sources, and thus can more accurately extract image information related to blood vessels. Secondly, the component images obtained by decomposition are mapped using a spectral mapping matrix to generate a second RGB image, which can effectively adjust the spectral characteristics of the image and enhance the display effect of specific wavelength signals, thereby improving the visualization quality of blood vessels. In addition, the second RGB image is color enhanced by a color adjustment matrix to obtain a target RGB image, which can significantly improve the color balance and visual effect of the image, making the details and color characteristics of tissue blood vessels more clearly visible. This series of image processing steps achieves a comprehensive improvement in contrast, clarity and color accuracy. This comprehensive image quality improvement effect can better meet the needs of clinical diagnosis for high-quality images in the field of endoscopic image processing.

[0120] Based on the same inventive concept, the present application embodiment provides an image processing device, such as Figure 3 FIG. 1 is a schematic diagram of a structure of an image processing device provided in an embodiment of the present application, comprising:

[0121] An acquisition module 301 is used to acquire a first RGB image formed by reflection of the output light of the endoscope illumination light source on the patient's tissue blood vessels, wherein the illumination light source includes a narrow-band amber light source and a narrow-band red light source;

[0122] A decomposition module 302 is used to decompose the R channel of the first RGB image to obtain a first component image with narrow-band red light as a main response component and a second component image with narrow-band amber light as a main response component;

[0123] A mapping module 303 is used to perform mapping processing on the first component image and the second component image based on the spectral mapping matrix to obtain a second RGB image;

[0124] The enhancement module 304 is used to perform color enhancement on the second RGB image based on the color adjustment matrix to obtain a target RGB image, and the target RGB image is used to observe and process tissue blood vessels.

[0125] In a possible embodiment, the central wavelength of the narrow-band amber light source is greater than or equal to 585 nm and less than or equal to 615 nm; the central wavelength of the narrow-band red light source is greater than or equal to 620 nm and less than or equal to 640 nm.

[0126] In a possible embodiment, the decomposition module 302 is used to:

[0127] Acquire a first spectral reflectance of tissue blood vessels in a first light band corresponding to narrow-band red light, and a first calibrated response value of a corresponding imaging signal; and acquire a second spectral reflectance of tissue blood vessels in a second light band corresponding to narrow-band amber light, and a second calibrated response value of a corresponding imaging signal; based on a ratio relationship between the first spectral reflectance and the first calibrated response value, decompose R channel data of a first RGB image to obtain a first component image; and based on a ratio relationship between the second spectral reflectance and the second calibrated response value, decompose R channel data to obtain a second component image.

[0128] In a possible embodiment, the decomposition module 302 is used to:

[0129] The first RGB image is input into a trained neural network model; wherein the training data set of the neural network model includes multiple training data pairs, the training data pairs include first training data and second training data, the first training data is R channel data corresponding to a narrow-band amber light source of one brightness level and a narrow-band red light source of one brightness level, and the second training data is R channel data corresponding to a narrow-band amber light source of one brightness level, or R channel data corresponding to a narrow-band red light source of one brightness level; the R channel data of the first RGB image is decomposed based on the neural network model to obtain a first component image and a second component image.

[0130] In a possible embodiment, the mapping module 303 is used to:

[0131] Determine the type of light source included in the illumination light source; determine the mapping parameters of the spectral mapping matrix based on the type of light source; wherein the mapping parameters at least satisfy a first constraint and a second constraint, the first constraint is used to distinguish between deep blood and shallow blood in tissue blood vessels, and the second constraint is used to prevent the second RGB image from being colored due to a B channel data deviation; based on the mapping parameters, perform mapping processing on the first component image and the second component image to obtain a second RGB image.

[0132] In a possible embodiment, the mapping module 303 is used to:

[0133] Based on the type of light source, determine the first row vector, the second row vector, the third row vector and the mapping parameters in each row vector contained in the spectral mapping matrix; wherein the product of the column vector composed of the first component image and the second component image with the first row vector and the second row vector, respectively, satisfies the first constraint condition; and the product of the column vector with the first row vector and the third row vector, respectively, satisfies the second constraint condition.

[0134] In a possible embodiment, the mapping module 303 is used to:

[0135] If the light source type consists of an amber light source and a red light source, then the spectral mapping matrix is ​​determined to consist of a first row vector, a second row vector and a third row vector; wherein each row vector contains two mapping parameters; then, the column vector is composed of a first component image and a second component image.

[0136] In a possible embodiment, the mapping module 303 is used to:

[0137] If the light source types consist of an amber light source, a red light source, and a green light source, then determine that the spectral mapping matrix consists of a first row vector, a second row vector, and a third row vector; wherein each row vector contains three mapping parameters, and the third column mapping parameter of the first row vector and the second row vector is 0; then, the column vector is composed of the first component image, the second component image, and the G channel data in the first RGB image.

[0138] In a possible embodiment, the mapping module 303 is used to:

[0139] If the light source type consists of at least one of a blue-violet light source and a blue light source, an amber light source, and a red light source, then the spectral mapping matrix is ​​determined to be composed of a first row vector, a second row vector, and a third row vector; wherein each row vector contains three mapping parameters, and the third column mapping parameter of the first row vector and the second row vector is 0; then, the column vector is composed of the first component image, the second component image, and the B channel data in the first RGB image.

[0140] In a possible embodiment, the mapping module 303 is used to:

[0141] If the light source type consists of at least one of a blue-violet light source and a blue light source, an amber light source, a red light source, and a green light source, then determine that the spectral mapping matrix consists of a first row vector, a second row vector, and a third row vector; wherein each row vector contains four mapping parameters, and the third column mapping parameter and the fourth column mapping parameter of the first row vector and the second row vector are 0; then, the column vector is composed of the first component image, the second component image, and the G channel data and the B channel data in the first RGB image.

[0142] In a possible embodiment, the enhancement module 304 is used to:

[0143] A color adjustment matrix is ​​determined, wherein the color adjustment matrix is ​​composed of three row vectors, and each row vector includes three color adjustment parameters; and the color adjustment matrix is ​​multiplied by a column vector corresponding to the second RGB image to obtain a target RGB image, wherein the column vector corresponding to the second RGB image is composed of R channel data, G channel data, and B channel data.

[0144] The technical effects achieved by the above-mentioned image processing device can be referred to the above-mentioned image processing method embodiment, which will not be described in detail here.

[0145] In some possible implementations, the image processing apparatus according to the present application may include at least a processor and a memory. The memory stores program code, and when the program code is executed by the processor, the processor executes the steps of the image processing method according to various exemplary implementations of the present application described in this specification. For example, the processor may execute the following steps: Figure 1 Follow the steps shown in .

[0146] Based on the same inventive concept, an electronic device is also provided in an embodiment of the present application, and the electronic device can implement the functions of the aforementioned image processing method and apparatus. Referring to the figure, the electronic device includes:

[0147] At least one processor 401, and a memory 402 connected to the at least one processor 401. The specific connection medium between the processor 401 and the memory 402 is not limited in the embodiment of the present application. Figure 4 In the example, the processor 401 and the memory 402 are connected via the bus 400. The bus 400 is Figure 4 The connection between other components is shown by bold lines, and is not intended to be limiting. The bus 400 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4Only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. Alternatively, the processor 401 can also be called a controller, and there is no limitation on the name.

[0148] In the embodiment of the present application, the memory 402 stores instructions that can be executed by at least one processor 401. The at least one processor 401 can execute the image processing method discussed above by executing the instructions stored in the memory 402. The processor 401 can implement Figure 3 The functions of each module in the device shown.

[0149] Among them, the processor 401 is the control center of the device, and can use various interfaces and lines to connect the various parts of the entire control device. By running or executing instructions stored in the memory 402 and calling the data stored in the memory 402, the various functions of the device and process data, the device can be monitored as a whole.

[0150] In one possible design, the processor 401 may include one or more processing units, and the processor 401 may integrate an application processor and a modem processor, wherein the application processor mainly processes an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the modem processor may not be integrated into the processor 401. In some embodiments, the processor 401 and the memory 402 may be implemented on the same chip, and in some embodiments, they may also be implemented separately on separate chips.

[0151] The processor 401 may be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component, and may implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the image processing method disclosed in the embodiments of the present application may be directly embodied as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

[0152] The memory 402 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The memory 402 may include at least one type of storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory, a random access memory (Random Access Memory, RAM), a static random access memory (Static Random Access Memory, SRAM), a programmable read-only memory (Programmable Read Only Memory, PROM), a read-only memory (Read Only Memory, ROM), an electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), a magnetic memory, a disk, an optical disk, etc. The memory 402 is any other medium that can be used to carry or store a desired program code in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. The memory 402 in the embodiment of the present application can also be a circuit or any other device that can realize a storage function, for storing program instructions and / or data.

[0153] By programming the processor 401, the code corresponding to the image processing method described in the above embodiment can be fixed into the chip, so that the chip can execute the image processing method when running. Figure 2 The steps of the image processing method of the embodiment shown are as follows: How to design and program the processor 401 is a technique well known to those skilled in the art and will not be described in detail here.

[0154] Based on the same inventive concept, an embodiment of the present application further provides a storage medium, which stores computer instructions. When the computer instructions are executed on a computer, the computer executes the image processing method discussed above.

[0155] In some possible implementations, various aspects of the image processing method provided by the present application may also be implemented in the form of a program product, which includes a program code. When the program product is run on an apparatus, the program code is used to enable the control device to execute the steps of the image processing method according to various exemplary implementations of the present application described above in this specification.

[0156] Based on the same inventive concept, the present application also provides an endoscope, such as Figure 5FIG. 5 is a schematic diagram of the structure of an endoscope provided in an embodiment of the present application, which includes an illumination unit 501, an imaging unit 502, and a processing unit 503. The illumination unit 501 is used to provide an illumination light source. After the emitted light of the illumination light source is reflected by the patient's tissue and blood vessels, a first RGB image is generated by the imaging unit 502. The processing unit 503 is used to perform processing on the first RGB image. Figure 1 The image processing method shown.

[0157] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0158] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram and the combination of the processes and / or blocks in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0159] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0160] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0161] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. An image processing method, characterized in that: The method comprises: Acquire a first RGB image formed by reflection of the output light of the endoscope illumination light source on the patient's tissue blood vessels, wherein the illumination light source includes a narrow-band amber light source and a narrow-band red light source; Decomposing the R channel of the first RGB image to obtain a first component image with narrow-band red light as a main response component and a second component image with narrow-band amber light as a main response component; Determine the type of light source included in the illumination light source, and determine the mapping parameters of the spectral mapping matrix based on the type of light source; wherein the mapping parameters at least satisfy a first constraint and a second constraint, the first constraint being used to distinguish the difference between deep blood and shallow blood in the tissue blood vessels, and the second constraint being used to prevent the second RGB image generated after the mapping process from having a color cast due to a deviation in the B channel data; Based on the mapping parameters, mapping processing is performed on the first component image and the second component image to obtain the second RGB image; Based on the color adjustment matrix, the second RGB image is color enhanced to obtain a target RGB image, and the target RGB image is used to observe the tissue blood vessels.

2. The method according to claim 1, characterized in that: Decomposing the R channel of the first RGB image to obtain a first component image with narrow-band red light as a main response component and a second component image with narrow-band amber light as a main response component includes: Acquiring a first spectral reflectance of the tissue blood vessel in a first light band corresponding to the narrow-band red light, and a first calibrated response value of a corresponding imaging signal; and, Acquire a second spectral reflectance of the tissue blood vessel in a second light band corresponding to the narrow-band amber light, and a second calibrated response value of a corresponding imaging signal; Decomposing R channel data of the first RGB image based on a ratio relationship between the first spectral reflectance and the first calibration response value to obtain the first component image; and Based on the ratio relationship between the second spectral reflectance and the second calibration response value, the R channel data is decomposed to obtain the second component image.

3. The method according to claim 1, characterized in that: Decomposing the R channel of the first RGB image to obtain a first component image with narrow-band red light as a main response component and a second component image with narrow-band amber light as a main response component includes: Input the first RGB image into a trained neural network model; wherein the training data set of the neural network model includes a plurality of training data pairs, the training data pairs include first training data and second training data, the first training data is R channel data corresponding to a narrow-band amber light source of one brightness level and a narrow-band red light source of one brightness level, and the second training data is R channel data corresponding to the narrow-band amber light source of one brightness level, or R channel data corresponding to the narrow-band red light source of one brightness level; The R channel data of the first RGB image is decomposed based on the neural network model to obtain the first component image and the second component image.

4. The method according to claim 1, characterized in that: The determining, based on the type of the light source, mapping parameters of the spectral mapping matrix comprises: Based on the type of the light source, determining a first row vector, a second row vector, a third row vector and a mapping parameter in each row vector included in the light-spectrometric mapping matrix; The products of the column vectors formed by the first component image and the second component image, the first row vector and the second row vector respectively, satisfy the first constraint condition; and The products of the column vector, the first row vector and the third row vector respectively satisfy the second constraint condition.

5. The method according to claim 4, characterized in that: The step of determining, based on the type of the light source, a first row vector, a second row vector, a third row vector and mapping parameters in each row vector included in the spectral mapping matrix, comprises: If the light source type consists of the amber light source and the red light source, then determining that the light splitting mapping matrix consists of the first row vector, the second row vector and the third row vector; wherein each row vector contains two mapping parameters; Then, the column vector is composed of the first component image and the second component image.

6. The method according to claim 4, characterized in that: The step of determining, based on the type of the light source, a first row vector, a second row vector, a third row vector and mapping parameters in each row vector included in the spectral mapping matrix, comprises: If the light source types consist of the amber light source, the red light source, and the green light source, then determining that the light splitting mapping matrix consists of the first row vector, the second row vector, and the third row vector; Each row vector includes three mapping parameters, and the third column mapping parameter of the first row vector and the second row vector is 0; Then, the column vector is composed of the first component image, the second component image and the G channel data in the first RGB image.

7. The method according to claim 4, characterized in that: The step of determining, based on the type of the light source, a first row vector, a second row vector, a third row vector and mapping parameters in each row vector included in the spectral mapping matrix, comprises: If the light source type is composed of at least one of a blue-violet light source and a blue light source, the amber light source, and the red light source, then determining that the light splitting mapping matrix is ​​composed of the first row vector, the second row vector, and the third row vector; Each row vector includes three mapping parameters, and the third column mapping parameter of the first row vector and the second row vector is 0; Then, the column vector is composed of the first component image, the second component image and the B channel data in the first RGB image.

8. The method according to claim 4, characterized in that: The step of determining, based on the type of the light source, a first row vector, a second row vector, a third row vector and mapping parameters in each row vector included in the spectral mapping matrix, comprises: If the light source type is composed of at least one of a blue-violet light source and a blue light source, the amber light source, the red light source, and a green light source, then determining that the light splitting mapping matrix is ​​composed of the first row vector, the second row vector, and the third row vector; Each row vector includes four mapping parameters, and the third column mapping parameter and the fourth column mapping parameter of the first row vector and the second row vector are 0; Then, the column vector is composed of the first component image, the second component image, and the G channel data and the B channel data in the first RGB image.

9. The method according to claim 1, characterized in that: The step of performing color enhancement on the second RGB image based on the color adjustment matrix to obtain a target RGB image includes: Determining the color adjustment matrix, wherein the color adjustment matrix is ​​composed of three row vectors, each row vector including three color adjustment parameters; The color adjustment matrix is ​​multiplied by a column vector corresponding to the second RGB image to obtain the target RGB image, wherein the column vector corresponding to the second RGB image is composed of R channel data, G channel data, and B channel data.

10. An image processing device, characterized in that: The device comprises: An acquisition module, used to acquire a first RGB image formed by reflection of the outgoing light of the endoscope illumination light source on the patient's tissue blood vessels, wherein the illumination light source includes a narrow-band amber light source and a narrow-band red light source; A decomposition module, used for decomposing the R channel of the first RGB image to obtain a first component image with narrow-band red light as a main response component and a second component image with narrow-band amber light as a main response component; a mapping module, used to determine the type of light source included in the illumination light source, and based on the type of light source, determine the mapping parameters of the spectral mapping matrix; wherein the mapping parameters at least satisfy a first constraint condition and a second constraint condition, the first constraint condition being used to distinguish the difference between deep blood and shallow blood in the tissue blood vessels, and the second constraint condition being used to prevent the second RGB image generated after the mapping process from having a color cast due to a B channel data deviation; based on the mapping parameters, the first component image and the second component image are mapped to obtain the second RGB image; The enhancement module is used to perform color enhancement on the second RGB image based on the color adjustment matrix to obtain a target RGB image, and the target RGB image is used to observe and process the tissue blood vessels.

11. An electronic device, characterized in that: It includes a processor and a memory, wherein the memory stores a program code, and when the program code is executed by the processor, the processor executes the method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that: The invention comprises a program code, and when the storage medium is run on an electronic device, the program code is used to enable the electronic device to execute the method described in any one of claims 1 to 9.

13. An endoscope, characterized in that: It includes a lighting unit, an imaging unit and a processing unit. The lighting unit is used to provide an illumination light source. After the emitted light of the illumination light source is reflected by the patient's tissue and blood vessels, a first RGB image is generated through the imaging unit. The processing unit is used to execute the method described in any one of claims 1 to 9 on the first RGB image.

Images