Image processing method and system

Abstract

The present invention discloses an image processing method and system, which are applied to the field of image processing technology. The method includes: obtaining image data of a target area as image data to be tested; performing spectral reconstruction on the image data to be tested to generate first image data with a first central wavelength, second image data with a second central wavelength, and third image data with a third central wavelength; performing differential calculation on the first image data and the third image data to generate differential image data, and weightedly fusing the differential image data with the second image data to form fused image data; and enhancing the fused image data using the third image data to form enhanced image data. The present invention uses differential detection to make the processed image more sensitive to bleeding points; and uses gain adjustable technology to adjust the gain parameters in real time, so that bleeding points in different states can be displayed and processed in the processed image.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an image processing method and system. Background Art

[0002] Endoscopic imaging and presentation to personnel has become a common practice in some surgical procedures. To facilitate better identification of bleeding points from endoscopic images, the prior art offers several technical solutions. Among them, Chinese patent application number 201780043850.1 discloses an image processing device that highlights bleeding points in an image by extracting blood accumulation areas within a region representing blood, where the blood concentration is lower than that of the bleeding point, and then enhancing the brightness of these areas. The 630nm wavelength employed in this device has relatively weak absorption by hemoglobin. Even though 630nm can penetrate deeper into tissue, the weak absorption coefficient makes it difficult to distinguish hemoglobin concentration. Meanwhile, 540nm has strong absorption by hemoglobin, penetrating tissue only shallowly. As a result, while hemoglobin concentration can be distinguished in the images processed using the aforementioned patent, the contrast is weak and susceptible to interference. Summary of the Invention

[0003] In order to at least overcome the above-mentioned deficiencies in the prior art, the purpose of this application is to provide an image processing method and system.

[0004] In a first aspect, an embodiment of the present application provides an image processing method, comprising:

[0005] Acquiring image data of the target area as image data to be measured;

[0006] Performing spectral reconstruction on the image data to be measured to generate first image data having a first central wavelength as the central wavelength, second image data having a second central wavelength as the central wavelength, and third image data having a third central wavelength as the central wavelength; wherein the first central wavelength, the second central wavelength, and the third central wavelength increase in sequence;

[0007] performing a difference calculation on the first image data and the third image data to generate difference image data, and performing a weighted fusion on the difference image data and the second image data to form fused image data;

[0008] The fused image data is enhanced by using the third image data to form enhanced image data.

[0009] In a possible implementation, the method further includes:

[0010] The first image data is output through a first channel; the enhanced image data is output through a second channel; the third image data is output through a third channel; the first channel, the second channel and the third channel are all one of a B channel, a G channel and an R channel, and are different from each other.

[0011] In a possible implementation, the first central wavelength is 520-550 nm; the second central wavelength is 600 nm; and the third central wavelength is 620-640 nm.

[0012] In a possible implementation, the first central wavelength is 540 nm; and the third central wavelength is 630 nm.

[0013] In a possible implementation, weighted fusion to form fused image data includes:

[0014] Weighted fusion is performed according to the following formula:

[0015]

[0016] Where, I f To fuse image data, I b is the first image data, I g is the second image data, I r is the third image data, k0 is the weight coefficient of the differential image data, and is between 0.3 and 0.8.

[0017] In a possible implementation, the enhanced image data is calculated according to the following formula:

[0018]

[0019] Where, I g To enhance image data, I f To fuse image data, K gain is the adjustable gain, and b is the fusion gain.

[0020] In a possible implementation, the fusion gain is calculated according to the following formula:

[0021]

[0022] Where, I r_ave is the average value of the pixel values ​​of the third image data, I f_ave is the average pixel value of the fused image data.

[0023] In a second aspect, the present application further provides an image processing system, comprising:

[0024] an acquisition unit, configured to acquire image data of a target area as image data to be measured;

[0025] a spectral reconstruction unit configured to perform spectral reconstruction on the image data to be measured to generate first image data having a first central wavelength, second image data having a second central wavelength, and third image data having a third central wavelength; wherein the first central wavelength, the second central wavelength, and the third central wavelength increase in sequence;

[0026] a fusion unit configured to perform a difference calculation on the first image data and the third image data to generate differential image data, and perform a weighted fusion on the differential image data and the second image data to form fused image data;

[0027] The enhancement unit is configured to enhance the fused image data using the third image data to form enhanced image data.

[0028] In a possible implementation, the fusion unit is further configured to:

[0029] Weighted fusion is performed according to the following formula:

[0030]

[0031] Where, I f To fuse image data, I b is the first image data, I g is the second image data, I r is the third image data, k0 is the weight coefficient of the differential image data, and is between 0.3 and 0.8.

[0032] In a possible implementation, the enhancement unit is further configured to:

[0033] The enhanced image data is calculated according to the following formula:

[0034]

[0035] Where, I G To enhance image data, I f To fuse image data, K gain is the adjustable gain, and b is the fusion gain.

[0036] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0037] The present invention provides an image processing method and system, which makes the processed image more sensitive to bleeding points through differential detection; and uses gain adjustable technology to adjust the gain parameters in real time so that bleeding points in different states can be displayed in the processed image. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The drawings described herein are used to provide a further understanding of the embodiments of the present invention, constitute a part of this application, and do not constitute a limitation of the embodiments of the present invention. In the drawings:

[0039] Figure 1 This is a schematic diagram of the steps of the method according to the embodiment of the present application;

[0040] Figure 2 This is a schematic diagram of the overall processing process of the embodiment of the present application;

[0041] Figure 3 This is a comparison chart of the processing effects of the embodiments of this application. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only serve the purpose of illustration and description and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate the operations implemented according to some embodiments of the embodiments of the present application. It should be understood that the operations of the flowcharts can be implemented out of sequence, and steps without logical context can be reversed or implemented simultaneously. In addition, those skilled in the art, under the guidance of the contents of this application, can add one or more other operations to the flowchart, or remove one or more operations from the flowchart.

[0043] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed application, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present application.

[0044] Please refer to Figure 1 , is a flow chart of an image processing method provided by an embodiment of the present invention. Furthermore, the image processing method may specifically include the contents described in the following steps S1 to S4.

[0045] S1: Acquire image data of the target area as image data to be measured;

[0046] S2: performing spectral reconstruction on the image data to be measured to generate first image data having a first central wavelength as the central wavelength, second image data having a second central wavelength as the central wavelength, and third image data having a third central wavelength as the central wavelength; wherein the first central wavelength, the second central wavelength, and the third central wavelength increase in sequence;

[0047] S3: performing a difference calculation on the first image data and the third image data to generate difference image data, and performing a weighted fusion on the difference image data and the second image data to form fused image data;

[0048] S4: enhancing the fused image data using the third image data to form enhanced image data.

[0049] When implementing the embodiments of the present application, it is necessary to first obtain the image data to be tested. It should be understood that the present application is mainly used in endoscopic imaging, and its main purpose is to provide enhanced image data to surgical personnel, so whether the enhanced image data can clearly show the bleeding point is very critical. When imaging through an endoscope, it is often necessary to illuminate the target area with a light source, which belongs to the existing technology and is not limited in the embodiments of the present application. In the existing technology, the reason why it is necessary to image three central wavelengths such as 415nm, 540nm and 630nm is that the light source of the endoscope is generally a laser diode light source of three colors RGB, which needs to be aligned by controlling the imaging moment and the illumination moment to achieve imaging under different color light sources. In the present application, due to the subsequent introduction of spectral reconstruction technology, there is no need to strictly control imaging and illumination, thereby reducing control costs.

[0050] In an embodiment of the present application, the spectrum reconstruction of the image data to be measured can be performed using a variety of existing technologies. For example, a linear method can be used for spectrum reconstruction. When you want to recover the 600nm signal, construct a 3×3 mapping matrix corresponding to 600nm to restore the input image signal to a 600nm image, restore 540nm, and then extract a 3×3 mapping matrix of 540nm for recovery. For example, a nonlinear method can also be used for spectrum reconstruction. When nonlinear reconstruction is used, a full-link network can be used in conjunction with a relu activation function to restore the spectrum graph. This belongs to the existing technology and is not limited in the embodiments of the present application. In an embodiment of the present application, a linear method is preferably used for spectrum reconstruction. The reason is that the light source corresponding to the image data to be measured is relatively simple, so when reconstructed in a linear manner, the spectrum reconstruction can also be effectively performed, and the amount of calculation is much lower than that of nonlinear reconstruction. It should be understood that spectrum reconstruction performed by other existing technologies should also be regarded as within the scope of protection of the present application.

[0051] In the embodiment of this application, the overall processing process can be found in Figure 2 Spectral reconstruction can generate three image data sets with different central wavelengths. Prior art techniques also generate three image data sets with different central wavelengths. However, in the embodiments of this application, the three central wavelengths are generally preferably 540 nm, 600 nm, and 630 nm, while images in the 415-460 nm band are omitted. This is because, unlike prior art techniques, this application does not focus on screening specific bleeding point regions, but rather on highlighting the bleeding points themselves. The first and third image data differ significantly in their absorption of the bleeding point spectrum. By performing a difference calculation on these two images, the resulting image can contain the spectral characteristics of the bleeding point and background blood in the corresponding central wavelength bands. At this point, the second image data is fused with the difference image. It should be understood that prior art techniques for processing the difference image using other images involve image enhancement, i.e., enhancing image brightness. However, this application utilizes a fusion method of the second image data and the difference image to highlight the bleeding points. This advantage is that the fused image can contain bleeding point information from all three images, compared to one or two images in prior art techniques. In the embodiment of the present application, the central wavelength of the second image data is preferably 600 nm. A 600 nm wavelength signal can penetrate deeper into tissue, but its weak absorption coefficient makes it difficult to distinguish hemoglobin concentration. The difference image can effectively compensate for this shortcoming, and by fusing the two images, the location of the bleeding point can be more accurately presented. It should be understood that the fusion process can be weighted fusion, maximum fusion, minimum fusion, gradient fusion, etc., and as long as they can produce similar results, they should be considered equivalent technical solutions to the present application.

[0052] In the embodiment of the present application, the image after fusion is generally darker, while the third image data is generally brighter, so it is necessary to enhance the fused image data through the third image data to increase the image brightness and form enhanced image data.

[0053] In a possible implementation, the method further includes:

[0054] The first image data is output through a first channel; the enhanced image data is output through a second channel; the third image data is output through a third channel; the first channel, the second channel and the third channel are all one of a B channel, a G channel and an R channel, and are different from each other.

[0055] When implementing the embodiment of the present application, the endoscope generally has three channels for image output, namely the B channel, the G channel, and the R channel. Therefore, in the embodiment of the present application, the first image data, the enhanced image data, and the third image data are generally output through these three channels respectively. For example, when the first center wavelength is 540nm, the second center wavelength is 600nm, and the third center wavelength is 630nm, the first image data is output through the B channel, the enhanced image data is output through the G channel, and the third image data is output through the R channel. For the final output result, please refer to Figure 3 , Figure 3 The figure on the left is the image output by the embodiment of the present application, and the image on the right is the image output by the prior art; the red dots shown in the figure are, from top to bottom, the situations of no bleeding points, one larger bleeding point, and multiple small bleeding points, all of which have blood accumulation. It can be seen from the figure that the prior art can well identify the blood accumulation situation, but cannot accurately see the specific location and situation of the bleeding points, while the present application can effectively display the location of the bleeding points.

[0056] In a possible implementation, the first central wavelength is 520-550 nm; the second central wavelength is 600 nm; and the third central wavelength is 620-640 nm.

[0057] In a possible implementation, the first central wavelength is 540 nm; and the third central wavelength is 630 nm.

[0058] In a possible implementation, weighted fusion to form fused image data includes:

[0059] Weighted fusion is performed according to the following formula:

[0060]

[0061] Where, I f To fuse image data, I b is the first image data, I g is the second image data, I r is the third image data, k0 is the weight coefficient of the differential image data, and is between 0.3 and 0.8.

[0062] When the embodiment of the present application is implemented, when calculating the differential image data, it can be achieved by calculating the difference between the pixel values ​​of the same pixel point in the third image data and the first image data; in the above formula, I r -I bis the differential image data. To avoid negative values, the absolute value of the data can be calculated. k0 is the weight coefficient of the differential image data, which is generally determined through experiments. It should be understood that the weight coefficient of the differential image data of different organs or parts can use different values. This allows the application to be better applied in various occasions.

[0063] In a possible implementation, the enhanced image data is calculated according to the following formula:

[0064]

[0065] Where, I G To enhance image data, I f To fuse image data, K gain is the adjustable gain, and b is the fusion gain.

[0066] In a possible implementation, the fusion gain is calculated according to the following formula:

[0067]

[0068] Where, I r_ave is the average value of the pixel values ​​of the third image data, I f_ave is the average pixel value of the fused image data.

[0069] When the embodiment of the present application is implemented, when the fused image is enhanced, the enhancement is performed based on the state of the third image data and the state of the fused image data, wherein the present application introduces an adjustable gain K gain The adjustable gain is set at the person viewing the endoscopic imaging results. This parameter can be adjusted to accommodate different bleeding situations. By adjusting the gain parameter in real time, even the faintest bleeding spots can be detected. After the above enhancement, the bleeding spot signal is amplified, and the overall brightness of the G channel is comparable to that of the R channel. This ensures that the overall image tone in bleeding spot mode is yellowish, while the bleeding spots appear red.

[0070] In a second aspect, the present application further provides an image processing system, comprising:

[0071] an acquisition unit, configured to acquire image data of a target area as image data to be measured;

[0072] a spectral reconstruction unit configured to perform spectral reconstruction on the image data to be measured to generate first image data having a first central wavelength, second image data having a second central wavelength, and third image data having a third central wavelength; wherein the first central wavelength, the second central wavelength, and the third central wavelength increase in sequence;

[0073] a fusion unit configured to perform a difference calculation on the first image data and the third image data to generate differential image data, and perform a weighted fusion on the differential image data and the second image data to form fused image data;

[0074] The enhancement unit is configured to enhance the fused image data using the third image data to form enhanced image data.

[0075] In a possible implementation, the fusion unit is further configured to:

[0076] Weighted fusion is performed according to the following formula:

[0077]

[0078] Where, I f To fuse image data, I b is the first image data, I g is the second image data, I r is the third image data, k0 is the weight coefficient of the differential image data, and is between 0.3 and 0.8.

[0079] In a possible implementation, the enhancement unit is further configured to:

[0080] The enhanced image data is calculated according to the following formula:

[0081]

[0082] Where, I g To enhance image data, I f To fuse image data, K gain is the adjustable gain, and b is the fusion gain.

[0083] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the composition and steps of each example according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0084] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, or can be electrical, mechanical or other forms of connection.

[0085] The units described as separate components may or may not be physically separated. As units, it is obvious that a person of ordinary skill in the art can realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0086] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0087] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or grid device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program code, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0088] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. An image processing method, characterized in that: include: Acquiring image data of the target area as image data to be measured; Performing spectral reconstruction on the image data to be measured to generate first image data having a first central wavelength as the central wavelength, second image data having a second central wavelength as the central wavelength, and third image data having a third central wavelength as the central wavelength; wherein the first central wavelength, the second central wavelength, and the third central wavelength increase in sequence; performing a difference calculation on the first image data and the third image data to generate difference image data, and performing a weighted fusion on the difference image data and the second image data to form fused image data; enhancing the fused image data by using the third image data to form enhanced image data; The enhanced image data is calculated according to the following formula: Where, I G To enhance image data, I f To fuse image data, K gain is the adjustable gain, b is the fusion gain; The fusion gain is calculated according to the following formula: Where, I r_ave is the average value of the pixel values ​​of the third image data, I f_ave is the average pixel value of the fused image data.

2. An image processing method according to claim 1, characterized in that: Also includes: outputting the first image data through a first channel; Outputting the enhanced image data through a second channel; outputting the third image data through a third channel; The first channel, the second channel, and the third channel are all one of the B channel, the G channel, and the R channel, and are different from each other.

3. The image processing method according to claim 1, wherein: The first central wavelength is 520-550 nm; the second central wavelength is 600 nm; and the third central wavelength is 620-640 nm.

4. An image processing method according to claim 3, characterized in that: The first central wavelength is 540 nm; the third central wavelength is 630 nm.

5. The image processing method according to claim 1, wherein: The fused image data formed by weighted fusion includes: Weighted fusion is performed according to the following formula: Where, I f To fuse image data, I b is the first image data, I g is the second image data, I r is the third image data, k0 is the weight coefficient of the differential image data, and is between 0.3 and 0.

8.

6. An image processing system, characterized in that: include: an acquisition unit, configured to acquire image data of a target area as image data to be measured; a spectral reconstruction unit configured to perform spectral reconstruction on the image data to be measured to generate first image data having a first central wavelength, second image data having a second central wavelength, and third image data having a third central wavelength; wherein the first central wavelength, the second central wavelength, and the third central wavelength increase in sequence; a fusion unit configured to perform a difference calculation on the first image data and the third image data to generate differential image data, and perform a weighted fusion on the differential image data and the second image data to form fused image data; an enhancing unit, configured to enhance the fused image data by using the third image data to form enhanced image data; The enhancement unit is further configured to: The enhanced image data is calculated according to the following formula: Where, I G To enhance image data, I f To fuse image data, K gain is the adjustable gain, b is the fusion gain; The fusion gain is calculated according to the following formula: Where, I r_ave is the average value of the pixel values ​​of the third image data, I f_ave is the average pixel value of the fused image data.

7. An image processing system according to claim 6, characterized in that: The fusion unit is further configured to: Weighted fusion is performed according to the following formula: Where, I f To fuse image data, I b is the first image data, I g is the second image data, I r is the third image data, k0 is the weight coefficient of the differential image data, and is between 0.3 and 0.8.

Images