Image processing method and system

Abstract

The invention discloses an image processing method and system, and is applied to the technical field of image processing, and the method comprises the steps: obtaining image data of a target region as to-be-detected image data; performing spectral reconstruction on the to-be-measured image data to generate first image data of which the central wavelength is a first central wavelength, second image data of which the central wavelength is a second central wavelength and third image data of which the central wavelength is a third central wavelength; performing differential calculation on the first image data and the third image data to generate differential image data, and performing weighted fusion on the differential image data and the second image data to form fused image data; and enhancing the fused image data through the third image data to form enhanced image data. According to the invention, through differential detection, the processed image has higher sensitivity to bleeding spots; through a gain adjustable technology, gain parameters are adjusted in real time, so that hemorrhagic spots in different states can be displayed and processed in the processed image.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an image processing method and system. Background Art

[0002] Endoscopic imaging and display to relevant personnel has become a common method in some surgeries. In order to facilitate relevant personnel to better identify bleeding points from endoscopic images, the prior art provides multiple sets of technical solutions. Among them, the Chinese patent application number 201780043850.1 discloses an image processing device, which extracts a blood accumulation area in the area representing blood with a lower blood concentration than the bleeding point, and enhances the brightness of the area to highlight the bleeding point in the image. The 630nm wavelength used by it has a relatively weak absorption of hemoglobin. Even if 630nm can penetrate deeper tissues, the weak absorption coefficient is not conducive to distinguishing the concentration of hemoglobin; at the same time, 540nm has a strong absorption of hemoglobin and a shallow penetration depth of the tissue. This will make the image processed by the above patent, although the hemoglobin concentration can be distinguished, the contrast will be relatively weak and easily interfered. Summary of the invention

[0003] In order to at least overcome the above-mentioned deficiencies in the prior art, the purpose of the present application is to provide an image processing method and system.

[0004] In a first aspect, an embodiment of the present application provides an image processing method, comprising: Acquire image data of the target area as image data to be tested; Performing spectral reconstruction on the image data to be measured to generate first image data having a first central wavelength as the central wavelength, second image data having a second central wavelength as the central wavelength, and third image data having a third central wavelength as the central wavelength; the first central wavelength, the second central wavelength, and the third central wavelength increase in sequence; Performing a differential calculation on the first image data and the third image data to generate differential image data, and performing a weighted fusion on the differential image data and the second image data to form fused image data; The fused image data is enhanced by using the third image data to form enhanced image data.

[0005] In a possible implementation, the method further includes: The first image data is output through a first channel; the enhanced image data is output through a second channel; the third image data is output through a third channel; the first channel, the second channel and the third channel are all one of the B channel, the G channel and the R channel, and are different from each other.

[0006] In a possible implementation, the first central wavelength is 520 - 550 nm; the second central wavelength is 600 nm; the third central wavelength is 620 - 640 nm.

[0007] In a possible implementation, the first central wavelength is 540 nm; the third central wavelength is 630 nm.

[0008] In a possible implementation, weighted fusion to form fused image data includes: Perform weighted fusion according to the following formula: In the formula, I f is the fused image data, I b is the first image data, I g is the second image data, I r is the third image data, and k0 is the weight coefficient of the differential image data, taking values from 0.3 to 0.8.

[0009] In a possible implementation, the enhanced image data is calculated according to the following formula: In the formula, I g is the enhanced image data, I f is the fused image data, K gain is the adjustable gain, and b is the fusion gain.

[0010] In a possible implementation, the fusion gain is calculated according to the following formula: In the formula, I r_ave is the average value of the pixel values of the third image data, I f_ave is the average value of the pixel values of the fused image data.

[0011] In a second aspect, the present application also provides an image processing system, including: An acquisition unit configured to acquire the image data of the target area as the image data to be measured; A spectral reconstruction unit configured to perform spectral reconstruction on the image data to be measured to generate first image data with a central wavelength of the first central wavelength, second image data with a central wavelength of the second central wavelength, and third image data with a central wavelength of the third central wavelength; the first central wavelength, the second central wavelength, and the third central wavelength increase in sequence; A fusion unit configured to perform differential calculation on the first image data and the third image data to generate differential image data, and perform weighted fusion on the differential image data and the second image data to form fused image data; An enhancement unit, configured to enhance the fused image data through the third image data to form enhanced image data.

[0012] In a possible implementation manner, the fusion unit is further configured to: Perform weighted fusion according to the following formula: In the formula, I f is the fused image data, I b is the first image data, I g is the second image data, I r is the third image data, k0 is the weight coefficient of the differential image data, taking a value of 0.3 to 0.8.

[0013] In a possible implementation manner, the enhancement unit is further configured to: Calculate the enhanced image data according to the following formula: In the formula, I G is the enhanced image data, I f is the fused image data, K gain is an adjustable gain, and b is a fusion gain.

[0014] Compared with the prior art, the present invention has the following advantages and beneficial effects: A method and system for image processing according to the present invention make the processed image more sensitive to bleeding points through differential detection; through the gain adjustable technology, the gain parameters are adjusted in real time, so that bleeding points in different states can be displayed in the processed image. Description of the Drawings

[0015] The drawings described herein are used to provide a further understanding of the embodiments of the present invention, form a part of this application, and do not constitute a limitation to the embodiments of the present invention. In the drawings: Figure 1 is a schematic diagram of the method steps of the embodiment of this application; Figure 2 is a schematic diagram of the overall processing process of the embodiment of this application; Figure 3 is a comparison diagram of the processing effects of the embodiment of this application. Detailed Embodiments

[0016] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of this application. It should be understood that the accompanying drawings in this application only serve the purposes of illustration and description, and are not used to limit the protection scope of this application. Additionally, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate the operations implemented in some embodiments according to the embodiments of this application. It should be understood that the operations in the flowchart may not be implemented in sequence, and steps without a logical context relationship may be reversed or implemented simultaneously. Furthermore, those skilled in the art may add one or more other operations to the flowchart or remove one or more operations from the flowchart under the guidance of the content of this application.

[0017] In addition, the described embodiments are only some embodiments of this application, not all of the embodiments. The components of the embodiments of this application generally described and illustrated in the accompanying drawings here may be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of this application claimed, but merely represents the selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative efforts fall within the protection scope of this application.

[0018] Please refer to Figure 1 , which is a schematic flowchart of an image processing method provided by an embodiment of the present invention. Further, the image processing method may specifically include the content described in the following steps S1 to S4.

[0019] S1: Obtain the image data of the target area as the image data to be measured; S2: Perform spectral reconstruction on the image data to be measured to generate first image data with a central wavelength of a first central wavelength, second image data with a central wavelength of a second central wavelength, and third image data with a central wavelength of a third central wavelength; the first central wavelength, the second central wavelength, and the third central wavelength increase in sequence; S3: Perform differential calculation on the first image data and the third image data to generate differential image data, and perform weighted fusion on the differential image data and the second image data to form fusion image data; S4: Enhance the fusion image data through the third image data to form enhanced image data.

[0020] When the embodiment of the present application is implemented, it is necessary to first obtain the image data to be measured. It should be understood that the present application is mainly applied in endoscopic imaging, and its main purpose is to provide enhanced image data to surgical personnel. Therefore, whether the enhanced image data can clearly display the bleeding point is very crucial. When imaging through an endoscope, it is often necessary to irradiate the target area with a light source, which belongs to the prior art and is not limited in detail in the embodiment of the present application; in the prior art, the reason for imaging three central wavelength images such as 415nm, 540nm, and 630nm is that the light source of the endoscope is generally a laser diode light source of three colors RGB, and it is necessary to control the alignment of the imaging time and the illumination time to achieve imaging under different color light sources; in the present application, due to the subsequent introduction of the spectral reconstruction technology, it is not necessary to strictly control the imaging and illumination, thereby reducing the control cost.

[0021] In the embodiment of the present application, various existing technologies can be used to perform spectral reconstruction on the image data to be measured. For example, spectral reconstruction can be performed in a linear manner. When attempting to recover the 600nm signal, a 3×3 mapping matrix corresponding to 600nm is constructed to restore the input image signal to a 600nm image, then for 540nm, another 3×3 mapping matrix for 540nm is refined for restoration. For example, spectral reconstruction can also be performed in a non-linear manner. When using non-linear reconstruction, a fully connected network combined with the relu activation function can be used to restore the spectral image, which belongs to the prior art and is not limited in detail in the embodiment of the present application. In the embodiment of the present application, a linear method is preferably used for spectral reconstruction because the light source corresponding to the image data to be measured is relatively simple. Therefore, spectral reconstruction can also be effectively performed by the linear method, and the computational complexity is much lower than that of non-linear reconstruction. It should be understood that performing spectral reconstruction through other existing technologies should also be regarded as within the protection scope of the present application.

[0022] In the embodiment of the present application, for the overall processing procedure, please refer to Figure 2, three image data with different central wavelengths can be obtained through spectral reconstruction. In the prior art, three image data with different central wavelengths are also obtained. Different from this, the three central wavelengths in the embodiments of the present application are generally preferably 540nm, 600nm, and 630nm, and the images in the 415 - 460nm band are abandoned. The reason is that different from the prior art solutions, in the present application, the screening of specific bleeding point regions is not concerned, but the highlighting of the bleeding points themselves is emphasized. There are significant differences in the absorption of the bleeding point spectra by the first image data and the third image data. The image obtained by calculating the difference between these two images can include the spectral characteristics of the bleeding points and background blood in the corresponding central band. At this time, the second image data is fused with the difference image. It should be understood that in the prior art, the processing of the difference image by other images is image enhancement, that is, enhancing the image brightness, while the present application uses the method of fusing the second image data and the difference image to highlight the bleeding points. Its advantage is that the fused image can include the bleeding point information in the three images, while only one or two images in the prior art. In the embodiments of the present application, the central wavelength of the second image data is preferably 600nm. The signal with a wavelength of 600nm can penetrate deeper tissues, but the weak absorption coefficient is not conducive to distinguishing the concentration of hemoglobin. The difference image can well make up for this defect. By fusing these two images, the position of the bleeding point can be presented more accurately. It should be understood that the fusion process can be weighted fusion, or maximum fusion, minimum fusion, gradient fusion, etc. As long as it can produce similar effects, it should be considered a technical solution equivalent to the present application.

[0023] In the embodiments of the present application, the fused image is generally darker, while the third image data is generally brighter. Therefore, at this time, it is necessary to perform image enhancement on the fused image data through the third image data to increase the image brightness and form enhanced image data.

[0024] In a possible implementation manner, it further includes: Output the first image data through the first channel; output the enhanced image data through the second channel; output the third image data through the third channel; the first channel, the second channel, and the third channel are each one of the B channel, the G channel, and the R channel, and are different from each other.

[0025] When the embodiment of the present application is implemented, the channels for the endoscope to output images are generally three, namely the B channel, the G channel, and the R channel. Therefore, in the embodiment of the present application, the first image data, the enhanced image data, and the third image data are generally output through these three channels respectively. For example, when the first central wavelength is 540 nm, the second central wavelength is 600 nm, and the third central wavelength is 630 nm, the first image data is output through the B channel, the enhanced image data is output through the G channel, and the third image data is output through the R channel. For the final output result, please refer to Figure 3 , Figure 3 In the figure on the left in Figure 3 is the image output by the embodiment of the present application, and the image on the right is the image output by the prior art; the red dots shown in the figure from top to bottom are the cases of no bleeding point, a relatively large bleeding point, and multiple small bleeding points, and there is blood accumulation on all of them. It can be seen from the figure that the prior art can well identify the blood accumulation situation, but cannot accurately see the specific position and situation of the bleeding point, while the present application can effectively display the position of the bleeding point.

[0026] In a possible implementation manner, the first central wavelength is 520 - 550 nm; the second central wavelength is 600 nm; the third central wavelength is 620 - 640 nm.

[0027] In a possible implementation manner, the first central wavelength is 540 nm; the third central wavelength is 630 nm.

[0028] In a possible implementation manner, weighted fusion to form the fused image data includes: Perform weighted fusion according to the following formula: In the formula, I f is the fused image data, I b is the first image data, I g is the second image data, I r is the third image data, and k0 is the weight coefficient of the differential image data, taking a value of 0.3 - 0.8.

[0029] When the embodiment of the present application is implemented, when calculating the differential image data, it can be achieved by calculating the difference in pixel values of the same pixel point in the third image data and the first image data; in the above formula, I r -I b is the differential image data. To avoid negative values, the absolute value calculation can be performed on this data. And k0 is the weight coefficient of the differential image data, which is generally determined by experiments. It should be understood that different values can be used for the weight coefficients of the differential image data of different organs or parts, so that the present application can be better applied to various occasions.

[0030] In a possible implementation, the enhanced image data is calculated according to the following formula: In the formula, I G is the enhanced image data, I f is the fused image data, K gain is the adjustable gain, and b is the fusion gain.

[0031] In a possible implementation, the fusion gain is calculated according to the following formula: In the formula, I r_ave is the average value of the pixel values of the third image data, and I f_ave is the average value of the pixel values of the fused image data.

[0032] When the embodiments of the present application are implemented, when enhancing the fused image, the enhancement is based on the states of the third image data and the fused image data. Among them, the present application introduces an adjustable gain K gain , and the adjustment of this adjustable gain is set at the person who views the endoscopic imaging result. The person can adjust this parameter to cope with different bleeding situations and adjust the gain parameter in real time, so that even extremely weak bleeding points can be detected. After the above enhancement, the signal of the bleeding point is amplified, and the overall brightness of the G channel is equivalent to that of the R channel, so that the overall tone of the image is yellowish in the bleeding point mode, and the color of the bleeding point is red.

[0033] In a second aspect, the present application further provides an image processing system, including: An acquisition unit, configured to acquire the image data of the target area as the image data to be measured; A spectral reconstruction unit, configured to perform spectral reconstruction on the image data to be measured to generate first image data with a first central wavelength, second image data with a second central wavelength, and third image data with a third central wavelength; the first central wavelength, the second central wavelength, and the third central wavelength increase in sequence; A fusion unit, configured to perform differential calculation on the first image data and the third image data to generate differential image data, and perform weighted fusion on the differential image data and the second image data to form fused image data; An enhancement unit, configured to enhance the fused image data through the third image data to form enhanced image data.

[0034] In a possible implementation, the fusion unit is further configured to: Perform weighted fusion according to the following formula: In the formula, I f is the fused image data, I b is the first image data, I g is the second image data, I r is the third image data, and k0 is the weight coefficient of the differential image data, taking a value of 0.3 to 0.8.

[0035] In a possible implementation manner, the enhancement unit is further configured to: Calculate the enhanced image data according to the following formula: In the formula, I g is the enhanced image data, I f is the fused image data, K gain is the adjustable gain, and b is the fusion gain.

[0036] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0037] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed couplings or direct couplings or communication connections to each other can be indirect couplings or communication connections through some interfaces, devices or units, and can also be electrical, mechanical or other forms of connection.

[0038] The unit described as a separation component may or may not be physically separated. Obviously, those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0039] In addition, the functional units in each embodiment of the present invention can be integrated into a processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0040] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a grid device, etc.) to execute all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs that can store program codes.

[0041] The specific embodiments described above further elaborate on the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above is only the specific embodiment of the present invention and is not used to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An image processing method, characterized in that, Including: Obtain the image data of the target area as the image data to be measured; Perform spectral reconstruction on the image data to be measured to generate first image data with a first central wavelength, second image data with a second central wavelength, and third image data with a third central wavelength; the first central wavelength, the second central wavelength, and the third central wavelength increase in sequence; Perform differential calculation on the first image data and the third image data to generate differential image data, and perform weighted fusion on the differential image data and the second image data to form fused image data; Enhance the fused image data through the third image data to form enhanced image data.

2. The image processing method according to claim 1, wherein Also including: Output the first image data through a first channel; Output the enhanced image data through a second channel; Output the third image data through a third channel; The first channel, the second channel, and the third channel are each one of the B channel, the G channel, and the R channel, and are mutually different.

3. The image processing method according to claim 1, wherein The first central wavelength is 520 - 550 nm; the second central wavelength is 600 nm; the third central wavelength is 620 - 640 nm.

4. The image processing method according to claim 3, wherein The first central wavelength is 540 nm; the third central wavelength is 630 nm.

5. A method for image processing according to claim 1, characterized in that, Weighted fusion to form fused image data includes: Perform weighted fusion according to the following formula: Where, I f is the fused image data, I b is the first image data, I g is the second image data, I r is the third image data, and k0 is the weight coefficient of the differential image data, taking values from 0.3 to 0.

8.

6. The image processing method according to claim 1, wherein The enhanced image data is calculated according to the following formula: Where, I g is the enhanced image data, I f is the fused image data, K gain is the adjustable gain, and b is the fusion gain.

7. An image processing method according to claim 6, characterized in that The fusion gain is calculated according to the following formula: Where, I r_ave is the average value of the pixel values of the third image data, and I f_ave is the average value of the pixel values of the fused image data.

8. An image processing system, characterized in that, Including: An acquisition unit configured to obtain the image data of the target area as the image data to be measured; A spectral reconstruction unit configured to perform spectral reconstruction on the image data to be measured to generate first image data with a first central wavelength, second image data with a second central wavelength, and third image data with a third central wavelength; the first central wavelength, the second central wavelength, and the third central wavelength increase in sequence; A fusion unit configured to perform differential calculation on the first image data and the third image data to generate differential image data, and perform weighted fusion on the differential image data and the second image data to form fused image data; An enhancement unit configured to enhance the fused image data through the third image data to form enhanced image data.

9. An image processing system according to claim 8, characterized in that, The fusion unit is further configured to: Perform weighted fusion according to the following formula: Where, I f is the fused image data, I b is the first image data, I g is the second image data, I r is the third image data, and k0 is the weight coefficient of the differential image data, taking values from 0.3 to 0.

8.

10. An image processing system according to claim 8, characterized in that, The enhancement unit is further configured to: Calculate the enhanced image data according to the following formula: Wherein, I G is the enhanced image data, I f is the fused image data, K gain is the adjustable gain, and b is the fusion gain.

Images