Medical image processing method and device, equipment, medium and program product

Abstract

The invention relates to the technical field of image processing, and discloses a medical image processing method, device and equipment, a medium and a program product, and the method comprises the steps: obtaining a first to-be-processed medical image which is a three-dimensional image with a first preset color gamut; determining a second to-be-processed medical image with a second preset color gamut according to the first to-be-processed medical image; for the set medical event, based on the second to-be-processed medical image, obtaining a binary image used for representing focus position information of the first to-be-processed medical image for the set medical event; and carrying out image fusion on the binary image and the first to-be-processed medical image to obtain a fusion image used for representing the focus contour area of the first to-be-processed medical image for the set medical event. The situation that light sources with different spectrums are used for detection or large-scale samples are used for training focus area feature values is avoided, and timeliness and accuracy of focus tracking are effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a medical image processing method, device, equipment, medium and program product. Background Art

[0002] In the related art, the lesion tracking method includes identifying lesion tissue based on artificial intelligence and using cold light sources to emit different spectra to identify lesions. Among them, the method of identifying lesion tissue based on artificial intelligence requires the characteristic values ​​of sample training based on large-scale data of lesions, and then identifying the position of lesion tissue in the video stream in real time. It is extremely difficult to implement in the current endoscopic system, and sufficient computing power is required to support the identification of lesion tissue in real-time video streams. It is also difficult to develop, and sufficient training samples are required to accurately identify lesions. The method of using cold light sources to identify lesions requires emitting light sources of different spectra, and tracking lesions by distinguishing the different chromaticity between lesions and normal human tissues. In this way, different spectra need to be emitted for different lesions, and hundreds of lesions correspond to light sources of very many spectra, and the compatibility of the solution is poor. Summary of the invention

[0003] In view of this, the present invention provides a medical image processing method, apparatus, device, medium and program product to solve the problems of poor compatibility of lesion tracking methods in related technologies.

[0004] In a first aspect, the present invention provides a medical image processing method, the method comprising: acquiring a first medical image to be processed, the first medical image to be processed being a three-dimensional image having a first preset color gamut; determining a second medical image to be processed based on the first medical image to be processed, the second medical image to be processed being an image having a second preset color gamut; for a set medical event, based on the second medical image to be processed, obtaining a binary image for representing the lesion location information of the first medical image to be processed for the set medical event; performing image fusion on the binary image and the first medical image to be processed to obtain a fused image for representing the lesion contour area of ​​the first medical image to be processed for the set medical event.

[0005] The medical image processing method provided by the embodiment of the present invention converts the first medical image to be processed, performs image processing in different color gamut spaces, obtains a binary image for characterizing the location information of the lesion, and then fuses the binary image with the first medical image to be processed to obtain a fused image for characterizing the contour area of ​​the lesion. Thus, the use of light sources with different spectra for detection is avoided, and the use of large-scale samples for training the characteristic values ​​of the lesion area is also avoided, which effectively improves the timeliness and accuracy of lesion tracking.

[0006] In some optional embodiments, based on the second medical image to be processed, a binary image is obtained for characterizing the lesion location information of the first medical image to be processed for a set medical event, including: acquiring lesion data for the set medical event from a lesion database, the lesion data being used to show the offset threshold of the second medical image to be processed for the set medical event; and comparing the second medical image to be processed with the lesion data to obtain a binary image.

[0007] The medical image processing method provided by the embodiment of the present invention compares the second medical image to be processed with the lesion data used to show the offset threshold of the second medical image to be processed for the set medical event to obtain a binary image. The first medical image to be processed and the second medical image to be processed have a certain conversion relationship, so the binary image here can be used to characterize the lesion location information of the first medical image to be processed for the set medical event. Thus, the lesion location information can be accurately and quickly determined.

[0008] In some optional embodiments, the second medical image to be processed is compared with the lesion data to obtain a binary image, including: when it is determined that the blue concentration offset and / or red concentration offset of a pixel of the second medical image to be processed meets the offset threshold, the pixel is determined to be a suspected lesion pixel; and a binary image is determined based on multiple suspected lesion pixels.

[0009] The medical image processing method provided by the embodiment of the present invention can quickly and accurately judge suspected lesion pixels by simply comparing multiple color concentration offsets of pixels with corresponding offset thresholds, and further obtain a binary image. The judgment method is simple and the judgment result is accurate.

[0010] In some optional embodiments, the binary image is fused with the first medical image to be processed to obtain a fused image for characterizing the lesion contour area of ​​the first medical image to be processed for a set medical event, including: when the binary image shows that a pixel is a suspected lesion pixel, the three color channel values ​​of the pixel in the first medical image to be processed are respectively increased by a first set brightening value to obtain a fused image.

[0011] The medical image processing method provided by the embodiment of the present invention increases the three color channel values ​​of the pixel in the first medical image to be processed by the first set brightening value respectively when the binary image shows that the pixel is a suspected lesion pixel, so as to obtain a fused image, effectively improving the speed of image processing and significantly enhancing the effect of image processing.

[0012] In some optional embodiments, the binary image is fused with the first medical image to be processed to obtain a fused image for representing the lesion contour area of ​​the first medical image to be processed for a set medical event, including: when the binary image shows that a pixel is a suspected lesion pixel, the color channel value of the pixel in the first medical image to be processed is set to increase by a second set brightening value to obtain a fused image.

[0013] The medical image processing method provided by the embodiment of the present invention increases the color channel value of the pixel in the first medical image to be processed by a second set brightening value when the binary image shows that the pixel is a suspected lesion pixel, thereby obtaining a fused image, effectively improving the speed of image processing and significantly enhancing the effect of image processing.

[0014] In some optional embodiments, determining a second medical image to be processed based on a first medical image to be processed includes: obtaining a mapping relationship between the first medical image to be processed and the second medical image to be processed, the mapping relationship being used to show a first relationship between the brightness value of the second medical image to be processed and the three-color channel values ​​of the first medical image to be processed, a second relationship between the blue concentration offset and the three-channel values ​​of the first medical image to be processed, and a third relationship between the red concentration offset and the three-color channel values ​​of the first medical image to be processed; determining the brightness values, blue concentration offsets, and red concentration offsets of multiple pixels of the second medical image to be processed based on the three-color channel values ​​of the first medical image to be processed and the mapping relationship.

[0015] The medical image processing method provided by the embodiment of the present invention determines the brightness value, blue concentration offset and red concentration offset of multiple pixels of the second medical image to be processed according to the three color channel values ​​and mapping relationship of the first medical image to be processed. Thus, the first medical image to be processed is converted into the second medical image to be processed including the brightness value, blue concentration offset and red concentration offset, which is conducive to determining the lesion location information based on the second medical image to be processed.

[0016] In a second aspect, the present invention provides a medical image processing device, comprising: an acquisition module, configured to acquire a first medical image to be processed, the first medical image to be processed being a three-dimensional image with a first preset color gamut; a conversion module, configured to determine a second medical image to be processed based on the first medical image to be processed, the second medical image to be processed being an image with a second preset color gamut; a determination module, configured to obtain, for a set medical event, a binary image for characterizing the lesion location information of the first medical image to be processed for the set medical event based on the second medical image to be processed; and a fusion module, configured to fuse the binary image with the first medical image to be processed to obtain a fused image for characterizing the lesion contour area of ​​the first medical image to be processed for the set medical event.

[0017] In a third aspect, the present invention provides a computer device comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the medical image processing method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0018] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the medical image processing method of the first aspect or any corresponding embodiment thereof.

[0019] In a fifth aspect, the present invention provides a computer program product, comprising computer instructions for causing a computer to execute the medical image processing method of the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0021] Figure 1 is a schematic flow chart of a medical image processing method according to an embodiment of the present invention;

[0022] Figure 2 is a flowchart of another medical image processing method according to an embodiment of the present invention;

[0023] Figure 3 is a flowchart of a specific application example of the medical image processing method according to an embodiment of the present invention;

[0024] Figure 4 A schematic diagram of an original three-dimensional RGB image showing a specific application example of the medical image processing method according to an embodiment of the present invention;

[0025] Figure 5 A schematic diagram of a grayscale image of an original three-dimensional RGB image showing a specific application example of the medical image processing method according to an embodiment of the present invention;

[0026] Figure 6 A schematic diagram of a fused image finally obtained in a specific application example of the medical image processing method according to an embodiment of the present invention is shown;

[0027] Figure 7A schematic diagram of a grayscale image of a fused image finally obtained in a specific application example of the medical image processing method according to an embodiment of the present invention is shown;

[0028] Figure 8 is a structural block diagram of a medical image processing device according to an embodiment of the present invention;

[0029] Fig. 9 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0031] In order to explain the technical solution in more detail, the image processing of medical instruments such as electronic endoscopes is taken as an example to illustrate the technical solution. However, in actual application, the medical image processing method disclosed in the present invention can also be applied to other applicable application scenarios, and the present invention does not make specific limitations on this.

[0032] The embodiment of the present invention provides a medical image processing method, which converts a first medical image to be processed, performs image processing in different color gamut spaces, obtains a binary image for characterizing lesion location information, and then fuses the binary image with the first medical image to be processed to obtain a fused image for characterizing the lesion contour area. Thus, the use of light sources with different spectra for detection is avoided, and the use of large-scale samples for training lesion area feature values ​​is also avoided, effectively improving the timeliness and accuracy of lesion tracking.

[0033] According to an embodiment of the present invention, an embodiment of a medical image processing method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0034] In this embodiment, a medical image processing method is provided, which can be used for the above-mentioned medical instruments, such as electronic endoscopes, etc. Figure 1 is a flowchart of a medical image processing method according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0035] Step S101: acquiring a first medical image to be processed, where the first medical image to be processed is a three-dimensional image with a first preset color gamut.

[0036] In this embodiment of the present invention, the first medical image to be processed is an image acquired by a medical instrument, such as a medical image acquired by an electronic endoscope. Here, the first medical image to be processed can be an image directly acquired by an image acquisition device of the medical instrument, or can be obtained through image preprocessing such as denoising.

[0037] In one embodiment of the present invention, the first preset color gamut may be an RGB color gamut, and the first medical image to be processed may be an image in a three-dimensional RGB format. For example, the three-dimensional RGB image may be image[1080,1920,3], wherein 1080 represents the total number of rows of image pixels, 1920 represents the total number of columns of image pixels, and 3 represents the number of channels of the image, namely, three channels of R, G, and B.

[0038] Step S102: determining a second medical image to be processed according to the first medical image to be processed, wherein the second medical image to be processed is an image having a second preset color gamut.

[0039] In this embodiment of the present invention, the second preset color gamut may be a YCbCr color gamut. For example, the second to-be-processed medical image is image_ycbcr[1080,1920,3], where 1080 represents the total number of rows of image pixels, 1920 represents the total number of columns of image pixels, and 3 represents the number of channels of the image, namely, Y, Cb, and Cr. Y represents the brightness and concentration of the color, and Cb and Cr represent the blue concentration offset and red concentration offset of the color, respectively.

[0040] In addition, the second preset color gamut space here can also be a YUV color gamut space, in which the brightness Y represents the brightness information of the image, and the chromaticity U and V represent the color information. The Y channel only contains brightness information, and the U and V channels contain color information. In the YUV color space, for an image, only the brightness information Y needs to be transmitted to ensure the transmission of the main content of the image. The chromaticity information U and V can be calculated by the brightness information Y, thereby achieving compression.

[0041] YCbCr color space is a variant of YUV color space, commonly used in digital image and video processing. In YCbCr color space, the Y channel is similar to the Y channel in YUV color space, representing brightness information, and the Cb and Cr channels represent chrominance information. Unlike YUV, the Cb and Cr channels have undergone a certain degree of chrominance subsampling to reduce the transmission volume and storage space of image data.

[0042] In summary, RGB color space is commonly used in computer graphics and display devices, YUV is mainly used in TV broadcasting and video transmission, and YCbCr is mainly used in digital image and video processing, including compression, encoding, decoding and display.

[0043] Here, taking the conversion of the first image to be processed in the RGB color domain to the second image to be processed in the YCbCr color domain as an example, the implementation method of determining the second medical image to be processed according to the first medical image to be processed is described. In actual application, the first preset color domain and the second preset color domain can both be other color domain spaces.

[0044] Specifically, the following formula may be used to convert the first image to be processed in the RGB color gamut into the second image to be processed in the YCbCr color gamut.

[0045] Y = 0.299R + 0.587G + 0.114B;

[0046] Cb = -0.1687R - 0.3313G + 0.5B;

[0047] Cr=0.5R-0.4187G-0.0813B;

[0048] Among them, R, G, and B represent the color values ​​of the red, green, and blue channels of a certain pixel respectively;

[0049] Y, Cb, Cr represent the values ​​of the three channels of the pixel, Y, Cb, Cr, respectively. The Y channel represents brightness, and its value range is 0 to 255, while Cb and Cr represent the blue concentration offset and red concentration offset, respectively, and their value range is -128 to 127. In specific use, it is usually necessary to add an offset of 128 to make its value range become 0 to 255, which is consistent with the Y channel. The purpose of this is to facilitate data transmission and processing.

[0050] Step S103 : for the set medical event, based on the second medical image to be processed, a binary image for representing the lesion location information of the first medical image to be processed for the set medical event is obtained.

[0051] In this embodiment of the present invention, a lesion database can be pre-configured, and different lesion data can be configured for multiple medical events such as appendicitis, gastroenteritis, and tumors. The lesion data can be a lesion threshold range for the three-channel values ​​of Y, Cb, and Cr for a pixel. When the first medical image to be processed is collected, it is usually determined that the image needs to be processed as a reference for a certain set medical event. Therefore, here, if the three values ​​of Y, Cb, and Cr of a pixel in the second image to be processed fall within the corresponding lesion threshold range from the lesion data, the pixel can be determined to be a suspected lesion pixel. Thus, different values ​​are assigned to suspected lesion pixels and non-suspected lesion pixels, respectively, to obtain a binary image for characterizing the lesion position information of the first medical image to be processed for the set medical event.

[0052] It should be noted that the pixel position identifiers of the first medical image to be processed and the second medical image to be processed are consistent, but the color gamut spaces are different, so the three channel values ​​are different. Based on the Y, Cb, Cr three-channel values ​​of the second medical image to be processed and the lesion threshold range of the Y, Cb, Cr three-channel values ​​in the lesion data for the set medical event, the pixel position identifier of the suspected lesion pixel can be determined, thereby obtaining a binary image for characterizing the lesion position information of the first medical image to be processed for the set medical event.

[0053] Step S104 , performing image fusion on the binary image and the first medical image to be processed to obtain a fused image for representing the lesion contour area of ​​the first medical image to be processed for the set medical event.

[0054] In this embodiment of the present invention, for the suspected lesion pixel, the values ​​of the three channels R, G and B can be increased by the set values ​​respectively to highlight the lesion area. It should be noted that the same value can be added to the three channels respectively, for example: the values ​​of the three channels R, G and B can be increased by 100 respectively. It is also possible to increase the set value only for one of the channels, and the values ​​of the other channels remain unchanged, for example: only the value of the R channel is increased by 100. Other applicable values ​​can also be selected or other applicable methods can be used to perform image fusion, as long as the image of the suspected lesion area is distinguished and displayed from the non-suspected lesion area, and the present invention does not make specific limitations on this.

[0055] It should be noted that the present invention only performs more reasonable image processing on medical images, so that the processed images have more reference value under set circumstances.

[0056] The medical image processing method provided by the embodiment of the present invention converts the first medical image to be processed, performs image processing in different color gamut spaces, obtains a binary image for characterizing the lesion location information, and then fuses the binary image with the first medical image to be processed to obtain a fused image for characterizing the lesion contour area. Thus, the use of light sources of different spectra for detection is avoided, and the use of large-scale samples to train the lesion area feature values ​​is avoided, which effectively improves the timeliness and accuracy of lesion tracking. At the same time, only simple image processing is required for the first image to be processed, and there is no need to select light sources of different spectra for tracking and detection according to different lesions, which significantly enhances the compatibility of medical image processing.

[0057] In this embodiment, a medical image processing method is provided, which can be used in medical instruments, such as electronic endoscopes, etc. Figure 2 is a flowchart of a medical image processing method according to an embodiment of the present invention. Figure 2 As shown, the process includes the following steps:

[0058] Step S201: acquiring a first medical image to be processed, where the first medical image to be processed is a three-dimensional image having a first preset color gamut.

[0059] For details, please see Figure 1 Step S101 of the illustrated embodiment will not be described in detail here.

[0060] Step S202: determining a second medical image to be processed according to the first medical image to be processed, wherein the second medical image to be processed is an image having a second preset color gamut.

[0061] In some optional implementations, the above step S202 includes:

[0062] Step S2021, obtaining a mapping relationship between the first medical image to be processed and the second medical image to be processed, the mapping relationship being used to show a first relationship between the brightness value of the second medical image to be processed and the three-color channel values ​​of the first medical image to be processed, a second relationship between the blue concentration offset and the three-channel values ​​of the first medical image to be processed, and a third relationship between the red concentration offset and the three-color channel values ​​of the first medical image to be processed.

[0063] In this embodiment of the present invention, the first image to be processed may be an image with an RGB color gamut, and the second image to be processed may be an image with a YCrCb color gamut. The image conversion with the RGB color gamut has a specific mapping relationship with the image with the YCrCb color gamut space.

[0064] Specifically, the first relationship between the brightness value of the second to-be-processed medical image and the three-color channel values ​​of the first to-be-processed medical image may be: Y=0.299R+0.587G+0.114B.

[0065] The second relationship between the blue density offset and the three-channel values ​​of the first to-be-processed medical image may be: Cb=-0.1687R-0.3313G+0.5B.

[0066] The third relationship between the red density offset and the three color channel values ​​of the first to-be-processed medical image may be: Cr=0.5R-0.4187G-0.0813B.

[0067] Among them, R, G, and B represent the color values ​​of the red, green, and blue channels of a certain pixel respectively;

[0068] Y, Cb, Cr represent the values ​​of the three channels of the pixel, Y, Cb, Cr, respectively. The Y channel represents brightness, and its value range is 0 to 255, while Cb and Cr represent the blue concentration offset and red concentration offset, respectively, and their value range is -128 to 127. In specific use, it is usually necessary to add an offset of 128 to make its value range become 0 to 255, which is consistent with the Y channel. The purpose of this is to facilitate data transmission and processing.

[0069] Step S2022: Determine the brightness values, blue concentration offsets, and red concentration offsets of multiple pixels of the second medical image to be processed according to the three color channel values ​​and the mapping relationship of the first medical image to be processed.

[0070] In this embodiment of the present invention, the mapping relationship in step S2021 can be used to convert the first image to be processed with the RGB color gamut into the second image to be processed with the YCbCr color gamut to obtain the brightness values, blue concentration offsets and red concentration offsets of multiple pixels of the second medical image to be processed.

[0071] The medical image processing method provided by the embodiment of the present invention determines the brightness value, blue concentration offset and red concentration offset of multiple pixels of the second medical image to be processed according to the three color channel values ​​and mapping relationship of the first medical image to be processed. Thus, the first medical image to be processed is converted into the second medical image to be processed including the brightness value, blue concentration offset and red concentration offset, which is conducive to determining the lesion location information based on the second medical image to be processed.

[0072] Step S203 : for the set medical event, based on the second medical image to be processed, a binary image for representing the lesion location information of the first medical image to be processed for the set medical event is obtained.

[0073] In some optional implementations, the above step S203 includes:

[0074] Step S2031 : acquiring lesion data for a set medical event from a lesion database, where the lesion data is used to indicate an offset threshold of a second to-be-processed medical image for the set medical event.

[0075] For example, the second data to be processed is an image with a YCbCr color gamut. The lesion database can pre-configure corresponding brightness value thresholds, blue concentration offset thresholds, and red concentration offset thresholds for different image analysis requirements and for different medical events such as appendicitis and routine physical examinations. It should be noted that at least one of the brightness value threshold, blue concentration offset threshold, and red concentration offset threshold can be configured according to actual needs, and the threshold can be a numerical range or two or more discontinuous numerical ranges. For example: for a certain medical event, the brightness value threshold can be (100, 110) and (130, 140). It is also possible to set a blue concentration offset threshold threshold_cb∈(100, 110) and a red concentration offset threshold threshold_cr∈(140, 150) for the chromaticity area of ​​a certain lesion area.

[0076] Step S2032, comparing the second medical image to be processed with the lesion data to obtain a binary image.

[0077] In some optional implementations, the above step S2032 includes:

[0078] Step a1: when it is determined that the blue density offset and / or the red density offset of a pixel of the second medical image to be processed meets the offset threshold, the pixel is determined to be a suspected lesion pixel.

[0079] Specifically, in actual application, different medical problems have different abnormal manifestations in different color domains. For example, for different medical events, it can be set that when the blue concentration offset and the red concentration offset of a certain pixel both meet the corresponding offset thresholds, the pixel is judged to be a suspected lesion pixel. It can also be set that as long as one of the blue concentration offset and the red concentration offset meets the corresponding offset threshold, the pixel is judged to be a suspected lesion pixel.

[0080] Step a2: determining a binary image based on a plurality of suspected lesion pixels.

[0081] In this embodiment of the present invention, the value of the suspected lesion pixel in the binary image may be set to 1, and the value of the non-suspected lesion pixel may be set to 0 accordingly.

[0082] For example, the blue concentration offset threshold threshold_cb∈(100, 110) and the red concentration offset threshold threshold threshold_cr∈(140, 150) for a certain lesion are obtained from the lesion database. When a loop is used to judge that 100≤image_cb[j,i]≤110 and 140≤image_cr[j,i]≤150, BW_ROI[j,i]=1.

[0083] Where j is the row number of the current pixel, and i is the column number of the current pixel. For example, image_cr[1043,576] is the cr value of the pixel in the 576th column of the 1043th row. According to this cycle, the blue density offset and red density offset of each pixel of the second image to be processed are judged to obtain a binary image BW_ROI[1080,1920]. The value of the pixel position in the binary image where the blue density offset meets the blue density offset threshold and / or the red density offset meets the red density offset threshold is 1, otherwise it is 0. For example, 100≤image_cb[526,343]≤110, 140≤image_cr[526,343]≤150, then BW_ROI[526,343]=1, BW_ROI[526,343]=1 means that the pixel in the 526th row and 343rd column in the binary image is a suspected lesion pixel. 100≤image_cb[526,344]≤110, 150<image_cr[526,344], then BW_ROI[526,344]=0, BW_ROI[526,344]=0 indicates that the pixel at the 526th row and the 343rd column in the binary image is a non-suspected lesion pixel. Thus, after judging the blue density offset and the red density offset of each pixel of the second image to be processed, a binary image for representing the lesion position information of the first medical image to be processed for the set medical event is obtained.

[0084] The medical image processing method provided by the embodiment of the present invention can quickly and accurately judge suspected lesion pixels by simply comparing multiple color concentration offsets of pixels with corresponding offset thresholds, and further obtain a binary image. The judgment method is simple and the judgment result is accurate.

[0085] The medical image processing method provided by the embodiment of the present invention compares the second medical image to be processed with the lesion data used to show the offset threshold of the second medical image to be processed for the set medical event to obtain a binary image. The first medical image to be processed and the second medical image to be processed have a certain conversion relationship, so the binary image here can be used to characterize the lesion location information of the first medical image to be processed for the set medical event. Thus, the lesion location information can be accurately and quickly determined.

[0086] Step S204 , performing image fusion on the binary image and the first medical image to be processed to obtain a fused image for representing the lesion contour area of ​​the first medical image to be processed for the set medical event.

[0087] In some optional implementations, the above step S204 may include:

[0088] Step b1, when the binary image shows that a pixel is a suspected lesion pixel, the three color channel values ​​of the pixel in the first medical image to be processed are respectively increased by a first set brightening value to obtain a fused image.

[0089] Specifically, the binary image can characterize the lesion location information of the first medical image to be processed for the set medical event. For example, for the binary image of the set medical event, BW_ROI[526,343]=1, then the pixel [526,343] of the first image to be processed is a suspected lesion pixel. If the first set brightening value is 100, the values ​​of the three channels R, G, and B of the pixel can be increased by 100 respectively. It should be noted that the first set brightening value can be set according to actual needs during actual application. For example: set the first set brightening value to 80, 90, or 95.

[0090] The medical image processing method provided by the embodiment of the present invention increases the three color channel values ​​of the pixel in the first medical image to be processed by the first set brightening value respectively when the binary image shows that the pixel is a suspected lesion pixel, so as to obtain a fused image, effectively improving the speed of image processing and significantly enhancing the effect of image processing.

[0091] In some optional implementations, the above step S204 may include:

[0092] Step c1, when the binary image shows that a pixel is a suspected lesion pixel, the color channel value of the pixel in the first medical image to be processed is increased by a second set brightening value to obtain a fused image.

[0093] The medical image processing method provided by the embodiment of the present invention increases the color channel value of the pixel in the first medical image to be processed by a second set brightening value when the binary image shows that the pixel is a suspected lesion pixel, thereby obtaining a fused image, effectively improving the speed of image processing and significantly enhancing the effect of image processing.

[0094] Specifically, the binary image can represent the lesion location information of the first medical image to be processed for the set medical event. For example, for the binary image of the set medical event, BW_ROI[526,343]=1, then the pixel [526,343] of the first image to be processed is a suspected lesion pixel, if the first set brightening value is 100.

[0095] Here, according to specific needs, only one of the color channel values ​​can be adjusted to highlight the lesion area. For example, the red channel value of the three channels R, G, and B of the pixel can be increased by 100, the green channel value of the three channels R, G, and B of the pixel can be increased by 100, and the blue channel value of the three channels R, G, and B of the pixel can be increased by 100. Of course, here, two of the three channel values ​​R, G, and B of the pixel can also be increased by corresponding values ​​according to needs, and the values ​​increased for the two color channels can be the same or different.

[0096] It should be noted that, in actual application, the second set brightness enhancement value can also be set according to actual needs. For example, the second set brightness enhancement value can be set to 80, 90 or 95.

[0097] The first set increment value and the second set increment value are set based on the fact that after image fusion is performed according to the first set increment value and the second set increment value, the obtained image can highlight the lesion area.

[0098] In this embodiment of the present application, the lesion area can also be optimized based on multiple suspected lesion pixels. For example, when multiple consecutive suspected lesion pixels and the area surrounded by them include a first set number of non-suspected lesion pixels, and the first set number is much smaller than the number of consecutive suspected lesion pixels, then the multiple consecutive suspected lesion pixels and the area surrounded by them can be regarded as the lesion area, and the first set number of non-suspected lesion pixels can also be processed as suspected lesion pixels, and the three color channel values ​​of the pixel in the first medical image to be processed can be increased by the first set brightening value respectively, or the color channel value of the pixel in the first medical image to be processed can be increased by the second set brightening value.

[0099] Similarly, only the area composed of suspected lesion pixels can be used as the lesion area, and only for the suspected lesion pixels, a first set number of non-suspected lesion pixels can be treated as suspected lesion pixels, and the three color channel values ​​of the pixel in the first medical image to be processed can be increased by the first set brightening value, or the color channel value of the pixel in the first medical image to be processed can be increased by the second set brightening value.

[0100] Step S205: output the fused image.

[0101] Specifically, according to the operation of step S204, after the color channel values ​​of the first image to be processed are processed, the obtained fused image can highlight the corresponding lesion area for the set medical event. At this time, the fused image can be output to the display device of the medical instrument such as the electronic endoscope for viewing and reference by medical staff.

[0102] It should be noted that step S205 is a step included in an optional embodiment of the present invention. In actual application, whether to include this operation step can be determined according to actual needs.

[0103] The medical image processing method provided by the embodiment of the present invention timely outputs a fused image for representing the lesion contour area of ​​the first medical image to be processed for a set medical event. The fused image clearly and effectively displays the lesion contour area based on the lesion location information.

[0104] In this embodiment, a medical image processing method is provided, which can be used for medical instruments such as electronic endoscopes. Figure 3 is a flowchart of a specific application example of the medical image processing method according to an embodiment of the present invention. Figure 3 As shown, a specific application example of the medical image processing method may include the following steps:

[0105] Step S301, input the original three-dimensional RGB image image[1080,1920,3].

[0106] In this specific application example of the present invention, the original three-dimensional RGB image image[1080,1920,3] is the first image to be processed. In the original three-dimensional RGB image image[1080,1920,3], 1080 represents the total number of rows of image pixels, 1920 represents the total number of columns of image pixels, and 3 represents the number of channels of the image, which are the color values ​​of the three channels of R, G and B.

[0107] Figure 4 A schematic diagram of an original three-dimensional RGB image is shown as a specific application example of the medical image processing method according to an embodiment of the present invention. Figure 5 FIG. 1 is a schematic diagram of a grayscale image of an original three-dimensional RGB image of a specific application example of the medical image processing method according to an embodiment of the present invention. Figure 4 and Figure 5 The schematic diagram of the original 3D RGB image shown does not highlight the outline of the lesion area in the image.

[0108] Step S302, performing color gamut conversion on the original three-dimensional RGB image image[1080,1920,3] to obtain a three-dimensional YCbCr image image_ycbcr[1080,1920,3].

[0109] For details, please see Figure 1 Step S102 and Figure 2 Step S202 of the illustrated embodiment will not be described in detail here.

[0110] Step S303: extracting the lesion area CB threshold and / or CR threshold for the current medical event from the lesion database.

[0111] Specifically, the lesion database can be a pre-configured database that stores lesion chromaticity intervals for different medical events. Here, the blue concentration offset threshold and the red offset threshold are also referred to as lesion chromaticity intervals. For example, the blue concentration offset threshold threshold_cb∈(100, 110) and the red concentration offset threshold threshold threshold_cr∈(140, 150) for a certain medical event. It should be noted here that the lesion chromaticity interval can also be multiple discontinuous intervals. For example: threshold_cr∈(100, 110) or (130, 140).

[0112] Step S304, determining whether the pixel is in the lesion chromaticity interval.

[0113] In some optional embodiments, the three-dimensional YCbCr image image_ycbcr[1080,1920,3] can be first converted into three two-dimensional images, namely, a two-dimensional cr image image_cr[1080,1920], a two-dimensional cb image image_cb[1080,1920] and a two-dimensional y image image_y[1080,1920], and it is determined in turn whether image_cr[j:i] and image_cb[j:i] belong to the corresponding lesion area CB threshold and CR threshold.

[0114] Specifically, a circular judgment can be used. If 100≤image_cb[j,i]≤110 and 140≤image_cr[j,i]≤150, then BW_ROI[j,i]=1. Where j is the row number of the current pixel, and i is the column number of the current pixel. For example, image_cr[1043,576] is the cr value of the pixel in the 576th column of the 1043th row. In this way, a judgment is made for each pixel in the original 3D RGB image of 1080p. A binary image BW_ROI[1080,1920] is obtained. The value of the pixel position in the lesion chromaticity interval in the binary image is 1, otherwise it is 0. For example, 100≤image_cb[526,343]≤110, 140≤image_cr[526,343]≤150, then BW_ROI[526,343]=1. For another example: 100≤image_cb[526,344]≤110, 150<image_cr[526,344], then BW_ROI[526,344]=0.

[0115] Step S305, obtaining a binary image of the lesion region BW_ROI[1080,1920].

[0116] According to the operation of step S304, after judging each pixel, a binary image BW_ROI[1080,1920] capable of showing the position information of the lesion region can be obtained.

[0117] Step S306, performing image fusion on the binary image BW_ROI[1080,1920] of the lesion region and the original three-dimensional RGB image image[1080,1920,3].

[0118] Specifically, when BW_ROI[j,i]=1, the R, G, and B channel values ​​in the original three-dimensional image image[j,i,3] of the pixel are increased by 100 respectively, and the RGB value of the pixel in the fused image post_image[j,i,3] can be obtained. Specifically, post_image[j,i,3]=image[j,i,3]+100. Thus, the lesion area in the fused image can be prominently displayed on the display device. Where post_image[j,i,3] is the R, G, and B channel values ​​of the pixel in the jth row and the ith column of the fused image post_image[1080,1920,3].

[0119] Specifically, when BW_ROI[j,i]=1, it is also possible to assign a value to only one of the three channel values ​​of R, G and B in the original three-dimensional image image[j,i,3] of the pixel according to actual needs, for example, increase the R value by 100. The RGB value of the pixel in the fused image post_image[j,i,3] can be obtained. Thus, the lesion area in the fused image can also be prominently displayed on the display device.

[0120] In addition, the lesion contour can be first determined according to the binary image, and then the determined lesion contour is fused with the original three-dimensional image to obtain a fused image that can highlight the lesion contour. Here, the lesion contour is first determined according to the binary image, mainly by modifying the BW_ROI[j,i] of the binary image corresponding to the sporadic non-suspected lesion pixels between multiple continuous suspected lesion pixels to 1. And the lesion contour is first determined according to the binary image, mainly by modifying the BW_ROI[j,i] of the binary image corresponding to the sporadic suspected lesion pixels between multiple non-continuous suspected lesion pixels to 0.

[0121] Step S307, outputting a three-dimensional RGB image post_image [1080, 1920, 3] for representing the lesion area.

[0122] Specifically, the binary image BW_ROI[1080,1920] of the lesion region and the original three-dimensional RGB image image[1080,1920,3] can be fused to output post_image[1080,1920,3] to a display screen for display.

[0123] Figure 6 A schematic diagram of a fused image finally obtained in a specific application example of the medical image processing method according to an embodiment of the present invention is shown. Figure 7 FIG. 2 is a schematic diagram of a grayscale image of a fused image finally obtained in a specific application example of the medical image processing method according to an embodiment of the present invention. Figure 6 and Figure 7 As shown, the final fused image can highlight the contour of the lesion area in the image.

[0124] In this embodiment, a medical image processing device is also provided, which is used to implement the above-mentioned embodiment, and the description has been made no further. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0125] This embodiment provides a medical image processing device, such as Figure 8 As shown, including:

[0126] The acquisition module 801 is configured to acquire a first medical image to be processed, where the first medical image to be processed is a three-dimensional image with a first preset color gamut.

[0127] The conversion module 802 is configured to determine a second medical image to be processed according to the first medical image to be processed, where the second medical image to be processed is an image with a second preset color gamut.

[0128] The determination module 803 is configured to obtain, for a set medical event, a binary image for representing lesion location information of the first medical image to be processed for the set medical event based on the second medical image to be processed.

[0129] The fusion module 804 is configured to perform image fusion on the binary image and the first medical image to be processed to obtain a fused image for representing the lesion contour area of ​​the first medical image to be processed for a set medical event.

[0130] In some optional embodiments, the determination module 803 includes: a data acquisition submodule, configured to acquire lesion data for a set medical event from a lesion database, the lesion data being used to show an offset threshold of a second medical image to be processed for the set medical event; and a comparison submodule, configured to compare the second medical image to be processed with the lesion data to obtain a binary image.

[0131] In some optional embodiments, the comparison submodule includes: a suspected pixel determination unit, configured to determine that a pixel is a suspected lesion pixel when the blue concentration offset and / or red concentration offset of the pixel of the second medical image to be processed meets the offset threshold; and an image determination unit, configured to determine a binary image based on multiple suspected lesion pixels.

[0132] In some optional embodiments, the fusion module 804 includes: a first fusion submodule, configured to increase the three color channel values ​​of the pixel in the first medical image to be processed by a first set brightening value when the binary image shows that the pixel is a suspected lesion pixel, so as to obtain a fused image.

[0133] In some optional embodiments, the fusion module 804 includes: a second fusion submodule, configured to increase the color channel value of the pixel in the first medical image to be processed by a second set brightening value when the binary image shows that the pixel is a suspected lesion pixel, so as to obtain a fused image.

[0134] In some optional embodiments, the conversion module 802 includes: a mapping acquisition submodule, configured to acquire a mapping relationship between a first medical image to be processed and a second medical image to be processed, the mapping relationship being used to show a first relationship between the brightness value of the second medical image to be processed and the three-color channel values ​​of the first medical image to be processed, a second relationship between the blue concentration offset and the three-channel values ​​of the first medical image to be processed, and a third relationship between the red concentration offset and the three-color channel values ​​of the first medical image to be processed; a conversion submodule, configured to determine the brightness values, blue concentration offsets, and red concentration offsets of multiple pixels of the second medical image to be processed based on the three-color channel values ​​of the first medical image to be processed and the mapping relationship.

[0135] In some optional embodiments, the apparatus further includes: an output module configured to output a fused image after obtaining the fused image representing the lesion contour area of ​​the first medical image to be processed for the set medical event.

[0136] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0137] The medical image processing device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0138] The embodiment of the present invention also provides a computer device having the above Fig. 9 The medical image processing device shown.

[0139] See also Fig. 9 , Fig. 9 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Fig. 9 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Fig. 9 A processor 10 is taken as an example.

[0140] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0141] The memory 20 stores instructions executable by at least one processor 10, so that at least one processor 10 executes the method shown in the above embodiment.

[0142] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0143] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0144] The computer device also includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 may be connected via a bus or other means. Fig. 9 The example of connecting through bus is taken in the following.

[0145] The input device 30 may receive the first image to be processed acquired by the image acquisition device of the electronic endoscope, etc. The output device 40 may include a display device, etc., for displaying the fused image output by the above-mentioned medical image processing method, etc. The above-mentioned display device includes but is not limited to a liquid crystal display, etc. In some optional embodiments, the display device may be a touch screen.

[0146] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0147] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0148] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A medical image processing method, characterized in that: The method comprises: Acquire a first medical image to be processed, where the first medical image to be processed is a three-dimensional image having a first preset color gamut; Determine a second medical image to be processed according to the first medical image to be processed, wherein the second medical image to be processed is an image having a second preset color gamut; For a set medical event, based on the second medical image to be processed, obtaining a binary image for representing lesion location information of the first medical image to be processed for the set medical event; The binary image is fused with the first medical image to be processed to obtain a fused image for representing a lesion contour area of ​​the first medical image to be processed for the set medical event.

2. The medical image processing method according to claim 1, characterized in that: The step of obtaining, based on the second medical image to be processed, a binary image for representing the lesion location information of the first medical image to be processed for the set medical event comprises: Acquire lesion data for the set medical event from a lesion database, wherein the lesion data is used to show an offset threshold of the second to-be-processed medical image for the set medical event; The second medical image to be processed is compared with the lesion data to obtain the binary image.

3. The medical image processing method according to claim 2, characterized in that: The step of comparing the second medical image to be processed with the lesion data to obtain the binary image includes: In the case where it is determined that the blue density offset and / or the red density offset of the pixel of the second medical image to be processed meets the offset threshold, determining that the pixel is a suspected lesion pixel; The binary image is determined according to the plurality of suspected lesion pixels.

4. The medical image processing method according to claim 1, characterized in that: The step of fusing the binary image with the first medical image to be processed to obtain a fused image for representing a lesion contour area of ​​the first medical image to be processed for the set medical event includes: When the binary image shows that a pixel is a suspected lesion pixel, the three color channel values ​​of the pixel in the first medical image to be processed are respectively increased by a first set brightening value to obtain the fused image.

5. The medical image processing method according to claim 1, characterized in that: The step of fusing the binary image with the first medical image to be processed to obtain a fused image for representing a lesion contour area of ​​the first medical image to be processed for the set medical event includes: When the binary image shows that a pixel is a suspected lesion pixel, the color channel value of the pixel in the first medical image to be processed is set to be increased by a second set brightening value to obtain the fused image.

6. The medical image processing method according to claim 1, characterized in that: The step of determining the second medical image to be processed according to the first medical image to be processed includes: Acquire a mapping relationship between the first to-be-processed medical image and the second to-be-processed medical image, the mapping relationship being used to show a first relationship between a brightness value of the second to-be-processed medical image and three-color channel values ​​of the first to-be-processed medical image, a second relationship between a blue concentration offset and the three-channel values ​​of the first to-be-processed medical image, and a third relationship between a red concentration offset and the three-color channel values ​​of the first to-be-processed medical image; According to the three color channel values ​​of the first medical image to be processed and the mapping relationship, the brightness values, blue concentration offsets, and red concentration offsets of multiple pixels of the second medical image to be processed are determined.

7. A medical image processing device, characterized in that: The device comprises: An acquisition module is configured to acquire a first medical image to be processed, where the first medical image to be processed is a three-dimensional image with a first preset color gamut; a conversion module, configured to determine a second medical image to be processed according to the first medical image to be processed, wherein the second medical image to be processed is an image having a second preset color gamut; a determination module configured to obtain, for a set medical event, a binary image for representing lesion location information of the first medical image to be processed for the set medical event based on the second medical image to be processed; The fusion module is configured to perform image fusion on the binary image and the first medical image to be processed to obtain a fused image for representing a lesion contour area of ​​the first medical image to be processed for the set medical event.

8. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the medical image processing method according to any one of claims 1 to 6 by executing the computer instructions.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the medical image processing method according to any one of claims 1 to 7.

10. A computer program product, characterized in that The method comprises computer instructions for causing a computer to execute the medical image processing method according to any one of claims 1 to 6.

Images