Medical endoscope equipment, image processing method and device and storage medium

Abstract

The invention provides medical endoscope equipment, an image processing method and device and a storage medium, and belongs to the technical field of computers. A highlight mask corresponding to an initial image can be determined, and a region type corresponding to each highlight region can be determined based on the number of pixels included in each highlight region of the highlight mask. In the embodiment of the invention, based on the restoration mode corresponding to the region type, the highlight region corresponding to any region type can be restored to obtain the restored image corresponding to any region type, and based on the obtained restored image corresponding to each region type, the target image can be generated. In the target image, the highlight area in the initial image is repaired, and a doctor can identify a lesion conveniently.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a medical endoscope device, an image processing method, a device and a storage medium. Background Art

[0002] There is no light source inside the human body. When using medical endoscope equipment for endoscopic surgery, an external light source needs to be taken into the body to obtain a viewable image on the display screen for surgical operations.

[0003] However, during the operation, reflections from body fluids and blood in the human body, as well as the surgical instruments used by the doctor, can cause bright areas to appear in the image, affecting the doctor's ability to identify lesions. Summary of the invention

[0004] In order to solve the above-mentioned problems in the prior art, the embodiments of the present application provide a medical endoscope device, an image processing method, an apparatus and a storage medium, which can repair the highlight area in the image and help doctors identify lesions.

[0005] In a first aspect, an embodiment of the present application provides a medical endoscope device, including: a display, a camera, and a camera host;

[0006] The camera is configured to: collect an initial image;

[0007] The camera host is configured to: determine a highlight mask corresponding to the acquired initial image;

[0008] Determining the region type corresponding to each highlight region based on the number of pixels included in each highlight region of the highlight mask;

[0009] For any region type, based on the restoration method corresponding to the region type, restoration processing is performed on the highlight region corresponding to the region type to obtain a restoration image corresponding to the region type;

[0010] Generate a target image based on the obtained restoration images corresponding to each region type;

[0011] The display is configured to display the target image.

[0012] In a possible implementation manner, the camera host is specifically configured as follows:

[0013] Determine a highlight image based on a maximum pixel value of each pixel in the initial image in the R channel, the G channel, and the B channel, and determine a low-brightness image based on a minimum pixel value of each pixel in the initial image in the R channel, the G channel, and the B channel;

[0014] Based on the highlight image and the low-brightness image, a highlight mask corresponding to the initial image is determined.

[0015] In a possible implementation manner, the camera host is specifically configured as follows:

[0016] The ratio of the pixel values ​​of the pixel points at the same position in the low-brightness image and the high-brightness image is used as the enhancement factor of the same position;

[0017] Determine new pixel values ​​at each position based on the pixel values ​​at each position in the initial image and the enhancement factors corresponding to each position, and obtain an intermediate image based on the new pixel values ​​at each position;

[0018] Converting the intermediate image from RGB color space to XYZ color space;

[0019] According to the pixel values ​​of each pixel in the intermediate image in the X channel, the Y channel and the Z channel, a highlight pixel is determined, and the pixel value of the highlight pixel is set to a first preset value, and the pixel values ​​of the remaining pixels except the highlight pixel are set to a second preset value, so as to obtain the highlight mask.

[0020] In a possible implementation manner, the camera host is specifically configured as follows:

[0021] Determine the sum of the pixel values ​​of each pixel point in the X channel, the Y channel and the Z channel;

[0022] If the sum of the pixel values ​​of any pixel point in the X channel, the Y channel and the Z channel is greater than or equal to the set threshold, the any pixel point is determined to be a highlight pixel point.

[0023] In a possible implementation manner, the camera host is specifically configured as follows:

[0024] Determine that the area type of the highlight area whose number of pixels is less than a set number threshold is a small area highlight type;

[0025] Determine that the region type of the highlight region whose pixel quantity is greater than or equal to the set quantity threshold is a large-area highlight type.

[0026] In a possible implementation manner, the camera host is specifically configured as follows:

[0027] For each first highlight region of the small-area highlight type, a restoration process and an inversion process are sequentially performed using a centroid filling restoration method to obtain a first restoration image;

[0028] For each second highlight region of the large-area highlight type, a restoration process is performed using a restoration method of reducing brightness to obtain a second restored image.

[0029] In a possible implementation manner, the camera host is specifically configured as follows:

[0030] The fused image of the repaired images corresponding to the various region types is used as the target image.

[0031] In a second aspect, an embodiment of the present application provides an image processing method, which is applied to a medical endoscope device, and the method includes:

[0032] Determine a highlight mask corresponding to the acquired initial image;

[0033] Determining the region type corresponding to each highlight region based on the number of pixels included in each highlight region of the highlight mask;

[0034] For any region type, based on the restoration method corresponding to the region type, restoration processing is performed on the highlight region corresponding to the region type to obtain a restoration image corresponding to the region type;

[0035] Based on the obtained restoration images corresponding to each region type, a target image is generated.

[0036] In a third aspect, an embodiment of the present application provides an image processing device, including:

[0037] A mask determination unit, used to determine a highlight mask corresponding to the acquired initial image;

[0038] The repair unit is used to determine the area type corresponding to each highlight area based on the number of pixels included in each highlight area of ​​the highlight mask, and for any area type, based on the repair method corresponding to the any area type, repair the highlight area corresponding to the any area type to obtain a repaired image corresponding to the any area type, and generate a target image based on the repaired images corresponding to each area type.

[0039] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a camera host, the method described in the second aspect is implemented.

[0040] The medical endoscope equipment, image processing method, device and storage medium provided by the embodiments of the present application can determine the highlight mask corresponding to the initial image, and based on the number of pixels included in each highlight area of ​​the highlight mask, the area type corresponding to each highlight area can be determined. Based on the repair method corresponding to the area type, the highlight area corresponding to any area type can be repaired to obtain a repaired image corresponding to any area type, and a target image can be generated based on the repaired images corresponding to each area type. In the target image, the highlight area in the initial image is repaired, which is helpful for doctors to identify lesions. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0042] Figure 1 A schematic diagram of the structure of a medical endoscope device provided in an embodiment of the present application;

[0043] Figure 2 A schematic diagram of a medical endoscope device diagnosis provided in an embodiment of the present application;

[0044] Figure 3 A flowchart of an image processing method provided in an embodiment of the present application;

[0045] Figure 4 A flowchart of an image processing method provided in an embodiment of the present application;

[0046] Figure 5 A flowchart of an image processing method provided in an embodiment of the present application;

[0047] Figure 6 A schematic diagram of a highlight mask provided in an embodiment of the present application;

[0048] Figure 7 A flowchart of an image processing method provided in an embodiment of the present application;

[0049] Figure 8 A schematic diagram of a pixel point that has not been expanded provided in an embodiment of the present application;

[0050] Fig. 9 A schematic diagram of a pixel point that has been expanded provided in an embodiment of the present application;

[0051] Fig.10 A schematic diagram of another pixel point that has undergone expansion processing provided in an embodiment of the present application;

[0052] Fig.11 A schematic diagram of a connected domain image provided in an embodiment of the present application;

[0053] Fig.12 A schematic diagram of an endoscopic image provided in an embodiment of the present application;

[0054] Fig.13 A schematic diagram of a connected domain image corresponding to an endoscopic image provided in an embodiment of the present application;

[0055] Fig.14 A flowchart of an image processing method provided in an embodiment of the present application;

[0056] Fig.15 A flowchart of an image processing method provided in an embodiment of the present application;

[0057] Fig.16 A schematic diagram of adjusting brightness provided in an embodiment of the present application;

[0058] Fig.17 A schematic diagram of an image processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solutions and advantages of this application clearer, this application will be further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0060] It should be noted that the application scenarios described in the following embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. A person of ordinary skill in the art can know that with the emergence of new application scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0061] There is no light source inside the human body. When doctors use medical endoscopes for endoscopic surgery, they need to use external light sources to assist in diagnosis. However, during the operation, the mucosal tissue in the cavity is smooth and there are body fluids and blood that reflect light, which will cause highlight areas to appear in the endoscopic image. In addition, the surgical instruments used by doctors can also cause reflections, resulting in highlight areas in the endoscopic image. These highlight areas are very unfavorable for doctors to perform surgery. Not only will they affect doctors' identification of lesions, but they can also easily cause visual fatigue.

[0062] Based on this, the embodiments of the present application provide a medical endoscope device, an image processing method, an apparatus and a storage medium. The highlight mask corresponding to the initial image can be determined, and based on the number of pixels included in each highlight area of ​​the highlight mask, the region type corresponding to each highlight area can be determined. Based on the repair method corresponding to the region type, the highlight area corresponding to any region type can be repaired to obtain a repaired image corresponding to any region type, and a target image can be generated based on the repaired images corresponding to each region type. In the target image, the highlight area in the initial image is repaired, which is beneficial for doctors to identify lesions and prevent doctors from visual fatigue.

[0063] Figure 1 The schematic diagram of the structure of a medical endoscope device in an embodiment of the present application is exemplarily shown, wherein the camera host is a device for photographing endoscopic examinations and surgeries to display the images of the examinations and surgeries, and its key components include a camera, a button, a camera host, a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor, a video cable, an optical adapter, a mirror rod, a communication port, and a video output interface. The camera host can be connected to the display through the video output interface, and connected to the cold light source host through the communication port. The cold light source is a device that provides illumination light to the endoscope during endoscopic examinations and surgeries, and its key components generally include: a cold light source host, a light guide, and a cold light source light output interface.

[0064] Combination Figure 2 As shown, in the actual operation stage, medical endoscope equipment inspection or surgery basically uses dynamic video to assist the doctor's operation. After the doctor operates the medical endoscope equipment to enter the human body, the image of the human body is collected and displayed through the camera on the medical endoscope equipment, and as the camera of the medical endoscope equipment moves, the range illuminated by the camera will be presented on the display connected to the medical endoscope equipment. The environment around the lesion can be clearly seen, and with the help of surgical instruments, movement and surgical operations can be performed in three dimensions and intuitively.

[0065] Figure 3 A flowchart of an image processing method provided by an embodiment of the present application is shown. The method can be applied to Figure 1 In the medical endoscope device shown, Figure 3 As shown, the method may include the following steps:

[0066] Step S301, determining a highlight mask corresponding to the acquired initial image.

[0067] During surgery using an endoscope, the reflection of body fluids and blood in the mucosal tissue will cause small areas of highlight, and the reflection of metal surgical instruments will cause large areas of highlight. Before repairing these highlight areas, it is necessary to determine the highlight masks corresponding to these highlight areas.

[0068] In an optional implementation, a method for determining a highlight mask corresponding to the acquired initial image is as follows: Figure 4 As shown, the following steps may be included:

[0069] Step S401, determining a highlight image based on the maximum pixel value of each pixel in the initial image in the R channel, the G channel and the B channel, and determining a low-brightness image based on the minimum pixel value of each pixel in the initial image in the R channel, the G channel and the B channel.

[0070] After obtaining the initial image, the maximum pixel value of each pixel in the initial image in the R channel, the G channel and the B channel can be determined, and the maximum pixel value of each pixel in the initial image in the R channel, the G channel and the B channel can be used as a new pixel value to obtain a highlight image.

[0071] For example, assuming that the pixel value of a pixel in the initial image in the RGB color space is (50, 100, 150), the pixel value of the pixel at the same position in the highlight image is 150.

[0072] Furthermore, after obtaining the highlight image, the minimum pixel value of each pixel in the initial image in the R channel, G channel and B channel can be determined, and the minimum pixel value of each pixel in the R channel, G channel and B channel can be used as a new pixel value to obtain a low-brightness image.

[0073] For example, assuming that the pixel value of a certain pixel point in the initial image in the RGB color space is (50, 100, 150), the pixel value of the pixel point at the same position in the low-brightness image is 50.

[0074] Step S402: The ratio of the pixel values ​​of the pixels at the same position in the low-brightness image and the high-brightness image is used as an enhancement factor at the same position.

[0075] After obtaining the low-brightness image and the high-brightness image corresponding to the initial image, the ratio of the pixel values ​​of the pixel points at the same position in the low-brightness image and the high-brightness image can be used as the enhancement factor at the same position.

[0076] In an optional implementation, before determining the enhancement factor, in order to facilitate data processing, the pixel values ​​of the pixels in the low-brightness image and the highlight image may be normalized, that is, the pixel values ​​of the pixels are converted into decimals in the interval (0,1), and then the ratio of the pixel values ​​of the pixels at the same position in the low-brightness image and the highlight image is used as the enhancement factor for the same position.

[0077] For example, assuming that the pixel value of a certain pixel in the low-brightness image is 0.2 after normalization, and the pixel value of the pixel at the same position in the high-brightness image is 0.8 after normalization, the enhancement factor corresponding to the position is 0.2÷0.8=0.25.

[0078] Step S403, based on the pixel value of each position in the initial image and the enhancement factor corresponding to each position, determine the new pixel value of each position, and obtain the intermediate image based on the new pixel value of each position.

[0079] In an optional embodiment, after determining the enhancement factor corresponding to each position in the initial image, the product of the pixel value of each position in the initial image and the corresponding enhancement factor can be used as the new pixel value of each position, and an intermediate image can be obtained based on the new pixel value of each position.

[0080] For example, the pixels in the initial image may be normalized before determining the intermediate image. Assuming that the normalized pixel value of a pixel in the initial image is (0.8, 0.6, 0.2), and the enhancement factor corresponding to the pixel is 0.25, the new pixel value of the pixel is (0.8, 0.6, 0.2) × 0.25 = (0.2, 0.15, 0.05).

[0081] After determining the new pixel value of each position in the initial image using the above method, an intermediate image can be obtained. In the intermediate image, the distinction between the highlight area and the low-brightness area can be enlarged to facilitate the subsequent more accurate determination of the highlight mask.

[0082] Step S404: convert the intermediate image from the RGB color space to the XYZ color space.

[0083] In an optional implementation, after the intermediate image is obtained, the intermediate image may be converted from the RGB color space to the XYZ color space.

[0084] Exemplarily, the intermediate image may be converted from the RGB color space to the XYZ color space using formula (1), where formula (1) may be expressed as:

[0085]

[0086] Step S405, according to the pixel values ​​of each pixel in the intermediate image in the X channel, the Y channel and the Z channel, the highlight pixel points are determined, the pixel values ​​of the highlight pixel points are set to the first preset value, and the pixel values ​​of the remaining pixel points except the highlight pixel points are set to the second preset value, so as to obtain a highlight mask.

[0087] In an optional implementation, after the intermediate image is converted into the XYZ color space, the highlight pixel point can be determined according to the pixel value of each pixel point in the intermediate image in the X channel, the Y channel and the Z channel, such as Figure 5 As shown, the method for determining the highlight pixel point may include the following steps:

[0088] Step S501, determining the sum of the pixel values ​​of each pixel point in the X channel, the Y channel and the Z channel.

[0089] After obtaining the pixel values ​​of each pixel in the intermediate image in the X channel, the Y channel, and the Z channel, the sum of the pixel values ​​of each pixel in the X channel, the Y channel, and the Z channel can be determined first, and then according to the size of the sum of the pixel values ​​of each pixel in the X channel, the Y channel, and the Z channel, it is determined whether each pixel is a highlight pixel.

[0090] Step S502: If the sum of the pixel values ​​of any pixel in the X channel, the Y channel and the Z channel is greater than or equal to a set threshold, then the pixel is determined to be a highlight pixel.

[0091] The threshold value may be set to 1.

[0092] For example, if the sum of the pixel values ​​of a certain pixel in the intermediate image in the X channel, Y channel, and Z channel is 0.8, it can be determined that the pixel is not a highlight pixel. If the sum of the pixel values ​​of a certain pixel in the intermediate image in the X channel, Y channel, and Z channel is 1, it can be determined that the pixel is a highlight pixel. If the sum of the pixel values ​​of a certain pixel in the intermediate image in the X channel, Y channel, and Z channel is 1.5, it can be determined that the pixel is a highlight pixel.

[0093] Furthermore, after determining the highlight pixel points in the intermediate image, the pixel values ​​of the highlight pixel points may be set to a first preset value, and the pixel values ​​of the remaining pixel points except the highlight pixel points may be set to a second preset value, so as to obtain a highlight mask.

[0094] Exemplarily, the first preset value may be 1, and the second preset value may be 0.

[0095] Figure 6 A schematic diagram of a highlight mask provided in an embodiment of the present application is shown, Figure 6 As shown, the white area in the figure is the highlighted area that needs to be repaired.

[0096] Step S302 : determining the region type corresponding to each highlight region based on the number of pixels included in each highlight region of the highlight mask.

[0097] In an optional implementation, after obtaining the highlight mask, each highlight region may be determined based on the highlight mask, such as Figure 7 As shown, the method for determining each highlight area may include the following steps:

[0098] Step S701, dilating the highlight mask with a radius of A to obtain a first dilated image.

[0099] For example, the value of A can be 2.

[0100] Figure 8 FIG. 1 shows a schematic diagram of a pixel point that has not been dilated provided by an embodiment of the present application. Further, after the highlight mask is dilated with a radius of 2, the first dilated image corresponding to the pixel point is as follows: Fig. 9 shown.

[0101] Step S702: dilate the highlight mask with a radius of B to obtain a second dilated image.

[0102] Among them, the value of B is greater than the value of A.

[0103] For example, the value of B may be 4. The highlight mask is expanded with a radius of 4, and the second expanded image corresponding to the above pixel point is as follows: Fig.10 shown.

[0104] Step S703: determine whether a difference image between the second dilated image and the first dilated image is a connected domain image.

[0105] Among them, the connected domain image is Fig.11 As shown, multiple white pixels form a closed curve, and the connected area inside the closed curve is a highlighted area.

[0106] Step S704: determine that the connected domain in the connected domain image is a highlight area.

[0107] Fig.12 A schematic diagram of an endoscopic image provided by an embodiment of the present application is shown, and the connected domain image corresponding to the image is as follows: Fig.13 As shown in Figure 2, the area inside the white closed curve is the highlighted area.

[0108] Furthermore, after determining each highlight region, the region type corresponding to each highlight region may be determined according to the number of pixels included in each highlight region.

[0109] Specifically, the region type of the highlight region whose number of pixels is less than a set number threshold may be determined as a small area highlight type, and the region type of the highlight region whose number of pixels is greater than or equal to the set number threshold may be determined as a large area highlight type. For example, the set number threshold may be 80, or 100.

[0110] For example, assuming that the number threshold is set to 80, if the number of pixels contained in a certain highlight area is 60, then the area type of the highlight area can be determined to be a small area highlight type. If the number of pixels contained in a certain highlight area is 120, then the area type of the highlight area can be determined to be a large area highlight type.

[0111] It should be noted that the above-mentioned set quantity threshold is only used as an example, and the present application does not impose any limitation on the size of the set quantity threshold. For example, the set quantity threshold can also be 150 or 200.

[0112] Step S303: for any region type, based on the restoration method corresponding to any region type, restoration processing is performed on the highlight region corresponding to any region type to obtain a restoration image corresponding to any region type.

[0113] For example, for each first highlight region of the small-area highlight type, a centroid filling repair method can be used to perform repair processing and inversion processing in sequence to obtain a first repaired image, such as Fig.14 As shown, the method for determining the first repaired image may include the following steps:

[0114] Step S1401, setting the pixel values ​​of the pixels to be repaired in each first highlight region to 1, and setting the pixel values ​​of the remaining pixels except the pixels in each first highlight region to 0, to obtain a first mask.

[0115] In an optional implementation, all pixels to be repaired in each first highlight area can be traversed, the pixel values ​​of all the pixels to be repaired are set to 1, and the pixel values ​​of the remaining pixels except the pixels in each first highlight area are set to 0 to obtain a first mask.

[0116] Step S1402, for each pixel to be repaired in the first mask, perform the following operations to obtain a filling image corresponding to each first highlight area: determine the target highlight area that is closest to the pixel to be repaired, determine the target pixel value of the centroid position of the target highlight area in the corresponding pixel of the initial image, and use the target pixel value to fill the pixel to be repaired.

[0117] After obtaining the first mask, it is necessary to use the centroid filling method to repair each pixel to be repaired in the first mask. The following is an example of using the centroid filling method to repair a pixel to be repaired.

[0118] Specifically, the coordinates of the centroid position of each first highlight area and the pixel value of the centroid position in the initial image can be determined first. Further, the coordinates (x, y) of the pixel to be repaired can be determined, and then the target highlight area closest to the pixel to be repaired can be determined, and then the target pixel value of the centroid position of the target highlight area at the same position in the initial image can be determined, and finally, the pixel to be repaired is filled with the target pixel value, and the repair of the pixel to be repaired is completed.

[0119] For example, the distance between the pixel to be repaired and any first highlight area can be determined using formula (2), which can be expressed as:

[0120] D=sqrt((x-x1)^2+(y-y1)^2)————Formula (2)

[0121] Wherein, D represents the distance between the pixel to be repaired and the highlight area, x is the horizontal coordinate of the pixel to be repaired, y is the vertical coordinate of the pixel to be repaired, x1 is the horizontal coordinate of the centroid of any first highlight area, and y1 is the vertical coordinate of the centroid of any first highlight area.

[0122] After repairing each pixel to be repaired in the first mask, a filling image corresponding to each first highlight area can be obtained.

[0123] Step S1403, invert the filled image to obtain a first repaired image.

[0124] In an optional implementation, before the filling image is inverted, a difference image between the highlight mask and the first mask may be used as a second mask, and then the filling image is inverted according to the second mask to obtain a first repaired image.

[0125] For example, the filled image can be inverted using formula (3), which can be expressed as:

[0126] img1=(1-large_mask)*filled_img————Formula (3)

[0127] Among them, img1 is the first repaired image, large_mask is the second mask, and filled_img is the filled image.

[0128] So far, the repair work for each first highlight region of the small-area highlight type has been completed.

[0129] Furthermore, for each second highlight region of the large-area highlight type, a restoration method of reducing brightness may be used to perform restoration processing to obtain a second restored image. The restoration method may include: Fig.15The following steps are shown:

[0130] Step S1501: taking a product image of the second mask and the initial image as a second image to be restored.

[0131] Exemplarily, the second image to be restored high_img can be expressed by formula (4), which can be expressed as:

[0132] high_img=large_mask*img————Formula (4)

[0133] Among them, high_img is the second image to be repaired, large_mask is the second mask, and img is the initial image.

[0134] Step S1502, reducing the brightness of the second image to be restored to obtain a second restored image.

[0135] In an optional implementation, before reducing the brightness of the second image to be restored, the second image to be restored may be converted from an RGB color space to an HSV color space.

[0136] Exemplarily, the following formula may be used to convert the second image to be restored from the RGB color space to the HSV color space:

[0137] V=max(R,G,B) / 255

[0138]

[0139]

[0140] After converting the second image to be restored into the HSV color space, the brightness value V of the second image to be restored may be reduced. For example, the brightness value may be adjusted to 0.9 times of the initial brightness value.

[0141] Exemplarily, the gradient of brightness value adjustment can be selected by the user.

[0142] like Fig.16 As shown, the target brightness value entered by the user in the target brightness input box can be obtained. For example, assuming that the target brightness value entered by the user is 9, the brightness value can be adjusted to 0.9 times the initial brightness value. For another example, assuming that the target brightness value entered by the user is 8, the brightness value can be adjusted to 0.8 times the initial brightness value.

[0143] After adjusting the brightness of the second image to be restored, the second image to be restored may be converted from the HSV color space to the RGB color space to obtain a second restored image img2.

[0144] Exemplarily, the following formula may be used to convert the second image to be restored from the HSV color space to the RGB color space:

[0145] C=V′×S′

[0146]

[0147] m=V′-C

[0148]

[0149] (R′, G′, B′) = ((R″+m)×255, (G″+m)×255, (B″+m)×255)

[0150] At this point, the repair work for each second highlight area of ​​the large-area highlight type has been completed.

[0151] Step S304: generating a target image based on the obtained restoration images corresponding to each region type.

[0152] In an optional implementation, a fused image of the restored images corresponding to each region type may be used as the target image.

[0153] Specifically, the first repaired image corresponding to the small-area highlight type and the second repaired image corresponding to the large-area highlight type may be fused to obtain a target image.

[0154] For example, the target image final_img can be expressed by formula (5), which can be expressed as:

[0155] final_img=img1+img2————Formula (5)

[0156] Among them, final_img is the target image, img1 is the first repaired image, and img2 is the second repaired image.

[0157] Furthermore, after the target image is generated, in order to remove the noise of the target image and make the target image smoother, the target image can be filtered using formula (6) to obtain a filtered image gaussian_filtered_img after filtering, where formula (6) can be expressed as:

[0158]

[0159] Among them, the standard deviation σ can be 4, μ x and μ y Can be 0.

[0160] Furthermore, the highlight mask may be subjected to a full 1 / 20 filter of a size of 10*10 to obtain a filtered highlight mask final_mask.

[0161] After filtering the target image and highlight mask, the filtered target image can be determined using the following formula:

[0162] inpainted_img=final_mask*gaussian_filtered_img+(1-final_mask)*img

[0163] Among them, inpainted_img is the filtered target image, final_mask is the filtered highlight mask final_mask, gaussian_filtered_img is the filtered image, and img is the initial image.

[0164] The target image after filtering is smoother and can present better visual effects. While displaying the real color feature image, it can prevent doctors from visual fatigue and improve user experience.

[0165] An image processing method provided by an embodiment of the present application can perform repair processing on a highlight area corresponding to any area type based on a repair method corresponding to an area type, obtain a repaired image corresponding to any area type, and generate a target image based on the repaired images corresponding to each area type obtained. In the target image, various types of highlight areas in the initial image are repaired, which can assist doctors in identifying lesions and avoid visual fatigue.

[0166] Based on the same inventive concept, an image processing device is also provided in the embodiment of the present application, such as Fig.17 As shown, the image processing device comprises:

[0167] The mask determination unit 1701 is used to determine the highlight mask corresponding to the acquired initial image;

[0168] The repair unit 1702 is used to determine the region type corresponding to each highlight region based on the number of pixels included in each highlight region of the highlight mask, and for any region type, based on the repair method corresponding to any region type, repair the highlight region corresponding to any region type to obtain a repaired image corresponding to any region type, and generate a target image based on the repaired images corresponding to each region type.

[0169] In a possible implementation manner, the mask determination unit 1701 is specifically configured to:

[0170] Based on the maximum pixel value of each pixel point in the initial image in the R channel, the G channel and the B channel, a highlight image is determined, and based on the minimum pixel value of each pixel point in the initial image in the R channel, the G channel and the B channel, a low-brightness image is determined;

[0171] Based on the highlight image and the low-brightness image, a highlight mask corresponding to the initial image is determined.

[0172] In a possible implementation manner, the mask determination unit 1701 is specifically configured to:

[0173] The ratio of the pixel values ​​of the pixels at the same position in the low-brightness image and the high-brightness image is used as the enhancement factor at the same position;

[0174] Based on the pixel value of each position in the initial image and the enhancement factor corresponding to each position, a new pixel value of each position is determined, and an intermediate image is obtained based on the new pixel value of each position;

[0175] Convert the intermediate image from RGB color space to XYZ color space;

[0176] According to the pixel values ​​of each pixel in the intermediate image in the X channel, the Y channel and the Z channel, the highlight pixel is determined, and the pixel value of the highlight pixel is set to the first preset value, and the pixel values ​​of the remaining pixels except the highlight pixel are set to the second preset value, so as to obtain a highlight mask.

[0177] In a possible implementation manner, the mask determination unit 1701 is specifically configured to:

[0178] Determine the sum of the pixel values ​​of each pixel in the X channel, Y channel and Z channel;

[0179] If the sum of the pixel values ​​of any pixel in the X channel, Y channel and Z channel is greater than or equal to the set threshold, then any pixel is determined to be a highlight pixel.

[0180] In a possible implementation, the repair unit 1702 is specifically configured to:

[0181] Determine that the area type of the highlight area whose pixel number is less than the set number threshold is a small area highlight type;

[0182] It is determined that the region type of the highlight region whose pixel quantity is greater than or equal to the set quantity threshold is a large area highlight type.

[0183] In a possible implementation, the repair unit 1702 is specifically configured to:

[0184] For each first highlight region of the small-area highlight type, a centroid filling repair method is used to perform repair processing and inversion processing in sequence to obtain a first repaired image;

[0185] For each second highlight region of the large-area highlight type, a restoration process is performed using a restoration method of reducing brightness to obtain a second restored image.

[0186] In a possible implementation, the repair unit 1702 is specifically configured to:

[0187] The fused image of multiple repaired images corresponding to the multiple region types is used as the target image.

[0188] An embodiment of the present application further provides a computer storage medium, in which computer executable instructions are stored. The computer executable instructions are used to implement the image processing method described in any embodiment of the present application.

[0189] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0190] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0191] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0192] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0193] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A medical endoscope device, characterized in that: include: Display, camera and camera host; The camera is configured to: collect an initial image; The camera host is configured to: determine a highlight mask corresponding to the acquired initial image; Determining the region type corresponding to each highlight region based on the number of pixels included in each highlight region of the highlight mask; For any region type, based on the restoration method corresponding to the region type, restoration processing is performed on the highlight region corresponding to the region type to obtain a restoration image corresponding to the region type; Generate a target image based on the obtained restoration images corresponding to each region type; The display is configured to display the target image.

2. The medical endoscope device according to claim 1, characterized in that: The camera host is specifically configured as follows: Determine a highlight image based on a maximum pixel value of each pixel in the initial image in the R channel, the G channel, and the B channel, and determine a low-brightness image based on a minimum pixel value of each pixel in the initial image in the R channel, the G channel, and the B channel; Based on the highlight image and the low-brightness image, a highlight mask corresponding to the initial image is determined.

3. The medical endoscope device according to claim 2, characterized in that: The camera host is specifically configured as follows: The ratio of the pixel values ​​of the pixel points at the same position in the low-brightness image and the high-brightness image is used as the enhancement factor of the same position; Determine new pixel values ​​at each position based on the pixel values ​​at each position in the initial image and the enhancement factors corresponding to each position, and obtain an intermediate image based on the new pixel values ​​at each position; Converting the intermediate image from RGB color space to XYZ color space; According to the pixel values ​​of each pixel in the intermediate image in the X channel, the Y channel and the Z channel, a highlight pixel is determined, and the pixel value of the highlight pixel is set to a first preset value, and the pixel values ​​of the remaining pixels except the highlight pixel are set to a second preset value, so as to obtain the highlight mask.

4. The medical endoscope device according to claim 3, characterized in that: The camera host is specifically configured as follows: Determine the sum of the pixel values ​​of each pixel point in the X channel, the Y channel and the Z channel; If the sum of the pixel values ​​of any pixel point in the X channel, the Y channel and the Z channel is greater than or equal to the set threshold, the any pixel point is determined to be a highlight pixel point.

5. The medical endoscope device according to claim 1, characterized in that: The camera host is specifically configured as follows: Determine that the area type of the highlight area whose number of pixels is less than a set number threshold is a small area highlight type; Determine that the region type of the highlight region whose pixel quantity is greater than or equal to the set quantity threshold is a large-area highlight type.

6. The medical endoscope device according to claim 5, characterized in that: The camera host is specifically configured as follows: For each first highlight region of the small-area highlight type, a restoration process and an inversion process are sequentially performed using a centroid filling restoration method to obtain a first restoration image; For each second highlight region of the large-area highlight type, a restoration process is performed using a restoration method of reducing brightness to obtain a second restored image.

7. The medical endoscope device according to any one of claims 1 to 6, characterized in that: The camera host is specifically configured as follows: The fused image of the repaired images corresponding to the various region types is used as the target image.

8. An image processing method, characterized in that: Applied to medical endoscope equipment, the method comprises: Determine a highlight mask corresponding to the acquired initial image; Determining the region type corresponding to each highlight region based on the number of pixels included in each highlight region of the highlight mask; For any region type, based on the restoration method corresponding to the region type, restoration processing is performed on the highlight region corresponding to the region type to obtain a restoration image corresponding to the region type; Based on the obtained restoration images corresponding to each region type, a target image is generated.

9. An image processing device, characterized in that: The device comprises: A mask determination unit, used to determine a highlight mask corresponding to the acquired initial image; The repair unit is used to determine the area type corresponding to each highlight area based on the number of pixels included in each highlight area of ​​the highlight mask, and for any area type, based on the repair method corresponding to the any area type, repair the highlight area corresponding to the any area type to obtain a repaired image corresponding to the any area type, and generate a target image based on the repaired images corresponding to each area type.

10. A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, characterized in that: When the computer program is executed by a camera host, the method described in claim 8 is implemented.

Images