An image processing method, device and computer readable storage medium

Abstract

Embodiments of the present application disclose an image processing method and device and a computer readable storage medium, and relate to the field of image processing. The method comprises: obtaining a sample image and a sample enhanced image for a target part in a human body; the sample enhanced image is obtained by performing image enhancement on the sample image; training based on the sample image and the sample enhanced image to determine a target weight function; determining a spatial transformation parameter of a collection component that collects the sample image; determining an image enhancement model based on the target weight function and the spatial transformation parameter; obtaining a to-be-processed image for the target part in the human body, and determining a target image for the target part in the human body based on the to-be-processed image and the image enhancement model; in this way, the visual effect of the target image is improved, and the accuracy of extracting lesion features from the target image is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the image processing technology in the field of image processing, and in particular to an image processing method, device and computer readable storage medium. BACKGROUND

[0002] At present, it is a mature medical technology to collect the stomach image in the human body through a gastroscope and determine whether the stomach of the human body is abnormal through the stomach image. However, the brightness of the stomach image collected by the gastroscope is not uniform, which leads to low accuracy of extracting lesion features from the stomach image. SUMMARY

[0003] To solve the above technical problems, the embodiments of the present application provide an image processing method, device and computer readable storage medium, which solve the problem of low accuracy of extracting lesion features in the stomach image and improve the accuracy of extracting lesion features in the stomach image.

[0004] The technical scheme of the present application is implemented as follows:

[0005] An image processing method, the method comprising:

[0006] obtaining a sample image and a sample enhanced image for a target part in a human body; wherein the sample enhanced image is obtained by performing image enhancement on the sample image;

[0007] training based on the sample image and the sample enhanced image to determine a target weight function;

[0008] determining a spatial transformation parameter of a collection component that collects the sample image;

[0009] determining an image enhancement model based on the target weight function and the spatial transformation parameter;

[0010] obtaining a to-be-processed image for the target part in the human body, and determining a target image for the target part in the human body based on the to-be-processed image and the image enhancement model.

[0011] In the above scheme, the sample image and the sample enhanced image for the target part in the human body are obtained, comprising:

[0012] obtaining a plurality of candidate images for the target part in the human body;

[0013] performing noise reduction processing on the plurality of candidate images to obtain the sample image;

[0014] performing enhancement processing on the sample image by using an image enhancer to obtain the sample enhanced image.

[0015] In the scheme, the training based on the sample image and the sample enhanced image to determine the target weight function comprises:

[0016] The sample image and the sample enhanced image are analyzed to determine a target function representing the difference between the sample image and the sample enhanced image.

[0017] The target weight function is determined based on the target function.

[0018] In the scheme, the sample image and the sample enhanced image are analyzed to determine a target function representing the difference between the sample image and the sample enhanced image, which comprises:

[0019] The sample image and the sample enhanced image are analyzed to determine a first sub-function representing the color difference between the sample image and the sample enhanced image, and a second sub-function representing the texture difference between the sample image and the sample enhanced image.

[0020] In the scheme, the target weight function is determined based on the target function, which comprises:

[0021] The first sub-function and the second sub-function are processed by a target convolutional neural network to obtain the target weight function.

[0022] In the scheme, the spatial transformation parameter of the acquisition component for acquiring the sample image is determined, which comprises:

[0023] The spatial function of the acquisition component for acquiring the sample image is obtained.

[0024] The spatial transformation parameter is determined based on the spatial function.

[0025] In the scheme, the image enhancement model is determined based on the target weight function and the spatial transformation parameter, which comprises:

[0026] The target weight function and the spatial transformation parameter are transformed to obtain a target logic function; wherein the target logic function represents the relationship between the to-be-enhanced image and the enhanced image.

[0027] In the case where the target logic function tends to converge, the target logic function is determined as the image enhancement model.

[0028] In the scheme, the to-be-processed image for the target part in the human body is obtained, which comprises:

[0029] The to-be-input image for the target part in the human body is obtained.

[0030] The denoising processing is performed on the to-be-input image to obtain the to-be-processed image.

[0031] In the scheme, the determining of the target image for the target part in the human body based on the to-be-processed image and the image enhancement model comprises:

[0032] The to-be-processed image is processed by using the image enhancement model to obtain a to-be-selected image.

[0033] The to-be-selected image is regionally marked to obtain the target image.

[0034] An image processing device, comprising a processor, a memory and a communication bus;

[0035] The communication bus is used to realize the communication connection between the processor and the memory.

[0036] The processor is used to execute the image processing program stored in the memory to realize the following steps:

[0037] A sample image and a sample enhanced image for a target part in a human body are obtained; wherein the sample enhanced image is obtained by performing image enhancement on the sample image;

[0038] The sample image and the sample enhanced image are used for training to determine a target weight function;

[0039] The spatial transformation parameters of a collection component used to collect the sample image are determined;

[0040] Based on the target weight function and the spatial transformation parameters, an image enhancement model is determined.

[0041] A to-be-processed image for the target part in the human body is obtained, and a target image for the target part in the human body is determined based on the to-be-processed image and the image enhancement model.

[0042] A computer readable storage medium stores one or more programs, which can be executed by one or more processors to realize the steps of the above image processing method.

[0043] The image processing method, apparatus, and computer-readable storage medium provided in this application acquire sample images and enhanced images of target areas within the human body. The enhanced image is obtained by enhancing the sample image. Training is performed based on the sample image and the enhanced image to determine a target weight function. Spatial transformation parameters of the acquisition component for acquiring the sample image are determined. An image enhancement model is determined based on the target weight function and the spatial transformation parameters. An image to be processed for the target area within the human body is acquired, and a target image for the target area is determined based on the image to be processed and the image enhancement model. Thus, by determining the image enhancement model based on the target weight function and spatial transformation parameters, and enhancing the image to be processed using the image enhancement model to obtain the target image, the visual effect of the target image is improved, thereby increasing the accuracy of extracting lesion features from the target image. Furthermore, using spatial transformation parameters to determine the image enhancement model considers the actual environmental factors of acquiring sample images for the target area, improving the accuracy of the determined image enhancement model. Attached Figure Description

[0044] Figure 1 A schematic flowchart of an image processing method provided in an embodiment of this application;

[0045] Figure 2 A flowchart illustrating another image processing method provided in an embodiment of this application;

[0046] Figure 3 This is a schematic diagram of the network structure corresponding to the WESPES algorithm provided in the embodiments of this application;

[0047] Figure 4 A schematic flowchart illustrating another image processing method provided in an embodiment of this application;

[0048] Figure 5 This is a schematic diagram of the structure of an image processing device provided in an embodiment of this application;

[0049] Figure 6 This is a schematic diagram of the structure of an image processing device provided in an embodiment of this application. Detailed Implementation

[0050] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0051] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0052] This application provides an image processing method, which can be applied to an image processing device, as shown below. Figure 1 As shown, the method includes the following steps:

[0053] Step 101, acquire a sample image and a sample enhanced image for a target part in a human body.

[0054] The sample enhanced image is obtained by performing image enhancement on the sample image.

[0055] In the embodiments of the present application, the image processing device can be a device with image acquisition and processing functions; the image acquisition component of the image acquisition device can scan the organs in the human body, determine the position of the target part in the human body, and based on the position of the target part, acquire images of the target part at different angles, then screen the images of the target part to obtain the sample image, and perform image enhancement processing on the sample image to obtain the sample enhanced image; the sample image can also be acquired by the acquisition component with image acquisition function on other devices and sent to the image processing device, and the sample enhanced image can be obtained by the image acquisition device after receiving the sample image, performing image enhancement processing on the sample image; wherein the sample enhanced image can also be obtained by the device with image enhancement capability after performing enhancement processing on the sample image and sent to the image processing device.

[0056] In a feasible implementation, a plurality of images of the stomach in the human body can be acquired at different acquisition angles by a gastroscope, and a value representing the uniformity of the brightness of each image is determined, the plurality of images are screened according to the value, an image with a value greater than a target threshold is determined as a sample image from the plurality of images, and the sample image is sent to the image processing device, and the image processing device receives the sample image and performs image enhancement on the sample image to obtain a sample enhanced image. Wherein, the greater the value, the better the uniformity of the brightness of the image.

[0057] Step 102, train based on the sample image and the sample enhanced image to determine a target weight function.

[0058] In the embodiments of the present application, the target network algorithm can be used to train the sample image and the sample enhanced image to obtain the target weight function. Wherein, the target network algorithm can be a weakly supervised photo enhancer algorithm (Weakly Supervised Photo Enhancer for Digital Cameras, WESPE) for digital cameras.

[0059] The WESPE algorithm improves a traditional generative adversarial network (GAN) algorithm. Unlike the traditional GAN algorithm, the WESPE algorithm does not need to retrain a model for each data set and is not universal. When the WESPE algorithm is used for training, the input data and the output data are low-quality images and high-quality images, respectively, but they do not need to correspond in content. A transitive CNN-GAN (convolutional neural network-generative adversarial network) structure is used to learn the mapping relationship between them.

[0060] Step 103: determining a spatial transformation parameter of a collection component that collects the sample image.

[0061] In the embodiment of the present application, the collection component that collects the sample image can be a gastroscope; the spatial transformation parameter can be determined by using the gastroscope in a simulated human stomach environment and an extracorporeal environment, and specifically can be determined according to a spatial function of the human stomach environment in which the gastroscope is located and a spatial function of the extracorporeal environment in which the gastroscope is located.

[0062] Step 104: determining an image enhancement model based on the target weight function and the spatial transformation parameter.

[0063] In the embodiment of the present application, the image processing device performs operation based on the target weight function and the spatial transformation parameter to obtain the image enhancement model.

[0064] Step 105: obtaining a to-be-processed image for a target part in a human body, and determining a target image for the target part in the human body based on the to-be-processed image and the image enhancement model.

[0065] It should be noted that the process of obtaining the to-be-processed image for the target part in the human body is similar to the process of obtaining the sample image for the target part in the human body in step 101, and the embodiment of the present application will not be repeated here.

[0066] In the embodiment of the present application, the to-be-processed image can be input into the image enhancement model, so that the image enhancement model performs image enhancement processing on the to-be-processed image to obtain the target image for the target part in the human body. The uniformity of the brightness and darkness of the target image is better than that of the sample image. In a feasible implementation manner, the target part can be the stomach of the human body.

[0067] The image processing method provided in the embodiments of the present application obtains a sample image and a sample enhanced image for a target part in a human body; the sample enhanced image is obtained by performing image enhancement on the sample image; a target weight function is determined based on the sample image and the sample enhanced image; a spatial transformation parameter of a collection component that collects the sample image is determined; an image enhancement model is determined based on the target weight function and the spatial transformation parameter; a to-be-processed image for the target part in the human body is obtained, and a target image for the target part in the human body is determined based on the to-be-processed image and the image enhancement model; in this way, the image enhancement model is determined based on the target weight function and the spatial transformation parameter, and the to-be-processed image is enhanced by using the image enhancement model to obtain the target image, thereby improving the visual effect of the target image and further improving the accuracy of extracting lesion features from the target image; moreover, the spatial transformation parameter is used to determine the image enhancement model, and the actual environmental factors for collecting the sample image for the target part are considered, thereby improving the accuracy of the determined image enhancement model.

[0068] Based on the foregoing embodiments, the embodiments of the present application provide an image processing method, as shown in Figure 2 The method comprises the following steps:

[0069] Step 201: An image processing device obtains multiple candidate images for a target part in a human body.

[0070] In the embodiments of the present application, the image processing device can collect multiple images with the target part in the human body at different collection angles, and perform target part recognition on each image with the target part in the human body to extract candidate images for the target part in the human body; the candidate images can also be collected by a collection component with an image collection function on other devices and sent to the image processing device.

[0071] In a feasible implementation manner, the candidate images include but are not limited to images of the stomach in the human body.

[0072] It should be noted that the candidate images are obtained at different collection angles, so that the target part can be presented in multiple directions in the image, avoiding the case that the collected target part is blocked or incomplete due to the complex environment in the human body, thereby improving the accuracy of the subsequently determined sample enhanced model.

[0073] Step 202: The image processing device performs noise reduction processing on the multiple candidate images to obtain a sample image.

[0074] In the embodiments of the present application, the image processing device can perform noise reduction processing on the multiple candidate images, and specifically can perform Gaussian blur processing on the multiple candidate images to obtain a sample image; the Gaussian blur processing can be two-dimensional Gaussian blur processing.

[0075] It should be noted that, by performing the noise reduction processing on the plurality of candidate images, the noise and the detail level in the candidate images can be reduced, the signal-to-noise ratio of the candidate images can be improved, and the original information of the candidate images can be maximally maintained. Other manners can also be used to perform the noise reduction processing on the plurality of candidate images, and the method used for the noise reduction processing in the embodiments of the present application is not limited.

[0076] Step 203: The image processing device performs enhancement processing on the sample image by using an image enhancer to obtain a sample enhanced image.

[0077] In the embodiments of the present application, the sample image can refer to a low-quality image, and the sample enhanced image refers to a high-quality image; the image processing device can perform enhancement processing on the sample image by using an image enhancer, and the image obtained after the enhancement processing is taken as the sample enhanced image. The enhancement processing can also be referred to as image enhancement processing, which can improve the sample image quality, enrich the information amount, and strengthen the image judgment and recognition effect.

[0078] In a feasible implementation manner, the image enhancer can be an image enhancer in a network structure corresponding to the WESPE algorithm.

[0079] Step 204: The image processing device analyzes the sample image and the sample enhanced image to determine a target function representing the difference between the sample image and the sample enhanced image.

[0080] In the embodiments of the present application, the image processing device can analyze the pixel values of the pixel points of the sample image and the sample enhanced image, determine the difference parameters between the sample image and the sample enhanced image, and determine the target function according to the difference parameters between the sample image and the sample enhanced image. The difference parameters include color difference parameters and texture difference parameters.

[0081] In a feasible implementation manner, the image processing device can analyze the sample image and the sample enhanced image by using a discriminator in a network structure corresponding to the WESPE algorithm to determine the target function.

[0082] It should be noted that, step 204 can be implemented by step a1.

[0083] Step a1: The image processing device analyzes the sample image and the sample enhanced image to determine a first sub-function representing the color difference between the sample image and the sample enhanced image, and to determine a second sub-function representing the texture difference between the sample image and the sample enhanced image.

[0084] In the embodiment of the present application, the image processing device can analyze the sample image and the sample enhanced image, determine a color difference parameter of the color difference between the sample image and the sample enhanced image, and determine the first sub-function based on the color difference parameter, determine a texture difference parameter of the texture difference between the sample image and the sample enhanced image, and determine the second sub-function based on the texture difference parameter; the sample image and the sample enhanced image can be used in determining the first sub-function and the second sub-function; the sample image and the denoised sample enhanced image can also be used, and the denoised sample enhanced image can reduce the influence of image noise.

[0085] The color difference can also be referred to as color loss, and is used to measure the color difference between the sample enhanced image and the sample image; the texture difference can also be referred to as texture loss, and is used to measure the quality difference of the texture between the sample enhanced image and the sample image.

[0086] In a feasible implementation manner, the first sub-function can be represented by formula (1):

[0087]

[0088] In formula (1), X is the candidate image; Y is the sample enhanced image without denoising processing; Xb is the sample image, and Yb is the denoised sample enhanced image.

[0089] In a feasible implementation manner, the second sub-function can be represented by formula (2):

[0090] Ltexture=-∑LogD(Fw(Is),It)

[0091] In formula (2), Fw is a generation sub-network in the WESPE algorithm, D is a discrimination network in the WESPE algorithm, Is is the source image, and It is the image obtained by performing enhancement processing on the source image.

[0092] In a feasible implementation manner, the source image can be the sample image, and the image obtained by performing enhancement processing on the source image can be the sample enhanced image.

[0093] Step 205: The image processing device determines the target weight function based on the target function.

[0094] In formula (3), the target function at least includes the first sub-function and the second sub-function.

[0095] In the embodiment of the present application, the first weight coefficient of the first sub-function and the second weight coefficient of the second sub-function can be obtained, and the to-be-processed function can be determined based on the first sub-function, the first weight coefficient, the second sub-function and the second weight coefficient, and the target weight function can be obtained by processing the to-be-processed function.

[0096] It should be noted that step 205 can be implemented by step a2.

[0097] Step a2, the image processing device processes the first sub-function and the second sub-function by using the target convolutional neural network to obtain the target weight function.

[0098] In the embodiment of the application, the first sub-function and the first weight coefficient can be multiplied to obtain a first to-be-processed sub-function, and the second sub-function and the second weight coefficient can be multiplied to obtain a second to-be-processed sub-function, the first to-be-processed sub-function and the second to-be-processed sub-function are added to obtain a to-be-processed function, and the target convolutional neural network is used to analyze the to-be-processed function to obtain the target weight function.

[0099] In a feasible implementation manner, the first sub-function is represented by Lcolor, the first weight coefficient is 1, the second sub-function is represented by Ltexture, and the second weight coefficient is 5*0.001. The to-be-processed function can be represented by formula (3):

[0100] L = Lcolor + 5*0.001*Ltexture formula (3)

[0101] The image processing device can process the to-be-processed function through the VGG19 network in the WESPE algorithm to obtain the target weight function, wherein the target weight function can be represented by formula (4):

[0102]

[0103] Wherein, W* represents the target weight function, n is the number of image pairs of the sample image and the sample enhanced image, Is is the source image, It is the image processed by enhancing the source image; n is a positive integer.

[0104] It should be noted that the sample image and the sample enhanced image form a one-to-one corresponding image pair. In a feasible implementation manner, the number of sample images is 10, the number of sample enhanced images is also 10, and the image pair n is 10.

[0105] Step 206, the image processing device acquires the spatial function of the acquisition component for acquiring the sample image.

[0106] Wherein, the spatial function can be the spatial function of the acquisition component for acquiring the sample image in the simulated human stomach environment and the spatial function in the human extracorporeal environment.

[0107] In the embodiment of the application, the image acquisition component can be used to acquire images for the standard color plate in the simulated human stomach environment and the extracorporeal environment, and the spatial function can be determined according to the acquired images.

[0108] In a feasible implementation, a tangent direction of an extension direction of a pipeline connected with the acquisition component can be taken as a W axis, a line connecting a camera center in a plane perpendicular to the W direction can be taken as a U axis, a direction perpendicular to the U axis in the plane can be taken as a v axis, a three-dimensional coordinate system u-v-w is constructed, and the acquisition component is used to respectively take pictures of 1024*1024 standard color panels in a simulated human stomach environment and an in-vitro environment. A fixed point S in the standard color panels is selected, an image coordinate of S on an imaging plane in the simulated human stomach environment is (ul, vl), and a coordinate of S on an imaging plane in the simulated in-vitro environment is (ur, vr). The spatial function can be represented by formula (5) and formula (6):

[0109]

[0110]

[0111] wherein (x, y, z) is a position coordinate of the fixed point S in the world coordinate system; Ml is a feature matrix of the image acquisition component in the human body; and Mr is a feature matrix of the image acquisition component outside the human body.

[0112] In step 207, the image processing device determines the spatial transformation parameter based on the spatial function.

[0113] In the embodiment of the present application, the image processing device can analyze the spatial function to determine the first feature parameter of the acquisition component outside the human body and the second feature parameter of the acquisition component in the human body, and process the first feature parameter and the second feature parameter in combination with the space-time transformation characteristics of Fourier to obtain the spatial transformation parameter.

[0114] In a feasible implementation, the first feature parameter and the second feature parameter can be represented in the form of a matrix; the first feature parameter can be a feature matrix Ml, and the second feature parameter can be a feature matrix Mr. In combination with the space-time transformation characteristics of Fourier, the spatial transformation parameter in the space-time domain can be obtained, and the spatial transformation parameter can be represented by formula (7):

[0115]

[0116] wherein M represents the spatial transformation parameter.

[0117] In step 208, the image processing device performs transformation processing on the target weight function and the spatial transformation parameter to obtain a target logic function.

[0118] The target logic function represents a relationship between the image to be enhanced and the enhanced image.

[0119] In the embodiment of the present application, the target weight parameter and the spatial transformation parameter can be multiplied in the time domain of Fourier to obtain the target logic function.

[0120] In a feasible implementation, a target weight function is denoted as W*, and a target logic function is denoted as W.

[0121]

[0122] Wherein, Is can represent a source image, It is an image processed by enhancing the source image; Fw can be a generation subnetwork in the WESPE algorithm; M represents a spatial transformation parameter; n is the number of image pairs of sample images and sample enhanced images, and L is a to-be-processed function.

[0123] Step 209, in a case where the target logic function tends to converge, the image processing device determines the target logic function as a determined image enhancement model.

[0124] In the embodiment of the present application, the target logic function can be analyzed, and when the target logic function tends to converge, the target logic function is taken as the image enhancement model.

[0125] Step 210, the image processing device acquires a to-be-input image for a target part in a human body.

[0126] In the embodiment of the present application, the image processing device can collect an image having multiple parts in a human body, and perform target part recognition on the image having multiple parts in the human body, to obtain the to-be-input image having only the target part.

[0127] Step 211, the image processing device performs noise reduction processing on the to-be-input image to obtain a to-be-processed image.

[0128] In the embodiment of the present application, the image processing device can perform noise reduction processing on the to-be-processed image, and specifically, can perform Gaussian blur processing on the to-be-input image to obtain the to-be-processed image, so as to reduce the influence of noise of the to-be-input image.

[0129] Step 212, the image processing device processes the to-be-processed image by using the image enhancement model to obtain a to-be-selected image.

[0130] In the embodiment of the present application, the to-be-processed image can be input to the image enhancement model, and the image enhancement model can output the to-be-selected image; wherein, the quality of the to-be-selected image is higher than that of the to-be-processed image.

[0131] Step 213, the image processing device performs region marking on the to-be-selected image to obtain a target image.

[0132] In the embodiment of the present application, the image processing device can analyze the target part in the to-be-selected image, divide the target part into multiple regions, and mark each region, and take the marked image as the target image.

[0133] In one feasible implementation, the stomach can be divided into upper, middle, and lower parts. An image processing device can analyze the stomach image, determining a first position of the upper region, a second position of the middle region, and a third position of the lower region. Based on these first, second, and third positions, the upper, middle, and lower regions in the stomach image are labeled to obtain the target image. The labeling can employ a nearest neighbor method to mark the upper, middle, and lower regions, avoiding errors in subjective judgment by doctors, reducing their workload, and improving the efficiency of identifying lesion features.

[0134] It should be noted that the descriptions of the same steps and contents as in other embodiments in this embodiment can be found in the descriptions in other embodiments, and will not be repeated here.

[0135] The following combination Figure 3 The network structure corresponding to the WESPE algorithm provided in the embodiments of this application will be explained in detail.

[0136] like Figure 3 As shown, the image intensifier and degrader are used as the core nodes of the feedforward architecture; the color difference discriminator is used to determine the color difference between the sample image and the sample augmented image, and the texture difference discriminator is used to determine the texture difference between the sample image and the sample augmented image; VGG19 is used as the target convolutional neural network to process the function to be processed; L is the function to be processed; X can be a candidate image, Y is the sample augmented image without denoising, X~ is the intermediate state image corresponding to the candidate image in the network; Y~ is the intermediate state image corresponding to the sample augmented image without denoising in the network structure.

[0137] This application also provides an image processing method, which is described below in conjunction with... Figure 4 This image processing method will be explained in detail.

[0138] like Figure 4 As shown, taking the stomach as an example, the image processing device can acquire candidate images and perform noise reduction on the candidate images to obtain sample images. The sample images and the noise-reduced sample-enhanced images (or un-noise-reduced sample-enhanced images) are then input into the network corresponding to the WESPE algorithm. The objective function representing the difference between the sample images and the sample-enhanced images is calculated, and the objective weight function W* is determined based on the objective function. Based on the spatial transformation parameters M of the acquisition component for acquiring sample images and the objective weight function W*, the objective logic function is determined. If the objective logic function converges, it is determined as the image enhancement model, and the image to be processed is input into the image enhancement model to obtain the target image. If the objective logic function does not converge, the training process continues.

[0139] The image processing method provided by the embodiment of the present application determines an image enhancement model based on the target weight function and the spatial transformation parameter, and performs enhancement processing on the to-be-processed image through the image enhancement model to obtain a target image, thereby improving the visual effect of the target image and further improving the accuracy of extracting lesion features from the target image. Moreover, the spatial transformation parameter is used to determine the image enhancement model, and the actual environmental factors for collecting sample images of the target part are considered, thereby improving the accuracy of the determined image enhancement model.

[0140] Based on the foregoing embodiments, the embodiment of the present application provides an image processing device which can be applied to Figures 1-2 The image processing method provided by the corresponding embodiment can refer to Figure 5 As shown in the figure, the image processing device 3 can include a processor 32, a memory 31 and a communication bus 33; wherein:

[0141] The communication bus 33 is used to realize the communication connection between the processor 32 and the memory 31;

[0142] The processor 32 is used to execute the image processing program stored in the memory 31 to realize the following steps:

[0143] Obtain a sample image and a sample enhanced image for a target part in a human body; wherein the sample enhanced image is obtained by performing image enhancement on the sample image;

[0144] Based on the sample image and the sample enhanced image, a target weight function is determined;

[0145] Determine the spatial transformation parameter of the collection component for collecting the sample image;

[0146] Based on the target weight function and the spatial transformation parameter, an image enhancement model is determined;

[0147] Obtain a to-be-processed image for the target part in the human body, and determine a target image for the target part in the human body based on the to-be-processed image and the image enhancement model.

[0148] In other embodiments of the present application, the processor 32 is used to execute the image processing program in the memory 31 to obtain a sample image and a sample enhanced image for a target part in a human body, to realize the following steps:

[0149] Obtain a plurality of candidate images for the target part in the human body;

[0150] Perform noise reduction processing on the plurality of candidate images to obtain the sample image;

[0151] Perform enhancement processing on the sample image using an image enhancer to obtain the sample enhanced image.

[0152] In other embodiments of the present application, the processor 32 is configured to execute the image processing program in the memory 31 to train based on the sample image and the sample enhanced image, determine the target weight function to achieve the following steps:

[0153] analyze the sample image and the sample enhanced image to determine a target function representing the difference between the sample image and the sample enhanced image;

[0154] determine the target weight function based on the target function.

[0155] In other embodiments of the present application, the processor 32 is configured to execute the image processing program in the memory 31 to analyze the sample image and the sample enhanced image to determine a target function representing the difference between the sample image and the sample enhanced image to achieve the following steps:

[0156] analyze the sample image and the sample enhanced image to determine a first sub-function representing the color difference between the sample image and the sample enhanced image, and determine a second sub-function representing the texture difference between the sample image and the sample enhanced image;

[0157] Correspondingly, in other embodiments of the present application, the processor 32 is configured to execute the image processing program in the memory 31 to determine the target weight function based on the target function to achieve the following steps:

[0158] process the first sub-function and the second sub-function using the target convolutional neural network to obtain the target weight function.

[0159] In other embodiments of the present application, the processor 32 is configured to execute the image processing program in the memory 31 to determine the spatial transformation parameter of the acquisition component for acquiring the sample image to achieve the following steps:

[0160] obtain a spatial function of the acquisition component for acquiring the sample image;

[0161] determine the spatial transformation parameter based on the spatial function.

[0162] In other embodiments of the present application, the processor 32 is configured to execute the image processing program in the memory 31 to determine the image enhancement model based on the target weight function and the spatial transformation parameter to achieve the following steps:

[0163] transform the target weight function and the spatial transformation parameter to obtain a target logic function; wherein the target logic function represents the relationship between the image to be enhanced and the enhanced image;

[0164] In the case where the target logic function converges, determine the target logic function as the determined image enhancement model.

[0165] In other embodiments of the present application, the processor 32 is configured to execute the image processing program in the memory 31 to obtain a to-be-processed image for a target site in a human body, so as to implement the following steps:

[0166] obtain a to-be-input image for the target site in the human body;

[0167] perform noise reduction processing on the to-be-input image to obtain the to-be-processed image.

[0168] In other embodiments of the present application, the processor 32 is configured to execute the image processing program in the memory 31 to determine a target image for the target site in the human body based on the to-be-processed image and an image enhancement model, so as to implement the following steps:

[0169] perform processing on the to-be-processed image by using the image enhancement model to obtain a to-be-selected image;

[0170] perform region marking on the to-be-selected image to obtain the target image.

[0171] It should be noted that the specific implementation process of the steps performed by the processor in this embodiment can refer to the implementation process in the image processing method provided by the corresponding embodiments, which will not be described here in detail. Figures 1-2

[0172] The image processing device provided by the embodiments of the present application determines an image enhancement model based on a target weight function and a spatial transformation parameter, and performs enhancement processing on a to-be-processed image by using the image enhancement model to obtain a target image, which improves the visual effect of the to-be-processed image and further improves the accuracy of extracting lesion features from the target image. Moreover, the spatial transformation parameter is used to determine the image enhancement model, which takes into account the actual environmental factors of collecting sample images for the target site, thereby improving the accuracy of the determined image enhancement model.

[0173] Based on the foregoing embodiments, the embodiments of the present application provide an image processing apparatus, which can be applied to Figures 1-2 the image processing method provided by the corresponding embodiments, and refer to Figure 6 The apparatus can include an obtaining unit 41 and a processing unit 42, where:

[0174] The obtaining unit 41 is configured to obtain a sample image and a sample enhanced image for a target site in a human body, where the sample enhanced image is obtained by performing image enhancement on the sample image.

[0175] The processing unit 42 is configured to perform training based on the sample image and the sample enhanced image to determine a target weight function.

[0176] The processing unit 42 is further configured to determine a spatial transformation parameter of a collection component that collects the sample image. ​

[0177] The processing unit 42 is further configured to determine an image enhancement model based on the target weight function and the spatial transformation parameter.

[0178] The processing unit 42 is further configured to obtain a to-be-processed image of the target part in the human body, and determine a target image of the target part in the human body based on the to-be-processed image and the image enhancement model.

[0179] In other embodiments of the present application, the obtaining unit 41 is further configured to perform the following steps:

[0180] Obtain a plurality of candidate images of the target part in the human body;

[0181] Perform noise reduction processing on the plurality of candidate images to obtain a sample image;

[0182] Perform enhancement processing on the sample image by using an image enhancer to obtain a sample enhanced image.

[0183] In other embodiments of the present application, the processing unit 42 is further configured to perform the following steps:

[0184] Perform analysis on the sample image and the sample enhanced image to determine a target function representing the difference between the sample image and the sample enhanced image;

[0185] Determine the target weight function based on the target function.

[0186] In other embodiments of the present application, the processing unit 42 is further configured to perform the following steps:

[0187] Perform analysis on the sample image and the sample enhanced image to determine a first sub-function representing the color difference between the sample image and the sample enhanced image, and determine a second sub-function representing the texture difference between the sample image and the sample enhanced image;

[0188] Process the first sub-function and the second sub-function by using the target convolutional neural network to obtain the target weight function.

[0189] In other embodiments of the present application, the processing unit 42 is further configured to perform the following steps:

[0190] Obtain a spatial function of a collection component that collects the sample image;

[0191] Determine the spatial transformation parameter based on the spatial function.

[0192] In other embodiments of the present application, the processing unit 42 is further configured to perform the following steps:

[0193] Perform transformation processing on the target weight function and the spatial transformation parameter to obtain a target logic function; wherein the target logic function represents the relationship between the to-be-enhanced image and the enhanced image;

[0194] In a case where the target logic function tends to converge, the target logic function is determined as the determined image enhancement model.

[0195] In other embodiments of the present application, the processing unit 42 is further configured to perform the following steps:

[0196] obtaining a to-be-input image for a target part in a human body;

[0197] performing noise reduction processing on the to-be-input image to obtain a to-be-processed image.

[0198] In other embodiments of the present application, the processing unit 42 is further configured to perform the following steps:

[0199] processing the to-be-processed image by using the image enhancement model to obtain a to-be-selected image;

[0200] performing region marking on the to-be-selected image to obtain a target image.

[0201] It should be noted that the specific implementation process of the steps performed by each unit in the present embodiment can refer to the implementation process of the image processing method provided in the corresponding embodiments, which will not be described here in detail. Figures 1-2 The specific implementation process of the steps performed by each unit in the present embodiment can refer to the implementation process of the image processing method provided in the corresponding embodiments, which will not be described here in detail.

[0202] The image processing apparatus provided by the embodiments of the present application determines the image enhancement model based on the target weight function and the spatial transformation parameter, and performs enhancement processing on the to-be-processed image by using the image enhancement model to obtain the target image, thereby improving the visual effect of the target image and further improving the accuracy of extracting the lesion feature from the target image. Moreover, the spatial transformation parameter is used to determine the image enhancement model, which takes into account the actual environmental factors for collecting the sample image of the target part, thereby improving the accuracy of the determined image enhancement model.

[0203] Based on the foregoing embodiments, the embodiments of the present application provide a computer-readable storage medium storing one or more programs, which can be executed by one or more processors to implement Figures 1-2 the steps of the image processing method provided in the corresponding embodiments.

[0204] It should be noted that the computer readable storage medium above can be a Read Only Memory (ROM), a Programmable Read-Only Memory (PROM), an Erasable Programmable Read-Only Memory (EPROM), an Electrically Erasable Programmable Read-Only Memory (EEPROM), a Ferromagnetic Random Access Memory (FRAM), a Flash Memory, a magnetic surface memory, an optical disc, or a Compact Disc Read-Only Memory (CD-ROM) memory, etc. It can also be various electronic devices including one or any combination of the above memories, such as a mobile phone, a computer, a tablet device, a personal digital assistant, etc.

[0205] It should be noted that in this paper, the term "including", "containing" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or includes elements inherent to such process, method, article or device. Without more limitations, the element defined by the statement "including a" does not exclude the presence of other identical elements in the process, method, article or device including the element.

[0206] The above-mentioned embodiment numbers of the present application are only for description, not representing the advantages and disadvantages of the embodiments.

[0207] From the above description of the embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be realized by software plus the necessary general hardware platform, of course, they can also be realized by hardware, but in many cases the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of software products, and the computer software product is stored in a storage medium (such as ROM / RAM, magnetic disc, optical disc), including a plurality of instructions to make a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) execute the methods described in various embodiments of the present application.

[0208] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart or flows and / or block diagram block or blocks. Figure 1 one or more flow or flows and / or block diagram block or blocks. Figure 1 one or more flow or flows and / or block diagram block or blocks.

[0209] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart or flows and / or block diagram block or blocks. Figure 1 one or more flow or flows and / or block diagram block or blocks. Figure 1 one or more flow or flows and / or block diagram block or blocks.

[0210] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart or flows and / or block diagram block or blocks. Figure 1 one or more flow or flows and / or block diagram block or blocks. Figure 1 one or more flow or flows and / or block diagram block or blocks.

[0211] The above merely provides the preferred embodiment of the present application, and is not intended to limit the patent scope of the present application, and any equivalent structure or equivalent flow transformation made according to the content of the present application specification and drawings, or direct or indirect application in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. An image processing method, characterized by, The method comprises: obtaining a sample image and a sample enhanced image of a target part in a human body; wherein the sample enhanced image is obtained by image enhancement on the sample image; training based on the sample image and the sample enhanced image to determine a target weight function; determining a spatial transformation parameter of a collection component for collecting the sample image; wherein the spatial transformation parameter is determined according to an in-vivo environment and an out-vivo environment of the collection component; determining an image enhancement model based on the target weight function and the spatial transformation parameter; obtaining a to-be-processed image of the target part in the human body, and determining a target image of the target part in the human body based on the to-be-processed image and the image enhancement model; wherein the determining of the image enhancement model based on the target weight function and the spatial transformation parameter comprises: multiplying the target weight function and the spatial transformation parameter in the time-space domain of Fourier to obtain a target logic function; wherein the target logic function represents the relationship between the to-be-enhanced image and the enhanced image; in the case that the target logic function tends to converge, determining the target logic function as the image enhancement model.

2. The method of claim 1, wherein, The obtaining of the sample image and the sample enhanced image of the target part in the human body comprises: obtaining a plurality of candidate images of the target part in the human body; performing noise reduction processing on the plurality of candidate images to obtain the sample image; performing enhancement processing on the sample image by using an image enhancer to obtain the sample enhanced image.

3. The method of claim 1, wherein, The training based on the sample image and the sample enhanced image to determine the target weight function comprises: analyzing the sample image and the sample enhanced image to determine a target function representing the difference between the sample image and the sample enhanced image; determining the target weight function based on the target function.

4. The method of claim 3, wherein, The analyzing of the sample image and the sample enhanced image to determine the target function representing the difference between the sample image and the sample enhanced image comprises: analyzing the sample image and the sample enhanced image to determine a first sub-function representing the color difference between the sample image and the sample enhanced image, and a second sub-function representing the texture difference between the sample image and the sample enhanced image; correspondingly, the determining of the target weight function based on the target function comprises: processing the first sub-function and the second sub-function by using a target convolutional neural network to obtain the target weight function.

5. The method of claim 1, wherein, The determining of the spatial transformation parameter of the collection component for collecting the sample image comprises: obtaining a spatial function of the collection component for collecting the sample image; determining the spatial transformation parameter based on the spatial function.

6. The method of claim 1, wherein, The obtaining of the to-be-processed image of the target part in the human body comprises: obtaining a to-be-input image of the target part in the human body; performing noise reduction processing on the to-be-input image to obtain the to-be-processed image.

7. The method of claim 6, wherein, The determining of the target image of the target part in the human body based on the to-be-processed image and the image enhancement model comprises: The image enhancement model is used to process the to-be-processed image, to obtain a to-be-selected image; The to-be-selected image is regionally marked to obtain the target image.

8. An image processing apparatus characterized by comprising: The device comprises a processor, a memory and a communication bus; The communication bus is used to realize the communication connection between the processor and the memory; The processor is used to execute the image processing program stored in the memory, to realize the following steps: Obtain a sample image and a sample enhanced image of a target part in a human body; wherein the sample enhanced image is obtained by image enhancement on the sample image; Based on the sample image and the sample enhanced image, training is performed to determine a target weight function; Determine the spatial transformation parameters of the acquisition component for acquiring the sample image; wherein the spatial transformation parameters are determined according to the environment in the human body and the environment outside the human body where the acquisition component is located; The target weight function and the spatial transformation parameters are multiplied in the time-space domain of Fourier to obtain a target logic function; wherein the target logic function represents the relationship between the to-be-enhanced image and the enhanced image; In the case where the target logic function tends to converge, the target logic function is determined as an image enhancement model; Obtain a to-be-processed image of the target part in the human body, and determine a target image of the target part in the human body based on the to-be-processed image and the image enhancement model.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores one or more programs which can be executed by one or more processors to realize the steps of the image processing method according to any one of claims 1 to 7.

Images