Convolutional neural network-based medical image enhancement method and related device

Abstract

The embodiment of the invention relates to the field of data processing, and provides a medical image enhancement method based on a convolutional neural network and a related device, and the method comprises the steps: obtaining a medical image data set; according to the medical image data set, a first convolutional layer of a convolutional neural network is adopted to output a medical image convolutional feature map data set; according to the medical image convolutional feature map data set, using improved pnmix to adjust the convolutional neural network to obtain a medical image enhancement model; and performing data enhancement on the medical image feature map by using the medical image enhancement model to obtain an enhanced medical image feature map, outputting a medical image feature data set according to a first convolutional layer of a convolutional neural network, and adjusting the convolutional neural network by using the improved pnmix enhanced medical image feature map to obtain a medical image feature data set. And a medical image enhancement model capable of identifying the medical image more accurately is obtained.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a medical image enhancement method and related devices based on convolutional neural networks. Background Art

[0002] In the field of medical image processing, image anomaly detection is a crucial task, directly related to early disease diagnosis and precise treatment. However, the unique nature of medical image data presents numerous challenges. Data augmentation techniques for medical images are crucial for improving the performance of anomaly detection models. Traditional data augmentation methods, such as rotation, scaling, and cropping, can increase data diversity to a certain extent, but for medical images, these methods often struggle to capture deeper changes in image features.

[0003] There are significant differences in data distribution between medical images and natural images. This is mainly reflected in the fact that medical images are mostly single-channel grayscale images with a lack of color information, and the texture and shape features in the images are often more complex and subtle. This characteristic makes traditional data augmentation methods ineffective when directly applied to medical images. Summary of the Invention

[0004] The embodiments of the present application provide a medical image enhancement method and related devices based on a convolutional neural network, which can output a medical image feature data set according to the first convolution layer of the convolutional neural network, and adjust the convolutional neural network using the medical image feature map enhanced by the improved PNmix, to obtain a medical image enhancement model that can more accurately identify medical images.

[0005] A first aspect of an embodiment of the present application provides a medical image enhancement method based on a convolutional neural network, the method comprising:

[0006] Obtain medical image datasets;

[0007] According to the medical image dataset, the first convolution layer of the convolutional neural network is used to output a medical image convolution feature map dataset;

[0008] Adjusting the convolutional neural network using an improved pnmix according to the medical image convolution feature map dataset to obtain a medical image enhancement model;

[0009] The medical image enhancement model is used to perform data enhancement on the medical image feature map to obtain an enhanced medical image feature map.

[0010] In a possible implementation, adjusting the CLIP network using pnmix according to the medical image feature dataset to obtain a medical image enhancement model includes:

[0011] Grouping the medical image convolution feature map dataset to obtain k medical image convolution feature map datasets;

[0012] Performing pnmix enhancement on the k medical image convolution feature map data sets in sequence to obtain k first medical image convolution feature map data sets;

[0013] Performing feature map pixel value calculation on the k first medical image convolution feature map data sets to obtain k feature map pixel value calculation results;

[0014] Sorting the k first medical image convolution feature map data sets according to calculation results of the k feature map pixel values to obtain a sorting result of the first medical image convolution feature map data set;

[0015] Extracting top N first medical image convolution feature map datasets from the sorting results of the first medical image convolution feature map datasets;

[0016] Performing integration processing on the N first medical image convolution feature map data sets to obtain an integrated second medical image convolution feature map data set;

[0017] The convolutional neural network is adjusted using a second medical image convolution feature map dataset to obtain a medical image enhancement model.

[0018] In a possible implementation, the feature map pixel values are calculated for the k first medical image convolution feature map data sets to obtain k feature map pixel value calculation results. The calculated feature pixel values are expressed as follows:

[0019]

[0020] Among them, H and W are the height and width of the feature map, i is the number of channels of the feature map, and m represents the mth group. It is the result feature map obtained by the mth group pnmix.

[0021] In a possible implementation, the k first medical image convolution feature map data sets are sorted according to the calculation results of the k feature map pixel values to obtain the sorting result of the first medical image convolution feature map data sets. The expression of sorting the k first medical image convolution feature map data sets is as follows:

[0022]

[0023] Among them, S sorted is the pixel value of the feature map.

[0024] In a possible implementation, the integration processing is performed on the N first medical image convolution feature map data sets to obtain an integrated second medical image convolution feature map data set. The expression of the integration processing on the N first medical image convolution feature map data sets is as follows:

[0025]

[0026] Among them, F avg is the second medical image convolution feature map dataset, N is the number of the first medical image convolution feature map datasets for integration processing, S m is the mth first medical image convolution feature map dataset.

[0027] In this example, a medical image dataset is first obtained, and the first convolution layer of the convolutional neural network is used to output a medical image convolution feature map dataset. The convolutional neural network is adjusted using pnmix based on the medical image convolution feature map dataset, and then a medical image enhancement model is obtained. The pnmix method can be used to effectively enhance single-channel grayscale medical images, thereby improving the effect of medical image enhancement and further improving the enhancement accuracy of the medical image enhancement model based on the medical image convolution feature map dataset.

[0028] A second aspect of an embodiment of the present application provides a medical image enhancement device based on a convolutional neural network, the device comprising:

[0029] an acquisition unit, configured to acquire a medical image dataset;

[0030] A first processing unit is configured to output a medical image convolution feature map dataset using a first convolution layer of a convolutional neural network according to the medical image dataset;

[0031] A second processing unit is configured to adjust the convolutional neural network using an improved pnmix according to the medical image convolution feature map dataset to obtain a medical image enhancement model;

[0032] The image enhancement unit is used to perform data enhancement on the medical image feature map using the medical image enhancement model to obtain an enhanced medical image feature map.

[0033] In one possible implementation, in the aspect of adjusting the convolutional neural network using the improved pnmix based on the medical image convolution feature map dataset to obtain the medical image enhancement model, the first processing unit is configured to:

[0034] Grouping the medical image convolution feature map dataset to obtain k medical image convolution feature map datasets;

[0035] Performing pnmix enhancement on the k medical image convolution feature map data sets in sequence to obtain k first medical image convolution feature map data sets;

[0036] Performing feature map pixel value calculation on the k first medical image convolution feature map data sets to obtain k feature map pixel value calculation results;

[0037] Sorting the k first medical image convolution feature map data sets according to calculation results of the k feature map pixel values to obtain a sorting result of the first medical image convolution feature map data set;

[0038] Extracting top N first medical image convolution feature map datasets from the sorting results of the first medical image convolution feature map datasets;

[0039] Performing integration processing on the N first medical image convolution feature map data sets to obtain an integrated second medical image convolution feature map data set;

[0040] The convolutional neural network is adjusted using a second medical image convolution feature map dataset to obtain a medical image enhancement model.

[0041] In one possible implementation, in the aspect of calculating feature map pixel values for the k first medical image convolution feature map data sets to obtain k feature map pixel value calculation results, in the first processing unit, the expression form of the calculated feature pixel value is as follows:

[0042]

[0043] Among them, H and W are the height and width of the feature map, i is the number of channels of the feature map, and m represents the mth group. It is the result feature map obtained by the mth group pnmix.

[0044] In a possible implementation, in the aspect of sorting the k first medical image convolution feature map data sets according to the calculation results of the k feature map pixel values to obtain the sorting result of the first medical image convolution feature map data sets, in the first processing unit, the expression form of sorting the k first medical image convolution feature map data sets is as follows:

[0045]

[0046] Among them, S sorted is the pixel value of the feature map.

[0047] In one possible implementation, in the aspect of performing integration processing on the N first medical image convolution feature map data sets to obtain an integrated second medical image convolution feature map data set, in the first processing unit, the expression of performing integration processing on the N first medical image convolution feature map data sets is as follows:

[0048]

[0049] Among them, F avg is the second medical image convolution feature map dataset, N is the number of the first medical image convolution feature map datasets for integration processing, S m is the mth first medical image convolution feature map dataset.

[0050] A third aspect of an embodiment of the present application provides a terminal, comprising a processor, an input device, an output device, and a memory, wherein the processor, input device, output device, and memory are interconnected, wherein the memory is used to store a computer program, the computer program comprising program instructions, and the processor is configured to call the program instructions to execute the step instructions of the medical image abnormality detection method of CLIP in the first aspect of the embodiment of the present application.

[0051] The fourth aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the above-mentioned computer-readable storage medium stores a computer program for electronic data exchange, wherein the above-mentioned computer program enables a computer to execute some or all of the steps described in the CLIP medical image abnormality detection method in the first aspect of the embodiment of the present application.

[0052] A fifth aspect of the present application provides a computer program product, wherein the computer program product includes a non-transitory computer-readable storage medium storing a computer program, wherein the computer program is operable to cause a computer to execute some or all of the steps described in the CLIP medical image anomaly detection method in the first aspect of the present application. The computer program product may be a software installation package. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0054] Figure 1 A schematic diagram of the overall process of a medical image enhancement method based on a convolutional neural network is provided for the embodiment of the present application;

[0055] Figure 2 The present invention provides a schematic diagram of a specific process of grouping medical image convolution feature maps in a medical image convolution feature map dataset in a medical image enhancement method based on a convolutional neural network, using an improved pnmix to adjust the convolutional neural network;

[0056] Figure 3A Schematic diagram of test results obtained by testing different medical anomaly detection methods provided by a medical image enhancement method based on a convolutional neural network in six medical anomaly detection datasets;

[0057] Figure 3B A schematic diagram of test results obtained using 10 groups of different medical image samples for a medical image enhancement method based on a convolutional neural network provided in an embodiment of the present application;

[0058] Figure 4 A schematic diagram of the structure of a medical image enhancement device based on a convolutional neural network provided in an embodiment of the present application;

[0059] Figure 5 A schematic diagram of the structure of a terminal provided in an embodiment of the present application;

[0060] Reference numerals:

[0061] Acquisition unit-1, first processing unit-2, second processing unit-3, image enhancement unit-4. DETAILED DESCRIPTION

[0062] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0063] The terms "first," "second," and the like in the specification and claims of this application and the accompanying drawings are used to distinguish between different objects, not to describe a particular order. Furthermore, the terms "including," "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or elements is not limited to the listed steps or elements but may optionally include steps or elements not listed, or may optionally include other steps or elements inherent to the process, method, product, or apparatus.

[0064] References to "embodiments" in this application mean that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described in this application may be combined with other embodiments.

[0065] In order to better understand the medical image enhancement method based on convolutional neural networks provided in the embodiments of the present application, the following first briefly introduces the scenarios in which the medical image enhancement method based on convolutional neural networks is applied. There are significant differences in data distribution between medical images and natural images, which is mainly reflected in the fact that medical images are mostly single-channel grayscale images with a lack of color information, and the texture and shape features in the images are often more complex and subtle. This characteristic makes traditional data enhancement methods, such as PNmix (a data enhancement method based on the idea of mixed images), ineffective when directly applied to medical images. Although the PNmix method can achieve significant results on natural images, it achieves fine color changes by adaptively changing the ratio between different channels of the input image, but this advantage cannot be fully utilized on medical images. Because the single-channel characteristics of medical images make the operation of PNmix in changing the channel ratio meaningless, it cannot effectively enhance the feature representation of the image.

[0066] The medical image enhancement method based on convolutional neural network is applied to the medical image enhancement device based on convolutional neural network. Figure 1 The figure shows a schematic diagram of the overall process of a medical image enhancement method based on convolutional neural network. Figure 1 Shown, including:

[0067] S1. Obtain a medical image dataset.

[0068] Among them, the medical image dataset can be connected to the medical system to obtain the patient's historical medical images for subsequent processing in this example.

[0069] S2. Based on the medical image dataset, use the first convolution layer of the convolutional neural network to output a medical image convolution feature map dataset.

[0070] The first convolutional layer of the CLIP network can be used for processing. Considering that the output feature maps of the CLIP network convolutional layer are essentially images, but with a different shape from the original image, and the feature maps obtained by the convolutional layer closer to the input layer, some texture and shape features of the original image can be seen, and different feature maps focus on different local textures or shapes. The principle of the pnmix algorithm used in this example is based on the positional normalization method. The mean and std feature maps extracted by this normalization method mainly reflect the texture and shape information of the original image. Therefore, the pnmix algorithm can be used to process the output feature maps of the CLIP network convolutional layer.

[0071] S3. Adjust the convolutional neural network using the improved pnmix according to the medical image convolution feature map dataset to obtain a medical image enhancement model.

[0072] Among them, pnmix can be performed on the output feature maps of the first convolutional layer of the CLIP network. Since the number of output feature maps of the first convolutional layer is large, a better way can be used to utilize the data of these channels. We divide the 1024 feature maps output by the first convolutional layer into different groups. This example uses a total of 11 different groups, 1, 2, 4, 8...1024, to calculate the pnmix results within the group. Different groups will obtain different new samples that meet the nearest neighbor distribution conditions. If the computing resources are sufficient, we can obtain a model for each different grouping and then take the optimal result.

[0073] However, considering the complexity of the grouping of 1024 feature maps, in order to save computing resources, it is also possible to group the model first (when grouping, the feature maps within the group can be randomly grouped. For example, the 1024 feature maps are divided into two large groups, the first group has 1 feature map, and the second group has 1023 feature maps. This example is only an example and is not limited). Then, the relatively optimal group is taken. Only one model can be trained. Without increasing the amount of additional calculations and parameters, the features of different groups can be used to obtain a suboptimal result (suboptimal means that the result is slightly lower than the result of training the model separately for different groups and then selecting the optimal one, but the result of taking the relatively optimal group while training within the model is still significantly improved compared to other methods). This result can be regarded as the optimal solution with a small amount of calculations and parameters.

[0074] S4. Use the medical image enhancement model to perform data enhancement on the medical image feature map to obtain an enhanced medical image feature map.

[0075] In this example, a medical image dataset is first obtained, and the first convolution layer of the convolutional neural network is used to output a medical image convolution feature map dataset. The convolutional neural network is adjusted using pnmix based on the medical image convolution feature map dataset, and then a medical image enhancement model is obtained. The pnmix method can be used to effectively enhance single-channel grayscale medical images, thereby improving the effect of medical image enhancement and further improving the enhancement accuracy of the medical image enhancement model based on the medical image convolution feature map dataset.

[0076] Figure 2 The figure shows a schematic diagram of the specific process of grouping medical image convolution feature maps in the process of adjusting the convolutional neural network using the improved pnmix. Figure 2 As shown, in one possible implementation, the convolutional neural network is adjusted using an improved pnmix based on the medical image convolution feature map dataset to obtain a medical image enhancement model, including:

[0077] S201, grouping the medical image convolution feature map dataset to obtain k medical image convolution feature map datasets;

[0078] S202, performing pnmix enhancement on the k medical image convolution feature map data sets in sequence to obtain k first medical image convolution feature map data sets;

[0079] Among them, the formula for pnmix is as follows:

[0080]

[0081] Among them, X b,c,h,w Represents the pixel value of the original image, μ b,h,w Represents the mean image obtained from different channels of the image. b,h,w Indicates that the image pixel value is about μ b,h,w The standard deviation of , where a very small value is added to prevent the denominator from being 0 during the calculation process. In the experiment, we take the value as 1e-5. and are the results of position normalization of images A and B, respectively, μ A and μ B are the position-normalized means of images A and B, σ A and σ B are the position-normalized standard deviations of images A and B, respectively.

[0082] The result of the pnmix algorithm is shown in the following formula:

[0083]

[0084] in, and are the results of position normalization of images A and B, respectively, μ A and μ B are the position-normalized means of images A and B, σ A and σ B are the position-normalized standard deviations of images A and B, respectively.

[0085] When the weight in the mixup algorithm is fixed at 0.5, the formula for mixup is as follows:

[0086]

[0087] Among them, X A and X B are the pixel values of images A and B respectively

[0088] Among them, X b,c,h,w Represents the pixel value of the original image, μ b,h,w Represents the mean image obtained from different channels of the image. b,h,w Indicates that the image pixel value is about μ b,h,w The standard deviation of μ is a small value added to prevent the denominator from being 0 during the calculation process. In the experiment, we take the value as 1e-5. It can be seen from formula (1) that when C=1, μ b,h,w In fact, it is equal to X b,c,h,w , that is, the mean image is equal to the original image. At this time, σ b,h,w is approximately equal to 0, since X A =μ A , and X B =μ B , from formulas (3) and (4), we can see that So we have:

[0089]

[0090] S203, performing feature map pixel value calculation on the k first medical image convolution feature map data sets to obtain k feature map pixel value calculation results;

[0091] S203. Sort the k first medical image convolution feature map data sets according to the calculation results of the k feature map pixel values to obtain a sorting result of the first medical image convolution feature map data sets;

[0092] S204: extracting top N first medical image convolution feature map datasets from the sorting results of the first medical image convolution feature map datasets;

[0093] S205. Integrate the N first medical image convolution feature map datasets to obtain an integrated second medical image convolution feature map dataset;

[0094] S206. Use the second medical image convolution feature map dataset to adjust the convolutional neural network to obtain a medical image enhancement model.

[0095] Specifically, in the convolution feature map, in most cases, the larger the pixel value of the feature image, the more significant this part of the feature is. Therefore, the process of selecting the optimal grouping scheme is as follows:

[0096]

[0097] where H and W are the height and width of the feature map, i is the number of channels of the feature map, m represents the mth group, is the result feature map obtained by pnmix for the mth group

[0098] Sort M S m to obtain:

[0099]

[0100] where each element within {} represents the sorted M groups of S m , a total of M groups

[0101] where, when 1 ≤ p < q ≤ M,

[0102]

[0103] where I = {j1, j2,..., j N}}, and is the index of the top N largest values taken from S sorted . [[ID=四十二]] is the result feature map obtained by pnmix for the mth group. Sum the result feature maps of each group to obtain the S of this group m , and then take the larger N groups to calculate the final result feature map F of the result pnmix avg .

[0104] In this example, first obtain a medical image dataset, use the first convolutional layer of the convolutional neural network to output a medical image convolution feature map dataset, and use pnmix to adjust the convolutional neural network according to the medical image convolution feature map dataset, thereby obtaining a medical image enhancement model, which can effectively enhance medical images of single-channel grayscale images through the pnmix method, so as to improve the effect of medical image enhancement, and further improve the enhancement accuracy of the medical image enhancement model based on the medical image convolution feature map dataset.

[0105] It should be noted that although Pnmix has shown better results than mixup in the field of natural images, in the field of medical images, since many images are single-channel images (even if a few medical images are three-channel images, the colors are also very single), the advantage of this method in terms of color information cannot be directly utilized. If pnmix is directly applied to medical images, the effect is not good. Figure 3A As shown, Figure 3A The data in Table 1 is a schematic diagram of the test results obtained by testing different medical anomaly detection methods on six medical anomaly detection data sets.

[0106] Furthermore, the inventors used the medical image enhancement method based on convolutional neural network of the present invention to conduct experiments on different specific samples for the first time. The experimental data showed that the results of the existing methods would fluctuate greatly under different specific sample conditions. However, no matter which specific samples are used, the method of the present invention can achieve different degrees of improvement in a relatively stable manner. The specific results are as follows: Figure 3B As shown, Figure 3B A schematic diagram of test results obtained by using 10 groups of different medical image samples for a medical image enhancement method based on convolutional neural networks is shown.

[0107] In line with the above, please see Figure 2 , Figure 2 The present invention provides a schematic diagram of a medical image enhancement device based on a convolutional neural network. Figure 2 As shown, the device includes:

[0108] An acquisition unit 1, configured to acquire a medical image dataset;

[0109] A first processing unit 2 is configured to output a medical image convolution feature map dataset using a first convolution layer of a convolutional neural network according to the medical image dataset;

[0110] A second processing unit 3 is configured to adjust the convolutional neural network using an improved pnmix according to the medical image convolution feature map dataset to obtain a medical image enhancement model;

[0111] The image enhancement unit 4 is used to perform data enhancement on the medical image feature map using the medical image enhancement model to obtain an enhanced medical image feature map.

[0112] In the aspect of adjusting the convolutional neural network using the improved pnmix according to the medical image convolution feature map dataset to obtain the medical image enhancement model, the first processing unit is configured to:

[0113] Grouping the medical image convolution feature map dataset to obtain k medical image convolution feature map datasets;

[0114] Performing pnmix enhancement on the k medical image convolution feature map data sets in sequence to obtain k first medical image convolution feature map data sets;

[0115] Performing feature map pixel value calculation on the k first medical image convolution feature map data sets to obtain k feature map pixel value calculation results;

[0116] Sorting the k first medical image convolution feature map data sets according to calculation results of the k feature map pixel values to obtain a sorting result of the first medical image convolution feature map data set;

[0117] Extracting top N first medical image convolution feature map datasets from the sorting results of the first medical image convolution feature map datasets;

[0118] Performing integration processing on the N first medical image convolution feature map data sets to obtain an integrated second medical image convolution feature map data set;

[0119] The convolutional neural network is adjusted using a second medical image convolution feature map dataset to obtain a medical image enhancement model.

[0120] In one possible implementation, in the aspect of calculating feature map pixel values for the k first medical image convolution feature map data sets to obtain k feature map pixel value calculation results, in the first processing unit, the expression form of the calculated feature pixel value is as follows:

[0121]

[0122] Among them, H and W are the height and width of the feature map, i is the number of channels of the feature map, and m represents the mth group. It is the result feature map obtained by the mth group pnmix.

[0123] In a possible implementation, in the aspect of sorting the k first medical image convolution feature map data sets according to the calculation results of the k feature map pixel values to obtain the sorting result of the first medical image convolution feature map data sets, in the first processing unit, the expression form of sorting the k first medical image convolution feature map data sets is as follows:

[0124]

[0125] Among them, S sorted is the pixel value of the feature map.

[0126] In one possible implementation, in the aspect of performing integration processing on the N first medical image convolution feature map data sets to obtain an integrated second medical image convolution feature map data set, in the first processing unit, the expression of performing integration processing on the N first medical image convolution feature map data sets is as follows:

[0127]

[0128] Among them, F avg is the second medical image convolution feature map dataset, N is the number of the first medical image convolution feature map datasets for integration processing, S m is the mth first medical image convolution feature map dataset.

[0129] For the same example as above, please refer to Figure 4 , Figure 4 A schematic structural diagram of a terminal provided in an embodiment of the present application, as shown in the figure, includes a processor, an input device, an output device, and a memory, the processor, the input device, the output device, and the memory being interconnected, wherein the memory is used to store a computer program, the computer program including program instructions, the processor being configured to call the program instructions, and the program including instructions for executing the following steps;

[0130] Obtain medical image datasets;

[0131] According to the medical image dataset, the first convolutional layer of the CLIP network is used to output a medical image feature dataset;

[0132] Adjusting the CLIP network using pnmix according to the medical image feature dataset to obtain a medical image enhancement model;

[0133] The medical image enhancement model is used to perform abnormality detection on the medical image to obtain a medical image abnormality detection result.

[0134] In this example, a medical image dataset is first obtained, and the first convolutional layer of the CLIP network is used to output the medical image feature dataset. The CLIP network is then adjusted using pnmix based on the medical image feature dataset, thereby obtaining a medical image enhancement model that can effectively improve the recognition of single-channel grayscale medical images, thereby improving the accuracy of medical image anomaly detection and recognition.

[0135] The above mainly introduces the scheme of the embodiment of the present application from the perspective of the execution process on the method side. It is understandable that, in order to implement the above functions, the terminal includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of the various examples described in the embodiments provided herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in a hardware or computer software driven hardware manner depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0136] The embodiment of the present application can divide the terminal into functional units according to the above method example. For example, each functional unit can be divided according to each function, or two or more functions can be integrated into one processing unit. The above integrated unit can be implemented in the form of hardware or in the form of software functional units. It should be noted that the division of units in the embodiment of the present application is schematic and is only a logical functional division. In actual implementation, there may be other division methods.

[0137] An embodiment of the present application also provides a computer storage medium, wherein the computer storage medium stores a computer program for electronic data exchange, and the computer program enables a computer to execute part or all of the steps of any one of the medical image enhancement methods based on convolutional neural networks as described in the above method embodiments.

[0138] An embodiment of the present application also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program enables a computer to execute part or all of the steps of any one of the medical image enhancement methods based on convolutional neural networks as described in the above method embodiments.

[0139] It should be noted that for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.

[0140] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0141] In the several embodiments provided in this application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0142] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0143] In addition, the functional units in the various embodiments of the application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software program modules.

[0144] If the integrated unit is implemented in the form of a software program module and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a memory, including a number of instructions for enabling a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned memory includes: various media that can store program codes, such as a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0145] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable memory, which may include: a flash drive, a read-only memory, a random access memory, a magnetic disk or an optical disk, etc.

[0146] The above is a detailed introduction to the embodiments of the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core idea of the present application. At the same time, for those skilled in the art, according to the idea of the present application, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A medical image enhancement method based on convolutional neural network, characterized in that: include: Obtain medical image datasets; According to the medical image dataset, the first convolution layer of the convolutional neural network is used to output a medical image convolution feature map dataset; Adjusting the convolutional neural network using an improved pnmix according to the medical image convolution feature map dataset to obtain a medical image enhancement model; The medical image enhancement model is used to perform data enhancement on the medical image feature map to obtain an enhanced medical image feature map.

2. The medical image enhancement method based on convolutional neural network according to claim 1, characterized in that: The method of adjusting the convolutional neural network using the improved pnmix according to the medical image convolution feature map dataset to obtain a medical image enhancement model includes: Grouping the medical image convolution feature map dataset to obtain k medical image convolution feature map datasets; Performing pnmix enhancement on the k medical image convolution feature map data sets in sequence to obtain k first medical image convolution feature map data sets; Performing feature map pixel value calculation on the k first medical image convolution feature map data sets to obtain k feature map pixel value calculation results; Sorting the k first medical image convolution feature map data sets according to calculation results of the k feature map pixel values to obtain a sorting result of the first medical image convolution feature map data set; Extracting top N first medical image convolution feature map datasets from the sorting results of the first medical image convolution feature map datasets; Performing integration processing on the N first medical image convolution feature map data sets to obtain an integrated second medical image convolution feature map data set; The convolutional neural network is adjusted using a second medical image convolution feature map dataset to obtain a medical image enhancement model.

3. The medical image enhancement method based on convolutional neural network according to claim 2, characterized in that: The feature map pixel values of the k first medical image convolution feature map data sets are calculated to obtain k feature map pixel value calculation results. The calculated feature pixel values are expressed as follows: Among them, H and W are the height and width of the feature map, i is the number of channels of the feature map, and m represents the mth group. It is the result feature map obtained by the mth group pnmix.

4. The medical image enhancement method based on convolutional neural network according to claim 2, characterized in that: The k first medical image convolution feature map data sets are sorted according to the calculation results of the k feature map pixel values to obtain the sorting result of the first medical image convolution feature map data sets. The expression form of sorting the k first medical image convolution feature map data sets is as follows: Among them, S sorted is the pixel value of the feature map.

5. The medical image enhancement method based on convolutional neural network according to claim 2, characterized in that: The integration processing is performed on the N first medical image convolution feature map data sets to obtain an integrated second medical image convolution feature map data set. The expression of the integration processing on the N first medical image convolution feature map data sets is as follows: Among them, F avg is the second medical image convolution feature map dataset, N is the number of the first medical image convolution feature map datasets for integration processing, S m is the mth first medical image convolution feature map dataset.

6. A medical image enhancement device based on convolutional neural network, characterized in that: include: an acquisition unit, configured to acquire a medical image dataset; A first processing unit is configured to output a medical image convolution feature map dataset using a first convolution layer of a convolutional neural network according to the medical image dataset; A second processing unit is configured to adjust the convolutional neural network using an improved pnmix according to the medical image convolution feature map dataset to obtain a medical image enhancement model; The image enhancement unit is used to perform data enhancement on the medical image feature map using the medical image enhancement model to obtain an enhanced medical image feature map.

7. The medical image enhancement device based on convolutional neural network according to claim 6, characterized in that: In the aspect of adjusting the convolutional neural network using the improved pnmix according to the medical image convolution feature map dataset to obtain the medical image enhancement model, the first processing unit is configured to: Grouping the medical image convolution feature map dataset to obtain k medical image convolution feature map datasets; Performing pnmix enhancement on the k medical image convolution feature map data sets in sequence to obtain k first medical image convolution feature map data sets; Performing feature map pixel value calculation on the k first medical image convolution feature map data sets to obtain k feature map pixel value calculation results; Sorting the k first medical image convolution feature map data sets according to calculation results of the k feature map pixel values to obtain a sorting result of the first medical image convolution feature map data set; Extracting top N first medical image convolution feature map datasets from the sorting results of the first medical image convolution feature map datasets; Performing integration processing on the N first medical image convolution feature map data sets to obtain an integrated second medical image convolution feature map data set; The convolutional neural network is adjusted using a second medical image convolution feature map dataset to obtain a medical image enhancement model.

8. The medical image enhancement device based on convolutional neural network according to claim 6, characterized in that: In the aspect of performing feature map pixel value calculation on the k first medical image convolution feature map data sets to obtain k feature map pixel value calculation results, in the first processing unit, the expression form of the calculated feature pixel value is as follows: Among them, H and W are the height and width of the feature map, i is the number of channels of the feature map, and m represents the mth group. It is the result feature map obtained by the mth group pnmix.

9. A terminal, characterized in that: The medical image enhancement method comprises a processor, an input device, an output device and a memory, wherein the processor, the input device, the output device and the memory are interconnected, wherein the memory is used to store a computer program, the computer program includes program instructions, and the processor is configured to call the program instructions to execute the medical image enhancement method based on a convolutional neural network as described in any one of claims 1 to 5.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which includes program instructions. When the program instructions are executed by a processor, the processor executes the medical image enhancement method based on convolutional neural network as described in any one of claims 1 to 5.

Images