An AI training method and system for big data denoising optimization

Abstract

The present invention provides an AI training method and system for big data denoising optimization, relating to the field of image denoising technology. The method includes: obtaining preset basic information of a preset filter; building M image denoising models; obtaining an image denoising target of the preset AI model, and obtaining an image sample set; obtaining N screening models; obtaining N denoised image sets respectively; performing loss analysis on the N denoised image sets in sequence to obtain N loss results, and analyzing and matching the optimal denoising model corresponding to the minimum loss result; performing image denoising on a first target image through the optimal denoising model to obtain an output denoised image, and training to obtain the preset AI model, which solves the technical problem in the prior art that the image denoising effect is poor and the distortion is serious, which leads to poor AI model training speed and accuracy, and achieves the technical effect of improving the image denoising effect and ensuring the AI ​​model training speed and accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of image denoising, and in particular to an AI training method and system for big data denoising optimization. Background Art

[0002] In most cases, noise is introduced during image acquisition or transmission, reducing image quality. Due to factors such as different transmission paths or sensors, the types of noise in noisy images vary, and even the same image can contain multiple types of noise. This noise can affect the training speed and accuracy of AI models, making it necessary to perform noise reduction on the images fed into the AI ​​model.

[0003] At present, there are technical problems in the existing technology due to inappropriate selection of denoising filters, poor image denoising effect and severe distortion, which in turn leads to poor AI model training speed and accuracy. Summary of the Invention

[0004] The present invention provides an AI training method and system for big data denoising optimization, which is used to solve the technical problem in the prior art that the image denoising effect is poor and the distortion is severe due to inappropriate selection of denoising filters, which leads to poor AI model training speed and accuracy.

[0005] According to a first aspect of the present invention, an AI training method for big data denoising optimization is provided, comprising: obtaining preset basic information of a preset filter, the preset basic information comprising filter type information and filter feature information; constructing the preset filter according to the filter type information to obtain M image denoising models, where M is an integer greater than 1; obtaining an image denoising target of the preset AI model, and obtaining an image sample set; screening the M image denoising models based on the image denoising target and the filter feature information to obtain N screening models, where 1<N≤M; using the image sample set as input information, sequentially inputting the N screening models to obtain output results, respectively, where the output results include N denoised image sets; introducing a preset loss function to perform loss analysis on the N denoised image sets in sequence to obtain N loss results, and analyzing the optimal denoising model corresponding to the matching minimum loss result; performing image denoising on the first target image through the optimal denoising model to obtain an output denoised image, and training to obtain the preset AI model.

[0006] According to a second aspect of the present invention, an AI training system for big data denoising optimization is provided, comprising: a preset basic information acquisition module, the preset basic information acquisition module is used to obtain preset basic information of a preset filter, the preset basic information including filter type information and filter feature information; an image denoising model building module, the image denoising model building module is used to build the preset filter according to the filter type information to obtain M image denoising models, where M is an integer greater than 1; an AI model analysis module, the AI ​​model analysis module is used to obtain an image denoising target of a preset AI model and obtain an image sample set; a model screening module, the model screening module is used to, based on the image denoising target and the filter feature information, build an image denoising model among the M image denoising models. The model is screened to obtain N screening models, wherein 1<N≤M; a denoised image set acquisition module, wherein the denoised image set acquisition module is used to take the image sample set as input information, input the N screening models in sequence, and obtain output results respectively, wherein the output results include N denoised image sets; a minimum loss result matching module, wherein the minimum loss result matching module is used to introduce a preset loss function to perform loss analysis on the N denoised image sets in sequence, obtain N loss results, and analyze and match the optimal denoising model corresponding to the minimum loss result; a preset AI model training module, wherein the preset AI model training module is used to perform image denoising on the first target image through the optimal denoising model to obtain an output denoised image, and train to obtain the preset AI model.

[0007] According to an AI training method for big data denoising optimization adopted by the present invention, the present invention obtains preset basic information of a preset filter, wherein the preset basic information includes filter type information and filter feature information, and builds the preset filter according to the filter type information to obtain M image denoising models, obtains the image denoising target of the preset AI model, and obtains an image sample set. Based on the image denoising target and the filter feature information, the M image denoising models are screened to obtain N screening models, and the image sample set is used as input information and sequentially input into the N screening models to obtain output results respectively, wherein the output results include N denoised image sets, and a preset loss function is introduced to perform loss analysis on the N denoised image sets in sequence to obtain N loss results, and the optimal denoising model corresponding to the minimum loss result is analyzed and matched, and the first target image is denoised by the optimal denoising model to obtain an output denoised image, and the preset AI model is trained to achieve the technical effect of ensuring the image denoising effect and improving the training speed and output accuracy of the preset AI model.

[0008] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely exemplary, and a person of ordinary skill in the art can obtain other drawings based on the provided drawings without creative work.

[0010] Figure 1 A flowchart of an AI training method for big data denoising optimization provided by an embodiment of the present invention;

[0011] Figure 2 Schematic diagram of the process of obtaining an image sample set in an embodiment of the present invention;

[0012] Figure 3 Schematic diagram of the process of obtaining the optimal denoising model in an embodiment of the present invention;

[0013] Figure 4 A schematic diagram of the structure of an AI training system for big data denoising and optimization provided by an embodiment of the present invention.

[0014] Explanation of the accompanying drawings: preset basic information acquisition module 11, image denoising model building module 12, AI model analysis module 13, model screening module 14, denoised image set acquisition module 15, minimum loss result matching module 16, preset AI model training module 17. DETAILED DESCRIPTION

[0015] The following description of exemplary embodiments of the present invention is provided in conjunction with the accompanying drawings, in which various details of the embodiments of the present invention are included to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0016] In order to solve the technical problem in the prior art that due to inappropriate selection of denoising filters, image denoising effect is poor and distortion is serious, which leads to poor AI model training speed and accuracy, the inventors of the present invention have obtained an AI training method and system for big data denoising optimization through creative work.

[0017] Example 1

[0018] Figure 1 An AI training method diagram for big data denoising optimization provided by an embodiment of the present invention is shown in FIG. Figure 1 As shown, the method includes:

[0019] Step S100: obtaining preset basic information of a preset filter, wherein the preset basic information includes filter type information and filter feature information;

[0020] Specifically, the preset filter includes a variety of different types of filters, which need to be selected according to actual conditions to obtain the preset basic information of the preset filter. The preset basic information includes filter type information and filter feature information. Simply put, different types of filters can achieve different filtering effects. For example, the mean filter is a linear filter that can eliminate sharp noise in the image and achieve image smoothing, blurring and other functions; the median filter can effectively eliminate salt and pepper noise (impulse noise), with the filtering effect as the filter feature information. The filter type information includes mean filter, median filter, Wiener filter, Gaussian filter, cone filter, etc. There is a one-to-one correspondence between the filter type information and the filter feature information.

[0021] Step S200: constructing the preset filter according to the filter type information to obtain M image denoising models, where M is an integer greater than 1;

[0022] In this embodiment of the present invention, step S200 further includes:

[0023] Step S210: setting the filter quantity constraint;

[0024] Step S220: constructing the preset filter in combination with the filter type information and the filter quantity constraint to obtain the M image denoising models.

[0025] Specifically, the preset filter is constructed according to the filter type information, and the image denoising model is obtained by constructing at least two types of preset filters. That is, the number of types of preset filters is determined according to the filter type information, and multiple different types of filters are integrated. For the same type of filters, the order of integration and construction is different, and the resulting image denoising models are different. For example, one image denoising model is: filter a→filter b, and the other image denoising model is: filter b→filter a, then the two image denoising models are also different, thereby obtaining M image denoising models, where M is an integer greater than 1.

[0026] When constructing the preset filters, a filter quantity constraint is first set. The filter quantity constraint includes a minimum and maximum filter quantity constraint value, which can be understood as a numerical range. For example, if an image denoising model is constructed using at least two types of preset filters, the minimum constraint value is set to 2. Meanwhile, if no maximum constraint is set on the number of filters, a large number of image denoising models can be obtained as long as the preset filter types are sufficiently numerous. However, integrating too many filters can lead to poor image denoising effects, severe image distortion, and a large amount of computation. Therefore, a maximum constraint value needs to be set, such as 4, so that no more than four types of preset filters can be used to construct the image denoising model. The preset filters are further constructed in combination with the filter type information and the filter quantity constraint condition to obtain the M image denoising models. The number of filter types contained in the M image denoising models is ensured to not exceed the filter quantity constraint range, thereby ensuring the denoising effect and reducing the computational complexity.

[0027] Step S300: Obtain an image denoising target of a preset AI model and obtain an image sample set;

[0028] In this embodiment of the present invention, step S310 includes:

[0029] Step S311: Analyze the preset AI model to determine image retention features, image acquisition environment, and image transmission path;

[0030] Step S312: analyzing the image noise type based on the image acquisition environment and the image transmission path;

[0031] Step S313: determining the image noise reduction target according to the image noise type and the image preservation feature.

[0032] Among them, such as Figure 2 As shown, step S320 of the embodiment of the present invention includes:

[0033] Step S321: obtaining an original image set;

[0034] Step S322: Analyze the image noise type to obtain a set of pure noise images;

[0035] Step S323: adding noise according to the pure noise image set and the original image to obtain a first noisy image;

[0036] Step S324: preprocessing the first noisy image to obtain a preprocessed image set;

[0037] Step S325: performing image enhancement processing on the pre-processed image set to obtain the image sample set, wherein the number of images in the image sample set is greater than that in the pre-processed image set and greater than that in the first noisy image set.

[0038] Specifically, the preset AI model needs to be determined based on the actual situation. It can be a facial recognition model, a defect recognition model, an image texture anomaly recognition model, etc., without limitation. The preset AI model is analyzed to determine the image denoising target. Generally speaking, different preset AI models aim to achieve different denoising effects. For example, a defect recognition model focuses on image edge features, so image denoising requires preserving these features. Furthermore, different preset AI models correspond to different types of noise in the input image. The image denoising target is to filter out noise while preserving the desired features. The image sample set is obtained by adding noise to the original image. Simply put, the image acquisition process necessarily requires image transmission. Although it is possible to artificially reduce the number of transmission paths, it is still difficult to obtain a completely clean, noise-free image. Therefore, the original image is treated as a noise-free image. Noise is added to the original image to simulate a noisy image. The denoising effect is then analyzed by comparing the denoised image with the original image during the subsequent denoising process.

[0039] Specifically, the process of obtaining the image denoising target of the preset AI model is as follows: first, the preset AI model is analyzed to determine the image retention features, image acquisition environment and image transmission path. The image retention features are the image features required by the preset AI model. For example, the defect recognition model needs to identify the defect location and needs to retain the image edge features and texture features; the image acquisition environment refers to the image acquisition device and the surrounding environment equipment, and the image transmission path refers to the process of transmitting the image from the acquisition device to other devices. The image noise type is analyzed based on the image acquisition environment and the image transmission path. That is to say, the image will inevitably be polluted by noise during the acquisition, device transmission and imaging process, and the type of noise generated needs to be analyzed. For example, there are many devices that generate image information (such as cameras, electronic screens, etc.). These devices will generate noise (such as Gaussian noise, channel-dependent noise, etc.) due to the device itself when generating images. During the image transmission process, sensitive components will generate electronic noise due to the thermal motion of electrons in the device. This type of noise is ubiquitous, and its characteristics can be simulated and characterized by Gaussian white noise. Based on this, the image noise reduction target is determined according to the image noise type and the image retention feature. The image noise reduction target refers to retaining the required features while filtering out different types of noise, providing data support for subsequent image noise reduction.

[0040] Specifically, the process of obtaining an image sample set is as follows: First, an original image set is obtained. The original image set can be obtained based on big data and the input images required by a preset AI model. The original image set contains multiple images. The image noise type is analyzed to obtain a pure noise image set. The pure noise image set is a pure noise spectrum obtained based on the image noise type. In other words, the pure noise image set only contains the noise spectrum corresponding to the image noise type and does not contain any useful information. The noise spectrum of the corresponding noise type can be obtained based on big data.

[0041] Further noise is added based on the pure noise image set and the original image. Simply put, it is a process of noise simulation. The type of noise that may be generated during the image acquisition process is determined according to the preset AI model, and the original image and the pure noise image set are superimposed. Specifically, an image may contain one or more types of noise. Therefore, when adding noise, different noise images can be added in combination to any original image based on the pure noise image set to obtain a first noisy image. The first noisy image contains multiple noisy images, and the noise types contained in any two noisy images are different.

[0042] The first noisy image is then preprocessed to obtain a preprocessed image set. The preprocessing process is as follows: the first noisy image is divided into smaller image blocks. For example, since the original image has large pixels, a sliding window with a step size of 100 can be used for cropping, or the image width and height can be divided into 256 segments each. A sliding window is used to take one pixel from each segment, so that the size of each pixel block is 256×256. Generally speaking, smaller image blocks have better denoising performance. Therefore, the first noisy image needs to be preprocessed to improve the noise reduction effect.

[0043] The preprocessed image set is subjected to image enhancement processing to obtain the image sample set. The image enhancement processing process is to enrich the number of image samples. That is, the number of image samples contained in the preprocessed image set may be insufficient, and the types of noise images are relatively few. Therefore, the image samples in the preprocessed image set can be subjected to multi-dimensional noise fusion by scaling, rotating, flipping, interpolation, etc., that is, different types of noise are fused to obtain an image sample set, which can enrich the number of image samples and the type of noise.

[0044] Step S400: Based on the image denoising target and the filter feature information, screening is performed among the M image denoising models to obtain N screening models, where 1<N≤M;

[0045] In this embodiment of the present invention, step S400 includes:

[0046] Step S410: performing filter matching in the preset filters based on the image denoising target and the filter feature information to obtain a first matched filter set;

[0047] Step S420: Obtain M groups of filter type information for the M image denoising models;

[0048] Step S430: Based on the first matched filter set, matching is performed in the M groups of filter type information to obtain the N screening models.

[0049] Specifically, based on the image denoising target and the filter feature information, screening is performed among the M image denoising models to obtain N screening models, wherein N is an integer greater than 1 and less than or equal to M. That is, different types of filters can filter out different types of noise. While filtering out noise, some image information may also be filtered out, such as filtering out image texture features. Therefore, it is necessary to determine the type of noise that needs to be filtered out and the image features that need to be retained according to the image denoising target. Based on this, the type of filter that can be used is determined, and then screening is performed among the M image denoising models according to the filter types used to build the M image denoising models to obtain N screening models.

[0050] Specifically, based on the image denoising target and the filter feature information, screening is performed in the M image denoising models to obtain N screening models as follows: based on the image denoising target and the filter feature information, filter matching is performed in the preset filter to obtain a first matched filter set. The first matched filter set includes a variety of different types of filters. Specifically, the type of noise to be filtered out and the image features to be retained are determined according to the image denoising target. Based on this, the types of filters that can be used are determined to form the first matched filter set. M groups of filter type information for the M image denoising models are further obtained. The filter type information is the filter type used to build any image denoising model. One image denoising model corresponds to one group of filter type information. Based on the first matched filter set, matching is performed in the M groups of filter type information, and the image denoising model corresponding to the matched filter type information is obtained as the N screening models. It should be noted that when matching the M groups of filter type information, as long as the filter type contained in the filter type information belongs to the first matched filter set, the group of filter type information can be considered to be matched successfully. For example, there are three types of filters in the first matched filter set, namely filter A, filter B, and filter C. Then, as long as any group of filter type information contains at least two types of filters, the match can be considered successful.

[0051] Step S500: using the image sample set as input information and sequentially inputting the N screening models to obtain output results, wherein the output results include N denoised image sets;

[0052] Specifically, N screening models are constructed from multiple filters and used to denoise images. An image sample set, containing multiple images with noise superimposed on them, is used as input and fed sequentially into the N screening models. Each of the N screening models then performs denoising on the multiple images in the set, generating output results. The output results include N sets of denoised images, with one screening model corresponding to each set.

[0053] Step S600: introducing a preset loss function to sequentially perform loss analysis on the N denoised image sets to obtain N loss results, and analyzing the optimal denoising model corresponding to the minimum loss result;

[0054] Among them, such as Figure 3 As shown, step S600 of the embodiment of the present invention includes:

[0055] Step S610: establishing a first mapping relationship between the original image set and the N denoised image sets;

[0056] Step S620: constructing the preset loss function according to the first mapping relationship;

[0057] Step S630: Obtaining the N loss results corresponding to the N denoised image sets according to the loss function;

[0058] Step S640: The screening model corresponding to the minimum loss result is used as the optimal denoising model.

[0059] In this embodiment of the present invention, step S620 includes:

[0060] Step S621: Obtain a first image denoising evaluation index;

[0061] Step S622: obtaining a first index value of the original image set according to the first image denoising evaluation index;

[0062] Step S623: Obtaining a second index value of the N denoised image sets;

[0063] Step S624: constructing the preset loss function using the first indicator value and the second indicator value.

[0064] Specifically, a preset loss function is introduced to perform loss analysis on the N denoised image sets in turn. By analyzing the differences between the original image and the images in the N denoised image sets, N loss results are obtained. The minimum loss result is screened out from the N loss results, and the screening model corresponding to the minimum loss result is used as the optimal denoising model.

[0065] Specifically, a preset loss function is introduced to sequentially perform loss analysis on the N denoised image sets to obtain N loss results, and the process of analyzing and matching the optimal denoising model corresponding to the minimum loss result is as follows: First, a first mapping relationship between the original image set and the N denoised image sets is established. It should be noted that the process of obtaining the image sample set is to superimpose the original image and the pure noise image, and the superimposed image is preprocessed and enhanced. Therefore, the number of sample images in the image sample set is much larger than the number of images in the original image set. One original image set corresponds to N denoised image sets. Therefore, when establishing the first mapping relationship between the original image set and the N denoised image sets, it is necessary to first establish a mapping relationship between the sample images in the sample image set and the original images in the original image set, specifically the corresponding positions of the sample images in the original images. Then, based on the corresponding relationship between the original image set and the N denoised image sets, a first mapping relationship between the original image set and the N denoised image sets is established. The first mapping relationship refers to the positions of the images in the N denoised image sets in the original images. The images in the original image set and the N denoised image sets are marked according to the first mapping relationship.

[0066] The preset loss function is further constructed based on the first mapping relationship. The preset loss function is the difference between the images in the original image set and the N denoised image sets. In layman's terms, the original image set is regarded as a clean image without noise. Noise is added to the original image to obtain a first noisy image. The first noisy image is then denoised using N screening models. Ideally, the images in the N denoised image sets obtained after denoising should be consistent with the original image set. Therefore, by comparing the differences between the N denoised image sets and the images in the original image set, the difference calculation formula between the two is used as the preset loss function. Then, the N loss results corresponding to the N denoised image sets are obtained according to the loss function. Finally, the N loss results are sorted in ascending order, and the screening model corresponding to the minimum loss result is matched as the optimal denoising model to ensure the image denoising effect and improve the training convergence speed and accuracy of the preset AI model.

[0067] The process of constructing the loss function based on the first mapping relationship is as follows: obtaining a first image denoising evaluation index, which refers to a reference index for denoising effects, such as an image's grayscale value, RGB color value, pixel coordinates, etc. Based on the first image denoising evaluation index, obtaining a first index value for the original image set and obtaining a second index value for the set of N denoised images. The first index value and the second index value can be one or more of multiple image denoising evaluation indexes, such as an image's grayscale value, RGB color value, pixel coordinates, etc. The preset loss function is constructed using the first index value and the second index value.

[0068] The preset loss function is as follows:

[0069]

[0070] Among them, LOSS(N) represents the loss result of the Nth denoised image set; X represents the denoised image set; Y represents the original image set; μ x Represents the mean grayscale value of the pixels in the denoised image set; μ y Represents the mean of the grayscale values ​​of the pixels in the original image set; Characterizes the variance of the grayscale values ​​of pixels in the denoised image set; Characterizes the variance of the grayscale values ​​of the pixels in the original image set; σ xy is the covariance of the grayscale values ​​of X and Y; C1 and C2 are constants (to avoid the denominator being zero and remain stable).

[0071] It should be noted that the above preset loss function is based on the comparison of the gray value of the pixel, and can also be based on the comparison of the RGB color value, the coordinates of the pixel point, etc., just by changing μ x 、μ y 、 and σ xy The meaning of the reference is transformed into RGB color values ​​or pixel coordinates, or the loss results corresponding to the grayscale values, RGB color values ​​and pixel coordinates can be calculated separately, and then the loss results of the three are averaged, and the average value is used as the final loss result, thereby obtaining the N loss results corresponding to the N denoised image sets. Of course, other indicators for evaluating the image denoising effect can also be selected, and there is no limitation here.

[0072] Step S700: Perform image denoising on the first target image using the optimal denoising model to obtain an output denoised image, and train to obtain the preset AI model.

[0073] Specifically, the first target image refers to the training image collected when the preset AI model needs to be trained, and is input into the optimal denoising model to obtain an output denoised image. The preset AI model is trained with the output denoised image to improve the training speed and output accuracy of the preset AI model.

[0074] Based on the above analysis, it can be seen that the present invention provides an AI training method for big data denoising optimization. In this embodiment, preset basic information of a preset filter is obtained, and the preset basic information includes filter type information and filter feature information. The preset filter is built according to the filter type information to obtain M image denoising models, and the image denoising target of the preset AI model is obtained. An image sample set is obtained, and based on the image denoising target and the filter feature information, screening is performed in the M image denoising models to obtain N screening models. The image sample set is used as input information and is sequentially input into the N screening models to obtain output results respectively, wherein the output results include N denoised image sets, and a preset loss function is introduced to perform loss analysis on the N denoised image sets in sequence to obtain N loss results, and the optimal denoising model corresponding to the minimum loss result is analyzed and matched. The first target image is subjected to image denoising by the optimal denoising model to obtain an output denoised image, and the preset AI model is trained to achieve the technical effect of ensuring the image denoising effect and improving the training speed and output accuracy of the preset AI model.

[0075] Example 2

[0076] Based on the same inventive concept as the AI ​​training method for big data denoising optimization in the aforementioned embodiment, Figure 4 As shown, the present invention also provides an AI training system for big data denoising optimization, the system comprising:

[0077] A preset basic information acquisition module 11 is used to acquire preset basic information of a preset filter, wherein the preset basic information includes filter type information and filter feature information;

[0078] An image denoising model building module 12 is configured to build the preset filter according to the filter type information to obtain M image denoising models, where M is an integer greater than 1;

[0079] An AI model analysis module 13, which is used to obtain an image denoising target of a preset AI model and obtain an image sample set;

[0080] A model screening module 14 is configured to screen the M image denoising models based on the image denoising target and the filter feature information to obtain N screening models, where 1<N≤M;

[0081] a denoised image set acquisition module 15, configured to take the image sample set as input information, sequentially input the N screening models, and obtain output results, wherein the output results include N denoised image sets;

[0082] A minimum loss result matching module 16 is configured to introduce a preset loss function to sequentially perform loss analysis on the N denoised image sets to obtain N loss results, and analyze and match the optimal denoising model corresponding to the minimum loss result;

[0083] The preset AI model training module 17 is used to perform image denoising on the first target image through the optimal denoising model to obtain an output denoised image, and train the preset AI model.

[0084] Furthermore, the system further comprises:

[0085] A constraint condition setting module, wherein the constraint condition setting module is used to set a filter quantity constraint condition;

[0086] A preset filter building module is used to build the preset filter in combination with the filter type information and the filter quantity constraint condition to obtain the M image denoising models.

[0087] Furthermore, the system further comprises:

[0088] An image feature analysis module, configured to analyze the preset AI model to determine image retention features, image acquisition environment, and image transmission path;

[0089] An image noise type analysis module, configured to analyze the image noise type based on the image acquisition environment and the image transmission path;

[0090] An image noise reduction target determination module is configured to determine the image noise reduction target according to the image noise type and the image preservation feature.

[0091] Furthermore, the system further comprises:

[0092] A pure noise image set acquisition module, configured to analyze the image noise type and acquire a pure noise image set;

[0093] a noise adding module, configured to add noise according to the pure noise image set and the original image to obtain a first noisy image;

[0094] a preprocessing module, configured to preprocess the first noisy image to obtain a preprocessed image set;

[0095] An image enhancement processing module is used to perform image enhancement processing on the preprocessed image set to obtain the image sample set, wherein the number of images in the image sample set is greater than that in the preprocessed image set and greater than that in the first noisy image set.

[0096] Furthermore, the system further comprises:

[0097] a filter matching module, configured to perform filter matching in the preset filter based on the image noise reduction target and the filter feature information to obtain a first matched filter set;

[0098] A filter construction type determination module, wherein the filter construction type determination module is used to obtain M groups of construction filter type information of the M image denoising models;

[0099] A filter type screening module is used to match the filter type information of the M groups based on the first matched filter set to obtain the N screening models.

[0100] Furthermore, the system further comprises:

[0101] a first mapping relationship establishing module, configured to establish a first mapping relationship between the original image set and the N denoised image sets;

[0102] A preset loss function building module, wherein the preset loss function building module is used to build the preset loss function according to the first mapping relationship;

[0103] A loss result output module, configured to obtain the N loss results corresponding to the N denoised image sets according to the loss function;

[0104] The optimal denoising model determination module is used to use the screening model corresponding to the minimum loss result as the optimal denoising model.

[0105] Furthermore, the system further comprises:

[0106] A first evaluation index acquisition module, wherein the first evaluation index acquisition module is used to obtain a first image denoising evaluation index;

[0107] a first index value acquisition module, configured to acquire a first index value of the original image set according to the first image denoising evaluation index;

[0108] a second index value acquisition module, configured to acquire a second index value of the N denoised image sets;

[0109] A function building module, wherein the function building module is used to build the preset loss function through the first indicator value and the second indicator value.

[0110] The specific example of an AI training method for big data denoising and optimization in the aforementioned embodiment 1 is also applicable to an AI training system for big data denoising and optimization in this embodiment. Through the aforementioned detailed description of an AI training method for big data denoising and optimization, those skilled in the art can clearly understand an AI training system for big data denoising and optimization in this embodiment, so for the sake of brevity of the specification, it will not be described in detail here.

[0111] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the present invention can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in the present invention can be achieved. This is not limited herein.

[0112] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. An AI training method for big data denoising optimization, characterized in that: The method comprises: Acquire preset basic information of a preset filter, wherein the preset basic information includes filter type information and filter characteristic information; The preset filter is constructed according to the filter type information to obtain M image denoising models, where M is an integer greater than 1; Get the image denoising target of the preset AI model and obtain the image sample set; Based on the image denoising target and the filter feature information, screening is performed among the M image denoising models to obtain N screening models, where 1<N≤M; Using the image sample set as input information, sequentially inputting the N screening models to obtain output results respectively, wherein the output results include N denoised image sets; Introducing a preset loss function to perform loss analysis on the N denoised image sets in sequence, obtaining N loss results, and analyzing the optimal denoising model corresponding to the minimum loss result; The first target image is subjected to image denoising by the optimal denoising model to obtain an output denoised image, and the preset AI model is trained.

2. The method according to claim 1, wherein The preset filter is constructed according to the filter type information to obtain M image denoising models, including: Set the filter quantity constraint; The preset filters are constructed in combination with the filter type information and the filter quantity constraint to obtain the M image denoising models.

3. The method according to claim 1, wherein Obtaining the image denoising target of the preset AI model includes: Analyzing the preset AI model to determine image retention features, image acquisition environment, and image transmission path; Analyzing the type of image noise based on the image acquisition environment and the image transmission path; The image noise reduction target is determined according to the image noise type and the image preservation feature.

4. The method according to claim 3, wherein The acquiring of the image sample set comprises: Get the original image set; Analyzing the image noise type to obtain a pure noise image set; Add noise according to the pure noise image set and the original image to obtain a first noisy image; Preprocessing the first noisy image to obtain a preprocessed image set; Performing image enhancement processing on the preprocessed image set to obtain the image sample set, wherein the number of images in the image sample set is greater than that in the preprocessed image set and greater than that in the first noisy image set.

5. The method according to claim 1, wherein The screening among the M image denoising models based on the image denoising target and the filter feature information to obtain N screening models includes: Based on the image noise reduction target and the filter characteristic information, filter matching is performed in the preset filter to obtain a first matched filter set; Obtaining M groups of filter type information for the M image denoising models; Based on the first matched filter set, matching is performed in the M groups of filter type information to obtain the N screening models.

6. The method according to claim 4, wherein The introduction of a preset loss function sequentially performs loss analysis on the N denoised image sets to obtain N loss results, and analyzes the optimal denoising model corresponding to the minimum loss result, including: Establishing a first mapping relationship between the original image set and the N denoised image sets; Building the preset loss function according to the first mapping relationship; Obtaining the N loss results corresponding to the N denoised image sets according to the loss function; The screening model corresponding to the minimum loss result is used as the optimal denoising model.

7. The method according to claim 6, wherein The step of constructing the loss function according to the first mapping relationship includes: Obtaining a first image denoising evaluation index; Obtaining a first index value of the original image set according to the first image denoising evaluation index; Obtaining a second index value of the N denoised image sets; The preset loss function is constructed using the first indicator value and the second indicator value.

8. An AI training system for big data denoising optimization, characterized by: The system comprises: A preset basic information acquisition module, wherein the preset basic information acquisition module is used to obtain preset basic information of a preset filter, wherein the preset basic information includes filter type information and filter characteristic information; An image denoising model building module, wherein the image denoising model building module is used to build the preset filter according to the filter type information to obtain M image denoising models, where M is an integer greater than 1; An AI model analysis module, which is used to obtain an image denoising target of a preset AI model and obtain an image sample set; A model screening module, the model screening module is used to screen the M image denoising models based on the image denoising target and the filter feature information to obtain N screening models, where 1<N≤M; a denoised image set acquisition module, the denoised image set acquisition module being configured to take the image sample set as input information, sequentially input the N screening models, and obtain output results respectively, wherein the output results include N denoised image sets; A minimum loss result matching module, wherein the minimum loss result matching module is used to introduce a preset loss function to sequentially perform loss analysis on the N denoised image sets to obtain N loss results, and analyze and match the optimal denoising model corresponding to the minimum loss result; A preset AI model training module is used to perform image denoising on the first target image through the optimal denoising model to obtain an output denoised image, and train to obtain the preset AI model.

Images