Working method for visual scene enhancement prediction for noisy images

Abstract

The present invention discloses a working method for visual scene enhancement prediction of noisy images, comprising the following steps: S1, obtaining an input image file through a cloud platform, remapping the dimension A representing the transparent channel in a 4-channel image (RGBA) to form an RGB image generation process; S2, after forming a 3-channel image (RGB), establishing a scene feature transformation model to output features of the scene contour, and performing transformation learning through a scene feature loss function; S3, performing a comprehensive score on the target feature image, setting a score threshold to filter and output the prediction results.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a working method for visual scene enhancement prediction of noisy images. Background Art

[0002] In the process of acquiring massive cloud-based image databases, the recognition content may be biased due to differences in image pixels and shooting angles. The image data analysis process needs to reduce the impact of image transparency. For example, the position, layout or shooting angle of the building in the image will affect the image information recognition process. In the case of stacked objects or sudden changes in exposure, how to accurately obtain image information urgently requires technical personnel in this field to solve the corresponding technical problems. Summary of the Invention

[0003] The present invention aims to at least solve the technical problems existing in the prior art, and in particular innovatively proposes a working method for visual scene enhancement prediction of noisy images.

[0004] In order to achieve the above-mentioned object of the present invention, the present invention provides a working method for visual scene enhancement prediction of noisy images, comprising the following steps:

[0005] S1, obtains the input image file through the cloud platform, remaps the dimension A representing the transparent channel in the 4-channel image (RGBA) to form the RGB image generation process;

[0006] S2, after forming a 3-channel image (RGB), establish a scene feature transformation model to output the features of the scene contour, and perform transformation learning through the scene feature loss function;

[0007] S3, perform comprehensive condition limitation of the target feature image, set the scoring threshold to filter and output the prediction results.

[0008] In the above technical solution, preferably, S1 includes:

[0009] S1-1, extracting the to-be-processed four-dimensional channel RGBA image set from the cloud platform, and capturing scene image information from the four-dimensional channel RGBA image set;

[0010] S1-2, the acquired scene image contains RGBA four-channel image data. The value range of the transparent channel Alpha is [0, 1]. A four-dimensional channel (R, G, B, A) quadruple is defined, and RGB is redefined and mapped through the transition of brightness and chroma; after repeated iterations, a mapping relationship from RGBA to RGB is formed.

[0011] The above technical solution is preferably, wherein S2 includes:

[0012] S2-1, after preparing the RGB mapping relationship of a large number of images, forming an RGB mapping relationship training set, extracting the training images that form the RGB mapping relationship from the training set to be processed, and performing layered processing on the training images to obtain foreground images and background images; respectively capturing the scene scenery information in the foreground image and background image to obtain integrated data for each scene in the training image.

[0013] The above technical solution is preferably, wherein S2 includes:

[0014] S2-2, obtaining scene angle types for the foreground images respectively, and extracting features of the scene angle types in the foreground images;

[0015] Angle type function for obtaining foreground image

[0016] Where i is the acquisition type of the foreground image, N is the number of acquisition types, L i is the offset value between the detected foreground image i and the preset foreground image, μ is the foreground image angle correction value, D(x θ ,y θ ) is the spatial distance between the detected foreground image angle θ and the preset reference angle;

[0017] in x θ The image coordinates x, y of the detected foreground image angle θ θ is the image coordinate y of the detected foreground image angle θ, x′ is the preset foreground image coordinate x, and y′ is the preset foreground image coordinate y;

[0018] By classifying the angle types of the foreground image, the scene recognition conditions of the foreground image are formed and the noise content is filtered.

[0019] The characteristic transformation strength value M θ =E U(θ) ·I·X θ +ω, where E U(θ) is the feature map vector selected from the angle category U(θ), ​​I is the feature map vector weight, X θ is the pixel parameter of the foreground image at angle θ, ω is the image transformation noise,

[0020]

[0021] Among them, X is the pixel parameter of the reference foreground image, X′ is the pixel parameter of the angle θ forward image, and w θ is the width of the foreground image at angle θ, h θis the angle θ of the foreground image, w is the width of the reference foreground image, and h is the height of the reference foreground image.

[0022] The above technical solution is preferably, wherein S2 further includes:

[0023] S2-3, forming the scene feature loss function,

[0024] Loss function Y = Y center +Y transfer , the loss function Y is set to two parts, where Y center is the scene center loss function, Y transfer Adjust the loss function for the scenario.

[0025] The above technical solution is preferably, wherein S2 further includes:

[0026] S2-4, scene center loss function

[0027]

[0028] Where, j is the total number of iterations of the foreground image center position, Z, β(*) is the weight of the foreground image angle feature type, C is the total number of angle types, K, is the scene center feature transformation strength of the jth foreground image iteration, q C is the central eigenvalue of the output forward tilt image angle category C.

[0029] The above technical solution is preferably characterized in that said S2 further includes:

[0030] S2-5, scene adjustment loss function Where s is the total number of iterations of adjusting the foreground image position W, Adjust the feature transformation strength for the scene of the sth foreground image iteration.

[0031] The above technical solution is preferably, wherein S3 further includes:

[0032] S3-1, after calculating the scene feature loss function, the foreground image features are abstracted; the features of the RGB image data are further abstracted, and after the global average pooling layer, the model performance is tested on the test set, setting the frame value of the foreground image input to 1, the feature map vector weight to 3.6, the number of iterations of the foreground image center position to 16*205, and the image transformation noise to 0.8.

[0033] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0034] By analyzing the complementarity of four-dimensional RGBA images and RGB images, the scene generation model is used to provide robustness of image output and assist in enhancing the recognition ability of the scene. Due to the presence of a considerable amount of interfering image information, the statistical scene generation model is used for feature learning to extract different environmental information and target extraction of scene contour features.

[0035] Additional aspects and advantages of the present invention will be set forth in part in the description which follows and, in part, will be obvious from the description which follows, or may be learned by practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the description of the embodiments with reference to the following drawings, in which:

[0037] Figure 1 It is an overall schematic diagram of the present invention;

[0038] Figure 2 This is the predicted ancient building effect diagram of the present invention;

[0039] Figure 3 This is another effect diagram of the ancient building predicted by the present invention;

[0040] Figure 4 This is another effect diagram of the ancient building predicted by the present invention;

[0041] Figure 5 This is another effect diagram of the ancient building predicted by the present invention;

[0042] Figure 6 This is another effect diagram of the ancient building predicted by the present invention;

[0043] Figure 7 This is another effect diagram of the ancient building predicted by the present invention;

[0044] Figure 8 This is another rendering of the noise ancient building of the present invention;

[0045] Figure 9 This is another rendering of the noise ancient building of the present invention;

[0046] Figure 10 It is a schematic diagram of a specific embodiment of the present invention. DETAILED DESCRIPTION

[0047] The following describes embodiments of the present invention in detail. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended only to explain the present invention and are not to be construed as limiting the present invention.

[0048] like Figure 1 and 3 As shown, the present invention discloses a working method for visual scene enhancement prediction of noisy images, comprising the following steps:

[0049] S1, obtains the input image file through the cloud platform, remaps the dimension A representing the transparent channel in the 4-channel image (RGBA) to form the RGB image generation process;

[0050] S2, after forming a 3-channel image (RGB), establishes a scene feature transformation model to output the features of the scene contour, and performs transformation learning through the scene feature loss function; the extracted environmental information includes: distant view, close view, daytime, nighttime, etc.

[0051] S3, perform comprehensive condition limitation of the target feature image, set the scoring threshold to filter and output the prediction results.

[0052] In the above technical solution, preferably, S1 includes:

[0053] S1-1, extracting the to-be-processed four-dimensional channel RGBA image set from the cloud platform, and capturing scene image information from the four-dimensional channel RGBA image set;

[0054] S1-2, the acquired scene image contains RGBA four-channel image data. The value range of the transparent channel Alpha is [0, 1]. A four-dimensional channel (R, G, B, A) quad is defined, and RGB is redefined and mapped through the transition of brightness and chrominance. During the pixel channel mapping process, the four-dimensional channel RGBA and the three-dimensional channel RGB cannot produce a completely consistent mapping relationship. The adjacent channel is found for mapping operation, and the mapping relationship from RGBA to RGB is formed after repeated iterations.

[0055] The above technical solution is preferably, wherein S2 includes:

[0056] S2-1, after preparing RGB mapping relationships for a large number of images, forming an RGB mapping relationship training set, extracting training images that form RGB mapping relationships from the training set to be processed, and performing layered processing on the training images to obtain foreground images and background images; respectively capturing scene information within the foreground images and background images to obtain integrated data for each scene in the training images;

[0057] During the collection of this training set, 3,500 scene images (front and side views), 1,852 front and top views, 2,360 oblique views, 1,223 oblique views, and 1,569 noisy images were extracted. While preserving the essential feature data of the scene images, the images were scaled to reduce system overhead and speed up processing. This training set, with RGB mapping relationships, was used for neural network learning. Deep image processing was performed on images from each characteristic angle of the scene, and scene images were classified into side views, front and top views, and oblique views.

[0058] S2-2, obtaining scene angle types for the foreground images respectively, and extracting features of the scene angle types in the foreground images;

[0059] Angle type function for obtaining foreground image

[0060] Where i is the acquisition type of the foreground image, N is the number of acquisition types, L i is the offset value between the detected foreground image i and the preset foreground image, μ is the foreground image angle correction value, D(x θ ,y θ ) is the spatial distance between the detected foreground image angle θ and the preset reference angle;

[0061] in x θ The image coordinates x, y of the detected foreground image angle θ θ is the image coordinate y of the detected foreground image angle θ, x′ is the preset foreground image coordinate x, and y′ is the preset foreground image coordinate y;

[0062] After classifying the angle types of the foreground image, the scene recognition conditions of the foreground image are formed and the noise content is filtered. Since the multi-view spatial image is based on a spatial image constructed in a nonlinear discrete state, the foreground image needs to be feature transformed;

[0063] The characteristic transformation strength value M θ =E U(θ) ·I·X θ +ω, where E U(θ) is the feature map vector selected from the angle category U(θ), ​​I is the feature map vector weight, X θ is the pixel parameter of the foreground image at angle θ, ω is the image transformation noise,

[0064]

[0065] Among them, X is the pixel parameter of the reference foreground image, X′ is the pixel parameter of the angle θ forward image, and w θ is the width of the foreground image at angle θ, hθ is the height of the foreground image at angle θ, w is the width of the reference foreground image, and h is the height of the reference foreground image;

[0066] S2-3, forming a scene feature loss function. According to the relationship of the loss function, in order to further converge the target features of the foreground image, a transformation function operation of the feature quantity is performed on the features of the corresponding angles to eliminate the foreground image noise, make the foreground image features more prominent, and be able to distinguish them from other features.

[0067] Loss function Y = Y center +Y transfer , the loss function Y is set to two parts, where Y center is the scene center loss function, Y transfer Adjust the loss function for the scenario;

[0068] S2-4, scene center loss function

[0069]

[0070] Where, j is the total number of iterations of the foreground image center position, Z, β(*) is the weight of the foreground image angle feature type, C is the total number of angle types, K, is the scene center feature transformation strength of the jth foreground image iteration, q C is the central eigenvalue of the output forward tilt image angle category C;

[0071] S2-5, scene adjustment loss function Where s is the total number of iterations of adjusting the foreground image position W, Adjust the feature transformation strength for the scene of the sth foreground image iteration,

[0072] By calculating the scene loss function, the relationship between the scene adjustment features and the scene center features is balanced, so that the corresponding class changes are classified according to the changes in the foreground image position, thereby improving robustness.

[0073] In the above technical solution, preferably, S3 includes:

[0074] S3-1, after calculating the scene feature loss function, the foreground image features are abstracted; the features of the RGB image data are further abstracted, and after the global average pooling layer, the model performance is tested on the test set. The frame value of the foreground image input is set to 1, the feature map vector weight is 3.6, the number of iterations of the foreground image center position is 16*205, the image transformation noise is 0.8, and a model is saved every 36 iterations. The training model with the highest training set accuracy is selected as the final model.

[0075] Setting a foreground image angle correction threshold, comparing the spatial distances of the angle types appearing in the foreground image based on the angle correction threshold, and traversing the foreground image features of the marked feature information; if the spatial distance of the foreground image is less than or equal to the correction value, deleting the foreground image;

[0076] like Figures 2 to 7 As shown, it is a schematic diagram of the specific effect of the present invention. Figure 8 and 9 This is a noise image. It can be clearly seen that the score is very low and it is removed after screening.

[0077] Display the prediction results in the form of pictures;

[0078] First, create a subgraph with one row and two columns. The first subgraph displays the original image data to facilitate comparison with the predicted results. In the second subgraph, the vertical axis represents the predicted probability and the horizontal axis represents the predicted result. After taking the top three predicted results by probability, display them in the second subgraph. Change the color of the bar graph of the result with the highest predicted probability to make the result clear at a glance. After adjusting the position of the predicted result bar graph, Figure 1 Start display.

[0079] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to the embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the claims and their equivalents.

Claims

1. A method for visual scene enhancement prediction of noisy images, characterized in that: include: S1, obtains the input image file through the cloud platform, remaps the dimension A representing the transparent channel in the 4-channel RGBA image to form the RGB image generation process; S2, after forming the 3-channel RGB image, establish a scene feature transformation model to output the features of the scene contour, and perform transformation learning through the scene feature loss function; The S2 includes: S2-1, after preparing RGB mapping relationships for a large number of images, forming an RGB mapping relationship training set, extracting training images that form RGB mapping relationships from the training set to be processed, and performing layered processing on the training images to obtain foreground images and background images; respectively capturing scene information within the foreground images and background images to obtain integrated data for each scene in the training images; S2-2, obtaining scene angle types for the foreground images respectively, and extracting features of the scene angle types in the foreground images; Angle type function for obtaining foreground image Where i is the acquisition type of the foreground image, N is the number of acquisition types, L i is the offset value between the detected foreground image i and the preset foreground image, μ is the foreground image angle correction value, D(x θ ,y θ ) is the spatial distance between the detected foreground image angle θ and the preset reference angle; in x θ The image coordinates x, y of the detected foreground image angle θ θ is the image coordinate y of the detected foreground image angle θ, x′ is the preset foreground image coordinate x, and y′ is the preset foreground image coordinate y; By classifying the angle types of the foreground image, the scene recognition conditions of the foreground image are formed and the noise content is filtered. The characteristic transformation strength value M θ =E U(θ) ·I·X θ +ω, where E U(θ) is the feature map vector selected from the angle category U(θ), ​​I is the feature map vector weight, X θ is the pixel parameter of the foreground image at angle θ, ω is the image transformation noise, Among them, X is the pixel parameter of the reference foreground image, X′ is the pixel parameter of the angle θ forward image, and w θ is the width of the foreground image at angle θ, h θ is the height of the foreground image at angle θ, w is the width of the reference foreground image, and h is the height of the reference foreground image; S3, perform comprehensive condition limitation of the target feature image, set the scoring threshold to filter and output the prediction results.

2. The method for visual scene enhancement prediction of noisy images according to claim 1, characterized in that: Said S1 comprises: S1-1, extracting the to-be-processed four-dimensional channel RGBA image set from the cloud platform, and capturing scene image information from the four-dimensional channel RGBA image set; S1-2, the acquired scene image contains RGBA four-channel image data. The value range of the transparent channel Alpha is [0, 1]. A four-dimensional channel (R, G, B, A) quadruple is defined, and RGB is redefined and mapped through the transition of brightness and chroma; after repeated iterations, a mapping relationship from RGBA to RGB is formed.

3. The method for visual scene enhancement prediction of noisy images according to claim 1, characterized in that: Said S2 further comprises: S2-3, forming the scene feature loss function, Loss function Y = Y center +Y transfer , the loss function Y is set to two parts, where Y center is the scene center loss function, Y transfer Adjust the loss function for the scenario.

4. The method for visual scene enhancement prediction of noisy images according to claim 1, characterized in that: Said S2 further comprises: S2-4, scene center loss function Where, j is the total number of iterations of the foreground image center position, Z, β(*) is the weight of the foreground image angle feature type, C is the total number of angle types, K, is the scene center feature transformation strength of the jth foreground image iteration, q C is the central eigenvalue of the output forward tilt image angle category C.

5. The method for visual scene enhancement prediction of noisy images according to claim 1, characterized in that: Said S2 further comprises: S2-5, scene adjustment loss function Where s is the total number of iterations of adjusting the foreground image position W, Adjust the feature transformation strength for the scene of the sth foreground image iteration.

6. The method for visual scene enhancement prediction of noisy images according to claim 1, characterized in that: Said S3 further comprises: S3-1, after calculating the scene feature loss function, the foreground image features are abstracted; the features of the RGB image data are further abstracted, and after the global average pooling layer, the model performance is tested on the test set, setting the frame value of the foreground image input to 1, the feature map vector weight to 3.6, the number of iterations of the foreground image center position to 16*205, and the image transformation noise to 0.8.

Images