Image replacement method and system based on sky recognition

Abstract

The invention discloses an image replacement method and system based on sky recognition, and the method comprises the steps: obtaining a picture uploaded by a user, carrying out the analysis of the picture, reading attribute parameters, and obtaining a picture data set; inputting the picture data set into the neural grid model, identifying whether a sky image is included or not, and outputting a sky position area in the identified image; obtaining a sky material library, displaying the sky material library in a front-end interface, and providing the sky material library for a user to operate; selecting materials in a sky material library, obtaining a sky position area, and performing replacement operation on the materials and the sky position area to obtain a preliminary effect picture; performing optimization processing on the initial effect picture to obtain a target effect picture; according to the method, the sky region segmentation image can be accurately obtained by adopting a neural network recognition algorithm, a large number of different sky material templates can be provided for a user for selection, and replacement and fusion are performed on the basis of the sky template image selected by the user to obtain a sky-replaced image.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to an image replacement method and system based on sky recognition. Background Art

[0002] The sky is an important part of some images, but due to the uncontrollable weather, it is often difficult to get the expected sky when shooting. Therefore, based on the algorithm, users can replace the unsatisfactory sky in the video or image with any desired one in a more convenient way, whether it is a starry sky, a bright moon, or thunder and lightning, various weather effects can be identified by the algorithm and replaced with materials.

[0003] In daily life, the sky often appears as the background of pictures, but the weather changes frequently, so it often looks like the weather is bad, the sky is not blue enough, etc. However, the existing sky replacement methods are not accurate enough in identifying the sky, the replacement is not precise enough, and the fusion is not natural enough. Summary of the invention

[0004] The purpose of the present invention is to solve the above problems and to design an image replacement method and system based on sky recognition.

[0005] A first aspect of the present invention provides an image replacement method based on sky recognition, the image replacement method based on sky recognition comprising the following steps:

[0006] Get the pictures uploaded by the user, parse the pictures, and read the attribute parameters to get the picture data set;

[0007] Inputting the picture data set into the neural grid model, identifying whether it contains a sky image, and outputting the sky position area in the identified image;

[0008] Obtain the sky material library, display it in the front-end interface, and provide it to users for operation;

[0009] Select a material in the sky material library, obtain the sky position area, and replace the material with the sky position area to obtain a preliminary effect image;

[0010] Optimizing the preliminary rendering to obtain a target rendering;

[0011] The target effect diagram is saved and stored in a folder set by the user.

[0012] Furthermore, in the above-mentioned image replacement method based on sky recognition, the step of obtaining the picture uploaded by the user, parsing the picture, and reading the attribute parameters to obtain the picture data set includes:

[0013] The user selects the image to be processed, submits the form data asynchronously to the backend server via AJAX, stores it in the server database, and obtains a sample image;

[0014] Get sample images from the server database and parse the images;

[0015] Read the attribute parameters of the sample image, including width, height, resolution, color mode, and EXIF ​​information;

[0016] The attribute parameters of the sample images read in are combined with the image files themselves to form an image dataset.

[0017] Furthermore, in the above-mentioned image replacement method based on sky recognition, the step of inputting the picture data set into the neural grid model, identifying whether the picture contains a sky image, and outputting the sky position area in the identified image includes:

[0018] After preprocessing, the image data set is input into a neural network model;

[0019] Image data propagates layer by layer in the neural network, undergoes convolution, pooling, and activation operations, and finally reaches the output layer;

[0020] The probability value output by the output layer is used as the recognition result of whether the image contains the sky;

[0021] The result of the judgment is whether the sky is recognized. If the sky is not recognized, the process ends. If the sky is recognized, the sky area is segmented.

[0022] Furthermore, in the above-mentioned image replacement method based on sky recognition, the selecting of the material in the sky material library, obtaining the sky position area, and performing a replacement operation between the material and the sky position area to obtain a preliminary effect image includes:

[0023] The materials stored in the sky material library are displayed on the front-end page, and users can select the corresponding materials according to their needs to obtain replacement materials;

[0024] Acquire the sky position area, apply the replacement material, scale the width of the replacement material to be consistent with the width of the original picture, keep the width and height of the replacement material in the original ratio, and use the replacement material as the bottom layer;

[0025] Use filters to process the image of the sky area to obtain a preliminary effect image for preview.

[0026] Furthermore, in the above-mentioned image replacement method based on sky recognition, the step of processing the sky position area image using a filter includes:

[0027] The scaled replacement material is superimposed with the image of the sky position area by using the multiply method;

[0028] Examine the color information in each channel and perform positive overlay processing on the underlying color. The color produced by the blending mode is always darker than the original. The formula is: result color = material color * underlying color / 255.

[0029] Furthermore, in the above-mentioned image replacement method based on sky recognition, the step of optimizing the preliminary effect image to obtain the target effect image further includes:

[0030] The preliminary effect image is optimized by performing edge optimization and white edge removal steps, and the user can still make manual adjustments, and finally save it as a result image;

[0031] The edge optimization is to blur the part where the replacement material and the sky position area image are connected, so as to achieve a natural transition effect;

[0032] The white edge removal is to determine the boundary point between the replacement material and the sky position area, shrink the boundary inward, and remove objects smaller than the structural element.

[0033] A second aspect of the present invention provides an image replacement system based on sky recognition, characterized in that the image replacement system based on sky recognition includes a picture acquisition module, a recognition module, a material display module, a replacement module, an optimization module, and an output module, wherein:

[0034] The image acquisition module is used to obtain the images uploaded by the user, parse the images, and read the attribute parameters to obtain the image data set;

[0035] A recognition module, used for inputting the picture data set into the neural grid model, identifying whether a sky image is included, and outputting a sky position area in the identified image;

[0036] The material display module is used to obtain the sky material library, display it in the front-end interface, and provide it to users for operation;

[0037] A replacement module, used for selecting a material in the sky material library, obtaining the sky position area, performing a replacement operation on the material and the sky position area, and obtaining a preliminary effect image;

[0038] An optimization module, used for optimizing the preliminary rendering to obtain a target rendering;

[0039] The output module is used to save the target effect diagram and store it in a folder set by the user.

[0040] Furthermore, in the above-mentioned image replacement system based on sky recognition, the replacement module includes a selection submodule, a ratio adjustment submodule, and a filter submodule, wherein:

[0041] The selection submodule is used to display the materials stored in the sky material library on the front-end page. Users can select the corresponding materials according to their needs to obtain replacement materials;

[0042] A ratio adjustment submodule, used for obtaining the sky position area, applying the replacement material, scaling the width of the replacement material to be consistent with the width of the original picture, maintaining the original ratio of the width and height of the replacement material, and using the replacement material as a bottom layer;

[0043] The filter submodule is used to process the image of the sky position area using a filter to obtain a preliminary effect image for preview.

[0045] Its beneficial effect lies in that, by acquiring the pictures uploaded by the user, the pictures are parsed, and the attribute parameters are read to obtain a picture data set; the picture data set is input into the neural grid model to identify whether it contains a sky image, and the sky position area in the identified image is output; the sky material library is acquired, displayed in the front-end interface, and provided to the user for operation; the material in the sky material library is selected, the sky position area is acquired, and the material and the sky position area are replaced to obtain a preliminary effect map; the preliminary effect map is optimized to obtain a target effect map; the target effect map is saved and stored in a folder set by the user. The method adopts a neural network recognition algorithm to accurately obtain a sky area segmentation map, and can provide users with a large number of different sky material templates to choose from. Replacement and fusion are performed on the basis of the sky template map selected by the user to obtain a picture with the sky replaced. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the following detailed description of the preferred embodiment.The drawings are only for the purpose of illustrating the preferred embodiments and are not to be construed as limiting the invention.

[0047] Figure 1 Schematic diagram of an embodiment of an image replacement method based on sky recognition in an embodiment of the present invention;

[0048] Figure 2 is a first schematic diagram of an image replacement system based on sky recognition in an embodiment of the present invention;

[0049] Figure 3 This is a second schematic diagram of an image replacement system based on sky recognition in an embodiment of the present invention. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0051] Those skilled in the art will appreciate that, unless otherwise stated, the singular forms "a", "an", "said" and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present invention refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0052] The present invention will be described in detail below in conjunction with the accompanying drawings. Figure 1 As shown, a method for replacing an image based on sky recognition comprises the following steps:

[0053] Step 101: Obtain the picture uploaded by the user, parse the picture, and read the attribute parameters to obtain the picture data set;

[0054] Specifically, in this embodiment, pictures uploaded by users are obtained, pictures are parsed, and attribute parameters are read to obtain picture data sets, including: the user selects the picture to be processed, and form data is asynchronously submitted to the back-end server through AJAX, which is stored in the server database to obtain sample pictures; the sample pictures in the server database are obtained and the pictures are parsed; the attribute parameters of the sample pictures are read, including width, height, resolution, color mode, and EXI F information; the attribute parameters of the sample pictures read and the picture file itself are constructed into a picture data set.

[0055] Step 102: Input the image data set into the neural grid model, identify whether it contains a sky image, and output the sky position area in the identified image;

[0056] Specifically, in this embodiment, the image data set is input into the neural grid model, whether the image contains the sky is identified, and the sky position area in the identified image is output, including: the image data set is input into the neural network model after preprocessing; the image data is propagated layer by layer in the neural network, and finally reaches the output layer after convolution, pooling, and activation operations; the probability value output by the output layer is used as the recognition result of whether the image contains the sky; the judgment result, whether the sky is recognized, if the sky is not recognized, the process ends, if the sky is recognized, the sky area is segmented.

[0057] Step 103: Obtain a sky material library, display it in the front-end interface, and provide it to the user for operation;

[0058] Specifically, in this embodiment, the image data set is input into the neural grid model, whether the image contains the sky is identified, and the sky position area in the identified image is output, including: the image data set is input into the neural network model after preprocessing; the image data is propagated layer by layer in the neural network, and finally reaches the output layer after convolution, pooling, and activation operations; the probability value output by the output layer is used as the recognition result of whether the image contains the sky; the judgment result, whether the sky is recognized, if the sky is not recognized, the process ends, if the sky is recognized, the sky area is segmented.

[0059] Step 104: select a material in the sky material library, obtain a sky position area, and replace the material with the sky position area to obtain a preliminary effect image;

[0060] Specifically, in this embodiment, materials in the sky material library are selected, the sky position area is obtained, and the material and the sky position area are replaced to obtain a preliminary effect diagram, including: displaying the materials stored in the sky material library on the front-end page, and users can select corresponding materials according to their own needs to obtain replacement materials; obtaining the sky position area, applying replacement materials, scaling the width of the replacement materials to be consistent with the width of the original image, maintaining the original width and height ratio of the replacement materials, and using the replacement materials as the bottom layer; using filters to process the sky position area image to obtain a preliminary effect image for preview.

[0061] Specifically, in this embodiment, a filter is used to process the image of the sky position area, including: superimposing the scaled replacement material and the sky position area image using a positive overlay; examining the color information in each channel, and performing a positive overlay process on the underlying color. The color generated by the mixed mode is always darker than the original, and the formula is: result color = material color * underlying color / 255.

[0062] Screen mode: Mix in the "additive mode" of color mixing principle. That is, for the screen mode, colors have an additive effect. For example, when red, green, and blue are all at their maximum value of 255, mixing in Screen mode will result in white with an RGB value of (255, 255, 255). On the contrary, black means 0. Therefore, mixing with black in this mode has no effect, while mixing with white will result in white with the maximum RGB value (RGB value 255, 255, 255).

[0063] Overlay Mode: Whether pixels are in Multiply or Screen mode depends on the underlying color. The colors are blended, but the brightness details of the highlights and shadows of the underlying color are preserved.

[0064] Step 105, optimizing the preliminary effect diagram to obtain a target effect diagram;

[0065] Specifically, in this embodiment, the preliminary effect image is optimized to obtain the target effect image, which also includes: optimizing the preliminary effect image, optimizing the edge and removing the white edge. The user can still adjust it manually, and finally save it as the result image; optimizing the edge is to blur the part where the replacement material and the sky position area image are connected, which plays a role of gradual change and achieves a natural transition effect; removing the white edge is to determine the boundary point between the replacement material and the sky position area, so that the boundary shrinks inward and removes objects smaller than the structural element. It should be noted that only the grayscale value of the target pixel and its adjacent four pixels above, below, left and right are considered to confirm whether it is consistent with the corroded structural element. If it is consistent, the pixel point is retained. Otherwise, the pixel point is deleted.

[0066] Step 106, saving the target effect diagram and storing it in a folder set by the user.

[0067] like Figure 2 As shown, it is a first schematic diagram of an image replacement system based on sky recognition. The image replacement system based on sky recognition includes a picture acquisition module, a recognition module, a material display module, a replacement module, an optimization module, and an output module, wherein:

[0068] The image acquisition module is used to obtain the images uploaded by the user, parse the images, and read the attribute parameters to obtain the image data set;

[0069] A recognition module is used to input the image data set into the neural grid model, identify whether it contains a sky image, and output the sky position area in the identified image;

[0070] The material display module is used to obtain the sky material library, display it in the front-end interface, and provide it to users for operation;

[0071] The replacement module is used to select a material in the sky material library, obtain the sky position area, and replace the material with the sky position area to obtain a preliminary effect image;

[0072] An optimization module is used to optimize the preliminary renderings to obtain the target renderings;

[0073] The output module is used to save the target rendering and store it in the folder set by the user.

[0074] like Figure 3 , which is a second schematic diagram of an image replacement system based on sky recognition.

[0075] In this embodiment, the replacement module includes a selection submodule, a ratio adjustment submodule, and a filter submodule, wherein:

[0076] The selection submodule is used to display the materials stored in the sky material library on the front-end page. Users can select the corresponding materials according to their needs to obtain replacement materials;

[0077] The proportion adjustment submodule is used to obtain the sky position area, apply the replacement material, scale the width of the replacement material to be consistent with the width of the original image, keep the width and height of the replacement material in the original proportion, and use the replacement material as the bottom layer;

[0078] The filter submodule is used to process the image of the sky position area using a filter to obtain a preliminary effect image for preview.

[0079] Through the implementation of the above scheme, the pictures uploaded by the user are obtained, the pictures are parsed, and the attribute parameters are read to obtain the picture data set; the picture data set is input into the neural grid model to identify whether it contains a sky image, and the sky position area in the identified image is output; the sky material library is obtained, displayed in the front-end interface, and provided to the user for operation; the material in the sky material library is selected, the sky position area is obtained, and the material and the sky position area are replaced to obtain a preliminary effect map; the preliminary effect map is optimized to obtain a target effect map; the target effect map is saved. This method uses a neural network recognition algorithm to accurately obtain a sky area segmentation map, and can provide users with a large number of different sky material templates to choose from. Replacement and fusion are performed on the basis of the sky template map selected by the user to obtain a picture with the sky replaced.

[0080] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments. The above embodiments and descriptions are only preferred examples of the present invention and are not intended to limit the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. An image replacement method based on sky recognition, characterized in that: The image replacement method based on sky recognition comprises the following steps: Get the pictures uploaded by the user, parse the pictures, and read the attribute parameters to get the picture data set; Inputting the picture data set into the neural grid model, identifying whether it contains a sky image, and outputting the sky position area in the identified image; Obtain the sky material library, display it in the front-end interface, and provide it to users for operation; Select a material in the sky material library, obtain the sky position area, and replace the material with the sky position area to obtain a preliminary effect image; Optimizing the preliminary rendering to obtain a target rendering; The target effect diagram is saved and stored in a folder set by the user.

2. The image replacement method based on sky recognition according to claim 1, characterized in that: The step of obtaining the picture uploaded by the user, parsing the picture, and reading the attribute parameters to obtain the picture data set includes: The user selects the image to be processed, submits the form data asynchronously to the backend server via AJAX, stores it in the server database, and obtains a sample image; Get sample images from the server database and parse the images; Read the attribute parameters of the sample image, including width, height, resolution, color mode, and EXIF ​​information; The attribute parameters of the sample images read in are combined with the image files themselves to form an image dataset.

3. The image replacement method based on sky recognition according to claim 1, characterized in that: The step of inputting the picture data set into the neural grid model, identifying whether a sky image is included, and outputting a sky position area in the identified image includes: After preprocessing, the image data set is input into a neural network model; Image data propagates layer by layer in the neural network, undergoes convolution, pooling, and activation operations, and finally reaches the output layer; The probability value output by the output layer is used as the recognition result of whether the image contains the sky; The result of the judgment is whether the sky is recognized. If the sky is not recognized, the process ends. If the sky is recognized, the sky area is segmented.

4. The image replacement method based on sky recognition according to claim 1, characterized in that: The selecting of the material in the sky material library, obtaining the sky position area, and replacing the material with the sky position area to obtain a preliminary effect map includes: The materials stored in the sky material library are displayed on the front-end page, and users can select the corresponding materials according to their needs to obtain replacement materials; Acquire the sky position area, apply the replacement material, scale the width of the replacement material to be consistent with the width of the original picture, keep the width and height of the replacement material in the original ratio, and use the replacement material as the bottom layer; Use filters to process the image of the sky area to obtain a preliminary effect image for preview.

5. The image replacement method based on sky recognition as claimed in claim 4, characterized in that: The step of using a filter to process the sky position area image comprises: The scaled replacement material is superimposed with the image of the sky position area by using the multiply method; Examine the color information in each channel and perform positive overlay processing on the underlying color. The color produced by the blending mode is always darker than the original. The formula is: result color = material color * underlying color / 255.

6. The image replacement method based on sky recognition according to claim 1, characterized in that: The optimizing process of the preliminary effect diagram to obtain the target effect diagram also includes: The preliminary effect image is optimized, and the edge optimization and white edge removal steps are performed. The user can still adjust it manually, and finally save it as a result image; The edge optimization is to blur the part where the replacement material and the sky position area image are connected, so as to achieve a gradual transition effect and achieve a natural transition effect; The white edge removal is to determine the boundary points between the replacement material and the sky position area, shrink the boundary inward, and remove objects smaller than the structural element.

7. An image replacement system based on sky recognition, characterized in that: The image replacement system based on sky recognition includes a picture acquisition module, a recognition module, a material display module, a replacement module, an optimization module, and an output module, wherein: The image acquisition module is used to obtain the images uploaded by the user, parse the images, and read the attribute parameters to obtain the image data set; A recognition module, used to input the picture data set into the neural grid model, identify whether it contains a sky image, and output a sky position area in the identified image; The material display module is used to obtain the sky material library, display it in the front-end interface, and provide it to users for operation; A replacement module, used for selecting a material in the sky material library, obtaining the sky position area, performing a replacement operation on the material and the sky position area, and obtaining a preliminary effect image; An optimization module, used for optimizing the preliminary rendering to obtain a target rendering; The output module is used to save the target effect diagram and store it in a folder set by the user.

8. The image replacement system based on sky recognition as claimed in claim 7, characterized in that: The replacement module includes a selection submodule, a ratio adjustment submodule, and a filter submodule, wherein: The selection submodule is used to display the materials stored in the sky material library on the front-end page. Users can select the corresponding materials according to their needs to obtain replacement materials; A ratio adjustment submodule, used for obtaining the sky position area, applying the replacement material, scaling the width of the replacement material to be consistent with the width of the original picture, maintaining the original ratio of the width and height of the replacement material, and using the replacement material as a bottom layer; The filter submodule is used to process the image of the sky position area using a filter to obtain a preliminary effect image for preview.

Images