Artificial intelligence-based mobile terminal integrated human-scene image generation system and method
Through the multi-dimensional feature acquisition and intelligent patch model construction of the artificial intelligence system, the problems of low segmentation accuracy, inconsistent light, and unnatural perspective effects in the generation of traditional human-scene-like images are solved, and the realism and fidelity of the image are improved .
Patent Information
- Application Number
- CN202411288386.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-14
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2044-09-14
AI Technical Summary
Traditionally, the combination of people and scenery pictures is difficult to accurately segment portraits and backgrounds in complex scenes, and the light is inconsistent and the perspective effect is unnatural, resulting in poor realistic feeling of the synthetic image.
Adopting a mobile portrait-in-one picture generation system based on artificial intelligence, through the target portrait acquisition module, the target background feature acquisition module, the output information acquisition module, the target light patch creation module and the perspective comparison result acquisition module, combined with the predetermined edge separation strategy , Multi-dimensional feature acquisition and random sampling consistency principles to build an intelligent patch model to accurately fusion and correction of portraits and backgrounds.
It realizes accurate separation between portraits and backgrounds and light consistency, improves the reality and visual effect of the synthetic image, and solves the problems of low segmentation accuracy, inconsistent light, and unnatural perspective effects.
Smart Images

Figure CN118799207B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image processing technology, and in particular to a system and method for generating a person-scene integrated picture on a mobile terminal based on artificial intelligence. Background Art
[0002] The generation of people-scenery images has broad application prospects in modern science and technology and applications. However, there are still some problems with the traditional generation of people-scenery images. First, the traditional image segmentation and edge processing methods often have difficulty in accurately segmenting the portrait and the background when processing complex scenes, and are prone to blurring, incompleteness and other phenomena; second, in the synthesis process, if the light of the portrait and the background is inconsistent, it will cause the synthetic image to have unnatural light problems, making the synthesis result appear unrealistic; third, if the perspective effect of the portrait and the background in the synthetic image does not match or is unnatural, the entire image will appear uncoordinated and lack a sense of reality. These technical problems affect the image quality of the generation of people-scenery images on mobile terminals. Summary of the invention
[0003] The present application provides a mobile terminal human-scene integrated picture generation system based on artificial intelligence, aiming to solve the technical problems in traditional human-scene integrated picture generation such as low segmentation accuracy, inconsistent lighting, unnatural perspective effect, etc., which make the synthetic image less realistic.
[0004] In view of the above problems, the present application provides a system and method for generating people-scene integrated pictures on a mobile terminal based on artificial intelligence.
[0005] The first aspect disclosed in the present application provides a mobile terminal person-scene integrated picture generation system based on artificial intelligence, the system comprising: a target portrait acquisition module, the target portrait acquisition module is used to call a predetermined edge separation strategy to perform edge separation processing on the initial frame separation block of the portrait picture to obtain the target portrait; a target background feature acquisition module, the target background feature acquisition module is used to read a predetermined picture feature dimension, and perform multi-dimensional acquisition of the background picture based on the predetermined picture feature dimension to obtain the target background feature; an output information acquisition module, the output information acquisition module is used to use the background light feature parameters in the target background feature as input information of a patch construction model to obtain output information, wherein the patch construction model is an intelligent model constructed based on the principle of random sampling consistency; a target light patch establishment module, the target light patch establishment module is used to adjust the patch model parameters in the output information in combination with the background color feature value and the background texture feature value in the target background feature, and establish the target light patch of the background image; an initial human-scene image acquisition module, the initial human-scene image acquisition module is used to fuse the target portrait with the background image based on the target light patch to obtain an initial human-scene image; a perspective comparison result acquisition module, the perspective comparison result acquisition module is used to analyze and compare the target portrait perspective information and the background perspective information of the target portrait to obtain a perspective comparison result; a target human-scene image acquisition module, the target human-scene image acquisition module is used to correct the initial human-scene image based on the perspective comparison result to obtain a target human-scene image.
[0006] Another aspect disclosed in the present application provides a method for generating a person-scene integrated image on a mobile terminal based on artificial intelligence, which is used in the above-mentioned person-scene integrated image generation system on a mobile terminal based on artificial intelligence, and the method includes: calling a predetermined edge separation strategy to perform edge separation processing on an initial frame separation block of a portrait image to obtain a target portrait; reading a predetermined image feature dimension, and performing multi-dimensional acquisition of a background image based on the predetermined image feature dimension to obtain a target background feature; using background light feature parameters in the target background feature as input information of a patch construction model to obtain output information, wherein the patch construction model is an intelligent model constructed based on the principle of random sampling consistency; adjusting the patch model parameters in the output information in combination with the background color feature value and the background texture feature value in the target background feature, and establishing a target light patch of the background image; fusing the target portrait with the background image based on the target light patch to obtain an initial person-scene image; analyzing and comparing the target portrait perspective information and the background perspective information of the target portrait to obtain a perspective comparison result; correcting the initial person-scene image based on the perspective comparison result to obtain a target person-scene image.
[0007] One or more technical solutions provided in this application have at least the following technical effects or advantages:
[0008] By calling the predetermined edge separation strategy to separate the border of the portrait image, the accurate separation of the portrait and the background is achieved, avoiding the problem of blurred edges or wrong segmentation in traditional methods; by reading the predetermined image feature dimensions and based on this, multi-dimensional acquisition of the background image is performed, richer and more accurate background features are obtained, which helps to improve the authenticity of the synthesized image; by using the random sampling consistency principle to build an intelligent patch model, it can more adaptively generate patches suitable for the current scene, improving the intelligence and efficiency of the synthesis process; combined with the color and texture feature values in the background features, the patch model parameters are adjusted to better match the background light and enhance the consistency of the synthesized image; based on the target light patch, the target portrait and the background image are fused, and the accurate fusion of the synthesized image is achieved through comparison of light direction, intensity, color temperature and other aspects; the perspective information of the target portrait and the background is analyzed and compared, and the image is corrected to improve the visual effect of the synthesized image. In summary, this AI-based mobile human-scenery image generation system solves the problems of low segmentation accuracy, inconsistent lighting, unnatural perspective effect, etc. in traditional human-scenery image generation through multi-dimensional feature collection, intelligent patch model construction, parameter adjustment and precise fusion, achieving the technical effect of improving the realism and fidelity of the synthesized image.
[0009] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS
[0010] Figure 1 A schematic diagram of the structure of a system for generating a person-scene integrated picture on a mobile terminal based on artificial intelligence is provided for the embodiment of the present application;
[0011] Figure 2 A schematic flow chart of a method for generating a person-scene integrated picture on a mobile terminal based on artificial intelligence is provided for an embodiment of the present application.
[0012] Description of reference numerals: target portrait acquisition module 10 , target background feature acquisition module 20 , output information acquisition module 30 , target light patch establishment module 40 , initial person-scene image acquisition module 50 , perspective comparison result acquisition module 60 , target person-scene image acquisition module 70 . DETAILED DESCRIPTION
[0013] The present application provides a system and method for generating a human-scene integrated picture on a mobile terminal based on artificial intelligence, thereby solving the technical problem that the segmentation accuracy is low, the light is inconsistent, the perspective effect is unnatural, and the realism of the synthesized image is poor.
[0014] After introducing the basic principles of the present application, various non-limiting implementation methods of the present application will be specifically described below in conjunction with the drawings in the specification.
[0015] Embodiment 1:
[0016] like Figure 1 As shown, the present application provides a system for generating a person-scene integrated picture on a mobile terminal based on artificial intelligence, the system comprising:
[0017] A target portrait acquisition module 10, wherein the target portrait acquisition module 10 is used to call a predetermined edge separation strategy to perform edge separation processing on the initial frame separation block of the portrait image to obtain a target portrait;
[0018] A target background feature acquisition module 20, the target background feature acquisition module 20 is used to read a predetermined picture feature dimension, and perform multi-dimensional acquisition of the background picture based on the predetermined picture feature dimension to obtain a target background feature;
[0019] An output information acquisition module 30, wherein the output information acquisition module 30 is used to use the background light characteristic parameters in the target background characteristics as input information of the patch construction model to obtain output information, wherein the patch construction model is an intelligent model constructed based on the principle of random sampling consistency;
[0020] A target light patch establishing module 40, the target light patch establishing module 40 is used to adjust the patch model parameters in the output information in combination with the background color feature value and the background texture feature value in the target background feature, and establish the target light patch of the background image;
[0021] An initial person-scene image acquisition module 50, the initial person-scene image acquisition module 50 is used to fuse the target person image with the background image based on the target light patch to obtain an initial person-scene image;
[0022] A perspective comparison result acquisition module 60, wherein the perspective comparison result acquisition module 60 is used to analyze and compare the target portrait perspective information and the background perspective information of the target portrait to obtain a perspective comparison result;
[0023] The target person-scene picture acquisition module 70 is used to correct the initial person-scene picture based on the perspective comparison result to obtain the target person-scene picture.
[0024] Furthermore, the predetermined edge separation strategy includes a predetermined hair edge separation strategy, a predetermined skin edge separation strategy and a predetermined clothing edge separation strategy, and the system includes:
[0025] An edge-related mesh set acquisition module, the edge-related mesh set acquisition module is used to screen and analyze multiple meshes in the mesh set obtained by dividing the initial border separation block to obtain an edge-related mesh set;
[0026] A clustering result acquisition module, wherein the clustering result acquisition module is used to perform cluster analysis on the edge mesh set to obtain edge mesh clustering results, wherein the edge mesh clustering results include hair clustering clusters, skin clustering clusters, and clothing clustering clusters;
[0027] a clothing separation result acquisition module, the clothing separation result acquisition module being used to separate the grids in the hair cluster based on the predetermined hair edge separation strategy to obtain a hair separation result, to separate the grids in the skin cluster based on the predetermined skin edge separation strategy to obtain a skin separation result, and to separate the grids in the clothing cluster based on the predetermined clothing edge separation strategy to obtain a clothing separation result;
[0028] A portrait acquisition module is used to obtain the target portrait according to the hair separation result, the skin separation result and the clothing separation result.
[0029] Furthermore, the predetermined image feature dimensions include color dimension, texture dimension, light dimension and perspective dimension, and the system includes:
[0030] A background color feature value acquisition module, the background color feature value acquisition module is used to acquire the background color feature value of the background image based on the color dimension;
[0031] A background texture feature value acquisition module, the background texture feature value acquisition module is used to acquire the background texture feature value of the background image based on the texture dimension;
[0032] A background light characteristic parameter acquisition module, the background light characteristic parameter acquisition module is used to read the predetermined light characteristics of the light dimension, and acquire the background light characteristic parameters of the background image based on the predetermined light characteristics;
[0033] A background perspective information acquisition module, the background perspective information acquisition module is used to acquire the background perspective information of the background image based on the perspective dimension;
[0034] The target background feature composition module is used for the background color feature value, the background texture feature value, the background light feature parameter and the background perspective information to jointly form the target background feature.
[0035] Further, the system comprises:
[0036] A background image processing module, the background image processing module is used to perform discrete cosine transform processing on the background image to obtain background discrete cosine transform coefficients;
[0037] A weighted calculation module is used to perform weighted calculation on the background DC coefficient in the background discrete cosine transform coefficient in combination with the color coefficient to obtain the background color feature value.
[0038] Furthermore, the background AC coefficient in the background discrete cosine transform coefficient is weightedly calculated in combination with the texture coefficient to obtain the background texture feature value, wherein the sum of the texture coefficient and the color coefficient is 1.
[0039] Furthermore, the predetermined light characteristics include light direction, light intensity, light color temperature, shadow distribution and highlight distribution.
[0040] Furthermore, the system comprises:
[0041] A first estimation information acquisition module, wherein the first estimation information acquisition module is used to randomly acquire first model parameter estimation information;
[0042] A first verification module, the first verification module is used to verify the first patch model generated according to the first model parameter estimation information based on a predetermined light feature to obtain first verification information;
[0043] A second estimation information acquisition module, the second estimation information acquisition module is used to randomly acquire second model parameter estimation information;
[0044] A second verification module, the second verification module is used to verify the second patch model generated according to the second model parameter estimation information based on the predetermined light feature to obtain second verification information;
[0045] A patch model parameter determination module is used for the patch construction model to determine the patch model parameters according to a comparison result obtained by comparing the first verification information with the second verification information.
[0046] Furthermore, the system comprises:
[0047] A first light characteristic parameter acquisition module, the first light characteristic parameter acquisition module is used to acquire the first light characteristic parameter of the first patch model based on the predetermined light characteristic;
[0048] A first verification information acquisition module is used to use a first similarity between the first light characteristic parameter and the background light characteristic parameter as the first verification information.
[0049] Furthermore, the system further comprises:
[0050] A fusion evaluation module is used to introduce a fusion evaluation function to perform fusion evaluation on the initial human-scene image to obtain an initial fusion degree, wherein the expression of the fusion evaluation function is as follows:
[0051] ,
[0052] Wherein, M refers to the initial fusion degree of the target person portrait in the initial person-scene picture and the background picture, and and and and They refer to the light direction fusion function, light intensity fusion function, light color temperature fusion function, light shadow distribution fusion function and light highlight distribution fusion function respectively. and and and and respectively refer to the light direction, light intensity, light color temperature, shadow distribution and highlight distribution of the background image in the initial human-scene image, and They respectively refer to the preset shadow index and preset highlight index of the background image in the initial human-scene image, and and and and They respectively refer to the light direction, light intensity, light color temperature, shadow distribution and highlight distribution of the target portrait in the initial human-scene picture, and respectively refer to the preset shadow index and preset highlight index of the target portrait in the initial human-scene picture, and and and are the first, second, third and fourth coefficients respectively, and refers to the shadow distribution characteristics of the background image in the initial human-scene image, including the shadow distribution shape, position, and softness and hardness. refers to the shadow distribution characteristics of the target person portrait in the initial person-scene image, including the shadow distribution shape, position, and softness and hardness. refers to the highlight distribution characteristics of the background image in the initial human-scene image, including the highlight distribution shape, position, and softness and hardness. Refers to the highlight distribution characteristics of the target person portrait in the initial person-scene image, including the highlight distribution shape, position, and softness and hardness;
[0053] An accuracy verification module is used to verify the accuracy of the target light patch according to the initial fusion degree.
[0054] This specification describes in detail the method for generating a person-scene integrated picture on a mobile terminal based on artificial intelligence through Example 2. Those skilled in the art can clearly understand the system for generating a person-scene integrated picture on a mobile terminal based on artificial intelligence in this embodiment. Since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the description of the method part.
[0055] Embodiment 2:
[0056] Based on the same inventive concept as the artificial intelligence-based mobile terminal person-scene integrated picture generation system in the aforementioned embodiment, Figure 2 As shown, the embodiment of the present application provides a method for generating a person-scene integrated picture on a mobile terminal based on artificial intelligence, and the method includes:
[0057] The predetermined edge separation strategy is called to perform edge separation processing on the initial border separation block of the portrait image to obtain the target portrait.
[0058] The predetermined edge separation strategy includes a predetermined hair edge separation strategy, a predetermined skin edge separation strategy and a predetermined clothing edge separation strategy. The portrait image is divided into a plurality of blocks, and each block is analyzed to generate an initial frame separation block. The edge information in each block is analyzed, and blocks containing significant edges are screened out. Different edge separation strategies are applied to the screened edge blocks to separate details such as hair, skin and clothing. The separated different edge areas are merged to generate a target portrait.
[0059] Further, the predetermined edge separation strategy includes a predetermined hair edge separation strategy, a predetermined skin edge separation strategy and a predetermined clothing edge separation strategy, and the method includes:
[0060] Screening and analyzing multiple grids in the grid set obtained by dividing the initial border separation block to obtain an edge-related grid set; performing cluster analysis on the edge-related grid set to obtain edge-related grid clustering results, wherein the edge-related grid clustering results include hair clustering clusters, skin clustering clusters, and clothing clustering clusters; separating the grids in the hair clustering cluster based on the predetermined hair edge separation strategy to obtain a hair separation result, separating the grids in the skin clustering cluster based on the predetermined skin edge separation strategy to obtain a skin separation result, and separating the grids in the clothing clustering cluster based on the predetermined clothing edge separation strategy to obtain a clothing separation result; obtaining the target portrait based on the hair separation result, the skin separation result, and the clothing separation result.
[0061] The portrait image is divided into multiple grids based on the initial border separation blocks. Each grid represents a small area in the image. This allows the edge features of different areas to be refined. The edge detection algorithm is used to identify the edges in the grid. The edge is the area where the pixel value changes dramatically. The edge strength in each grid is calculated. Grids with high strength contain more edge information. Grids with edge strength higher than the preset threshold are selected to form an edge-related grid set, which contains the edge information in the image.
[0062] The feature vectors of the edge mesh set are standardized to ensure the consistency of dimensions between different features. The clustering process is performed using a clustering algorithm. For example, the K-means algorithm is used to select the number of K-means clusters k according to image features, and k cluster centers are initialized. Through an iterative process, each mesh is assigned to the nearest cluster center, and then the cluster center position is updated until the cluster center no longer changes or changes very little. According to the output of the clustering algorithm, the edge mesh set is divided into hair clustering clusters, skin clustering clusters, and clothing clustering clusters, which represent different edge types respectively.
[0063] The predetermined hair edge separation strategy includes utilizing the unique texture and color features of the hair in combination with an edge detection algorithm to perform fine separation of the hair region. For example, the color, texture and gradient features of the hair region are extracted, the gradient features are used to further refine the edges of the hair region, an edge detection algorithm is applied to identify the edges of the hair region, the hair edges are strengthened, interference from the background and other regions is eliminated, morphological operations such as dilation and erosion are used to refine the hair edges, a region growing algorithm is used to further determine the hair region, noise is eliminated, and the above processing results are combined to obtain a hair separation result.
[0064] Similarly, the predetermined skin edge separation strategy includes utilizing the color consistency and smoothness characteristics of the skin, combining edge detection and region growing algorithms, to separate the skin region and obtain skin separation results; the predetermined clothing edge separation strategy includes utilizing the texture and color differences of clothing, combining edge detection and image segmentation algorithms, to separate the clothing region and obtain clothing separation results. These separation results provide accurate edge information for the subsequent fusion of portrait and background.
[0065] The hair separation result, the skin separation result and the clothing separation result are combined, for example, by using image synthesis technology to superimpose the pixel point sets of the three to form a complete target portrait.
[0066] The predetermined image feature dimensions are read, and the background image is multi-dimensionally collected based on the predetermined image feature dimensions to obtain target background features.
[0067] The predetermined image feature dimensions include color dimension, texture dimension, light dimension and perspective dimension. Based on the color dimension, texture dimension, light dimension and perspective dimension, the color feature of the background image is obtained, which usually includes information such as color distribution and main color; the texture feature of the background image is obtained, including information such as texture direction and roughness; the light feature of the background image is obtained, including information such as light direction, intensity, color temperature, etc.; the perspective feature of the background image is obtained, including information such as perspective point and vanishing point. The features of the above dimensions are integrated to form a complete target background feature.
[0068] Furthermore, the predetermined image feature dimensions include color dimension, texture dimension, light dimension and perspective dimension, and the method includes:
[0069] Based on the color dimension, the background color characteristic value of the background image is obtained; based on the texture dimension, the background texture characteristic value of the background image is obtained; the predetermined light characteristics of the light dimension are read, and the background light characteristic parameters of the background image are obtained based on the predetermined light characteristics; based on the perspective dimension, the background perspective information of the background image is obtained; the background color characteristic value, the background texture characteristic value, the background light characteristic parameters and the background perspective information together constitute the target background characteristics.
[0070] The background image is converted from the RGB color space to other color spaces, such as the Lab color space, and the converted image is subjected to discrete cosine transform to obtain background discrete cosine transform coefficients. Based on the color dimension, the background DC coefficient in the background discrete cosine transform coefficient is weightedly calculated in combination with the color coefficient to obtain the background color feature value; based on the texture dimension, the background AC coefficient in the background discrete cosine transform coefficient is weightedly calculated in combination with the texture coefficient to obtain the background texture feature value, wherein the sum of the texture coefficient and the color coefficient is 1.
[0071] The predetermined light characteristics include light direction, light intensity, light color temperature, shadow distribution and highlight distribution. According to the predetermined light characteristics, the actual light performance in the background image is analyzed, and light characteristic parameters such as direction angle, intensity value, color temperature value, etc. are collected to obtain the background light characteristic parameters.
[0072] Use perspective transformation or perspective projection technology to analyze the perspective information in the background image, including the distance relationship of the scene, perspective distortion, changes in object size and shape, etc., and convert the extracted perspective information into a parameter form suitable for processing, including parameters such as perspective center, perspective angle, perspective ratio, etc., to obtain background perspective information.
[0073] The background color feature value, background texture feature value, background light feature parameter and background perspective information are integrated to form a target background feature to provide a more comprehensive and integrated background description.
[0074] Further, the method comprises:
[0075] The background image is processed by discrete cosine transform to obtain background discrete cosine transform coefficients; the background DC coefficients in the background discrete cosine transform coefficients are weightedly calculated in combination with color coefficients to obtain the background color feature values.
[0076] Discrete cosine transform is a commonly used image processing technology that can convert image data from the time domain to the frequency domain and is used for local processing of the image. Specifically, the background image is divided into blocks of the same size, for example, into blocks of 8×8 pixels, and a discrete cosine transform is performed on each 8×8 image block based on the discrete cosine transform formula to obtain the corresponding discrete cosine transform coefficient matrix.
[0077] In discrete cosine transform, the background DC coefficient refers to the DC component in discrete cosine transform, which corresponds to the low-frequency information in the image. The DC component represents the overall brightness or color shift in the image. The texture coefficient and color coefficient are parameters used to perform weighted calculation on the discrete cosine transform coefficients. The sum of the two is 1, which can ensure a comprehensive analysis of the image features. The color coefficient represents the color information of the image and is used to highlight the color features of the image. The background DC coefficient is weightedly calculated in combination with the color coefficient to obtain the background color feature value, which is used to describe the background color feature information of the image.
[0078] Furthermore, the background AC coefficient in the background discrete cosine transform coefficient is weightedly calculated in combination with the texture coefficient to obtain the background texture feature value, wherein the sum of the texture coefficient and the color coefficient is 1.
[0079] In discrete cosine transform, the background AC coefficient refers to the AC component in discrete cosine transform, which corresponds to the high-frequency information in the image. The AC component represents the details and texture in the image. The texture coefficient represents the texture information of the image and is used to highlight the texture features in the image. The background AC coefficient is weightedly calculated in combination with the texture coefficient to obtain the background texture eigenvalue. The background texture eigenvalue is used to describe the background texture feature information of the image.
[0080] Furthermore, the predetermined light characteristics include light direction, light intensity, light color temperature, shadow distribution and highlight distribution.
[0081] The predetermined light characteristics include light direction, light intensity, light color temperature, shadow distribution and highlight distribution, among which light direction refers to the direction in which the light source illuminates the surface of an object. In image processing, the light direction can affect the object's shadow, reflection, brightness and other characteristics; light intensity describes the brightness or intensity of the light source. The stronger the light intensity, the brighter the light reflected from the object surface; light color temperature refers to the color characteristics of the light emitted by the light source, which is usually used to describe the degree of warmth or coolness of the light. A light source with a high color temperature presents a blue hue, while a light source with a low color temperature presents a warm hue; shadow distribution describes the distribution of shadows produced on the surface of an object under light irradiation. The shape, size and position of the shadows will be affected by the direction and intensity of the light; highlight distribution refers to the area with high brightness and strong reflection produced on the surface of an object under strong light irradiation. The position and size of the highlight will also be affected by the direction and intensity of the light.
[0082] The background light feature parameters in the target background feature are used as input information of the patch construction model to obtain output information, wherein the patch construction model is an intelligent model constructed based on the principle of random sampling consistency.
[0083] The random sampling consistency principle is an iterative method for model estimation, which is used to estimate when there are a large number of outliers in the data. The core idea is to continuously optimize parameters through random sampling and model verification to find the best fitting model. The goal of the patch construction model is to generate light patches that match the background light characteristics for fusion with the target portrait.
[0084] Specifically, the initial light patch model parameters are randomly selected. These parameters are used to describe the properties of the light patch, including direction, intensity, color temperature, etc. The initial light patch is generated based on the randomly initialized parameters. This patch is a virtual illumination model that contains the assumed illumination conditions. The generated light patch is compared with the background light characteristic parameters and the matching degree is calculated. A high matching degree means that the light patch is more consistent with the background light characteristics. Based on the principle of random sampling consistency, multiple random sampling and model verification are performed, and the parameters of the light patch are continuously adjusted. After multiple iterations and verifications, a set of light patch parameters are finally determined. These parameters can most accurately reflect the background light characteristics. The optimal parameters are used to generate the final light patch model as the output information. This model is used for subsequent portrait and background fusion.
[0085] The patch model parameters in the output information are adjusted in combination with the background color feature value and the background texture feature value in the target background feature, and a target light patch of the background image is established.
[0086] The patch model parameters include light direction, light intensity, light color temperature, etc. The light patch model parameters are adjusted according to the background color characteristics and background texture characteristics to make the light patch more naturally integrated into the background image. The target light patch is generated according to the adjusted light patch model parameters. This light patch can show a natural lighting effect on the background image, with both correct light direction and intensity, and shadow and highlight distribution that matches the background texture and color.
[0087] Further, including:
[0088] Randomly obtain first model parameter estimation information; based on predetermined light characteristics, verify the first patch model generated according to the first model parameter estimation information to obtain first verification information; randomly obtain second model parameter estimation information; based on the predetermined light characteristics, verify the second patch model generated according to the second model parameter estimation information to obtain second verification information; the patch construction model determines the patch model parameters according to the comparison result obtained by comparing the first verification information with the second verification information.
[0089] The parameter range of the first model is defined, including predetermined light characteristics such as light direction, intensity, color temperature, shadow distribution, and highlight distribution. These parameters can be continuous values or discrete values. For example, the light direction can be a set of angle values. According to the defined parameter range, a random number generator is used to generate random parameter values within the range as the first model parameter estimation information.
[0090] Using the generated first model parameter estimation information, a first patch model is constructed according to predetermined light characteristics, that is, according to the predetermined light characteristics, the effect of light on the first patch model is simulated, including simulating the direction, intensity, color temperature, shadow distribution and highlight distribution of the light, etc. The simulated light effect is compared with the actual background light characteristics, for example, a similarity calculation is performed, to obtain first verification information.
[0091] According to the defined parameter range, a random parameter value within the range is generated by a random number generator to obtain second model parameter estimation information. In the same way, a second patch model is generated according to the second model parameter estimation information, and a verification is performed to obtain second verification information.
[0092] Compare the first verification information with the second verification information to determine which model is more consistent with the actual background light characteristics, that is, determine the verification information with higher similarity between the first verification information and the second verification information, obtain the corresponding model parameter estimation information as the current most fitting parameter, repeat the above steps until the expected similarity is reached, or when the preset number of iterations is reached, obtain the final model parameter estimation information as the patch model parameter.
[0093] Further, the method comprises:
[0094] Based on the predetermined light feature collection, a first light feature parameter of the first patch model is obtained; and a first similarity between the first light feature parameter and the background light feature parameter is used as the first verification information.
[0095] According to the predetermined light characteristics, first light characteristic parameters of the first patch model are collected, including light direction parameters, light intensity parameters, light color temperature parameters, shadow distribution parameters, and highlight distribution parameters.
[0096] The first light characteristic parameters of the first patch model are similar to the background light characteristic parameters, and the calculated first similarity is used as the first verification information. This similarity is used to evaluate the matching degree between the light characteristics of the first patch model and the actual background light characteristics, thereby evaluating the accuracy of the model.
[0097] Based on the target light patch, the target portrait and the background image are merged to obtain an initial person-scene image;
[0098] Apply the lighting effect of the target light patch to the target portrait so that the lighting effect of the portrait is consistent with the lighting characteristics of the background, including adjusting the lighting direction of the portrait according to the light source direction of the target light patch; adjusting the brightness of the portrait according to the lighting intensity of the target light patch to match the brightness of the background; adjusting the color temperature of the portrait according to the light color temperature of the target light patch to make it consistent with the background color. The adjusted target portrait is merged with the background image to generate the initial human-scene image, so that the processed human-scene image has a high degree of naturalness and consistency.
[0099] The target portrait perspective information and the background perspective information of the target portrait are analyzed and compared to obtain a perspective comparison result.
[0100] Based on the perspective dimension, the background perspective information of the background image is obtained, and the perspective information is extracted from the target portrait. The perspective information mainly includes the vanishing point, viewing angle, and scaling ratio. The perspective information of the target portrait and the background are analyzed and compared. For example, by analyzing the lines in the target portrait and the background image, the positions of the vanishing points of the two are determined, and it is judged whether the positions of the vanishing points of the two are consistent. If they are inconsistent, it is necessary to adjust the position of the portrait or the perspective parameters of the background; analyze the viewing angles of the target portrait and the background image, determine the observation angle, and judge whether the viewing angles of the two are consistent. If they are inconsistent, it is necessary to adjust the rotation angle of the portrait or the viewing angle parameters of the background; analyze the scaling ratio of the target portrait and the background image, determine the far and near scaling effects of the two, and judge whether the scaling ratios are consistent. If they are inconsistent, it is necessary to adjust the size of the portrait or the scaling parameters of the background. Integrate the comparison results of the vanishing point, viewing angle, and scaling ratio of the target portrait and the background image to obtain the perspective comparison result.
[0101] The initial person-scene picture is corrected based on the perspective comparison result to obtain a target person-scene picture.
[0102] The initial human-scene image is corrected based on the perspective comparison result. Specifically, if the perspective comparison result shows that the vanishing points are inconsistent, the position of the portrait is moved to make the vanishing point of the portrait consistent with the vanishing point of the background; if the perspective is inconsistent, the portrait is rotated to make its perspective consistent with the background; if the scaling ratio is inconsistent, the size of the portrait is adjusted to make its ratio consistent with the background. All perspective, perspective, and scaling ratio adjustments are applied to the initial human-scene image to generate the corrected target human-scene image.
[0103] Furthermore, the method further comprises:
[0104] A fusion evaluation function is introduced to perform fusion evaluation on the initial human-scene image to obtain an initial fusion degree, wherein the expression of the fusion evaluation function is as follows:
[0105] ,
[0106] Wherein, M refers to the initial fusion degree of the target person portrait in the initial person-scene picture and the background picture, and and and and They refer to the light direction fusion function, light intensity fusion function, light color temperature fusion function, light shadow distribution fusion function and light highlight distribution fusion function respectively. and and and and respectively refer to the light direction, light intensity, light color temperature, shadow distribution and highlight distribution of the background image in the initial human-scene image, and They respectively refer to the preset shadow index and preset highlight index of the background image in the initial human-scene image, and and and and They respectively refer to the light direction, light intensity, light color temperature, shadow distribution and highlight distribution of the target portrait in the initial human-scene picture, and respectively refer to the preset shadow index and preset highlight index of the target portrait in the initial human-scene picture, and and and are the first, second, third and fourth coefficients respectively, and refers to the shadow distribution characteristics of the background image in the initial human-scene image, including the shadow distribution shape, position, and softness and hardness. refers to the shadow distribution characteristics of the target person portrait in the initial person-scene image, including the shadow distribution shape, position, and softness and hardness. refers to the highlight distribution characteristics of the background image in the initial human-scene image, including the highlight distribution shape, position, and softness and hardness. It refers to the highlight distribution characteristics of the target portrait in the initial human-scene picture, including the highlight distribution shape, position, and hardness; the accuracy of the target light patch is checked according to the initial fusion degree.
[0107] Specifically, a fusion evaluation function is introduced to perform fusion evaluation on the initial human-scene image to obtain an initial fusion degree, wherein the expression of the fusion evaluation function is as follows:
[0108] ,
[0109] The fusion evaluation function is a tool for evaluating the fusion of the initial human-scene image. It determines the degree of fusion by evaluating different aspects of light characteristics and factors such as shadow and highlight distribution. This function comprehensively evaluates the degree of fusion between the target portrait and the background image by combining light characteristics such as light direction, intensity, color temperature, and the distribution of shadows and highlights. Each feature has a corresponding fusion function to evaluate its contribution, and in the light and shadow distribution fusion function Import the preset shadow index of the background image , preset shadow index for target portrait , these two values are quantitative parameters that reflect the overall lighting conditions of the background image and the target portrait. They serve as the basis for quantifying the shadows in the background image and the target portrait. For example, on a cloudy day, the shadows are softer and more dispersed, while under direct sunlight, the shadows are more obvious and have clear boundaries. By setting , , the shadow intensity in the background image can be adjusted according to these conditions. Import the preset highlight index of the background image , preset highlight index for target portrait , and the same effect is achieved. Finally, the influence of each feature is balanced by adjusting different coefficients to obtain the final fusion evaluation value M.
[0110] An accuracy evaluation standard, such as a fusion degree threshold, is set to determine whether the target light patch meets the expected fusion effect, and the initial fusion degree is compared with the fusion degree threshold. According to the evaluation result, if the initial fusion degree meets the requirements, it means that the target light patch has passed the accuracy check; if the initial fusion degree does not meet the requirements, it means that the target light patch has not passed the accuracy check and needs to be adjusted and optimized.
[0111] In summary, the method for generating a person-scene integrated picture on a mobile terminal based on artificial intelligence provided by the embodiment of the present application has the following technical effects:
[0112] 1. By calling the predetermined edge separation strategy to separate the border of the portrait image, the portrait and the background are accurately separated, avoiding the edge blur or wrong segmentation problems existing in traditional methods;
[0113] 2. By reading the predetermined image feature dimensions and collecting the background image in multiple dimensions based on them, richer and more accurate background features are obtained, which helps to improve the authenticity of the synthesized image;
[0114] 3. Using the principle of random sampling consistency to build an intelligent patch model, it can more adaptively generate patches suitable for the current scene, improving the intelligence and efficiency of the synthesis process;
[0115] 4. Combine the color and texture feature values in the background features to adjust the patch model parameters to better match the background light and enhance the consistency of the synthesized image;
[0116] 5. Based on the target light patch, the target portrait and the background image are fused, and the precise fusion of the synthetic image is achieved through comparison of light direction, intensity, color temperature and other aspects;
[0117] 6. Analyze and compare the perspective information of the target portrait and the background, and use it to correct the image, thereby improving the visual effect of the composite image.
[0118] In summary, this AI-based mobile human-scenery image generation system solves the problems of low segmentation accuracy, inconsistent lighting, unnatural perspective effect, etc. in traditional human-scenery image generation through multi-dimensional feature collection, intelligent patch model construction, parameter adjustment and precise fusion, achieving the technical effect of improving the realism and fidelity of the synthesized image.
[0119] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A mobile terminal human-scene integrated picture generation system based on artificial intelligence, characterized in that: include: A target portrait acquisition module, wherein the target portrait acquisition module is used to call a predetermined edge separation strategy to perform edge separation processing on the initial frame separation block of the portrait image to obtain a target portrait; A target background feature acquisition module, the target background feature acquisition module is used to read a predetermined picture feature dimension, and perform multi-dimensional acquisition of the background picture based on the predetermined picture feature dimension to obtain a target background feature; An output information acquisition module, wherein the output information acquisition module is used to use background light characteristic parameters in the target background characteristics as input information of a patch construction model to obtain output information, wherein the patch construction model is an intelligent model constructed based on the principle of random sampling consistency; A target light patch establishment module, the target light patch establishment module is used to adjust the patch model parameters in the output information in combination with the background color feature value and the background texture feature value in the target background feature, and establish the target light patch of the background image; An initial person-scene image acquisition module, the initial person-scene image acquisition module is used to fuse the target person image and the background image based on the target light patch to obtain an initial person-scene image; A perspective comparison result acquisition module, the perspective comparison result acquisition module is used to analyze and compare the target portrait perspective information and the background perspective information of the target portrait to obtain a perspective comparison result; A target person-scene picture acquisition module is used to correct the initial person-scene picture based on the perspective comparison result to obtain a target person-scene picture.
2. The system according to claim 1, characterized in that: The predetermined edge separation strategy includes a predetermined hair edge separation strategy, a predetermined skin edge separation strategy and a predetermined clothing edge separation strategy, and the system includes: An edge-related mesh set acquisition module, the edge-related mesh set acquisition module is used to screen and analyze multiple meshes in the mesh set obtained by dividing the initial border separation block to obtain an edge-related mesh set; A clustering result acquisition module, wherein the clustering result acquisition module is used to perform cluster analysis on the edge mesh set to obtain edge mesh clustering results, wherein the edge mesh clustering results include hair clustering clusters, skin clustering clusters, and clothing clustering clusters; a clothing separation result acquisition module, the clothing separation result acquisition module being used to separate the grids in the hair cluster based on the predetermined hair edge separation strategy to obtain a hair separation result, to separate the grids in the skin cluster based on the predetermined skin edge separation strategy to obtain a skin separation result, and to separate the grids in the clothing cluster based on the predetermined clothing edge separation strategy to obtain a clothing separation result; A portrait acquisition module is used to obtain the target portrait according to the hair separation result, the skin separation result and the clothing separation result.
3. The system according to claim 1, characterized in that: The predetermined image feature dimensions include color dimension, texture dimension, light dimension and perspective dimension, and the system includes: A background color feature value acquisition module, the background color feature value acquisition module is used to acquire the background color feature value of the background image based on the color dimension; A background texture feature value acquisition module, the background texture feature value acquisition module is used to acquire the background texture feature value of the background image based on the texture dimension; A background light characteristic parameter acquisition module, the background light characteristic parameter acquisition module is used to read the predetermined light characteristics of the light dimension, and acquire the background light characteristic parameters of the background image based on the predetermined light characteristics; A background perspective information acquisition module, the background perspective information acquisition module is used to acquire the background perspective information of the background image based on the perspective dimension; The target background feature composition module is used for the background color feature value, the background texture feature value, the background light feature parameter and the background perspective information to jointly form the target background feature.
4. The system according to claim 3, characterized in that: The system includes: A background image processing module, the background image processing module is used to perform discrete cosine transform processing on the background image to obtain background discrete cosine transform coefficients; A weighted calculation module is used to perform weighted calculation on the background DC coefficient in the background discrete cosine transform coefficient in combination with the color coefficient to obtain the background color feature value.
5. The system according to claim 4, characterized in that: The background AC coefficient in the background discrete cosine transform coefficient is weightedly calculated in combination with the texture coefficient to obtain the background texture feature value, wherein the sum of the texture coefficient and the color coefficient is 1.
6. The system according to claim 3, characterized in that: The predetermined light characteristics include light direction, light intensity, light color temperature, shadow distribution and highlight distribution.
7. The system according to claim 1, characterized in that: include: A first estimation information acquisition module, wherein the first estimation information acquisition module is used to randomly acquire first model parameter estimation information; A first verification module, the first verification module is used to verify the first patch model generated according to the first model parameter estimation information based on a predetermined light feature to obtain first verification information; A second estimation information acquisition module, the second estimation information acquisition module is used to randomly acquire second model parameter estimation information; A second verification module, the second verification module is used to verify the second patch model generated according to the second model parameter estimation information based on the predetermined light feature to obtain second verification information; A patch model parameter determination module is used for the patch construction model to determine the patch model parameters according to a comparison result obtained by comparing the first verification information with the second verification information.
8. The system according to claim 7, characterized in that the system include: A first light characteristic parameter acquisition module, the first light characteristic parameter acquisition module is used to acquire the first light characteristic parameter of the first patch model based on the predetermined light characteristic; A first verification information acquisition module is used to use a first similarity between the first light characteristic parameter and the background light characteristic parameter as the first verification information.
9. The system according to claim 1, characterized in that: The system also includes: A fusion evaluation module is used to introduce a fusion evaluation function to perform fusion evaluation on the initial human-scene image to obtain an initial fusion degree, wherein the expression of the fusion evaluation function is as follows: , Wherein, M refers to the initial fusion degree of the target person portrait in the initial person-scene picture and the background picture, and and and and They refer to the light direction fusion function, light intensity fusion function, light color temperature fusion function, light shadow distribution fusion function and light highlight distribution fusion function respectively. and and and and respectively refer to the light direction, light intensity, light color temperature, shadow distribution and highlight distribution of the background image in the initial human-scene image, and They respectively refer to the preset shadow index and preset highlight index of the background image in the initial human-scene image, and and and and They respectively refer to the light direction, light intensity, light color temperature, shadow distribution and highlight distribution of the target portrait in the initial human-scene picture, and respectively refer to the preset shadow index and preset highlight index of the target portrait in the initial human-scene picture, and and and are the first, second, third and fourth coefficients respectively, and refers to the shadow distribution characteristics of the background image in the initial human-scene image, including the shadow distribution shape, position, and softness and hardness. refers to the shadow distribution characteristics of the target person portrait in the initial person-scene image, including the shadow distribution shape, position, and softness and hardness. refers to the highlight distribution characteristics of the background image in the initial human-scene image, including the highlight distribution shape, position, and softness and hardness. Refers to the highlight distribution characteristics of the target person portrait in the initial person-scene image, including the highlight distribution shape, position, and softness and hardness; An accuracy verification module is used to verify the accuracy of the target light patch according to the initial fusion degree.
10. A method for generating a human-scene integrated picture on a mobile terminal based on artificial intelligence, characterized in that: The method is implemented by the artificial intelligence-based mobile terminal person-scene integrated picture generation system according to any one of claims 1 to 9, and the method comprises: Retrieving a predetermined edge separation strategy to perform edge separation processing on the initial border separation block of the portrait image to obtain a target portrait; Reading a predetermined image feature dimension, and performing multi-dimensional acquisition of the background image based on the predetermined image feature dimension to obtain a target background feature; Using background light feature parameters in the target background feature as input information of a patch construction model to obtain output information, wherein the patch construction model is an intelligent model constructed based on the principle of random sampling consistency; Adjust the patch model parameters in the output information in combination with the background color feature value and the background texture feature value in the target background feature, and establish the target light patch of the background image; Based on the target light patch, the target portrait and the background image are merged to obtain an initial person-scene image; Analyzing and comparing the target portrait perspective information and the background perspective information of the target portrait to obtain a perspective comparison result; The initial person-scene picture is corrected based on the perspective comparison result to obtain a target person-scene picture.
