Content generation method and system based on AI

Abstract

The invention relates to an AI-based content generation method and system. The method comprises the following steps: acquiring a separated image and a background image, and separating the separated image to obtain a feature object; establishing a first illumination model belonging to the separation image and a second illumination model belonging to the background image; restoring the feature object by using a first illumination model to obtain a basic feature object; processing the basic feature object by using a second illumination model to obtain a fused feature object; and transferring the fused feature object to a background image, dividing a fusion domain at the peripheral edge of the fused feature object, and processing the fusion domain by using a generative adversarial network, so that the fused feature object is fused with the background image. According to the AI-based content generation method and generation system disclosed by the invention, the generation content is converted by combining the environment features with the generative adversarial network, and the generative adversarial network can be limited by the mode, so that high-fidelity content can be obtained.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to an AI-based content generation method and generation system. Background Art

[0002] The research field of AI (artificial intelligence) is broad and in-depth, covering multiple sub-fields such as machine learning, deep learning, natural language processing, computer vision, robotics, etc., and these technologies have been widely used in our daily lives, such as voice recognition in smartphones, self-driving cars, smart home devices, and medical diagnostic systems.

[0003] In the field of productivity, the use of AI is becoming more and more extensive. For example, in the field of content (text, pictures, videos, etc.) generation, AI can widely replace repetitive operations and generate the required content according to instructions. However, in the generation of picture and video content, AI has the problem of generating unreal content.

[0004] Taking conversion generation as an example, there is a big difference between the conversion content and the converted content. Simple copy and paste will produce unreality. At this time, introducing a generative adversarial network for correction can solve the problem to a certain extent, but it is easy to have the problem of local optimal solution. Summary of the invention

[0005] The present application provides an AI-based content generation method and generation system, which uses environmental features combined with a generative adversarial network to transform and generate content. This method can limit the generative adversarial network to obtain high-fidelity content.

[0006] The above-mentioned purpose of the present application is achieved through the following technical solutions:

[0007] In a first aspect, the present application provides an AI-based content generation method, comprising:

[0008] Acquire a separation image and a background image and separate a feature object from the separation image;

[0009] Establishing a first illumination model belonging to the separated image and a second illumination model belonging to the background image;

[0010] Use the first illumination model to restore the feature object to obtain a basic feature object;

[0011] Process the basic feature object using the second illumination model to obtain a fused feature object;

[0012] Transferring the fused feature object to the background image and dividing the fusion domain at the surrounding edge of the fused feature object;

[0013] The fusion domain is processed using a generative adversarial network to fuse the fusion feature object with the background image.

[0014] In a possible implementation manner of the first aspect, establishing a first illumination model belonging to the separated image includes:

[0015] After grayscale processing is performed on the feature object, multiple regions are selected on the feature object, each region including at least one pixel point;

[0016] Use regions to establish datum reference lines;

[0017] Rotate the reference line and calculate the pixel value change trend on the reference line, so that at least two reference lines with the same pixel value change trend are parallel or intersecting;

[0018] A first lighting model is established using the rotated base reference line.

[0019] In a possible implementation manner of the first aspect, restoring the feature object using the first illumination model to obtain the basic feature object includes:

[0020] Adjusting the first illumination model so that the light sources in the first illumination model tend to be consistent and establishing a restoration direction line for the adjusted first illumination model;

[0021] Calculate the enhancement or reduction degree on the restoration direction line;

[0022] The feature object is adjusted using the enhancement or reduction degree to obtain the basic feature object.

[0023] In a possible implementation manner of the first aspect, after calculating the enhancement degree or reduction degree on the restoration direction line, the method further includes:

[0024] Determine the cross-influence area and blank influence area of ​​the restoration direction line;

[0025] determining a restoration direction line associated with the blank impact area;

[0026] Use the associated restoring direction line to create a secondary restoring direction line and adjust the blank influence area based on the secondary restoring direction line.

[0027] In a possible implementation manner of the first aspect, using the second illumination model to process the basic feature object to obtain the fused feature object includes:

[0028] Determine the brightness parameter of the transfer position of the fused feature object to the background image;

[0029] Adjust the underlying feature object using the brightness parameter;

[0030] Adjust the base feature object again using the second lighting model;

[0031] An adjusted feature object is obtained, and the adjusted feature object is recorded as a fused feature object.

[0032] In a possible implementation manner of the first aspect, determining a brightness parameter of a transfer position of the fused feature object to the background image includes:

[0033] Sequentially establishing multiple annular analysis reference lines inside the feature object and transferring the multiple annular analysis reference lines to the background image, in a direction close to the edge of the feature object, a previous annular analysis reference line is located inside a next annular analysis reference line;

[0034] Calculate the pixel value changes on the annular analysis reference line and determine the strongest and weakest illumination points according to the pixel value changes;

[0035] Use the brightest and weakest points to create brightness parameters.

[0036] In a possible implementation manner of the first aspect, when the fusion domain is processed using a generative adversarial network, the method further includes adjusting a width of the fusion domain.

[0037] In a second aspect, the present application provides an AI-based content generation device, comprising:

[0038] A first processing unit is used to obtain a separated image and a background image and separate a feature object from the separated image;

[0039] An illumination model establishing unit, used to establish a first illumination model belonging to the separated image and a second illumination model belonging to the background image;

[0040] A second processing unit is used to restore the feature object using the first illumination model to obtain a basic feature object;

[0041] A third processing unit is used to process the basic feature object using the second illumination model to obtain a fused feature object;

[0042] a fourth processing unit, configured to transfer the fused feature object to the background image and divide a fusion domain at the peripheral edge of the fused feature object;

[0043] The fusion processing unit is used to process the fusion domain using a generative adversarial network to fuse the fusion feature object with the background image.

[0044] In a third aspect, the present application provides a content generation system, the system comprising:

[0045] one or more memories for storing instructions; and

[0046] One or more processors, used to call and run the instructions from the memory to execute the method as described in the first aspect and any possible implementation of the first aspect.

[0047] In a fourth aspect, the present application provides a computer-readable storage medium, the computer-readable storage medium comprising:

[0048] Program, when the program is executed by a processor, the method described in the first aspect and any possible implementation of the first aspect is executed.

[0049] In a fifth aspect, the present application provides a computer program product, comprising program instructions. When the program instructions are executed by a computing device, the method described in the first aspect and any possible implementation of the first aspect is executed.

[0050] In a sixth aspect, the present application provides a chip system, which includes a processor for implementing the functions involved in the above aspects, for example, generating, receiving, sending, or processing the data and / or information involved in the above methods.

[0051] The chip system may be composed of chips, or may include chips and other discrete devices.

[0052] In a possible design, the chip system also includes a memory, which is used to store necessary program instructions and data. The processor and the memory can be decoupled and respectively set on different devices, connected by wired or wireless means, or the processor and the memory can also be coupled on the same device. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 It is a schematic flowchart of the steps of a content generation method provided by this application.

[0054] Figure 2 It is a schematic diagram of an establishment area provided by this application.

[0055] Figure 3 It is a schematic diagram of a rotational datum reference line provided in this application.

[0056] Figure 4 This is a schematic diagram of obtaining a point light source and a range light source provided by the present application.

[0057] Figure 5 It is a schematic diagram of a blank impact area provided in this application.

[0058] Figure 6 This is a schematic diagram provided by the present application for establishing a secondary restoration direction line based on the edge of a coverage area.

[0059] Figure 7 This is a schematic diagram provided by the present application for determining the strongest illumination point and the weakest illumination point according to the change of pixel values ​​on a circular analysis reference line. DETAILED DESCRIPTION

[0060] First, the local optimal solution of the generative adversarial network is explained.

[0061] GANs consist of two neural networks that compete with each other: a generator and a discriminator. The goal of the generator is to generate fake data that approximates real data, while the goal of the discriminator is to distinguish between real data and fake data.

[0062] This generative adversarial training method enables GANs to learn the distribution of data and generate high-quality new data. However, since the generator and discriminator need to play multiple rounds of adversarial games during training, they are prone to falling into local optimal solutions.

[0063] The specific reasons are as follows:

[0064] Complex training process: The training process of GANs involves the competition and adaptation of two networks. This complex dynamic relationship may cause the training to fall into a local optimum.

[0065] Design of loss function: The loss function of GANs is usually designed based on maximizing the error of the discriminator and minimizing the error of the generator. However, this design may lead to instability and the emergence of local optimal solutions during training.

[0066] Choice of model structure: Different model structures have a great impact on the training effect of GANs. If the model structure is not designed properly, it may also cause the training to fall into a local optimum.

[0067] The technical solution in this application is further described in detail below in conjunction with the accompanying drawings.

[0068] This application discloses an AI-based content generation method, see Figure 1 In some examples, the AI-based content generation method disclosed in this application includes the following steps:

[0069] S101, acquiring a separation image and a background image and obtaining a feature object by separation from the separation image;

[0070] S102, establishing a first illumination model belonging to the separated image and a second illumination model belonging to the background image;

[0071] S103, using the first illumination model to restore the feature object to obtain a basic feature object;

[0072] S104, using the second illumination model to process the basic feature object to obtain a fused feature object;

[0073] S105, transferring the fused feature object to the background image and dividing the fusion domain at the surrounding edge of the fused feature object;

[0074] S106, using a generative adversarial network to process the fusion domain so that the fusion feature object is fused with the background image.

[0075] In general, the technical solution disclosed in the present application first uses the illumination model to process the feature object to be transferred, and then uses the generative adversarial network to process the fusion domain, rather than directly using the generative adversarial network to process the fusion domain to process the entire feature object.

[0076] The advantage of this processing method is that it greatly compresses the processing area, so that the generative adversarial network only needs to process a limited fusion domain.

[0077] In step S101, a separation image and a background image are first acquired and a feature object is separated from the separation image. Then, in step S102, a first illumination model belonging to the separation image and a second illumination model belonging to the background image are established.

[0078] In step S103, the first illumination model is used to restore the feature object to obtain the basic feature object. The purpose of restoring the feature object is to remove the influence of illumination in the environment to which the separation image belongs on the basic feature object.

[0079] In step S104, the second illumination model is used to process the basic feature object to obtain a fused feature object. The purpose of processing the basic feature object with the second illumination model is to apply the illumination in the environment to which the background image belongs to the basic feature object.

[0080] In step S105, the fused feature object is transferred to the background image and a fusion domain is divided at the surrounding edge of the fused feature object. The fusion domain here needs to be processed using a generative adversarial network, that is, the content in step S106. After the generative adversarial network processes the fusion domain, the fused feature object can be fused with the background image.

[0081] From the above description, it can be seen that using the first illumination model and the second illumination model to process the feature object is to perform a fusion processing on the feature object based on the background image; and using the generative adversarial network to process the fusion domain is to perform a secondary fusion processing on the feature object based on the background image.

[0082] The purpose of the two fusion processes is to solve the overall problem and local problem in the fusion process respectively.

[0083] Before performing step S101, if the clarity of the separated image and the background image is different, it is necessary to first adjust the clarity to be the same, and then in the subsequent steps, the low clarity is used as the basis for adjusting the clarity. When the clarity of the separated image is low, the clarity of the background image is lowered, and when the clarity of the background image is low, the clarity of the separated image is lowered.

[0084] In some examples, a first illumination model belonging to the separated image is established in a specific manner:

[0085] S201, after grayscale processing of the feature object, multiple regions are selected on the feature object, each region including at least one pixel point;

[0086] S202, using the area to establish a baseline reference line;

[0087] S203, rotating the baseline reference line and calculating the pixel value change trend on the baseline reference line, so that at least two baseline reference lines with the same pixel value change trend are parallel or intersecting;

[0088] S204: Establish a first illumination model using the rotated baseline reference line.

[0089] In step S201 and step S202, the feature object is firstly grayscale processed. The purpose of grayscale processing is to reduce the data volume of the feature object. Then, multiple regions are selected on the feature object. Figure 2 , the selected area is as close to the edge of the feature object as possible and is randomly distributed.

[0090] Each region includes at least one pixel point. The region is used to establish a baseline reference line. The baseline reference line starts from or passes through the region. When the region includes multiple pixels, the center point of the region is used to establish the baseline reference line.

[0091] In step S203, refer to Figure 3 , the baseline reference line will be rotated and the pixel value change trend on the baseline reference line will be calculated, so that at least two baseline reference lines with the same pixel value change trend are parallel or intersecting. The pixel value change trend here can be either increasing or decreasing.

[0092] For the changing trend of pixel value changes, one calculation method is to randomly select a point or a segment on the baseline reference line (the values ​​of the pixel points in this segment are averaged), and then calculate the changing trend of the pixel value changes in sequence.

[0093] Of course, the slope corresponding to the change trend obtained at this time will not be a stable and unchanging value. In this application, the value is required to fluctuate within a certain range, and the error is generally allowed to be between ±0.2-0.6. Of course, the error can also be changed. This is only an example and is not a limitation.

[0094] The purpose of rotating the datum reference line is to find the light source that affects the feature object. The number of light sources may be one or more. There are also two types of light sources, namely bright light sources and dark light sources. The function of the bright light source is to emit light, and the function of the dark light source is to absorb light.

[0095] The obtained values ​​of the bright light source and the dark light source directly use the values ​​at the corresponding positions on the baseline reference line.

[0096] In addition, the result of the rotation is to stabilize the change trend (increase and decrease) of the pixel value on the reference line, that is, the change trend of the pixel value on the reference line is a linear change.

[0097] There are two situations when the base reference line is rotated, namely intersection and parallel. Intersection indicates the appearance of point light source, while parallel indicates the appearance of range light source.

[0098] Finally, the first illumination model is established using the rotated baseline reference line. The first illumination model is based on the above-mentioned records, and the point light source and the range light source are recorded, and the intensity of the point light source and the range light source are recorded at the same time.

[0099] The specific method of restoring the feature object using the first illumination model to obtain the basic feature object is:

[0100] S301, adjusting the first illumination model to make the light sources in the first illumination model tend to be consistent and establishing a restoration direction line for the adjusted first illumination model;

[0101] S302, calculating and obtaining the enhancement degree or reduction degree on the restoration direction line;

[0102] S303, adjusting the feature object using the enhancement degree or reduction degree to obtain a basic feature object.

[0103] Specifically, it is necessary to first adjust the first illumination model, then make the light sources in the first illumination model consistent and give the adjusted first illumination model to establish a restoration direction line. Making the light sources in the first illumination model consistent here means making the intensity of the light sources in the first illumination model consistent. Of course, the influence of the distance parameter also needs to be considered here.

[0104] Then, the enhancement or reduction degree on the restoration direction line is calculated. The calculation method here is based on the linear calculation method. Finally, the enhancement or reduction degree is used to adjust the feature object. The specific adjustment process is to make the intensity of the light source in the first illumination model tend to be consistent. The numerical value here uses the brightness parameter of the transfer position of the fused feature object to the background image.

[0105] The enhancement degree corresponds to a dark light source, and the reduction degree corresponds to a bright light source. The enhancement degree and the reduction degree are a fixed value, that is, the slope of the oblique line. The influence of the light source on the feature object stops when the value on the restoration direction line decreases to zero or when it crosses the edge of the feature object.

[0106] After calculating the enhancement or reduction degree on the restoration direction line, the following contents are added:

[0107] S401, determining the cross-influence area and blank influence area of ​​the restoration direction line;

[0108] S402, determining a restoration direction line associated with the blank impact area;

[0109] S403: Use the associated restoration direction line to create a secondary restoration direction line and adjust the blank influence area based on the secondary restoration direction line.

[0110] In step S401 to step S403, the cross-influence area and the blank influence area of ​​the restored direction line are first determined. The cross-influence area refers to the area affected by at least two light sources, and the blank influence area refers to the area not affected by any light source.

[0111] For the cross-influence area, use the associated light source to process it respectively, while for the blank influence area, it is necessary to use the associated restoration direction line to create a secondary restoration direction line and adjust the blank influence area based on the secondary restoration direction line.

[0112] See also Figure 5 and Figure 6 The specific process is to establish a secondary restoration direction line based on the edge of the cross-affected area, and the secondary restoration direction line is perpendicular to the associated restoration direction line ( Figure 6 The starting parameters of the secondary restoration direction line are the parameters of the position of the starting point on the restoration direction line.

[0113] In some examples, the specific steps of using the second illumination model to process the basic feature object to obtain the fused feature object are as follows:

[0114] S501, determining a brightness parameter of a transfer position of a fused feature object to a background image;

[0115] S502, adjusting the basic feature object using the brightness parameter;

[0116] S503, adjusting the basic feature object again using the second illumination model;

[0117] S504, obtaining an adjusted feature object, and recording the adjusted feature object as a fused feature object.

[0118] In step S501 to step S504, the basic feature object is adjusted twice using the brightness parameter and the second illumination model respectively. The first adjustment is to adjust the entire basic feature object, and the second adjustment is to adjust a part of the basic feature object.

[0119] Through the adjustment in the above manner, the influence of the lighting on the separation image on the basic feature object can be eliminated while the basic feature object is affected by the lighting on the background image.

[0120] The specific method of determining the brightness parameter of the transfer position of the fusion feature object to the background image is:

[0121] S601, sequentially establish multiple annular analysis reference lines inside the feature object and transfer the multiple annular analysis reference lines to the background image, in a direction close to the edge of the feature object, the previous annular analysis reference line is located inside the next annular analysis reference line;

[0122] S602, calculating the pixel value change on the annular analysis reference line and determining the strongest illumination point and the weakest illumination point according to the pixel value change;

[0123] S603: Create brightness parameters using the point with the strongest illumination and the point with the weakest illumination.

[0124] See also Figure 7 In step S601 to step S603, firstly, a plurality of annular analysis reference lines are sequentially established inside the feature object ( Figure 7 The dotted lines in FIG. 2 are used to transfer the multiple annular analysis reference lines to the background image, and then the brightness parameters are obtained by using the multiple annular analysis reference lines established in sequence.

[0125] The specific method is to calculate the change of pixel value on the circular analysis reference line and determine the strongest and weakest lighting points according to the change of pixel value, and then use the strongest and weakest lighting points to create brightness parameters. The value of the strongest lighting point radiates to the surrounding area according to a linear relationship, and the value tends to decrease during the radiating process. The value of the weakest lighting point also radiates to the surrounding area according to a linear relationship, and the value tends to increase during the radiating process.

[0126] When the two meet, the mean processing is performed at the meeting position and then the value decrease speed and value increase speed in the dispersion process are adjusted in reverse.

[0127] In some examples, when using a generative adversarial network to process the fusion domain, it also includes adjusting the width of the fusion domain. The adjustment of the width of the fusion domain occurs under the premise that the processing result of the generative network cannot be recognized by the adversarial network.

[0128] This is because the width of the fusion domain directly limits the processing range of the generated network. A too narrow width will make the change trend in the fusion domain steep. Therefore, the width of the fusion domain needs to be appropriately increased. Conversely, when the processing results of the generated network are recognized by the adversarial network, the width of the fusion domain needs to be reduced.

[0129] The present application also provides an AI-based content generation device, comprising:

[0130] A first processing unit is used to obtain a separated image and a background image and separate a feature object from the separated image;

[0131] An illumination model establishing unit, used to establish a first illumination model belonging to the separated image and a second illumination model belonging to the background image;

[0132] A second processing unit is used to restore the feature object using the first illumination model to obtain a basic feature object;

[0133] A third processing unit is used to process the basic feature object using the second illumination model to obtain a fused feature object;

[0134] a fourth processing unit, configured to transfer the fused feature object to the background image and divide a fusion domain at the peripheral edge of the fused feature object;

[0135] The fusion processing unit is used to process the fusion domain using a generative adversarial network to fuse the fusion feature object with the background image.

[0136] Furthermore, it also includes:

[0137] A region selection unit, used to select a plurality of regions on the feature object after grayscale processing of the feature object, each region including at least one pixel point;

[0138] A datum reference line establishing unit, used for establishing a datum reference line using an area;

[0139] A reference line rotation unit, used to rotate the reference line and calculate the pixel value change trend on the reference line, so that at least two reference lines with the same pixel value change trend are parallel or intersecting;

[0140] The first illumination model establishing unit is used to establish a first illumination model using the rotated baseline reference line.

[0141] Furthermore, it also includes:

[0142] A first illumination model adjustment unit, used for adjusting the first illumination model so that the light sources in the first illumination model tend to be consistent and establishing a restoration direction line for the adjusted first illumination model;

[0143] A first calculation unit is used to calculate the enhancement degree or reduction degree on the restoration direction line;

[0144] The first adjustment unit is used to adjust the feature object using an enhancement degree or a reduction degree to obtain a basic feature object.

[0145] Furthermore, it also includes:

[0146] A first area determination unit, used for determining a cross-influence area and a blank influence area of ​​the restoration direction line;

[0147] A second area determination unit, used to determine a restoration direction line associated with the blank influence area;

[0148] The second adjustment unit is used to create a secondary restoration direction line using the associated restoration direction line and adjust the blank influence area based on the secondary restoration direction line.

[0149] Furthermore, it also includes:

[0150] A parameter determination unit, used to determine a brightness parameter of a transfer position of the fused feature object to the background image;

[0151] A third adjustment unit, configured to adjust the basic feature object using a brightness parameter;

[0152] A fourth adjustment unit, configured to adjust the basic feature object again using the second illumination model;

[0153] The obtaining unit is used to obtain an adjusted feature object, and the adjusted feature object is recorded as a fused feature object.

[0154] Furthermore, it also includes:

[0155] A first processing unit is used to sequentially establish a plurality of annular analysis reference lines inside the feature object and transfer the plurality of annular analysis reference lines to the background image, wherein a previous annular analysis reference line is located inside a next annular analysis reference line in a direction close to an edge of the feature object;

[0156] A second processing unit is used to calculate the pixel value change on the annular analysis reference line and determine the strongest illumination point and the weakest illumination point according to the pixel value change;

[0157] The brightness parameter creation unit is used to create the brightness parameter using the strongest lighting point and the weakest lighting point.

[0158] Furthermore, when the fusion domain is processed using a generative adversarial network, the width of the fusion domain is also adjusted.

[0159] In one example, the unit in any of the above devices can be one or more integrated circuits configured to implement the above methods, such as: one or more application specific integrated circuits (ASICs), or, one or more digital signal processors (DSPs), or, one or more field programmable gate arrays (FPGAs), or a combination of at least two of these integrated circuit forms.

[0160] For another example, when the units in the device can be implemented in the form of a processing element scheduling program, the processing element can be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call a program. For another example, these units can be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0161] Various objects such as various messages / information / equipment / network elements / systems / devices / actions / operations / processes / concepts that may appear in this application are named. It can be understood that these specific names do not constitute a limitation on the relevant objects. The names assigned may change with factors such as scenarios, contexts or usage habits. The understanding of the technical meaning of the technical terms in this application should be mainly determined from the functions and technical effects embodied / executed in the technical scheme.

[0162] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0163] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0164] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0165] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0166] It should also be understood that in various embodiments of the present application, the first, second, etc. are only used to indicate that multiple objects are different. For example, the first time window and the second time window are only used to indicate different time windows. They should not have any impact on the time window itself, and the first, second, etc. mentioned above should not impose any limitations on the embodiments of the present application.

[0167] It should also be understood that in the various embodiments of the present application, unless otherwise specified or there is a logical conflict, the terms and / or descriptions between different embodiments are consistent and can be referenced to each other, and the technical features in different embodiments can be combined to form new embodiments according to their internal logical relationships.

[0168] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a computer-readable storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned computer-readable storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0169] The present application also provides a content generation system, the system comprising:

[0170] one or more memories for storing instructions; and

[0171] One or more processors are used to call and run the instructions from the memory to execute the method as described above.

[0172] The present application also provides a computer program product, which includes instructions. When the instructions are executed, the terminal device and the network device perform operations of the terminal device and the network device corresponding to the above method.

[0173] The present application also provides a chip system, which includes a processor for implementing the functions involved in the above content, such as generating, receiving, sending, or processing the data and / or information involved in the above method.

[0174] The chip system may be composed of chips, or may include chips and other discrete devices.

[0175] The processor mentioned in any of the above places can be a CPU, a microprocessor, an ASIC, or one or more integrated circuits for executing programs for controlling the above-mentioned feedback information transmission method.

[0176] In a possible design, the chip system also includes a memory, which is used to store necessary program instructions and data. The processor and the memory can be decoupled and respectively set on different devices, connected by wire or wireless means to support the chip system to implement various functions in the above embodiments. Alternatively, the processor and the memory can also be coupled on the same device.

[0177] Optionally, the computer instructions are stored in a memory.

[0178] Optionally, the memory is a storage unit within the chip, such as a register, a cache, etc. The memory can also be a storage unit within the terminal located outside the chip, such as a ROM or other types of static storage devices that can store static information and instructions, RAM, etc.

[0179] It can be understood that the memory in the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories.

[0180] The non-volatile memory may be a ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), or a flash memory.

[0181] The volatile memory may be a RAM, which is used as an external cache. There are many different types of RAM, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct memory bus random access memory.

[0182] The embodiments of this specific implementation method are all preferred embodiments of the present application, and are not intended to limit the protection scope of the present application. Therefore, all equivalent changes made based on the structure, shape, and principle of the present application should be included in the protection scope of the present application.

Claims

1. A content generation method based on AI, characterized in that: include: Acquire a separation image and a background image and separate a feature object from the separation image; Establishing a first illumination model belonging to the separated image and a second illumination model belonging to the background image; Use the first illumination model to restore the feature object to obtain a basic feature object; Process the basic feature object using the second illumination model to obtain a fused feature object; Transferring the fused feature object to the background image and dividing the fusion domain at the surrounding edge of the fused feature object; The fusion domain is processed using a generative adversarial network to fuse the fusion feature object with the background image.

2. The AI-based content generation method according to claim 1, characterized in that: Establishing a first illumination model belonging to the separated image includes: After grayscale processing is performed on the feature object, multiple regions are selected on the feature object, each region including at least one pixel point; Use regions to establish datum reference lines; Rotate the reference line and calculate the pixel value change trend on the reference line, so that at least two reference lines with the same pixel value change trend are parallel or intersecting; A first lighting model is established using the rotated base reference line.

3. The AI-based content generation method according to claim 2, characterized in that: The first illumination model is used to restore the feature object, and the basic feature object includes: Adjusting the first illumination model so that the light sources in the first illumination model tend to be consistent and establishing a restoration direction line for the adjusted first illumination model; Calculate the enhancement or reduction degree on the restoration direction line; The feature object is adjusted using the enhancement or reduction degree to obtain the basic feature object.

4. The AI-based content generation method according to claim 3, characterized in that: After calculating the enhancement or reduction degree on the restoration direction line, the following is also included: Determine the cross-influence area and blank influence area of ​​the restoration direction line; determining a restoration direction line associated with the blank impact area; Use the associated restoring direction line to create a secondary restoring direction line and adjust the blank influence area based on the secondary restoring direction line.

5. The AI-based content generation method according to claim 1, characterized in that: The second illumination model is used to process the basic feature objects, and the fused feature objects obtained include: Determine the brightness parameter of the transfer position of the fused feature object to the background image; Adjust the underlying feature object using the brightness parameter; Adjust the base feature object again using the second lighting model; An adjusted feature object is obtained, and the adjusted feature object is recorded as a fused feature object.

6. The AI-based content generation method according to claim 5, characterized in that: The brightness parameters that determine the position where the fused feature object is transferred to the background image include: Sequentially establishing multiple annular analysis reference lines inside the feature object and transferring the multiple annular analysis reference lines to the background image, in a direction close to the edge of the feature object, a previous annular analysis reference line is located inside a next annular analysis reference line; Calculate the pixel value changes on the annular analysis reference line and determine the strongest and weakest illumination points according to the pixel value changes; Use the brightest and weakest points to create brightness parameters.

7. The AI-based content generation method according to claim 1, characterized in that: When using a generative adversarial network to process the fusion domain, it also includes adjusting the width of the fusion domain.

8. An AI-based content generation device, characterized in that: include: A first processing unit is used to obtain a separated image and a background image and separate a feature object from the separated image; An illumination model establishing unit, used to establish a first illumination model belonging to the separated image and a second illumination model belonging to the background image; A second processing unit is used to restore the feature object using the first illumination model to obtain a basic feature object; A third processing unit is used to process the basic feature object using the second illumination model to obtain a fused feature object; a fourth processing unit, configured to transfer the fused feature object to the background image and divide a fusion domain at the peripheral edge of the fused feature object; The fusion processing unit is used to process the fusion domain using a generative adversarial network to fuse the fusion feature object with the background image.

9. A content generation system, characterized in that: The system comprises: one or more memories for storing instructions; and One or more processors, configured to call and execute the instructions from the memory to perform the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer readable storage medium comprises: The program, when the program is executed by a processor, the method according to any one of claims 1 to 7 is executed.

Images