Content generation method and apparatus, computer device, and storage medium

Abstract

The present disclosure provides a content generation method and device, computer equipment and a storage medium, wherein the method comprises: obtaining a first reference material matched with a content theme; performing feature extraction on the first reference material to generate target feature data; processing the target feature data using determined target semantic information of the content theme to generate a target material matched with the target semantic information and a material style indicated by the first reference material; and generating to-be-delivered data corresponding to the content theme based on the target material.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and more specifically, to a content generation method, apparatus, computer device, and storage medium. Background Technology

[0002] Currently, in the field of content creation, content creation still heavily relies on designers. For example, designers can manually create content that meets business needs and is innovative and aesthetically pleasing. However, creating content through manual design requires high levels of professional skill from designers, incurs high time and financial costs, and has low efficiency in content generation. Summary of the Invention

[0003] This disclosure provides at least one content generation method, apparatus, computer device, and storage medium.

[0004] In a first aspect, embodiments of this disclosure provide a content generation method, the method comprising:

[0005] Obtain primary reference material that matches the content theme;

[0006] Feature extraction is performed on the first reference material to generate target feature data;

[0007] Using the target semantic information of the determined content theme, the target feature data is processed to generate target material that matches the target semantics and the material style indicated by the first reference material;

[0008] Based on the target material, generate the data to be delivered corresponding to the content theme.

[0009] In one possible implementation, the target semantic information of the content topic is determined according to the following steps:

[0010] Obtain a second reference material containing thematic elements related to the aforementioned content theme;

[0011] The second reference material is parsed to determine the semantic information included in the second reference material;

[0012] Based on the semantic information included in the second reference material, the target semantic information of the content topic is determined.

[0013] In one possible implementation, obtaining second reference material containing thematic elements related to the content theme includes:

[0014] Obtain multiple candidate reference materials that match the content theme, and a delivery performance indicator for each candidate reference material; the delivery performance indicator is determined based on the interaction data corresponding to the candidate reference material after it is delivered.

[0015] Based on the delivery performance metrics corresponding to each of the candidate reference materials, a second reference material is determined from the plurality of candidate reference materials.

[0016] In one possible implementation, when the second reference material comprises multiple source images, the step of parsing the second reference material to determine the semantic information included in the second reference material includes:

[0017] Each image in the second reference material is analyzed to determine the semantic information corresponding to each image.

[0018] The semantic information corresponding to each of the material images is integrated to generate the semantic information included in the second reference material.

[0019] In one possible implementation, determining the target semantic information of the content topic based on the semantic information included in the second reference material includes:

[0020] Based on the content theme, the semantic information included in the second reference material is expanded to generate updated semantic information;

[0021] Based on the updated semantic information, the target semantic information of the content topic is determined.

[0022] In one possible implementation, determining the target semantic information of the content topic based on the semantic information included in the second reference material includes:

[0023] In response to the triggered adjustment operation, the semantic information included in the second reference material is preprocessed to generate processed semantic information; the preprocessing includes one of the following: filtering, deduplication, and addition;

[0024] Based on the processed semantic information, the target semantic information of the content topic is determined.

[0025] In one possible implementation, the target semantic information of the content topic is determined according to the following steps:

[0026] Based on the content theme and the set initial semantic template, initial semantic information is generated; the initial semantic template includes multiple structured parameters, and the initial semantic information includes parameter information that matches the content theme for the multiple structured parameters.

[0027] Based on the initial semantic information, the target semantic information of the content topic is determined.

[0028] In one possible implementation, generating the content-to-be-delivered data corresponding to the content theme based on the target material includes:

[0029] Obtain multimedia information matching the content theme; the multimedia information includes at least one of the following: audio information, image information, text information, and video information;

[0030] The multimedia information is fused with the target material to generate the data to be delivered corresponding to the content theme.

[0031] In one possible implementation, the target material is generated using a target diffusion model, and the method further includes:

[0032] Based on the historical training samples of the trained diffusion model, new semantics not included in the historical training samples are determined from the target semantic information of the content topic.

[0033] Obtain new training samples that match the new semantics;

[0034] Using the newly added training samples, the model parameters of the trained diffusion model are adjusted to generate an adjusted diffusion model.

[0035] If the adjusted diffusion model meets the training cutoff condition, the adjusted diffusion model will be determined as the target diffusion model for content topic matching.

[0036] Secondly, embodiments of this disclosure also provide a content generation apparatus, comprising:

[0037] The acquisition module is used to acquire the first reference material that matches the content theme;

[0038] The feature extraction module is used to extract features from the first reference material and generate target feature data;

[0039] The first generation module is used to process the target feature data using the target semantic information of the determined content theme, and generate target material that matches the target semantics and the material style indicated by the first reference material;

[0040] The second generation module is used to generate the content-to-be-delivered data corresponding to the content theme based on the target material.

[0041] Thirdly, embodiments of this disclosure also provide a computer device, including: a processor, a memory, and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processor communicates with the memory via the bus, and when the machine-readable instructions are executed by the processor, the steps of the first aspect above, or any possible implementation of the first aspect, are performed.

[0042] Fourthly, embodiments of this disclosure also provide a computer-readable storage medium storing a computer program that, when executed by a processor, performs the steps of the first aspect or any possible implementation of the first aspect.

[0043] The content generation method, apparatus, computer equipment, and storage medium provided in this disclosure extract features from a first reference material that matches the content theme, generating target feature data that characterizes the style of the first reference material. Then, using the target semantic information of the determined content theme, the target feature data is processed to generate target material that matches the target semantics and the style indicated by the first reference material. This eliminates the need for manual intervention in the material generation process, ensuring that the generated target material conforms to the target semantics and achieving automatic material generation, thus improving efficiency. Furthermore, the first reference material matches the content theme, its style is adapted to the content theme, and the determined target semantic information provides material content that meets the content theme requirements. Therefore, the content generation method proposed in this disclosure can control the style and content of the target material according to the needs of the content theme, generating target material that conforms to the content theme. This allows the method to flexibly adapt to various content themes, improving the accuracy and flexibility of the generated target material. It can also generate high-quality content theme-specific data for deployment based on the target material, thus meeting the needs of business scenarios.

[0044] To make the above-mentioned objects, features and advantages of this disclosure more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0045] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings used in the embodiments will be briefly described below. These drawings are incorporated in and constitute a part of this specification. They illustrate embodiments conforming to this disclosure and, together with the specification, serve to explain the technical solutions of this disclosure. It should be understood that the following drawings only show some embodiments of this disclosure and should not be considered as limiting the scope. Those skilled in the art can obtain other related drawings based on these drawings without creative effort.

[0046] Figure 1 A schematic diagram of a background material with an uncontrollable material style provided in an embodiment of this disclosure is shown;

[0047] Figure 2 A flowchart of a content generation method provided by an embodiment of this disclosure is shown;

[0048] Figure 3a This illustration shows a schematic diagram of a first reference material in a content generation method provided by an embodiment of the present disclosure;

[0049] Figure 3b This illustration shows a schematic diagram of the target material in a content generation method provided by an embodiment of the present disclosure;

[0050] Figure 4 The flowchart shown is a method for training a target diffusion model in a content generation method provided by an embodiment of this disclosure;

[0051] Figure 5 A schematic diagram of a content generation apparatus provided in an embodiment of this disclosure is shown;

[0052] Figure 6 A schematic diagram of the structure of a computer device provided in an embodiment of this disclosure is shown. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. The components of the embodiments of this disclosure described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this disclosure provided in the accompanying drawings is not intended to limit the scope of the claimed disclosure, but merely represents selected embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without inventive effort are within the scope of protection of this disclosure.

[0054] Currently, in the field of content creation, content creation still heavily relies on designers. Designers can manually create content that meets business needs and is both innovative and aesthetically pleasing. However, creating content through manual design requires a high level of professional skill from designers and incurs significant time and financial costs.

[0055] Generally, models can be used to generate materials to reduce generation time and increase the quantity of materials generated. In related technologies, semantic descriptions are input into a diffusion model to generate materials. However, the style of materials generated using these methods is difficult to control, making it difficult to meet business needs and resulting in low efficiency in material generation. For example, if the input semantic information is "a photo of background design," the output background material might look like this: Figure 1 The three background images shown have a depressing color tone and lack any sense of design in their texture.

[0056] To alleviate the above problems, this disclosure provides a content generation method, apparatus, computer equipment, and storage medium.

[0057] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0058] In this document, the term "and / or" merely describes a relationship, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Furthermore, the term "at least one" in this document means any combination of at least two of any one or more elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0059] To facilitate understanding of this embodiment, a content generation method disclosed in this disclosure will first be described in detail. The execution subject of the content generation method provided in this disclosure is generally a computer device with certain computing capabilities, such as a terminal device or a server. In some possible implementations, the content generation method can be implemented by a processor calling computer-readable instructions stored in memory.

[0060] The following describes the content generation method provided in this disclosure embodiment, taking the server as the execution subject as an example.

[0061] See Figure 2 The diagram shows a flowchart of a content generation method provided in this embodiment of the disclosure. The method includes steps S201-S204, wherein:

[0062] S201. Obtain the first reference material that matches the content theme.

[0063] Here, the first reference material can be any material obtained from a material library that matches the content theme, or it can be open-source material searched by the user that matches the content theme; this embodiment of the disclosure does not limit the source of the first reference material. The style of the first reference material can be determined according to the needs of the business scenario, that is, the first reference material matches the content theme, that is, the first reference material can express the style of the material that the user expects to generate; for example, if the content theme is ink painting style, the first reference material can be ink painting, and the colors can include black and white, etc.; if the content theme is Mid-Autumn Festival, the first reference material can be an image of a night containing the moon, etc.

[0064] S202. Extract features from the first reference material to generate target feature data.

[0065] In implementation, after obtaining the first reference material, it can be input into the target diffusion model. The target diffusion model then extracts features from the first reference material to generate target feature data. For example, at least one convolutional layer can be used to extract features from the first reference material to generate target feature data; alternatively, the first reference material can be subjected to at least one noise addition process to generate noisy material, which serves as the target feature data. The generated target feature data can be used to characterize the style of the first reference material. Here, the target diffusion model can be an acquired, trained diffusion model, or it can be a target diffusion model obtained by updating an acquired, trained diffusion model.

[0066] S203. Using the target semantic information of the determined content theme, process the target feature data to generate target material that matches the target semantics and the material style indicated by the first reference material.

[0067] Furthermore, the target semantic information of the determined content theme can be used to process the target feature data to generate target materials that match the target semantics and the material style indicated by the first reference material. Among them, the target semantic information can be used to represent the material content that meets the needs of the business scenario, that is, the target semantic information of the content theme. For example, if the target material is used to display the Mid-Autumn Festival scene, the target semantic information can include the moon, mooncakes, Mid-Autumn Festival, etc.

[0068] For example, target semantic information can be input into a target diffusion model, and noise processing can be performed on the target semantic information using the target diffusion model to obtain semantic noise data. The semantic noise data can be fused with target feature data, such as by adding the semantic noise data to the target feature data, that is, by adding the noise matrix of the semantic noise data to the feature matrix of the target feature data to obtain fused feature data. The fused feature data can be subjected to at least one noise reduction process to generate target material. The semantic information of the target material matches the target semantic information included in the target semantic information of the content theme, and the material style of the target material matches the material style indicated by the first reference material.

[0069] In practice, the style of the materials can include at least one of the following: material color, material layout, and material texture. For example, if the material color of the first reference material is pink, then the color of the target material can also be pink; or, if the material layout of the first reference material is: there is material content on the left and blank on the right, then the material layout of the target material can also be: there is material content on the left that is semantically related to the content theme, and blank on the right.

[0070] For example, the target semantic information input is various cosmetics, and the first reference material input is such as... Figure 3a As shown, the content is positioned on the left side of the image, with a green background. Therefore, using the content generation method provided in this embodiment, the generated target material can be as follows: Figure 3b The images shown are arranged on the left side, and the images feature various cosmetics against a green background. Figure 3a and Figure 3b The background color is not displayed.

[0071] S204. Based on the target materials, generate the data to be delivered for the content theme.

[0072] After generating target materials that are semantically matched with the content theme and style that suits the content theme, data to be deployed corresponding to the content theme can be generated based on the target materials. Here, the data to be deployed is determined according to the actual application scenario. For example, if the data to be deployed is used for illustration applications, then the data to be deployed can be image information; if the data to be deployed is used for video display, then the data to be deployed can be video information.

[0073] In practice, based on the target material, data to be delivered corresponding to the content theme is generated, which may include: obtaining multimedia information matching the content theme; the multimedia information includes at least one of the following: audio information, image information, text information, and video information; and integrating the multimedia information with the target material to generate data to be delivered corresponding to the content theme.

[0074] During implementation, multimedia information matching the content theme can be obtained. For example, multimedia information may include at least one of audio information, image information, text information, and video information. Then, the multimedia information can be fused with the target material to generate data to be delivered corresponding to the content theme. For example, when the multimedia information includes video information, the target material can be fused with the video information, that is, the target material can be used as a video frame in the video information, or the target material can replace any video frame in the video information to generate data to be delivered corresponding to the content theme.

[0075] The above describes the content generation methods provided for S201 to S204. By extracting features from the first reference material that matches the content theme, target feature data representing the style of the first reference material is generated. Then, using the target semantic information of the determined content theme, the target feature data is processed to generate target materials that match the target semantics and the style indicated by the first reference material. This eliminates the need for manual intervention in the material generation process, ensuring that the generated target materials conform to the target semantics and achieving automatic material generation, thus improving efficiency. Furthermore, the first reference material matches the content theme, its style is adapted to the content theme, and the determined target semantic information provides material content that meets the content theme requirements. Therefore, the content generation method proposed in this disclosure can control the style and content of the target material according to the needs of the content theme, generating target materials that conform to the content theme. This allows the method to flexibly adapt to various content themes, improving the accuracy and flexibility of the generated target materials. It can also generate high-quality content theme-related data for deployment based on the target materials, thus meeting the needs of business scenarios.

[0076] The steps S201 to S204 described above will be explained in detail below based on specific embodiments.

[0077] In one optional implementation, for S203, the target semantic information of the content topic can be determined in the following two ways.

[0078] In a first alternative implementation, the target semantic information of the content topic can be determined according to the following steps:

[0079] Step A1: Obtain a second reference material containing thematic elements related to the content theme.

[0080] Step A2: Analyze the second reference material to determine the semantic information included in the second reference material.

[0081] Step A3: Based on the semantic information included in the second reference material, determine the target semantic information of the content topic.

[0082] Here, the second reference material can be the same as or different from the first reference material. For example, when the second reference material is the same as the first reference material, by parsing the acquired second reference material, the semantic information included in the second reference material can be determined, and based on the semantic information included in the second reference material, the target semantic information can be determined; this allows the second reference material to provide both material style and semantic information, thus improving the utilization rate of the material.

[0083] During implementation, second reference materials can be obtained. These second reference materials may contain thematic elements related to the content theme, meaning the content of the second reference materials matches the content of the target material expected to be generated by the user. For example, if the content theme is the Mid-Autumn Festival, thematic elements related to the content theme may include the moon, mooncakes, etc.; or if the content theme is cosmetics, thematic elements related to the content theme may include various types of cosmetics, makeup mirrors, facial images containing makeup styles, etc.; or if the content theme is music, thematic elements related to the content theme may include musical notes, musical instruments, etc. Here, the second reference materials can be obtained from a material library or open-source materials obtained by the user through search; however, this embodiment of the disclosure does not limit the source of the second reference materials.

[0084] Furthermore, the second reference material can be analyzed to determine its semantic information. For example, the second reference material can be input into a target diffusion model, and the Contrastive Language-Image Pre-Training (CLIP) module included in the model can be used to encode the material, obtaining a feature vector. Based on the feature vector and a preset text feature vector, at least one target text feature vector corresponding to the second reference material can be determined. For instance, the cosine similarity between the feature vector and the preset text feature vector can be calculated, and text feature vectors with a cosine similarity greater than a similarity threshold can be identified as the target text feature vectors corresponding to the second reference material. Furthermore, based on at least one target text feature vector, the semantic information included in the second reference material can be determined. Finally, the target semantic information can be determined based on the semantic information included in the second reference material. For example, the semantic information included in the second reference material can be identified as the target semantic information; or, a portion of the semantic information included in the second reference material can be selected as the target semantic information.

[0085] Here, by analyzing the second reference material containing thematic elements related to the content theme, the semantic information included in the second reference material is determined, and the target semantic information is determined based on the semantic information included in the second reference material. This can alleviate the tedious problem caused by manually writing semantic information, improve the efficiency of determining the target semantic information, and reduce labor and time costs.

[0086] In a specific implementation, for step A1, obtaining the second reference material containing thematic elements related to the content theme may include:

[0087] Step A11: Obtain multiple candidate reference materials that match the content theme, and the delivery performance index of each candidate reference material; the delivery performance index is determined based on the interaction data corresponding to the candidate reference material after it is delivered.

[0088] Step A12: Based on the delivery performance metrics corresponding to each of the candidate reference materials, determine the second reference material from the plurality of candidate reference materials.

[0089] During implementation, multiple candidate reference materials matching the content theme and the performance metrics for each candidate reference material can be obtained. Performance metrics can be determined based on the interaction data corresponding to the candidate reference materials after they are deployed; for example, usage, readership, and viewership can be used to determine performance metrics. Furthermore, a second reference material can be determined from multiple candidate reference materials based on their respective performance metrics. Specifically, candidate reference materials whose performance metrics exceed a threshold can be designated as second reference materials. Alternatively, based on the material type of each candidate reference material, at least one candidate reference material matching each material type can be identified. Then, for each material type, the candidate reference type with the highest performance metric among the at least one candidate reference material matching that material type can be designated as the second reference type.

[0090] Here, the performance metrics are determined based on the interaction data corresponding to the candidate reference materials after they are delivered. In other words, the performance metrics can characterize the degree of user interest in the candidate reference materials. Therefore, based on multiple candidate reference materials that match the content theme and the performance metrics of each candidate reference material, the second reference material determined from multiple candidate reference materials can better meet user needs, resulting in better quality target materials generated based on the second reference material.

[0091] In practice, the second reference material may include multiple material images, and the semantic information included in the second reference material may be determined based on the semantic information included in each material image.

[0092] In one possible implementation, regarding step A2, when the second reference material contains multiple source images, parsing the second reference material to determine the semantic information included in the second reference material may include:

[0093] Step A21: Analyze each image in the second reference material to determine the semantic information corresponding to each image.

[0094] Step A22: Integrate the semantic information corresponding to each of the material images to generate the semantic information included in the second reference material.

[0095] When the second reference material contains multiple images, each image can be parsed to determine its corresponding semantic information, thus identifying the semantic information for each image. The process of determining the semantic information for each image can be found in step A2, and will not be repeated here. Furthermore, after obtaining the semantic information for each image, this information can be integrated. For example, the semantic information can be concatenated, adjusted, or combined to generate the semantic information included in the second reference material.

[0096] Here, when the second reference material contains multiple material images, the semantic information corresponding to each material image is integrated, and the generated second reference material contains richer semantic information, so that the target semantic information of the content theme can be generated more accurately based on the rich semantic information.

[0097] In specific implementation, for step A3, the determination of the target semantic information of the content topic based on the semantic information included in the second reference material can be determined in the following two ways.

[0098] In the first approach, determining the target semantic information based on the semantic information included in the second reference material may include:

[0099] Step B1: Based on the content theme, expand the semantic information included in the second reference material to generate updated semantic information.

[0100] Step B2: Based on the updated semantic information, determine the target semantic information of the content topic.

[0101] In implementation, after determining the semantic information included in the second reference material, the semantic information can be expanded to generate updated semantic information. For example, the semantic information included in the second reference material can be input into the target diffusion model, and the expansion function module included in the target diffusion model can be used to expand the semantic information included in the second reference material. For instance, if the semantic information is "a picture of cosmetics and cosmetics products on a table, an airbrush painting," the updated semantic information could be "a picture of cosmetics and cosmetics products on a table, an airbrush painting, inspired by Zhang Shunzi, pork meat, product introduction photos, screengrab, with merchant logo, half-closed eyes, faint red lips, the walls are pink, white waist apron and undershirt." Furthermore, based on the updated semantic information, target semantic information can be determined. For example, the updated semantic information can be determined as the target semantic information of the content theme, or the words or phrases contained in the updated semantic information can be deduplicated to generate the target semantic information of the content theme.

[0102] Here, by expanding the semantic information included in the second reference material, updated semantic information can be generated. The updated semantic information is richer in content. Therefore, when determining the target semantic information based on the updated semantic information, the determined target semantic information can be richer and more diverse, so that the target material can be generated more accurately based on the rich target semantic information.

[0103] In the second approach, determining the target semantic information of the content topic based on the semantic information included in the second reference material may further include:

[0104] Step C1: In response to the triggered adjustment operation, preprocess the semantic information included in the second reference material to generate processed semantic information; the preprocessing includes at least one of the following: filtering, deduplication, and addition.

[0105] Step C2: Based on the processed semantic information, determine the target semantic information of the content topic.

[0106] In implementation, after determining the semantic information included in the second reference material, it can also be manually adjusted. That is, in response to the triggered adjustment operation, the semantic information included in the second reference material can be preprocessed. For example, at least one of the following can be performed: filtering, deduplication, and addition processing, to generate processed semantic information. For example, if the second reference material includes various cosmetics, the semantic information determined by the CLIP module can be "apicture of cosmetics and cosmetics products on a table, an airbrush painting". After addition processing, the generated processed semantic information can be "a picture of cosmetics and cosmetics products on a table, product introduction photos, screengrab, with merchant logo, half-closed eyes, faint red lips, the background is pink". Furthermore, based on the processed semantic information, the target semantic information of the content theme can be determined. For example, the processed semantic information can be determined as the target semantic information of the content theme. Alternatively, the words or phrases contained in the processed semantic information can be adjusted, for example, the words or phrases contained in the processed semantic information can be adjusted to words or phrases that are more consistent with the content theme, thereby generating the target semantic information of the content theme.

[0107] Here, by responding to the triggered adjustment operation, the semantic information included in the second reference material is preprocessed, which makes the generated processed semantic information more accurate and more suitable for the content theme. Therefore, based on the processed semantic information, the determined target semantic information can better meet the needs of the business scenario.

[0108] In specific implementation, after determining the semantic information included in the second reference material, the semantic information can be expanded using the expansion function module to generate updated semantic information. Then, in response to manually triggered adjustment operations, the updated semantic information can be preprocessed to generate processed semantic information. Further, based on the processed semantic information, the target semantic information of the content theme can be determined. Alternatively, after determining the semantic information included in the second reference material, the semantic information can also be preprocessed in response to manually triggered adjustment operations to generate processed semantic information. Then, the processed semantic information can be expanded using the expansion function module to generate updated semantic information. Finally, based on the updated semantic information, the target semantic information of the content theme can be determined, making the target semantic information generated according to the above process richer and more accurate.

[0109] In a second optional implementation, the target semantic information of the content topic can also be determined according to the following steps: generating initial semantic information based on the content topic and the set initial semantic template; the initial semantic template includes multiple structured parameters, and the initial semantic information includes parameter information that matches the content topic for the multiple structured parameters; and determining the target semantic information of the content topic based on the initial semantic information.

[0110] In practice, initial semantic information can be generated using the content theme and a set initial semantic template. This initial semantic template can include various structured parameters, such as adjective parameters, noun parameters, etc. Specifically, parameter information matching the content theme can be input into the initial semantic template to generate initial semantic information. Further, this initial semantic information can be determined as the target semantic information for the content theme. Alternatively, the initial semantic information can be input into a target diffusion model, and the expansion function module included in the target diffusion model can be used to expand the initial semantic information to generate the target semantic information for the content theme, and so on.

[0111] In the above method, by inputting parameter information that matches the content theme into the set initial semantic template, initial semantic information can be generated quickly and accurately. Furthermore, based on the initial semantic information, the determined target semantic information can be better adapted to the content theme, so that target materials can be generated more accurately based on the target semantic information that is better adapted to the content theme.

[0112] In one possible implementation, the target material can be generated using a target diffusion model, wherein the process of determining the target diffusion model can be found in [reference needed]. Figure 4 As shown:

[0113] S401. Based on the historical training samples of the trained diffusion model, determine new semantics not included in the historical training samples from the target semantic information of the content topic.

[0114] S402. Obtain new training samples that match new semantics.

[0115] S403. Using the newly added training samples, adjust the model parameters of the trained diffusion model to generate the adjusted diffusion model.

[0116] S404. If the adjusted diffusion model meets the training cutoff condition, the adjusted diffusion model shall be determined as the target diffusion model for content topic matching.

[0117] In implementation, the model parameters of the trained diffusion model can be adjusted based on the target semantic information of the content theme. This allows the adjusted diffusion model to generate target materials that match the content theme, resulting in better performance. For example, the historical training samples of the trained diffusion model contain semantic information of various landscapes, such as the sea, grassland, and mountains. However, the content theme is animals, and the target semantic information of the content theme includes various animals. In this case, the trained diffusion model cannot accurately generate animal-related materials. Therefore, new training samples matching the new semantics can be obtained, i.e., samples containing the new semantic information can be acquired. Semantic information can be new training samples of various animals, and these new training samples can be used to adjust the model parameters of the already trained diffusion model. For example, the historical training samples of the already trained diffusion model contain semantic information of various cartoon characters, such as Doraemon, Pikachu, and Mickey Mouse. The content theme is based on the newly appeared cartoon characters Xiong Da and Xiong Er, and the target semantic information of the content theme includes Xiong Da and Xiong Er. Therefore, new training samples that match the new semantics can be obtained, that is, new training samples containing the semantic information of Xiong Da and Xiong Er can be obtained, and the model parameters of the already trained diffusion model can be adjusted using these new training samples.

[0118] In specific implementation, new semantics not included in the historical training samples of the trained diffusion model can be determined from the target semantic information of the content topic. New training samples matching these new semantics can be obtained, and the model parameters of the trained diffusion model can be adjusted using these new training samples to generate an adjusted diffusion model. Then, if the adjusted diffusion model meets the training cutoff conditions, it can be determined as the target diffusion model for content topic matching. Specifically, when adjusting the model parameters of the trained diffusion model, loss values ​​can be generated, such as mean squared error loss, mean absolute error loss, and cross-entropy loss. Based on these loss values, the model parameters of the trained diffusion model can be adjusted until the training cutoff conditions are met, resulting in the target diffusion model for content topic matching. For example, training cutoff conditions include, but are not limited to, training times greater than or equal to a threshold, and loss function convergence. Here, the trained diffusion model can be a fully trained and converged diffusion model. The model structure of the diffusion model can be set as needed, and this embodiment does not impose specific limitations.

[0119] Here, by identifying new semantics not included in the historical training samples of the trained diffusion model from the target semantic information of the content topic, and then using newly added training samples matching these new semantics to retrain the already trained diffusion model, the resulting target diffusion model can more accurately generate target materials that match the target semantic information, thus ensuring a better match between the target materials and the content topic. Simultaneously, by adjusting the model parameters of the already trained diffusion model using newly added training samples matching new semantics, the target diffusion model can be flexibly adapted to various business scenarios.

[0120] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0121] Based on the same inventive concept, this disclosure also provides a content generation device corresponding to the content generation method. Since the principle of the device in this disclosure for solving the problem is similar to the content generation method described above in this disclosure, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.

[0122] Reference Figure 5 The diagram shown is an architectural schematic of a content generation device provided in an embodiment of this disclosure. The device includes: an acquisition module 501, a feature extraction module 502, a first generation module 503, and a second generation module 504; wherein,

[0123] Module 501 is used to obtain the first reference material that matches the content theme;

[0124] Feature extraction module 502 is used to extract features from the first reference material and generate target feature data;

[0125] The first generation module 503 is used to process the target feature data using the target semantic information of the determined content theme, and generate target material that matches the target semantics and the material style indicated by the first reference material;

[0126] The second generation module 504 is used to generate data to be delivered corresponding to the content theme based on the target material.

[0127] In an optional embodiment, the apparatus further includes: a determining module 505; the determining module 505 is configured to determine the target semantic information of the content topic according to the following steps:

[0128] Obtain a second reference material containing thematic elements related to the aforementioned content theme;

[0129] The second reference material is parsed to determine the semantic information included in the second reference material;

[0130] Based on the semantic information included in the second reference material, the target semantic information of the content topic is determined.

[0131] In an optional implementation, the determining module 505, when acquiring second reference material containing thematic elements related to the content theme, is used to:

[0132] Obtain multiple candidate reference materials that match the content theme, and a delivery performance indicator for each candidate reference material; the delivery performance indicator is determined based on the interaction data corresponding to the candidate reference material after it is delivered.

[0133] Based on the delivery performance metrics corresponding to each of the candidate reference materials, a second reference material is determined from the plurality of candidate reference materials.

[0134] In an optional implementation, when the second reference material contains multiple source images, the determining module 505, when parsing the second reference material and determining the semantic information included in the second reference material, is used to:

[0135] Each image in the second reference material is analyzed to determine the semantic information corresponding to each image.

[0136] The semantic information corresponding to each of the material images is integrated to generate the semantic information included in the second reference material.

[0137] In an optional implementation, the determining module 505, when determining the target semantic information of the content topic based on the semantic information included in the second reference material, is configured to:

[0138] Based on the content theme, the semantic information included in the second reference material is expanded to generate updated semantic information;

[0139] Based on the updated semantic information, the target semantic information of the content topic is determined.

[0140] In an optional implementation, the determining module 505, when determining the target semantic information of the content topic based on the semantic information included in the second reference material, is configured to:

[0141] In response to a triggered adjustment operation, the semantic information included in the second reference material is preprocessed to generate processed semantic information; the preprocessing includes at least one of the following: filtering, deduplication, and addition;

[0142] Based on the processed semantic information, the target semantic information of the content topic is determined.

[0143] In an optional implementation, the determining module 505 is further configured to determine the target semantic information of the content topic according to the following steps:

[0144] Based on the content theme and the set initial semantic template, initial semantic information is generated; the initial semantic template includes multiple structured parameters, and the initial semantic information includes parameter information that matches the content theme for the multiple structured parameters.

[0145] Based on the initial semantic information, the target semantic information of the content topic is determined.

[0146] In one optional implementation, the second generation module 504, when generating the content-to-be-delivered data corresponding to the content theme based on the target material, is used to:

[0147] Obtain multimedia information matching the content theme; the multimedia information includes at least one of the following: audio information, image information, text information, and video information;

[0148] The multimedia information is fused with the target material to generate the data to be delivered corresponding to the content theme.

[0149] In one optional embodiment, the target material is generated using a target diffusion model, and the device further includes: an adjustment module 506; the adjustment module 506 is used for:

[0150] Based on the historical training samples of the trained diffusion model, new semantics not included in the historical training samples are determined from the target semantic information of the content topic.

[0151] Obtain new training samples that match the new semantics;

[0152] Using the newly added training samples, the model parameters of the trained diffusion model are adjusted to generate an adjusted diffusion model.

[0153] If the adjusted diffusion model meets the training cutoff condition, the adjusted diffusion model will be determined as the target diffusion model for content topic matching.

[0154] The processing flow of each module in the device and the interaction flow between each module can be referred to the relevant descriptions in the above method embodiments, and will not be detailed here.

[0155] Based on the same technical concept, this disclosure also provides a computer device. (See also...) Figure 6 The diagram shows the structure of a computer device 600 provided in this embodiment of the present disclosure, including a processor 601, a memory 602, and a bus 603. The memory 602 stores execution instructions and includes main memory 6021 and external memory 6022. The main memory 6021, also called internal memory, is used to temporarily store computational data in the processor 601 and data exchanged with external memory 6022 such as a hard disk. The processor 601 exchanges data with the external memory 6022 through the main memory 6021. When the computer device 600 is running, the processor 601 and the memory 602 communicate through the bus 603, causing the processor 601 to execute the following instructions:

[0156] Obtain primary reference material that matches the content theme;

[0157] Feature extraction is performed on the first reference material to generate target feature data;

[0158] Using the target semantic information of the determined content theme, the target feature data is processed to generate target material that matches the target semantics and the material style indicated by the first reference material;

[0159] Based on the target material, generate the data to be delivered corresponding to the content theme.

[0160] In one possible implementation, the instructions executed by processor 601 determine the target semantic information of the content topic according to the following steps:

[0161] Obtain a second reference material containing thematic elements related to the aforementioned content theme;

[0162] The second reference material is parsed to determine the semantic information included in the second reference material;

[0163] Based on the semantic information included in the second reference material, the target semantic information of the content topic is determined.

[0164] In one possible implementation, the instructions executed by processor 601, including obtaining second reference material containing thematic elements related to the content theme, include:

[0165] Obtain multiple candidate reference materials that match the content theme, and a delivery performance indicator for each candidate reference material; the delivery performance indicator is determined based on the interaction data corresponding to the candidate reference material after it is delivered.

[0166] Based on the delivery performance metrics corresponding to each of the candidate reference materials, a second reference material is determined from the plurality of candidate reference materials.

[0167] In one possible implementation, in the instructions executed by processor 601, when the second reference material contains multiple material images, the step of parsing the second reference material to determine the semantic information included in the second reference material includes:

[0168] Each image in the second reference material is analyzed to determine the semantic information corresponding to each image.

[0169] The semantic information corresponding to each of the material images is integrated to generate the semantic information included in the second reference material.

[0170] In one possible implementation, the instructions executed by processor 601, wherein determining the target semantic information of the content topic based on the semantic information included in the second reference material, includes:

[0171] Based on the content theme, the semantic information included in the second reference material is expanded to generate updated semantic information;

[0172] Based on the updated semantic information, the target semantic information of the content topic is determined.

[0173] In one possible implementation, the instructions executed by processor 601, wherein determining the target semantic information of the content topic based on the semantic information included in the second reference material, includes:

[0174] In response to a triggered adjustment operation, the semantic information included in the second reference material is preprocessed to generate processed semantic information; the preprocessing includes at least one of the following: filtering, deduplication, and addition;

[0175] Based on the adjusted semantic information, the target semantic information of the content topic is determined.

[0176] In one possible implementation, the instructions executed by processor 601 determine the target semantic information of the content topic according to the following steps:

[0177] Based on the content theme and the set initial semantic template, initial semantic information is generated; the initial semantic template includes multiple structured parameters, and the initial semantic information includes parameter information that matches the content theme for the multiple structured parameters.

[0178] Based on the initial semantic information, the target semantic information of the content topic is determined.

[0179] In one possible implementation, the instructions executed by processor 601, including generating the content-to-be-delivered data corresponding to the content theme based on the target material, include:

[0180] Obtain multimedia information matching the content theme; the multimedia information includes at least one of the following: audio information, image information, text information, and video information;

[0181] The multimedia information is fused with the target material to generate the data to be delivered corresponding to the content theme.

[0182] In one possible implementation, the target material in the instructions executed by processor 601 is generated using a target diffusion model, and the method further includes:

[0183] Based on the historical training samples of the trained diffusion model, new semantics not included in the historical training samples are determined from the target semantic information of the content topic.

[0184] Obtain new training samples that match the new semantics;

[0185] Using the newly added training samples, the model parameters of the trained diffusion model are adjusted to generate an adjusted diffusion model.

[0186] If the adjusted diffusion model meets the training cutoff condition, the adjusted diffusion model will be determined as the target diffusion model for content topic matching.

[0187] This disclosure also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the content generation method described in the above method embodiments. The storage medium can be a volatile or non-volatile computer-readable storage medium.

[0188] This disclosure also provides a computer program product carrying program code. The program code includes instructions that can be used to execute the steps of the content generation method described in the above method embodiments. For details, please refer to the above method embodiments, which will not be repeated here.

[0189] The aforementioned computer program product can be implemented through hardware, software, or a combination thereof. In one optional embodiment, the computer program product is specifically embodied in a computer storage medium; in another optional embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0190] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and devices described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. In the several embodiments provided in this disclosure, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division; in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection may be through some communication interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

[0191] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0192] In addition, the functional units in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0193] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0194] Finally, it should be noted that the above-described embodiments are merely specific implementations of this disclosure, used to illustrate the technical solutions of this disclosure, and not to limit it. The protection scope of this disclosure is not limited thereto. Although this disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this disclosure. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this disclosure, and should all be covered within the protection scope of this disclosure. Therefore, the protection scope of this disclosure should be determined by the protection scope of the claims.

Claims

1. A content generation method, characterized in that, include: Obtain primary reference materials that match the content theme; Feature extraction is performed on the first reference material to generate target feature data that characterizes the style of the first reference material; Using the target semantic information of the determined content theme, the target feature data is processed to generate target material that matches the target semantics and the material style indicated by the first reference material. The target semantic information is used to characterize the material content of the target material that meets the needs of the business scenario. The target semantic information is determined based on the second reference material. Based on the target material, generate the data to be delivered corresponding to the content theme.

2. The method according to claim 1, characterized in that, The target semantic information of the content topic is determined according to the following steps: Obtain a second reference material containing thematic elements related to the aforementioned content theme; The second reference material is parsed to determine the semantic information included in the second reference material; Based on the semantic information included in the second reference material, the target semantic information of the content topic is determined.

3. The method according to claim 2, characterized in that, The acquisition of second reference material containing thematic elements related to the content theme includes: Obtain multiple candidate reference materials that match the content theme, and a delivery performance indicator for each candidate reference material; the delivery performance indicator is determined based on the interaction data corresponding to the candidate reference material after it is delivered. Based on the delivery performance metrics corresponding to each of the candidate reference materials, a second reference material is determined from the plurality of candidate reference materials.

4. The method according to claim 2, characterized in that, When the second reference material contains multiple image materials, the step of parsing the second reference material to determine the semantic information included in the second reference material includes: Each image in the second reference material is analyzed to determine the semantic information corresponding to each image. The semantic information corresponding to each of the material images is integrated to generate the semantic information included in the second reference material.

5. The method according to claim 2, characterized in that, The step of determining the target semantic information of the content topic based on the semantic information included in the second reference material includes: Based on the content theme, the semantic information included in the second reference material is expanded to generate updated semantic information; Based on the updated semantic information, the target semantic information of the content topic is determined.

6. The method according to claim 2 or 3, characterized in that, The step of determining the target semantic information of the content topic based on the semantic information included in the second reference material includes: In response to a triggered adjustment operation, the semantic information included in the second reference material is preprocessed to generate processed semantic information; the preprocessing includes at least one of the following: filtering, deduplication, and addition; Based on the processed semantic information, the target semantic information of the content topic is determined.

7. The method according to claim 1, characterized in that, The target semantic information of the content topic is determined according to the following steps: Based on the content theme and the set initial semantic template, initial semantic information is generated; the initial semantic template includes multiple structured parameters, and the initial semantic information includes parameter information that matches the content theme for the multiple structured parameters. Based on the initial semantic information, the target semantic information of the content topic is determined.

8. The method according to claim 1, characterized in that, The step of generating the content-to-be-delivered data corresponding to the content theme based on the target material includes: Obtain multimedia information matching the content theme; the multimedia information includes at least one of the following: audio information, image information, text information, and video information; The multimedia information is fused with the target material to generate the data to be delivered corresponding to the content theme.

9. The method according to claim 1, characterized in that, The target material is generated using a target diffusion model, and the method further includes: Based on the historical training samples of the trained diffusion model, new semantics not included in the historical training samples are determined from the target semantic information of the content topic. Obtain new training samples that match the new semantics; Using the newly added training samples, the model parameters of the trained diffusion model are adjusted to generate an adjusted diffusion model. If the adjusted diffusion model meets the training cutoff condition, the adjusted diffusion model will be determined as the target diffusion model for content topic matching.

10. A content generation apparatus, characterized in that, include: The acquisition module is used to acquire the first reference material that matches the content theme; The feature extraction module is used to extract features from the first reference material and generate target feature data that characterizes the style of the first reference material. The first generation module is used to process the target feature data using the target semantic information of the determined content theme, and generate target material that matches the target semantic and the material style indicated by the first reference material, wherein the target semantic information is used to characterize the material content of the target material that meets the needs of the business scenario, and the target semantic information is determined based on the second reference material; The second generation module is used to generate the content-to-be-delivered data corresponding to the content theme based on the target material.

11. A computer device, characterized in that, include: The computer device includes a processor, a memory, and a bus, wherein the memory stores machine-readable instructions executable by the processor, and the processor communicates with the memory via the bus when the computer device is running, and the machine-readable instructions, when executed by the processor, perform the steps of the content generation method as described in any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the content generation method as described in any one of claims 1 to 9.

Images