Live broadcast material generation method and device, equipment and storage medium
By obtaining the material requirements text, determining the target material template and style parameters, generating stylized images, and cropping them to obtain the target material, the problem of high threshold for live streaming material production is solved, and efficient live streaming material generation is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SHANGHAI JIUCHI NETWORK TECH CO LTD
- Filing Date
- 2026-01-20
- Publication Date
- 2026-05-15
AI Technical Summary
The production of live streaming materials is difficult and inefficient, requiring streamers to rely on professional designers and resulting in a long production time.
By obtaining the material requirements text, determining the target material template and style parameters, generating a stylized image, segmenting the foreground and background elements, setting the background transparency, and cropping to obtain the target live broadcast material.
It lowers the barrier to entry for creating live streaming materials, increases the diversity and efficiency of materials production, and shortens the production time.
Smart Images
Figure CN122053933A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of computer technology, and in particular to a method, apparatus, device and storage medium for generating live streaming materials. Background Technology
[0002] With the development of computer and internet technologies, live streaming platforms are offering increasingly richer features and visual effects. For example, streamers can add live streaming materials to their screens to meet their diverse viewing needs.
[0003] As the live streaming industry continues to expand, streamers are demanding more personalized content for their live streams. However, the production of live stream content remains challenging, often requiring streamers to rely on professional designers for tasks such as theme stylization, content cropping, and exporting transparent images. The process from conception to usable content is lengthy, resulting in low production efficiency. Summary of the Invention
[0004] This application provides a method, apparatus, device, and storage medium for generating live streaming materials, in order to solve the technical problems of long production time and low production efficiency of live streaming materials in related technologies. It can effectively shorten the production time of live streaming materials and improve the production efficiency of live streaming materials.
[0005] In a first aspect, embodiments of this application provide a method for generating live streaming materials, including: Obtain material requirement text, and determine the target material template and style parameters in the preset live streaming material library based on the material requirement text. The preset live streaming material library records multiple preset material templates. Generate a stylized image based on the target material template and the style parameters; Identify the foreground and background elements in the stylized image, and set the transparency of the background elements to a preset transparency. The target live stream material is obtained by cropping the stylized image based on the foreground elements.
[0006] In a second aspect, embodiments of this application provide a live streaming material generation device, including a demand response module, an image generation module, a background setting module, and a material generation module, wherein: The demand response module is used to obtain material demand text, and determine the target material template and style parameters in the preset live broadcast material library based on the material demand text. The preset live broadcast material library records multiple preset material templates. The image generation module is used to generate a stylized image based on the target material template and the style parameters; The background setting module is used to determine the foreground elements and background elements in the stylized image, and set the transparency of the background elements to a preset transparency. The material generation module is used to crop the stylized image based on the foreground elements to obtain the target live streaming material.
[0007] In a third aspect, embodiments of this application provide a live streaming material generation device, including: a memory and one or more processors; The memory is used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the live streaming material generation method as described in the first aspect.
[0008] In a fourth aspect, embodiments of this application provide a storage medium for storing computer-executable instructions, which, when executed by a computer processor, are used to perform the live streaming material generation method as described in the first aspect.
[0009] This application embodiment determines the target material template and style parameters in a preset live streaming material library based on the material requirement text, generates a stylized image based on the target material template and style parameters, determines the foreground and background elements in the stylized image, sets the transparency of the background elements to a preset transparency, and crops the stylized image based on the foreground and background elements to obtain the target live streaming material. The target live streaming material can meet the user's style requirements for live streaming material while conforming to the format requirements of the live streaming material template, effectively reducing the production threshold of live streaming material, increasing the diversity of live streaming material, shortening the production time of live streaming material, and improving the production efficiency of live streaming material. Attached Figure Description
[0010] Figure 1 This is a flowchart of a live streaming material generation method provided in an embodiment of this application; Figure 2 This is a flowchart of another live streaming material generation method provided in the embodiments of this application; Figure 3 This is a schematic diagram of the structure of a live streaming material generation device provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of a live streaming material generation device provided in an embodiment of this application. Detailed Implementation
[0011] To make the objectives, technical solutions, and advantages of this application clearer, specific embodiments of this application will be described in further detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely for explaining this application and not for limiting it. It should also be noted that, for ease of description, only the parts relevant to this application are shown in the drawings, not all of them. Before discussing exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe operations (or steps) as sequential processes, many of these operations can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but additional steps not included in the drawings may also be present. The above processes can correspond to methods, functions, procedures, subroutines, subroutines, etc.
[0012] Figure 1 A flowchart of a live streaming material generation method provided in this application embodiment is given. The live streaming material generation method provided in this application embodiment can be executed by a live streaming material generation device, which can be implemented by hardware and / or software and integrated into a live streaming material generation device.
[0013] The following description uses a live streaming material generation device as an example to illustrate the live streaming material generation method. (Reference) Figure 1 The method for generating live stream footage includes: S110: Obtain the material requirement text, and determine the target material template and style parameters in the preset live broadcast material library based on the material requirement text. The preset live broadcast material library records multiple preset material templates.
[0014] This application provides multiple preset material templates (user interface templates (UI templates)). These preset material templates are stored in a preset live streaming material library, which can be configured locally on the live streaming material generation device or on a server. Each preset material template includes a reference image, a unique template ID, a function type (e.g., leaderboard, chat box, voting panel, etc.), and preset element information (key element definitions, keyElements) used to describe the layout structure. For example, a leaderboard material template can define preset element information such as "main frame element," "left and right decorative elements," and "title area decoration." The reference image can demonstrate the spatial structure of the UI layout in the preset material, such as the position of the leaderboard frame and the relative size of the title area.
[0015] For example, the natural language input of a user (e.g., a live streamer) is obtained as the material requirement text. The user's natural language input can be text or voice input (voice text obtained through speech recognition). The material requirement text can reflect the user's style requirements for the generated live stream materials. For example, the user can input "winter snow and ice style game leaderboard" or "cyberpunk style chat box layout" as material requirement text. The live stream materials provided in this application can be understood as image materials used to be displayed in the live stream room, and the live stream materials can be PNG format images.
[0016] In one embodiment, a target material template is determined from a preset live streaming material library based on the obtained material requirement text, and style parameters are determined based on the material requirement text. Optionally, the preset material template that best meets the material requirement text can be selected as the target material template based on the function type corresponding to each preset material template in the preset live streaming material library. For example, when the material requirement text is "winter ice and snow style game leaderboard", a preset material template whose function type includes leaderboard can be selected as the target material template.
[0017] The style parameters can be used to reflect the style of the material as indicated by the material requirement text. For example, the style parameters can be constructed by extracting words as visual style from the material requirement text. Optionally, the style parameters may include style keywords, style color information, and style element information. This application determines the target material template and style parameters in the same semantic space based on the material requirement text, which can effectively avoid the problem of the material template and material style being separate and improve the quality of live broadcast material generation.
[0018] S120: Generate a stylized image based on the target material template and style parameters.
[0019] For example, based on the target material template and style parameters determined above, while preserving the overall layout structure of the reference image in the target material template, style transfer and element replacement are performed on the reference image to obtain a stylized image.
[0020] Optionally, a stylized image can be generated using a trained image generation model. For example, the template ID and style parameters corresponding to the target material template can be sent to the trained image generation model so that the image generation model follows the structural layout of the reference image corresponding to the target material template to create a stylized image that satisfies the style parameters.
[0021] S130: Determine the foreground and background elements in the stylized image, and set the transparency of the background elements to a preset transparency.
[0022] For example, using a preset foreground / background segmentation algorithm or a trained foreground / background segmentation model, the generated stylized image is segmented to determine foreground and background elements in the stylized image. The foreground elements include multiple pixels belonging to the foreground, and the background elements include multiple pixels belonging to the background.
[0023] In one embodiment, after determining the foreground and background elements in the stylized image, an alpha channel is added to the stylized image, and the transparency of the background elements is set to a preset transparency (the preset transparency can be 80% to 100%, for example, setting it to 100% transparency, that is, setting the background elements to be completely transparent).
[0024] S140: Obtain the target live streaming material by cropping the stylized image based on the foreground elements.
[0025] For example, the distribution area of the foreground element in the stylized image can be determined based on the foreground and background elements. This distribution area is then cropped to obtain one or more target live stream clips, and the position information (absolute and / or relative position) of these target live stream clips in the stylized image is recorded. After obtaining the target live stream clips, they can be saved to a preset media library. The live streaming system (e.g., OpenBroadcaster Software, OBS) can then call the target live stream clips from the preset media library, and when calling the target live stream clips, the display position of the target live stream clips can be set according to the position information.
[0026] The above describes a process where, based on the material requirements text, a target material template and style parameters are determined from a pre-set live streaming material library. A stylized image is then generated based on the target material template and style parameters. Foreground and background elements within the stylized image are identified, and the transparency of the background elements is set to a preset transparency. The stylized image is then cropped based on the foreground and background elements to obtain the target live streaming material. This process satisfies the user's style requirements for live streaming materials while conforming to the format requirements of the live streaming material template. This effectively lowers the barrier to entry for live streaming material production, increases the diversity of live streaming materials, shortens production time, and improves production efficiency.
[0027] Based on the above embodiments, Figure 2 A flowchart of another live stream material generation method provided in this application embodiment is given, which is a specific embodiment of the above-described live stream material generation method. (Reference) Figure 2 The method for generating live stream footage includes: S210: Obtain the material requirement text, and generate style prompt words based on the material requirement text and the preset template information of multiple preset material templates in the preset live broadcast material library.
[0028] S220: Send style cue words to the trained style generation model. The style generation model analyzes and processes the style cue words to obtain the target material template and style parameters. The style parameters include style keywords, style color information and style element information.
[0029] For example, after obtaining the material requirement text, the material requirement text and the preset template information of each preset material template in the preset live broadcast material library are combined to obtain a structured style prompt.
[0030] In one embodiment, style cue words may require a style generation model (e.g., a large language model LLM) to determine a target material template that meets the requirements of the material requirement text from multiple preset material templates, and to generate style parameters based on the material requirement text. The style parameters include style keywords (which can be used as overall style keywords), style color information (style_elements, which can be used as style descriptions of multiple decorative or replacement elements), and style element information (color_palette, which can represent color words in a color scheme).
[0031] Furthermore, style cue words are sent to the trained style generation model, which then analyzes and processes these cue words to obtain the target material template and style parameters. The target material template can be represented by a corresponding template ID. This application generates style cue words based on the material requirement text and preset template information, and uses a single call to the style generation model to analyze and process these cue words to obtain the target material template and style parameters. By determining the target material template and style parameters within the same semantic space based on the material requirement text, this effectively avoids the problem in related live streaming material generation schemes where the material template needs to be determined first, and then the material style needs to be separated and parsed, leading to a disconnect between the material template and the material style. This significantly improves the quality of live streaming material generation.
[0032] In one embodiment, after obtaining the target material template and style parameters, a consistency check can be performed on the target material template and style parameters. For example, it can check whether the target material template and style parameters contain all required fields, check whether the selected template ID is in the template ID candidate list of the preset live streaming material library, check whether the number of style color information meets the requirements of the preset element information of the target material template, and verify whether the first few style color information can be semantically mapped one-to-one with the preset element information.
[0033] When the target material template and style parameters pass the consistency check, the subsequent stylized image generation steps are performed. However, if the target material template or style parameters fail the consistency check, the style generation model is required to re-output the target material template and style parameters. Alternatively, the reason for failing the consistency check can be fed back to the style generation model, requiring it to re-output the target material template and style parameters until a target material template and style parameters that pass the consistency check are obtained. This ensures the correct generation of stylized images and guarantees the quality of live broadcast material generation.
[0034] S230: Based on the preset element information in the target material template and the style element information in the style parameters, generate an element conversion instruction that indicates the conversion from preset element information to style element information.
[0035] S240: Generate image prompts based on element conversion instructions and style keywords and style color information in the target material template.
[0036] S250: Send image prompts to the trained image generation model, which then generates a stylized image based on the image prompts and a reference image of the target material template.
[0037] For example, preset element information and style element information from the target material template are extracted, and element conversion instructions are generated based on the preset element information and style element information. These element conversion instructions can be used to instruct the element corresponding to the preset element information to be converted to the element corresponding to the style element information.
[0038] Furthermore, by integrating element conversion instructions with style keywords and style color information in the target material template, an image prompt is obtained. The image prompt can instruct the image generation model to perform element and style conversion on the reference image of the target material template according to the element conversion instructions, style keywords, and style color information, and output the corresponding stylized image.
[0039] In one embodiment, after generating image prompts, the image prompts can be sent to the trained image generation model, which then generates a stylized image based on the image prompts and a reference image of the target material template.
[0040] Optionally, multiple style element information that matches the number of preset element information in the target material template can be determined from multiple style element information in the style parameters (e.g., multiple style element information that matches the number of preset element information in the order). The preset element information and style element information that match the number can be combined in pairs (this can be a random combination or a combination according to the order of the preset element information and style element information) to generate corresponding element conversion instructions. If there is any remaining style element information (i.e. style element information that does not form an element conversion instruction), an element attachment instruction that indicates the addition of elements can be generated based on the remaining style element information.
[0041] For example, assuming the target template contains preset element information for two decorative elements, "cat" and "star," and style element information includes "winter bird," "snowflake," "ice crystal," and "snow cap," then element transformation instructions such as "cat transform to winter bird" (meaning to convert the "cat" element to the "winter bird" element) and "star transform to snowflake" (meaning to convert the "star" element to the "snowflake" element) can be generated. The remaining style element information can be used as additional decorations to generate element attachment instructions such as "apply ice crystal" (meaning to add the "ice crystal" element) and "apply snow cap" (meaning to add the "snow cap" element), so as to attach the remaining style elements to the blank or decorative areas of the overall layout.
[0042] Furthermore, the element conversion instructions, element attachment instructions, and style keywords and style color information in the target material template can be integrated to obtain image prompts. The image prompts can instruct the image generation model to perform element and style conversion on the reference image of the target material template, add new elements, and output the corresponding stylized image based on the element conversion instructions, element attachment instructions, style keywords, and style color information.
[0043] This application generates element conversion instructions based on preset element information and style element information, and generates image prompts based on element conversion instructions, style keywords, and style color information. It then uses an image generation model to generate stylized images based on the image prompts and reference images of the target material template. The stylized images maintain the structural layout of the reference images while realizing the conversion from preset elements to style elements, the conversion of image style, and the conversion of style color, thereby improving the flexibility of live streaming material generation.
[0044] S260: Identify the foreground and background elements in the stylized image, and set the transparency of the background elements to a preset transparency.
[0045] In one possible embodiment, the live streaming material generation method provided in this application determines the foreground elements and background elements in a stylized image by performing foreground and background segmentation processing on the stylized image to obtain candidate foreground elements and background elements, and then merging the candidate foreground elements to obtain the foreground elements.
[0046] For example, the stylized image can be converted to grayscale and binarized, and the stylized image after grayscale conversion and binarization can be segmented into foreground and background elements to obtain one or more candidate foreground elements and one or more background elements in the stylized image. Optionally, noise in the candidate foreground and background elements can be removed and any broken contours can be connected through morphological operations.
[0047] After performing foreground and background segmentation on a stylized image to obtain candidate foreground and background elements, multiple candidate foreground elements are merged based on their area and position (e.g., merging multiple candidate foreground elements with areas smaller than a preset area threshold and distances between them smaller than a preset threshold into one foreground element), resulting in one or more foreground elements. A foreground element can be composed of one or more candidate foreground elements. This application obtains foreground elements by merging candidate foreground elements in a stylized image, reducing the output of fragmented foreground elements, decreasing the generation of useless live stream footage, and improving the quality of live stream footage generation.
[0048] In one embodiment, the live streaming material generation method provided in this application merges candidate foreground elements to obtain foreground elements, which may involve: determining the border elements and the internal elements within the border elements in the candidate foreground elements; and merging the border elements and the internal elements into foreground elements.
[0049] For example, a pre-defined contour detection algorithm or contour extraction model is used to extract border elements with hollow structures (such as border elements formed by leaderboard frames, panel borders, etc.) from candidate foreground elements. Candidate foreground elements distributed within these border elements are then identified, and these candidate foreground elements are determined as the internal elements corresponding to the border elements. The border elements are then merged with their internal elements to form a foreground element. This application reduces the need to split a complete frame into multiple fragments by merging border elements and internal elements into foreground elements, ensuring that a complete frame corresponds to a single foreground element, thus improving the quality of live streaming content generation.
[0050] S270: Obtain the target live streaming material by cropping a stylized image based on foreground elements.
[0051] In one possible embodiment, the live streaming material generation method provided in this application obtains the target live streaming material by cropping a stylized image based on foreground elements. This may involve determining a foreground region based on foreground elements and cropping the stylized image based on the foreground region to obtain the target live streaming material.
[0052] For example, one or more foreground regions are determined in the stylized image based on the position, size, and shape of each foreground element. For instance, a rectangular area enclosing the foreground element is determined in the stylized image based on the position, size, and shape of the foreground element, and this rectangular area is defined as the foreground region corresponding to the foreground element. The foreground region includes the pixels of the corresponding foreground element and the pixels of the background element adjacent to the foreground element.
[0053] Furthermore, the stylized image is cropped according to the aforementioned determined foreground regions to obtain the target live streaming material (e.g., a PNG image) corresponding to each foreground region. The target live streaming material includes pixels corresponding to foreground elements and pixels corresponding to background elements within the foreground region. The background element pixels have a preset transparency (e.g., 100% transparent). After loading the target live streaming material onto the live streaming screen, the image corresponding to the foreground elements can be observed, and the background elements do not obstruct the view. This application obtains the target live streaming material by determining the foreground region based on the foreground elements and cropping the stylized image according to the foreground region. The target live streaming material satisfies the user's style requirements for live streaming material while conforming to the format requirements of the live streaming material template, effectively improving the production efficiency of live streaming material.
[0054] In one possible embodiment, the live stream material generation method provided in this application, after obtaining the target live stream material by cropping a stylized image based on foreground elements, further includes: S281: Determine the number and type of target elements in the target live streaming material.
[0055] S282: Verify the layout consistency of the target live streaming materials based on the number and type of target elements, as well as the preset quantity and type requirements of the target material template.
[0056] S283: If the target live streaming material fails the layout consistency verification, regenerate the target live streaming material based on the material requirement text.
[0057] In one embodiment, preset quantity requirements and preset type requirements can be set for each preset material template in the preset live streaming material library in advance (the preset quantity requirements and preset type requirements can be saved as Ground Truth metadata to describe the expected number and type of elements, for example, a leaderboard template may require at least one main frame element and two decorative elements). The preset quantity requirements can be used to limit the quantity range of materials, the quantity range of each element type, and the element quantity limit for different preset positions. The preset type requirements are used to limit the minimum element types to be included, incompatible element types, and element type limits for different preset positions.
[0058] For example, after cropping the stylized image to obtain the target live stream material, the number and type of target elements corresponding to the target live stream material are determined. The target element type of the target live stream material can be determined based on preset element information and style element information. For instance, if the foreground element corresponding to the target live stream material is obtained by transforming preset element information using style element information, or by adding (attaching) elements using style element information, then the target element type of the target live stream material is style element information; if the foreground element corresponding to the target live stream material is a preset element that already exists in the reference image, then the target element type of the target live stream material is the preset element information of that preset element recorded in the target material template.
[0059] Furthermore, obtain the preset quantity requirements and preset type requirements corresponding to the target material template, and determine whether the number and type of target elements of the target live streaming material meet the preset quantity requirements and preset type requirements corresponding to the target material template.
[0060] If the number of target elements meets the preset quantity requirement and the type of target elements meets the preset type requirement, then the target live streaming material is considered to have passed the layout consistency verification and can be output. The target live streaming material is then saved to the preset material library for the live streaming system to use.
[0061] If the number of target elements does not meet the preset quantity requirement, and / or the type of target elements does not meet the preset type requirement, the target live streaming material is considered to have failed the layout consistency verification. Then, the target live streaming material is regenerated according to the material requirement text (i.e., steps S210-S270 are re-executed) until the target live streaming material that has passed the layout consistency verification is obtained.
[0062] Optionally, when regenerating the target live streaming material, the style generation model and / or image generation model can be provided with the reasons for failing the layout consistency verification, so that the style generation model can output a more suitable target material template and style parameters, and / or the image generation model can generate a more suitable stylized image.
[0063] This application verifies layout consistency based on the number and type of target elements in the target live streaming material, and regenerates the target live streaming material if it fails the layout consistency verification. This eliminates the need for users to manually regenerate the live streaming material, ensuring that the target live streaming material meets the preset quantity and type requirements of the target material template. This improves the quality of live streaming material generation, not only generating live streaming material with an appearance that matches the user's theme preferences, but also meeting the special requirements of the live streaming user interface in terms of function and structure, thus enhancing the automation and reliability of live streaming material production.
[0064] This application uses the user's natural language description as the text input for material requirements. Functional template selection and style parameter extraction are completed uniformly in a single large language model call. Combined with element semantic mapping mechanism, image generation instruction formatting process, and multi-stage automatic cropping and layout verification process, it realizes full-process automation from natural language to transparent PNG functional materials, significantly reducing the input of manual operation. Through layout semantic preservation and automatic verification mechanism, it ensures that the generated results can be directly integrated into the live streaming system in terms of form and structure, effectively lowering the threshold for anchors to produce materials and improving the diversity and production efficiency of live streaming content.
[0065] The above describes a process where, based on the material requirements text, a target material template and style parameters are determined from a pre-set live streaming material library. A stylized image is then generated based on these parameters. Foreground and background elements within the stylized image are identified, and the background element's transparency is set to a preset level. The stylized image is then cropped based on the foreground and background elements to obtain the target live streaming material. This ensures that the target live streaming material meets the user's style requirements while conforming to the format requirements of the live streaming material template. This effectively lowers the barrier to entry for live streaming material production, increases its diversity, shortens production time, and improves production efficiency. Furthermore, by verifying layout consistency based on the number and type of target elements in the target live streaming material, and regenerating the material if it fails the consistency verification, the system guarantees that the target live streaming material meets the preset quantity and type requirements of the target material template, thus enhancing the automation and reliability of live streaming material production.
[0066] Figure 3 A schematic diagram of a live streaming material generation device according to an embodiment of this application is provided. (Reference) Figure 3 The live streaming material generation device includes a demand response module 31, an image generation module 32, a background setting module 33, and a material generation module 34.
[0067] The system includes a demand response module 31, which is used to obtain material demand text and determine the target material template and style parameters in the preset live streaming material library based on the material demand text. The preset live streaming material library records multiple preset material templates. The image generation module 32 is used to generate a stylized image based on the target material template and style parameters. The background setting module 33 is used to determine the foreground and background elements in the stylized image and set the transparency of the background elements to a preset transparency. The material generation module 34 is used to crop the stylized image based on the foreground elements to obtain the target live streaming material.
[0068] The above describes a process where, based on the material requirements text, a target material template and style parameters are determined from a pre-set live streaming material library. A stylized image is then generated based on the target material template and style parameters. Foreground and background elements within the stylized image are identified, and the transparency of the background elements is set to a preset transparency. Finally, the stylized image is cropped based on the foreground and background elements to obtain the target live streaming material. This process satisfies the user's style requirements for live streaming materials while also conforming to the format requirements of the live streaming material template. This effectively lowers the barrier to entry for live streaming material production, increases the diversity of live streaming materials, shortens production time, and improves production efficiency.
[0069] In one possible embodiment, the demand response module 31 determines the target material template and style parameters from a preset live streaming material library based on the material demand text, including: Style prompts are generated based on the material requirements text and the preset template information of multiple preset material templates in the preset live streaming material library; Style prompts are sent to the trained style generation model, which then analyzes and processes them to obtain the target material template and style parameters. The style parameters include style keywords, style color information, and style element information.
[0070] In one possible embodiment, the image generation module 32 generates a stylized image based on the target material template and style parameters, including: Based on the preset element information in the target material template and the style element information in the style parameters, generate an element conversion instruction that indicates the conversion from preset element information to style element information; Based on the element conversion instructions, as well as the style keywords and style color information in the target material template, generate image prompts; Send image prompts to the trained image generation model, which then generates stylized images based on the prompts and reference images of the target template.
[0071] In one possible embodiment, the material generation module 34 obtains the target live streaming material by cropping a stylized image based on foreground elements, including: Determine the foreground region based on foreground elements; The target live stream material is obtained by cropping the stylized image based on the foreground area.
[0072] In one possible embodiment, the background setting module 33 determines foreground elements and background elements in the stylized image, including: Perform foreground and background segmentation on the stylized image to obtain candidate foreground and background elements; The candidate foreground elements are merged to obtain the foreground elements.
[0073] In one possible embodiment, the background setting module 33 merges candidate foreground elements to obtain foreground elements, including: Identify the border elements in the candidate foreground elements, as well as the internal elements within the border elements; Combine the border element and the inner element into a foreground element.
[0074] In one possible embodiment, the live streaming material generation device further includes a layout verification module, which is used for: Determine the number and type of target elements in the target live stream material; Based on the number and type of target elements, as well as the preset quantity and type requirements of the target material template, the layout consistency of the target live streaming material is verified. If the target live stream material fails the layout consistency verification, regenerate the target live stream material based on the material requirement text.
[0075] It is worth noting that in the embodiments of the above-mentioned live streaming material generation device, the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the protection scope of the embodiments of this application.
[0076] This application also provides a live streaming material generation device, which can integrate the live streaming material generation apparatus provided in this application. Figure 4 This is a schematic diagram of the structure of a live streaming material generation device provided in an embodiment of this application. (Reference) Figure 4The live stream material generation device includes: an input device 43, an output device 44, a memory 42, and one or more processors 41; the memory 42 is used to store one or more programs; when one or more programs are executed by one or more processors 41, the one or more processors 41 implement the live stream material generation method provided in the above embodiments. The input device 43, output device 44, memory 42, and processors 41 can be connected via a bus or other means. Figure 4 Taking the example of a connection between China and Israel via a bus.
[0077] The memory 42, as a computing device readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as program instructions / modules corresponding to the live streaming material generation method provided in any embodiment of this application (e.g., the demand response module 31, image generation module 32, background setting module 33, and material generation module 34 in the live streaming material generation device). The memory 42 may primarily include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a function; the data storage area may store data created based on the use of the device, etc. Furthermore, the memory 42 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 42 may further include memory remotely located relative to the processor 41, and these remote memories can be connected to the device via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.
[0078] Input device 43 can be used to receive input digital or character information, and to generate key signal inputs related to user settings and function control of the device. Output device 44 may include display devices such as a display screen.
[0079] The processor 41 executes various functional applications and data processing of the device by running software programs, instructions and modules stored in the memory 42, thereby realizing the above-mentioned live broadcast material generation method.
[0080] The live streaming material generation apparatus, device, and computer provided above can be used to execute the live streaming material generation method provided in any of the above embodiments, and have corresponding functions and beneficial effects.
[0081] This application embodiment also provides a storage medium for storing computer-executable instructions. When executed by a computer processor, the computer-executable instructions are used to execute the live streaming material generation method provided in the above embodiment. The live streaming material generation method includes: obtaining material requirement text; determining a target material template and style parameters in a preset live streaming material library based on the material requirement text; the preset live streaming material library records multiple preset material templates; generating a stylized image based on the target material template and style parameters; determining foreground elements and background elements in the stylized image; setting the transparency of the background elements to a preset transparency; and cropping the stylized image based on the foreground elements to obtain the target live streaming material.
[0082] Storage medium – any type of memory device or storage device. The term “storage medium” is intended to include: mounting media, such as CD-ROMs, floppy disks, or magnetic tape devices; computer system memory or random access memory, such as DRAM, DDR RAM, SRAM, EDO RAM, Rambus RAM, etc.; non-volatile memory, such as flash memory, magnetic media (e.g., hard disks or optical storage); registers or other similar types of memory elements, etc. Storage media may also include other types of memory or combinations thereof. Furthermore, storage media may reside in a first computer system in which a program is executed, or may reside in a different second computer system connected to the first computer system via a network (such as the Internet). The second computer system can provide program instructions to the first computer for execution. The term “storage medium” can include two or more storage media that may reside in different locations (e.g., in different computer systems connected via a network). Storage media may store program instructions (e.g., specifically implemented as a computer program) executable by one or more processors.
[0083] Of course, the computer-executable instructions provided in the embodiments of this application are not limited to the live broadcast material generation method provided above, but can also execute related operations in the live broadcast material generation method provided in any embodiment of this application.
[0084] The live streaming material generation apparatus, device, and storage medium provided in the above embodiments can execute the live streaming material generation method provided in any embodiment of this application. For technical details not described in detail in the above embodiments, please refer to the live streaming material generation method provided in any embodiment of this application.
[0085] The above description is merely a preferred embodiment and the technical principles employed in this application. This application is not limited to the specific embodiments provided herein, and various obvious changes, readjustments, and substitutions that can be made by those skilled in the art will not depart from the scope of protection of this application. Therefore, although this application has been described in detail through the above embodiments, this application is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of this application, the scope of which is determined by the scope of the claims.
Claims
1. A method for generating live streaming materials, characterized in that, include: Obtain material requirement text, and determine the target material template and style parameters in the preset live streaming material library based on the material requirement text. The preset live streaming material library records multiple preset material templates. Generate a stylized image based on the target material template and the style parameters; Identify the foreground and background elements in the stylized image, and set the transparency of the background elements to a preset transparency. The target live stream material is obtained by cropping the stylized image based on the foreground elements.
2. The live streaming material generation method according to claim 1, characterized in that, The step of determining the target material template and style parameters in the preset live streaming material library based on the material requirement text includes: Style prompts are generated based on the material requirement text and the preset template information of multiple preset material templates in the preset live streaming material library; The style prompt words are sent to the trained style generation model, which then analyzes and processes them to obtain the target material template and style parameters. The style parameters include style keywords, style color information, and style element information.
3. The live streaming material generation method according to claim 1, characterized in that, The step of generating a stylized image based on the target material template and the style parameters includes: Based on the preset element information in the target material template and the style element information in the style parameters, generate an element conversion instruction that indicates the conversion from the preset element information to the style element information; Based on the element conversion instructions and the style keywords and style color information in the target material template, generate image prompts; The image prompts are sent to the trained image generation model, which then generates a stylized image based on the image prompts and a reference image of the target material template.
4. The live streaming material generation method according to claim 1, characterized in that, The process of cropping the stylized image based on the foreground elements to obtain the target live stream material includes: Determine the foreground region based on the foreground elements; The target live stream material is obtained by cropping the stylized image based on the foreground region.
5. The live streaming material generation method according to claim 1, characterized in that, Determining the foreground and background elements in the stylized image includes: The stylized image is subjected to foreground and background segmentation to obtain candidate foreground and background elements. The candidate foreground elements are merged to obtain the foreground elements.
6. The live streaming material generation method according to claim 5, characterized in that, The process of merging the candidate foreground elements to obtain foreground elements includes: Determine the border elements among the candidate foreground elements, and the internal elements within the border elements; The border element and the inner element are merged into a foreground element.
7. The live streaming material generation method according to claim 1, characterized in that, After obtaining the target live stream material by cropping the stylized image based on the foreground elements, the method further includes: Determine the number and type of target elements in the target live stream material; Based on the number and type of the target elements, as well as the preset quantity and type requirements of the target material template, the layout consistency of the target live streaming material is verified. If the target live streaming material fails the layout consistency verification, the target live streaming material will be regenerated based on the material requirement text.
8. A live streaming material generation device, characterized in that, It includes a demand response module, an image generation module, a background setting module, and a material generation module, among which: The demand response module is used to obtain material demand text, and determine the target material template and style parameters in the preset live broadcast material library based on the material demand text. The preset live broadcast material library records multiple preset material templates. The image generation module is used to generate a stylized image based on the target material template and the style parameters; The background setting module is used to determine the foreground elements and background elements in the stylized image, and set the transparency of the background elements to a preset transparency. The material generation module is used to crop the stylized image based on the foreground elements to obtain the target live streaming material.
9. A live streaming material generation device, characterized in that, include: Memory and one or more processors; The memory is used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the live streaming material generation method as described in any one of claims 1-7.
10. A storage medium for storing computer-executable instructions, characterized in that, The computer-executable instructions, when executed by a computer processor, are used to perform the live streaming material generation method as described in any one of claims 1-7.