Material generation method and device, electronic equipment and medium
By automatically determining and displaying the copy display area in the material generation method, the problem of low material generation efficiency in the prior art is solved, and more efficient and visually effective material generation is achieved.
Patent Information
- Application Number
- CN202411455004.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-17
- Publication Date
- 2025-02-28
AI Technical Summary
When generating promotional materials, developers need to manually select pictures and write copywriting, resulting in time consumption and low generation efficiency.
By obtaining the pending stills and target copy, the display area of the copy is determined based on the distance between the center of the target character's face and the edge of the stills, and the copy is automatically displayed in the area to generate the target material.
It saves manual selection and arrangement time, improves the efficiency and visual effect of material generation, and ensures that copywriting and characters are not blocked.
Smart Images
Figure 00000000_0000_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer technology, and in particular to a material generation method, device, electronic equipment and medium. Background Art
[0002] Promotional materials refer to various elements used for promotional purposes, including text, pictures, audio, video and animation, etc., which are designed to attract the attention of the target audience and promote the sales of products or services or the improvement of brand awareness. Compared with materials in the form of text, audio and other forms, picture materials can directly present images, have strong visual impact, and can convey a large amount of information in a concise and intuitive way, with high information transmission efficiency. Therefore, picture materials are more popular and are widely used in many fields such as advertising, marketing, and education. For example, in the field of film and television drama promotion, designers usually combine stills or promotional copy to create visually attractive promotional materials. However, high-quality picture materials require developers to manually select suitable pictures, then select or generate refined, concise and powerful copy for the picture, and design eye-catching application styles (such as eye-catching display position and font color). This process is not only time-consuming, but also affects the efficiency of material generation and the final visual effect. Summary of the invention
[0003] In order to solve the above technical problems or at least partially solve the above technical problems, the embodiments of the present invention provide a material generation method, device, electronic device and medium.
[0004] In a first aspect, an embodiment of the present invention provides a material generation method, comprising:
[0005] Obtaining a still photo to be processed and a target text corresponding to a target person in the still photo to be processed;
[0006] Determining a display area of the target text on the still to be processed according to the distance between the face center of the target person and the edge of the still to be processed; the overlap rate between the display area and the area occupied by the target person is less than a preset ratio;
[0007] The target text is displayed in the display area of the stills to be processed to generate target material.
[0008] Optionally, the method further includes: segmenting the target text according to the semantics of the target text, and determining the segmentation position and line number of the target text;
[0009] The displaying of the target text in the display area of the stills to be processed to generate the target material includes: displaying the target text in the display area of the stills to be processed according to the sentence positions and the number of lines to generate the target material.
[0010] Optionally, the method further comprises: determining a line height according to an area of the display region and the number of lines of the target text; and determining a font size of the target text according to the line height;
[0011] The step of displaying the target text in the display area of the stills to be processed according to the sentence positions and the number of lines to generate the target material comprises: displaying the target text in the display area of the stills to be processed according to the sentence positions, the number of lines and the font size to generate the target material.
[0012] Optionally, the method further includes: acquiring color feature information of the display area of the still to be processed, the color feature information including the brightness value of each pixel of the display area; calculating an average brightness value of the display area according to the brightness value of each pixel of the display area; determining a font color and a stroke color of the target text according to the average brightness value of the display area;
[0013] The displaying of the target text in the display area of the stills to be processed to generate the target material includes: displaying the target text in the display area of the stills to be processed according to the font color and stroke color of the target text to generate the target material.
[0014] Optionally, determining the font color and stroke color of the target text according to the average brightness value of the display area includes: when the average brightness value of the display area is greater than or equal to a preset threshold, determining the font color of the target text to be black and the stroke color of the target text to be white; when the average brightness value of the display area is less than a preset threshold, determining the font color of the target text to be yellow and the stroke color of the target text to be black.
[0015] Optionally, obtaining a target text corresponding to a target character on the still to be processed includes: obtaining a plurality of text data related to the target character on the still to be processed; and filtering out the target text from the plurality of text data according to a preset filtering strategy.
[0016] Optionally, the step of filtering out a target copy from the plurality of text data according to a preset filtering strategy includes:
[0017] According to a preset emotion recognition model, identifying the emotion category of the target person in the stills to be processed;
[0018] From the plurality of text data, text data matching the emotion category of the target person is determined, and the text data matching the emotion category of the target person is used as the target copy.
[0019] Optionally, the step of filtering out a target copy from the plurality of text data according to a preset filtering strategy includes:
[0020] Determining clothing feature information of the target person according to a preset clothing recognition model;
[0021] Determining the portrait information of the target person according to the clothing feature information;
[0022] From the multiple text data, text data matching the portrait information of the target person is determined, and the text data matching the portrait information of the target person is used as the target copy.
[0023] Optionally, obtaining a plurality of text data related to the target person in the stills to be processed includes:
[0024] Performing face recognition on the stills to be processed;
[0025] In the case where a face is identified on the still to be processed, the identified face is used as a target person;
[0026] In the case where multiple faces are identified on the still to be processed, determining a target person among the multiple faces according to the face area and / or the face definition;
[0027] A plurality of text data related to the target character is obtained from a general copy library and / or a dialogue library corresponding to the stills to be processed.
[0028] Optionally, after generating the target material, the method further includes: determining whether there is a special effect matching the target copy and / or the target character in a preset special effects library based on the expression characteristics of the target copy and / or the target character; the special effects include a variety of stickers and / or filters; and if there is a special effect matching the target copy and / or the target character, applying the special effect to the target material.
[0029] In a second aspect, an embodiment of the present invention provides a material generation device, including:
[0030] An acquisition module, used to acquire a still to be processed and a target text corresponding to a target character in the still to be processed;
[0031] A layout module, for determining a display area of the target text on the still to be processed according to the distance between the face center of the target person and the edge of the still to be processed; the display area does not overlap with the area occupied by the target person;
[0032] A synthesis module is used to display the target text in the display area of the stills to be processed to generate target material.
[0033] Optionally, the device further comprises a line number determination module, which is used to: segment the target text according to the semantics of the target text, and determine the segmentation position and line number of the target text;
[0034] The synthesis module is further used for: displaying the target text in the display area of the stills to be processed according to the sentence segmentation positions and the number of lines to generate target material.
[0035] Optionally, the device further comprises a font size determination module, which is used to: determine a line height according to the area of the display area and the number of lines of the target text; and determine a font size of the target text according to the line height;
[0036] The synthesis module is further used to: display the target text in the display area of the stills to be processed according to the sentence segmentation positions, the number of lines and the font size to generate target material.
[0037] Optionally, the device further comprises a color determination module, configured to obtain color feature information of the display area of the still to be processed, wherein the color feature information comprises a brightness value of each pixel of the display area; calculate an average brightness value of the display area according to the brightness value of each pixel of the display area; and determine a font color and a stroke color of the target text according to the average brightness value of the display area;
[0038] The synthesis module is further used to: display the target text in the display area of the stills to be processed according to the font color and stroke color of the target text to generate a target material.
[0039] Optionally, the color determination module is used to: when the average brightness value of the display area is greater than or equal to a preset threshold, determine that the font color of the target copy is black and the stroke color of the target copy is white; when the average brightness value of the display area is less than a preset threshold, determine that the font color of the target copy is yellow and the stroke color of the target copy is black.
[0040] Optionally, the acquisition module is further used to: acquire a plurality of text data related to the target character on the stills to be processed; and filter out a target copy from the plurality of text data according to a preset filtering strategy.
[0041] Optionally, the acquisition module is also used to: identify the emotion category of the target character in the stills to be processed according to a preset emotion recognition model; determine the text data that matches the emotion category of the target character from the multiple text data, and use the text data that matches the emotion category of the target character as the target copy.
[0042] Optionally, the acquisition module is also used to: determine the clothing feature information of the target person according to a preset clothing recognition model; determine the portrait information of the target person according to the clothing feature information; determine the text data that matches the portrait information of the target person from the multiple text data, and use the text data that matches the portrait information of the target person as the target copy.
[0043] Optionally, the acquisition module is also used to: perform face recognition on the stills to be processed; when a face is identified on the stills to be processed, use the identified face as a target person; when multiple faces are identified on the stills to be processed, determine the target person among the multiple faces based on the face area and / or face clarity; and obtain multiple text data related to the target person from a general copy library and / or a dialogue library corresponding to the stills to be processed.
[0044] Optionally, the device also includes a special effects module, which is used to: determine whether there is a special effect matching the target copy and / or the target character in a preset special effects library based on the expression characteristics of the target copy and / or the target character; the special effects include a variety of stickers and / or filters; if there is a special effect matching the target copy and / or the target character, apply the special effect to the target material.
[0045] In a third aspect, an embodiment of the present invention provides an electronic device, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; the memory is used to store computer programs; and the processor is used to implement the material generation method provided in any embodiment of the present invention when executing the program stored in the memory.
[0046] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the material generation method provided by any embodiment of the present invention.
[0047] The technical solution provided by the embodiments of the present invention brings at least the following beneficial effects:
[0048] The material generation method provided by the embodiment of the present invention determines the display area of the target text corresponding to the target person on the stills to be processed by the distance between the face center of the target person on the stills to be processed and the edge of the stills to be processed, and automatically synthesizes the stills to be processed and the target text to generate the target material. The method can determine the display area of the target text according to the position of the face center of the target person, saving labor time and cost, improving the efficiency of material generation, and the overlap rate between the display area and the area occupied by the target person is less than a preset ratio, and the occlusion of the target person by the target text is controlled by controlling the overlap rate, thereby ensuring the visual effect of the material. BRIEF DESCRIPTION OF THE DRAWINGS
[0049] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art are briefly introduced below.
[0050] Figure 1 A schematic diagram showing a flow chart of a material generation method according to an embodiment of the present invention;
[0051] Figure 2 A schematic diagram showing a flow chart of a material generation method according to another embodiment of the present invention;
[0052] Figure 3 A schematic diagram showing the structure of a material generating device according to an embodiment of the present invention is shown;
[0053] Figure 4 A schematic structural diagram of an electronic device according to an embodiment of the present invention is shown. DETAILED DESCRIPTION
[0054] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.
[0055] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.
[0056] Figure 1 FIG. 1 is a flow chart showing a material generation method provided by an embodiment of the present invention. Figure 1 As shown, the method includes:
[0057] Step S101: Acquire a still photo to be processed and a target text corresponding to a target character in the still photo to be processed.
[0058] Among them, the stills to be processed can be photos used to summarize the main plot or character image of the film and television work, can include photos of a certain scene or shot in the film and television work, and can also include central stills, characters, character type photos and on-set life photos, etc. The stills to be processed can include one character or multiple characters. When there is only one character on the stills to be processed, the character is the target character. When there are multiple characters on the stills to be processed, the character with the largest area can be used as the target character, or the character with the highest face clarity can be used as the target character. The target copy corresponding to the target character on the stills to be processed can be the target copy that best matches the plot and character image of the film and television work shown in the stills to be processed. As an optional example, the target copy can be a line of the target character in the film and television work, or it can be an introduction to the character image of the target character in the film and television work.
[0059] Step S102: determining a display area of the target text on the still to be processed according to the distance between the face center of the target person and the edge of the still to be processed; the overlap rate between the display area and the area occupied by the target person is less than a preset ratio.
[0060] Optionally, the stills to be processed can be identified according to a preset face recognition model to determine the face position of the target person, and then determine the position of the face center. Calculate the distance between the face center and the four sides of the stills to be processed, and take the direction of the side with the largest distance as the placement position of the target copy, that is, the display area of the target copy on the stills to be processed. For example, if the distance between the face center and the upper side of the stills to be processed is greater than the distance between the face center and the left side of the stills to be processed, and the distance between the face center and the right side of the stills to be processed is greater than the distance between the face center and the left side of the stills to be processed, then the display area of the target copy on the stills to be processed is the area to the upper left of the face center. In order to better display the target person and the target copy, the overlap rate between the display area of the target copy and the area occupied by the target person is less than a preset ratio, such as 10%. Furthermore, the preset ratio is 0, and the display area of the target copy and the area occupied by the target person do not overlap, and the two do not block each other.
[0061] In an optional embodiment, after determining the display area of the target text on the stills to be processed, the font, font size, color, sentence breaks and line breaks of the target text can also be determined. As an optional example, semantic understanding technology can be used to intelligently break sentences and line breaks the target text to ensure that the meaning of the target text is complete. The optimal display position of the target text in the stills to be processed can be calculated based on the face position of the target person to avoid blocking key visual elements such as the face of the target person. The font size of the target text can be calculated based on the sentence breaks and line breaks and the optimal display position of the target text to ensure that it is presented in the optimal display position with the best visual effect. The color of the target text can be selected from a pre-set fixed color, for example, the main color of the target text is white with a black border, or the main color of the target text is yellow with a black border. The color of the target text can also be determined based on the color of the stills to be processed and / or the main color of the target person. For example, by identifying the color block with the highest hue and brightness of the stills to be processed, the color block with the highest hue and brightness is used as the color of the stills to be processed, and the color with the greatest difference from the color block with the highest hue and brightness is used as the color of the target copy.
[0062] Step S103: Displaying the target text in the display area of the still to be processed to generate target material.
[0063] Optionally, the stills to be processed can be drawn on the first layer, the target text can be drawn on the second layer according to the position of the display area, the first layer and the second layer can be superimposed, and the target text can be displayed on the display area of the stills to be processed to obtain the target material. The material generation method provided in the embodiment of the present invention determines the display area of the target text corresponding to the target person on the stills to be processed by the distance between the face center of the target person on the stills to be processed and the edge of the stills to be processed, and automatically synthesizes the stills to be processed and the target text to generate the target material. The method can determine the display area of the target text according to the position of the face center of the target person, saves labor time and cost, improves the efficiency of material generation, and the overlap rate between the display area and the area occupied by the target person is less than a preset ratio. By controlling the overlap rate, the target text can be controlled to block the target person, thereby ensuring the visual effect of the material.
[0064] In an optional embodiment, the material generation method further includes: segmenting the target text according to the semantics of the target text, determining the segmentation position and the number of lines of the target text, and displaying the target text in the display area of the stills to be processed according to the segmentation position and the number of lines to generate the target material.
[0065] Alternatively, this can be done using the TextWrap module. The TextWrap module provides functions and methods to handle the wrapping and padding of text strings. The main function of this module is to split a long string into multiple lines according to a specified width. Still optionally, the sentence segmentation position and number of lines of the target text can be determined according to the following process: Use regular expressions to split the target text into sentences based on punctuation marks such as periods, exclamation marks, and question marks. Since sentences may be split into two parts (text and punctuation) by punctuation marks, use the join method to recombines punctuation marks and text. Perform word segmentation on each sentence. Initialize current_line and current_length to store the text and length of the current line. Traverse the words after word segmentation and use the tokenizer.tokenize(word) method in the BERT model to segment the words (convert the words into word units). Check whether the length after the word is added to the current_line exceeds the specified length. If it exceeds the specified length, add the current_line to the specified list and reset the current_line to the current word. If the length does not exceed, the word is added to the current_line and the length is updated until all the words are traversed. Then, the sentence segmentation position and line number of the target text are determined according to the current_line in the specified list.
[0066] In this embodiment, by understanding the semantics of the target text and segmenting it into sentences, and displaying it in multiple lines, the visual impact of the target text can be improved, thereby improving the visual effect of the material.
[0067] In an optional embodiment, the material generation method may further include: determining the line height according to the area of the display area and the number of lines of the target text; determining the font size of the target text according to the line height; and displaying the target text in the display area of the stills to be processed according to the sentence position, the number of lines and the font size to generate the target material. In this embodiment, the font size of the target text is determined according to the line height, which can ensure that each line of the target text can be fully displayed.
[0068] In an optional embodiment, the material generation method may further include:
[0069] Acquire color feature information of a display area of the still to be processed, where the color feature information includes a brightness value of each pixel in the display area;
[0070] Calculate the average brightness value of the display area according to the brightness value of each pixel in the display area;
[0071] Determine the font color and stroke color of the target text based on the average brightness value of the display area;
[0072] In the display area of the stills to be processed, the target text is displayed according to the font color and stroke color of the target text to generate the target material.
[0073] For example, a sub-image of the optimal display position for placing the target text is extracted from the stills to be processed, and the sub-image is converted into a grayscale image, in which each pixel represents a brightness value (0-255). The average brightness of all pixels in the grayscale image is calculated. According to the calculated average brightness value, the following logic is used to determine the font color and border color: If the average brightness of the sub-image is higher than a certain threshold, for example, it is set to 128 (medium brightness), it means that the background is relatively bright. In the case of a bright background, in order to ensure the clarity of the text and have a high contrast on the high-brightness background, a black font (RGB: (0,0,0)) is used, and in order to increase the clarity of the text edge, a white stroke (RGB: (255,255,255)) is used; if the average brightness is lower than 128, it means that the background is relatively dark. In the case of a dark background, in order to ensure the clarity of the text and have a high visibility on the low-brightness background, a yellow font (RGB: (255,255,0)) is used, and in order to increase the edge contrast, a black stroke (RGB: (0,0,0)) is used.
[0074] In an optional embodiment, the process of obtaining the target text corresponding to the target character in the still to be processed in step S101 includes: obtaining multiple text data related to the target character in the still to be processed; and filtering out the target text from the multiple text data according to a preset filtering strategy.
[0075] Among them, the multiple text data related to the target person may include, for example, the role information of the target person in the film and television works, the lines of the target person, etc. The screening strategy is used to screen out the target copy that best matches the film and television plot and character image shown in the stills to be processed from the multiple text data related to the target person. In some optional embodiments, the text data that best matches the expression characteristics of the target person can be screened out from the multiple text data, and the text data that best matches the expression characteristics of the target person is used as the target copy. In other optional embodiments, the text data that best matches the clothing characteristics of the target person can also be screened out from the multiple text data, and the text data that best matches the clothing characteristics of the target person is used as the target copy.
[0076] In some optional embodiments, the target copy is screened out from a plurality of text data according to a preset screening strategy, including:
[0077] According to the preset emotion recognition model, the emotion category of the target person in the stills to be processed is identified;
[0078] From the plurality of text data, text data matching the emotion category of the target person is determined, and the text data matching the emotion category of the target person is used as the target copy.
[0079] Optionally, the preset emotion recognition model can recognize, for example, including but not limited to the following emotion categories: sadness, fear, happiness, disgust, surprise, anger, etc. The stills to be processed are input as input data into the preset emotion recognition model, and the emotion recognition model extracts the facial expression features of the target person in the stills to be processed, and identifies the target person's emotional category based on the extracted facial expression features. After identifying the target person's emotional category, the semantics of each text data related to the target person is determined according to the semantic understanding technology, for example, the emotions expressed by each text data are understood, and the target copy is screened out from multiple text data according to the target person's emotional category and the semantics of each text data. The number of target copies can be one or more. In the case of obtaining multiple target copies, the application style of each target copy is determined respectively, and each target copy is synthesized with the stills to be processed respectively to obtain a pair of target materials.
[0080] In some optional embodiments, the target copy is screened out from a plurality of text data according to a preset screening strategy, including:
[0081] Determine the target person's clothing feature information based on a preset clothing recognition model;
[0082] Determine the portrait information of the target person based on the clothing feature information;
[0083] From the plurality of text data, text data matching the portrait information of the target person is determined, and the text data matching the portrait information of the target person is used as the target copy.
[0084] The clothing feature information is used to indicate the characteristics of the clothes and accessories worn by the target person. As an optional example, the clothing feature information may include, but is not limited to: clothing color information, clothing style information, clothing texture information, and clothing material information. Clothing color information can indicate the color of the clothes worn by the target person. Clothing style information can indicate the style of the clothing worn by the target person, such as skirts (such as long skirts, short skirts, dresses, etc.), trousers, etc. Clothing texture information can indicate stripes, plaids, solid colors, etc. Clothing material information can indicate the material of the clothing worn by the target person, such as cotton fabrics, leather, metal, etc. Portrait information can be used to indicate the personality (such as openness, extroversion, sensitivity, etc.), preferences, etc. of the target person. According to the semantic understanding technology, the semantics of each text data related to the target person are determined, and the target copy is screened out from multiple text data according to the portrait information of the target person and the semantics of each text data.
[0085] Figure 2FIG. 2 is a flow chart of a material generation method provided by another embodiment of the present invention. Figure 2 As shown, the method includes:
[0086] Step S201: Acquire stills to be processed and perform face recognition on the stills to be processed. Optionally, the stills to be processed may be recognized according to a preset face recognition model to determine the number and positions of faces on the stills to be processed.
[0087] Step S202: when a face is identified on the still to be processed, the identified face is used as a target person.
[0088] Step S203: When multiple faces are identified in the still to be processed, a target person among the multiple faces is determined based on the face area and / or face definition. Optionally, the person with the largest face area can be used as the target person, or the person with the highest face definition can be used as the target person, or the target person can be determined based on a combination of face area and face definition.
[0089] Step S204: Acquire multiple text data related to the target person from the general copy library and / or the dialogue library corresponding to the stills to be processed. The general copy library may record a resource library of various creative copy. The dialogue library corresponding to the stills to be processed is a collection of dialogues of the film and television works corresponding to the stills to be processed, and the dialogue library may include the dialogues of each character in the film and television works. In an optional embodiment, the text data corresponding to the target person may be acquired from the general copy library according to the subject type of the film and television works and the role information of the target person, and then the dialogue of the target person may be acquired from the dialogue library corresponding to the stills to be processed.
[0090] Step S205: Filter out the target copy from the multiple text data according to the preset screening strategy. For example, the target person's emotion category can be identified by a preset emotion recognition model, and the target copy can be filtered out according to the emotion category. The preset emotion recognition model can identify, for example, but not limited to the following emotion categories: sadness, fear, happiness, disgust, surprise, anger, etc. The stills to be processed are input into the preset emotion recognition model as input data, and the emotion recognition model extracts the expression features of the target person on the stills to be processed, and identifies the emotion category of the target person according to the extracted expression features. After identifying the emotion category of the target person, the semantics of each text data related to the target person is determined according to the semantic understanding technology, for example, the emotions expressed by each text data are understood, and the target copy is filtered out from the multiple text data according to the emotion category of the target person and the semantics of each text data. For another example, the portrait information of the target person can be determined according to the clothing feature information of the target person, and the target copy can be filtered out according to the portrait information. The clothing feature information is used to indicate the characteristics of the clothes and accessories worn by the target person. Clothing feature information may include, but is not limited to, clothing color information, clothing style information, clothing texture information, and clothing material information. Clothing color information may indicate the color of the clothing worn by the target person. Clothing style information may indicate the style of clothing worn by the target person, such as skirts (such as long skirts, short skirts, dresses, etc.), trousers, etc. Clothing texture information may indicate stripes, plaids, solid colors, etc. Clothing material information may indicate the material of clothing worn by the target person, such as cotton fabrics, leather, metal, etc. Portrait information may be used to indicate the personality (such as openness, extroversion, sensitivity, etc.), preferences, etc. of the target person. The semantics of each text data related to the target person is determined according to the semantic understanding technology, and the target copy is screened out from multiple text data according to the portrait information of the target person and the semantics of each text data.
[0091] Step S206: determining a display area of the target text on the still to be processed according to the distance between the face center of the target person and the edge of the still to be processed; the overlap rate between the display area and the area occupied by the target person is less than a preset ratio.
[0092] Step S207: According to the semantics of the target text, the target text is segmented to determine the segmentation position and number of lines of the target text. Optionally, the TextWrap module can be used to complete it. The TextWrap module provides some functions and methods to handle the packaging and filling of text strings. The main function of this module is to divide a long string into multiple lines according to a specified width. Still optionally, the segmentation position and number of lines of the target text can be determined according to the following process: Use regular expressions to divide the target text into sentences according to punctuation marks such as periods, exclamation marks, and question marks. Since a sentence may be split into two parts (text and punctuation marks) by punctuation marks, the join method is used to recombines punctuation marks and text. Perform word segmentation on each sentence. Initialize current_line and current_length to store the text and length of the current line. Traverse the words after word segmentation and use the tokenizer.tokenize(word) method in the BERT model to segment the words (convert the words into subword units). Check whether the length of the subword after adding it to current_line exceeds the specified length. If it exceeds the specified length, add current_line to the specified list and reset current_line to the current word. If it does not exceed the length, add the word to current_line and update the length until all words are traversed. Then, determine the sentence break position and line number of the target text based on current_line in the specified list.
[0093] Step S208: determining the font size of the target text according to the display area of the target text, the sentence segmentation position and the number of lines.
[0094] Step S209: Acquire color feature information of the still to be processed.
[0095] Step S210: Determine the display color of the target text on the stills to be processed based on the color feature information of the stills to be processed. Optionally, the background brightness of the optimal display position of the target text can be obtained, and the display color on the stills to be processed can be determined based on the background brightness. For example, extract a sub-image for placing the optimal display position of the target text from the stills to be processed, and convert the sub-image into a grayscale image, in which each pixel of the grayscale image represents a brightness value (0-255). Calculate the average brightness of all pixels in the grayscale image. According to the calculated average brightness value, the following logic is used to determine the font color and border color: if the average brightness of the sub-image is higher than a certain threshold, for example, it is set to 128 (medium brightness), it means that the background is relatively bright. In the case of a brighter background, in order to ensure clear text and high contrast on the high-brightness background, use black fonts (RGB: (0,0,0)), and in order to increase the clarity of the text edge, use white strokes (RGB: (255,255,255)); if the average brightness is lower than 128, it means that the background is relatively dark. In the case of a darker background, in order to ensure clear text and high visibility on the low-brightness background, use yellow fonts (RGB: (255,255,0)), and in order to increase the edge contrast, use black strokes (RGB: (0,0,0)).
[0096] Step S211: synthesizing the stills to be processed and the target text, so that the target text is displayed in the display area on the stills to be processed according to the sentence segmentation positions and line numbers, the font size, and the display color, to obtain the target material.
[0097] For steps S206-S210, the optimal display position of the target text in the stills to be processed can be calculated according to the face position of the target person, so as to avoid blocking key visual elements such as the face of the target person. Using semantic understanding technology, the target text is intelligently segmented and wrapped to ensure that the meaning of the target text is complete. The font size of the target text can be calculated according to the segmentation and wrapping of the target text and the optimal display position to ensure that it is presented at the optimal display position with the best visual effect. The color of the target text can be selected from a pre-set fixed color, such as the main color of the target text is white with a black stroke, or the main color of the target text is yellow with a black stroke. The color of the target text can also be determined according to the color of the stills to be processed and / or the main color of the target person. For example, by identifying the color block with the highest hue brightness of the stills to be processed, the color block with the highest hue brightness is used as the color of the stills to be processed, and the color with the greatest difference from the color block with the highest hue brightness is used as the color of the target text.
[0098] The material generation method provided in the embodiment of the present invention automatically screens suitable target texts through screening strategies, and automatically determines the optimal display style of the target texts on the stills to be processed through typesetting strategies, thereby saving labor time and costs and improving material generation efficiency and visual effects.
[0099] In an optional embodiment, after generating the target material, the material generating method further includes:
[0100] According to the expression features of the target text and / or the target person, determining whether there is a special effect matching the target text and / or the target person in a preset special effect library; the special effect includes a variety of stickers and / or filters;
[0101] In the case where there is a special effect that matches the target text and / or the target person, the special effect is applied to the target material.
[0102] In this embodiment, it can be determined whether there is a special effect matching the target text in the special effect library according to the semantics of the target text, and if so, the special effect can be applied to the target material. It can also be determined whether there is a special effect matching the target person in the special effect library according to the facial expression characteristics of the target person (the facial expression characteristics can be obtained according to the emotion recognition model mentioned above), and if so, the special effect can be applied to the target material.
[0103] Figure 3 FIG. 2 shows a schematic diagram of the structure of a material generating device provided by an embodiment of the present invention. Figure 3 As shown, the material generating device 300 includes:
[0104] An acquisition module 301 is used to acquire a still photo to be processed and a target text corresponding to a target character in the still photo to be processed;
[0105] The layout module 302 is used to determine the display area of the target text on the still to be processed according to the distance between the face center of the target person and the edge of the still to be processed; the display area and the area occupied by the target person do not overlap each other;
[0106] The synthesis module 303 is used to display the target text in the display area of the stills to be processed to generate target material.
[0107] Optionally, the device further comprises a line number determination module, which is used to: segment the target text according to the semantics of the target text, and determine the segmentation position and line number of the target text;
[0108] The synthesis module is further used for: displaying the target text in the display area of the stills to be processed according to the sentence segmentation positions and the number of lines to generate target material.
[0109] Optionally, the device further comprises a font size determination module, which is used to: determine a line height according to the area of the display area and the number of lines of the target text; and determine a font size of the target text according to the line height;
[0110] The synthesis module is further used for: displaying the target text in the display area of the stills to be processed according to the sentence segmentation positions, the number of lines and the font size to generate target material.
[0111] Optionally, the device further comprises a color determination module, configured to obtain color feature information of the display area of the still to be processed, wherein the color feature information comprises a brightness value of each pixel of the display area; calculate an average brightness value of the display area according to the brightness value of each pixel of the display area; and determine a font color and a stroke color of the target text according to the average brightness value of the display area;
[0112] The synthesis module is further used to: display the target text in the display area of the still to be processed according to the font color and stroke color of the target text to generate a target material.
[0113] Optionally, the color determination module is used to: when the average brightness value of the display area is greater than or equal to a preset threshold, determine that the font color of the target copy is black and the stroke color of the target copy is white; when the average brightness value of the display area is less than a preset threshold, determine that the font color of the target copy is yellow and the stroke color of the target copy is black.
[0114] Optionally, the acquisition module is further used to: acquire a plurality of text data related to the target character on the stills to be processed; and filter out a target copy from the plurality of text data according to a preset filtering strategy.
[0115] Optionally, the acquisition module is also used to: identify the emotion category of the target character in the stills to be processed according to a preset emotion recognition model; determine the text data that matches the emotion category of the target character from the multiple text data, and use the text data that matches the emotion category of the target character as the target copy.
[0116] Optionally, the acquisition module is also used to: determine the clothing feature information of the target person according to a preset clothing recognition model; determine the portrait information of the target person according to the clothing feature information; determine the text data that matches the portrait information of the target person from the multiple text data, and use the text data that matches the portrait information of the target person as the target copy.
[0117] Optionally, the acquisition module is also used to: perform face recognition on the stills to be processed; when a face is identified on the stills to be processed, use the identified face as a target person; when multiple faces are identified on the stills to be processed, determine the target person among the multiple faces based on the face area and / or face clarity; and obtain multiple text data related to the target person from a general copy library and / or a dialogue library corresponding to the stills to be processed.
[0118] Optionally, the device also includes a special effects module, which is used to: determine whether there is a special effect matching the target copy and / or the target character in a preset special effects library based on the expression characteristics of the target copy and / or the target character; the special effects include a variety of stickers and / or filters; if there is a special effect matching the target copy and / or the target character, apply the special effect to the target material.
[0119] The above device can execute the method provided in the embodiment of the present invention, and has the functional modules and beneficial effects corresponding to the execution method. For technical details not described in detail in this embodiment, please refer to the material generation method provided in the embodiment of the present invention.
[0120] Figure 4 FIG. 1 is a schematic diagram showing the structure of an electronic device according to an embodiment of the present invention. Figure 4 As shown, the electronic device includes:
[0121] Processor 401, communication interface 402, memory 403 and communication bus 404, wherein processor 401, communication interface 402, memory 403 communicate with each other via communication bus 404.
[0122] Memory 403, used for storing computer programs;
[0123] The processor 401 is used to execute the program stored in the memory 403 to implement the following steps:
[0124] Obtaining a still photo to be processed and a target text corresponding to a target person in the still photo to be processed;
[0125] Determining a display area of the target text on the still to be processed according to the distance between the face center of the target person and the edge of the still to be processed; the overlap rate between the display area and the area occupied by the target person is less than a preset ratio;
[0126] The target text is displayed in the display area of the stills to be processed to generate target material.
[0127] The communication bus mentioned in the above terminal can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.
[0128] The communication interface is used for communication between the above terminal and other devices.
[0129] The memory may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.
[0130] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
[0131] In another embodiment provided by the present invention, a computer-readable storage medium is provided, in which instructions are stored. When the computer-readable storage medium is run on a computer, the computer executes the material generation method described in any of the above embodiments.
[0132] In another embodiment of the present invention, a computer program product including instructions is provided, which, when executed on a computer, enables the computer to execute any of the material generation methods described in the above embodiments.
[0133] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk (SSD)), etc.
[0134] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.
[0135] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.
[0136] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.
Claims
1. A material generation method, characterized in that: include: Obtaining a still photo to be processed and a target text corresponding to a target person in the still photo to be processed; Determining a display area of the target text on the still to be processed according to the distance between the face center of the target person and the edge of the still to be processed; the overlap rate between the display area and the area occupied by the target person is less than a preset ratio; The target text is displayed in the display area of the stills to be processed to generate target material.
2. The method according to claim 1, characterized in that The method further comprises: According to the semantics of the target text, segment the target text to determine the segmentation position and line number of the target text; The step of displaying the target text in the display area of the stills to be processed to generate target material includes: In the display area of the stills to be processed, the target text is displayed according to the sentence segmentation positions and line numbers to generate target material.
3. The method according to claim 2, characterized in that The method further comprises: Determine the row height according to the area of the display area and the number of rows of the target text; Determining the font size of the target text according to the line height; The step of displaying the target text in the display area of the stills to be processed according to the sentence positions and the number of lines to generate the target material includes: In the display area of the stills to be processed, the target text is displayed according to the sentence segmentation positions, the number of lines and the font size to generate target material.
4. The method according to claim 1, characterized in that The method further comprises: Acquire color feature information of the display area of the still to be processed, wherein the color feature information includes a brightness value of each pixel in the display area; Calculating an average brightness value of the display area according to the brightness value of each pixel in the display area; Determining the font color and stroke color of the target text according to the average brightness value of the display area; The step of displaying the target text in the display area of the stills to be processed to generate target material includes: In the display area of the still to be processed, the target text is displayed according to the font color and stroke color of the target text to generate a target material.
5. The method according to claim 4, characterized in that The step of determining the font color and the stroke color of the target text according to the average brightness value of the display area includes: When the average brightness value of the display area is greater than or equal to a preset threshold, determining that the font color of the target text is black and the stroke color of the target text is white; When the average brightness value of the display area is less than a preset threshold, the font color of the target text is determined to be yellow and the stroke color of the target text is determined to be black.
6. The method according to any one of claims 1 to 5, characterized in that: Obtaining the target text corresponding to the target person on the stills to be processed, including: Acquire multiple text data related to the target person in the stills to be processed; According to a preset screening strategy, a target copy is screened out from the plurality of text data.
7. The method according to claim 6, characterized in that The step of selecting a target text from the plurality of text data according to a preset screening strategy includes: According to a preset emotion recognition model, identifying the emotion category of the target person in the stills to be processed; From the plurality of text data, text data matching the emotion category of the target person is determined, and the text data matching the emotion category of the target person is used as the target copy.
8. The method according to claim 7, characterized in that The step of selecting a target text from the plurality of text data according to a preset screening strategy includes: Determining clothing feature information of the target person according to a preset clothing recognition model; Determining the portrait information of the target person according to the clothing feature information; From the multiple text data, text data matching the portrait information of the target person is determined, and the text data matching the portrait information of the target person is used as the target copy.
9. The method according to claim 6, characterized in that Acquire multiple text data related to the target person in the stills to be processed, including: Performing face recognition on the stills to be processed; In the case where a face is identified on the still to be processed, the identified face is used as a target person; In the case where multiple faces are identified on the still to be processed, determining a target person among the multiple faces according to the face area and / or the face definition; A plurality of text data related to the target character is obtained from a general copy library and / or a dialogue library corresponding to the stills to be processed.
10. The method according to claim 1, characterized in that After generating the target material, the method further includes: According to the expression features of the target text and / or the target person, determining whether there is a special effect matching the target text and / or the target person in a preset special effect library; the special effect includes a variety of stickers and / or filters; In the case where there is a special effect that matches the target text and / or the target person, the special effect is applied to the target material.
11. A material generating device, characterized in that: include: An acquisition module, used to acquire a still to be processed and a target text corresponding to a target character in the still to be processed; A layout module, for determining a display area of the target text on the still to be processed according to the distance between the face center of the target person and the edge of the still to be processed; the display area does not overlap with the area occupied by the target person; A synthesis module is used to display the target text in the display area of the stills to be processed to generate target material.
12. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, configured to implement the method according to any one of claims 1 to 10 when executing a program stored in a memory.
13. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 10 is implemented.