Comment content processing method and device, readable storage medium and program product
By splitting the comment content into target keywords and combining image frames and audio data, digital people can control their body movements and facial expressions to display, solving the problem that the comment content is difficult to express vividly and vividly, achieving a more vivid comment presentation effect.
Patent Information
- Application Number
- CN202410993007.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-23
- Publication Date
- 2024-11-12
AI Technical Summary
In the prior art, the content of the comment is difficult to express vividly and vividly in the form of text, and cannot effectively convey the dynamics and emotions of the comment.
By splitting the comment content into target keywords, matching image frames and audio data are obtained, body movements and facial expressions are determined, and digital people are controlled to display the body movements and facial expressions to vividly present the comment content.
It realizes that the user comments are presented more vividly and vividly through digital human body movements and facial expressions, improving the viewing and fun of the comments.
Smart Images

Figure CN118945435A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present application relate to the field of multimedia communication technology, and in particular, to a method, device, readable storage medium, and program product for processing comment content. Background Art
[0002] With the rapid development of Internet technology, people have various interactions through the Internet. For example, users can post their own comments while watching a video, and the interaction between video viewers can be achieved based on the comments. However, in the related art, the comment content in the comment interface is displayed in the form of text, such as the comment interface of the video platform displays multiple text contents in the form of scrolling or list, but the way of displaying comments in text content is difficult to express the comment content vividly. Summary of the invention
[0003] The embodiments of the present application provide a method, device, readable storage medium and program product for processing comment content to solve the problem that it is difficult to express the comment content vividly and vividly by displaying the comment content in text content.
[0004] In a first aspect, an embodiment of the present application provides a method for processing comment content, including:
[0005] Split the first comment content into at least two target keywords;
[0006] Acquire at least one set of data matching each of the target keywords, each set of data comprising a target image frame, wherein the target image frame comprises limb position information;
[0007] Determining a first limb movement corresponding to the first comment content according to limb position information of at least two of the target image frames;
[0008] The first digital human is controlled to show the first body movement.
[0009] Optionally, each set of data further includes a target audio file corresponding to the target image frame; and determining the first body movement corresponding to the first comment content according to the body position information of at least two target image frames includes:
[0010] Determine the target limb starting position and the target limb ending position corresponding to the first comment content according to the limb position information of at least two of the target image frames;
[0011] Determining a change speed between a starting position of the target limb and an ending position of the target limb according to a duration of the target audio file;
[0012] A first limb movement corresponding to the first comment content is obtained according to the target limb starting position, the target limb ending position and the change speed.
[0013] Optionally, determining the target limb starting position and the target limb ending position corresponding to the first comment content according to the limb position information of at least two target image frames includes:
[0014] According to the speech rate level corresponding to the target audio file of the at least one group of data, the target image frames in the at least one group of data are grouped to obtain at least one group of image frames, wherein the speech rate level of the target audio files corresponding to the target image frames in the same group of image frames is the same, and the speech rate levels of the target audio files corresponding to the target image frames in different groups of image frames are different;
[0015] According to the limb position information of the target image frame in each group of image frames, the limb starting position information and the limb ending position information corresponding to each group of image frames are obtained;
[0016] According to the limb starting position information and limb ending position information corresponding to the target group image frames, the target limb starting position and target limb ending position corresponding to the first comment content are determined; wherein the speaking speed level corresponding to the target group image frames is the target speaking speed level, and the target speaking speed level is the speaking speed level that appears most times among the speaking speed levels of the target audio file of the at least one group of data.
[0017] Optionally, determining the change speed between the target limb starting position and the target limb ending position according to the duration of the target audio file includes:
[0018] Determine the duration of the limb position change corresponding to each group of image frames according to the duration of the target audio file corresponding to the target image frame in each group of image frames;
[0019] Determine the limb position change speed corresponding to each group of image frames according to the limb starting position information, the limb ending position information and the limb position change duration corresponding to each group of image frames;
[0020] The change speed between the target limb starting position and the target limb ending position is determined according to the change speed of the limb position corresponding to the target group image frames.
[0021] Optionally, the method of the embodiment of the present application further includes:
[0022] Converting the first comment content into a phoneme sequence, wherein the phoneme sequence includes at least one phoneme, and each phoneme corresponds to a facial expression;
[0023] Determining the facial expression of the first digital person according to the facial expression corresponding to each phoneme in the phoneme sequence;
[0024] The first digital human is controlled to show the facial expression.
[0025] Optionally, determining the facial expression of the first digital person according to the facial expression corresponding to each phoneme in the phoneme sequence includes:
[0026] Acquire at least one image frame matching the image information of the first digital human;
[0027] The facial expression corresponding to each phoneme is adjusted according to the facial expression in the image frame to obtain the facial expression of the first digital person.
[0028] Optionally, the method of the embodiment of the present application further includes:
[0029] When receiving a first input from a user to the first digital human, the first digital human is controlled to voice broadcast the first comment content.
[0030] Optionally, controlling the first digital human to show the first body movement includes:
[0031] Controlling the first digital human to display the first body movement in a first display area of the comment interface, wherein the first display area is a display area corresponding to the first comment content, and different comment contents correspond to different display areas;
[0032] Alternatively, the first digital human is controlled to display the first body movement in a second display area of the comment interface, wherein the second display area is a display area corresponding to at least two comment contents in the comment interface.
[0033] The present application also provides a review content processing device, including:
[0034] A first processing module, used to split the first comment content into at least two target keywords;
[0035] A first acquisition module, used to acquire at least one set of data matching each of the target keywords, each set of data including a target image frame, wherein the target image frame includes limb position information;
[0036] A first determination module, configured to determine a first limb movement corresponding to the first comment content according to limb position information of at least two target image frames;
[0037] The second processing module is used to control the first digital human to show the first body movement.
[0038] An embodiment of the present application also provides an electronic device, including: a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the steps of the method for processing comment content as described above when executed by the processor.
[0039] An embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method for processing comment content as described above are implemented.
[0040] The embodiment of the present application also provides a computer program product, including computer instructions, which, when executed by a processor, implement the steps of the method for processing comment content as described above.
[0041] In the embodiment of the present application, the first comment content is split into at least two target keywords; at least one set of data matching each of the target keywords is obtained, each set of data includes a target image frame, and the target image frame includes body position information; according to the body position information of at least two target image frames, the first body movement corresponding to the first comment content is determined; and the first digital human is controlled to show the first body movement. Thus, the purpose of presenting user comments through the body movements of the digital human is achieved, and the user's comment content can be presented more vividly through the body movements of the digital human. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0043] Figure 1 A flowchart showing a method for processing comment content provided in an embodiment of the present application;
[0044] Figure 2 One of the display schematic diagrams of the digital human in the embodiment of the present application is shown;
[0045] Figure 3 A second schematic diagram showing the display of a digital human in an embodiment of the present application;
[0046] Figure 4 A schematic diagram showing a module of a device for processing review content provided in an embodiment of the present application;
[0047] Figure 5 A structural block diagram of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION
[0048] In order to make the technical problems, technical solutions and advantages to be solved by the application clearer, the following will be described in detail in conjunction with the accompanying drawings and specific embodiments. In the following description, specific details such as specific configurations and components are provided only to help fully understand the embodiments of the application. Therefore, it should be clear to those skilled in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the application. In addition, for clarity and brevity, the description of known functions and structures has been omitted.
[0049] It should be understood that the references to "one embodiment" or "an embodiment" throughout the specification mean that the specific features, structures, or characteristics associated with the embodiment are included in at least one embodiment of the present application. Therefore, the references to "in one embodiment" or "in an embodiment" appearing throughout the specification do not necessarily refer to the same embodiment. In addition, these specific features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.
[0050] In various embodiments of the present application, it should be understood that the order of execution of the following processes does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application. In addition, the terms "system" and "network" are often used interchangeably in this article.
[0051] like Figure 1 As shown, an embodiment of the present application provides a method for processing comment content, which is applied to an electronic device, and the electronic device may be a terminal device. The processing method includes the following steps:
[0052] Step 101: Split the first comment content into at least two target keywords.
[0053] In an embodiment of the present application, after the user inputs the first comment content, the first comment content can be split into at least two target keywords based on keyword types, and the keyword types include characters, scenes, actions, emotions, etc.
[0054] Step 102: Acquire at least one set of data matching each of the target keywords, each set of data comprising a target image frame, wherein the target image frame comprises limb position information.
[0055] The limb position information in the embodiment of the present application includes position information such as head angle, gesture height, body amplitude and leg movement.
[0056] As an implementation manner, at least one set of data matching the target keyword is obtained in a reference data set, and the reference data set stores keywords and image frames stored according to a corresponding relationship.
[0057] Step 103: Determine a first limb movement corresponding to the first comment content according to limb position information of at least two target image frames.
[0058] The limb position information in the target image frame is used to indicate the limb position in the target image frame, such as the head position, hand position, leg position, etc. Based on the limb position information of each target image frame, the limb position change information is obtained, and then the first limb movement corresponding to the first comment content is determined based on the limb position change information. For example, if the limb position change information is the rotation information of the head, then the first limb movement is determined to be head twisting based on the head rotation information. Alternatively, if the limb position change information is the information of the leg gradually being raised, then the first limb movement is determined to be leg raising based on the information of the leg gradually being raised, etc.
[0059] Step 104: Control the first digital human to show the first body movement.
[0060] Optionally, the image of the first digital human is a target image.
[0061] In an embodiment of the present application, the target image can be determined by obtaining the image information of the digital human based on user input information, for example, by obtaining the image information of the digital human based on photo information uploaded by the user. It can also be determined based on video content, or the image can be determined based on user preference information, for example, a digital human image corresponding to a corresponding team is generated based on a ball game, or a digital human image is generated based on characters in a film or television drama.
[0062] In the embodiment of the present application, the first comment content is split into at least two target keywords; at least one set of data matching each of the target keywords is obtained, each set of data includes a target image frame, and the target image frame includes body position information; according to the body position information of at least two target image frames, the first body movement corresponding to the first comment content is determined; and the first digital human is controlled to show the first body movement. Thus, the purpose of presenting user comments through the body movements of the digital human is achieved, and the user's comment content can be presented more vividly through the body movements of the digital human.
[0063] Optionally, each set of data further includes a target audio file corresponding to the target image frame; and determining the first body movement corresponding to the first comment content according to the body position information of at least two target image frames includes:
[0064] Determine the target limb starting position and the target limb ending position corresponding to the first comment content according to the limb position information of at least two of the target image frames;
[0065] Determining a change speed between a starting position of the target limb and an ending position of the target limb according to a duration of the target audio file;
[0066] A first limb movement corresponding to the first comment content is obtained according to the target limb starting position, the target limb ending position and the change speed.
[0067] In an embodiment of the present application, the limb position information in the above-mentioned target image frame is used to indicate the position of the limb displayed in the image frame. The target limb starting position and the target limb ending position can be obtained by combining the limb position information of each target image frame, and combined with the above-mentioned change speed, the above-mentioned first limb movement can be further obtained, which makes it easier for the subsequent digital human to display the first limb movement.
[0068] Optionally, determining the target limb starting position and the target limb ending position corresponding to the first comment content according to the limb position information of at least two target image frames includes:
[0069] According to the speech rate level corresponding to the target audio file of the at least one group of data, the target image frames in the at least one group of data are grouped to obtain at least one group of image frames, wherein the speech rate level of the target audio files corresponding to the target image frames in the same group of image frames is the same, and the speech rate levels of the target audio files corresponding to the target image frames in different groups of image frames are different;
[0070] According to the limb position information of the target image frame in each group of image frames, the limb starting position information and the limb ending position information corresponding to each group of image frames are obtained;
[0071] According to the limb starting position information and limb ending position information corresponding to the target group image frames, the target limb starting position and target limb ending position corresponding to the first comment content are determined; wherein the speaking speed level corresponding to the target group image frames is the target speaking speed level, and the target speaking speed level is the speaking speed level that appears most times among the speaking speed levels of the target audio file of the at least one group of data.
[0072] Optionally, the reference data set stores keywords, image frames and audio files stored in a corresponding relationship.
[0073] In the embodiment of the present application, a crawler tool can be written to crawl video materials from video websites at regular intervals. Before the materials are stored in the library, they are voice recognized to generate subtitle timestamps and text content corresponding to the audio of each video. Finally, the video and text content are matched one by one and stored in the system material library as a reference data set for the subsequent generation of digital human body movements. The above text content is the keyword.
[0074] Exemplarily, the presentation form of the above reference data set is shown in Table 1.
[0075] Table 1
[0076]
[0077] Exemplarily, the speech speed levels in the embodiment of the present application include three levels: fast, medium and slow. The target image frames in at least one set of data are grouped based on the three levels, as shown in Table 2, assuming that the target keywords include "Xiao Ming", "operation" and "playing football". The target keyword "Xiao Ming" is used as an example to illustrate how to determine its corresponding limb position information, and the process of determining the corresponding limb position information for other target keywords is the same.
[0078] As shown in Table 2, according to the speech speed level corresponding to the target audio file of the at least one group of data, the target image frames in the at least one group of data are grouped to obtain three groups of image frames, wherein a group of image frames corresponding to the slow speed level includes target image frames in 7 groups of data, a group of image frames corresponding to the medium speed level includes target image frames in 5 groups of data, and a group of image frames corresponding to the fast speed level includes target image frames in 2 groups of data.
[0079] Table 2
[0080]
[0081]
[0082] As an implementation method, according to the limb position information of the target image frame in each group of image frames, the limb starting position information and limb ending position information corresponding to each group of image frames are obtained, including: obtaining the average value of the limb position information of the target image frame in each group of image frames, the average value of the limb position information includes the average value of the limb initial position information and the average value of the limb ending position information, the average value of the limb starting position information is used as the limb starting position information corresponding to each group of image frames, and the average value of the limb ending position information is used as the limb ending position information corresponding to each group of image frames, wherein the limb starting position information is determined based on the limb position information of the first target image frame in at least one group of data included in each group of image frames, and the limb ending position information is determined based on the limb position information of the last target image frame in at least one group of data included in each group of image frames. In this way, the limb positions before and after the changes of each body part at different speech speeds for each target keyword can be obtained.
[0083] Assume that a group of image frames includes target image frames in m groups of data, the first target image frame in each group of data corresponds to a limb starting position information, and the last target image frame in each group of data corresponds to a limb ending position information, that is, the group of image frames corresponds to m limb starting position information and m limb ending position information, the limb starting position information corresponding to the group of image frames is obtained based on the average value of the above m limb starting position information, and the limb ending position information corresponding to the group of image frames is obtained based on the average value of the above m limb ending position information.
[0084] For example, the limb starting position information corresponding to a group of image frames corresponding to the slow speed level is obtained based on the average value of the limb position information of the first target image frame in the above 7 groups of data, and the limb ending position information corresponding to a group of image frames corresponding to the slow speed level is obtained based on the average value of the limb position information of the last target image frame in the above 7 groups of data. That is, the limb starting position information corresponding to a group of image frames corresponding to the slow speed level is obtained based on the average value of the 7 limb starting position information included in the group of image frames, and the limb ending position information corresponding to the group of image frames is obtained based on the average value of the 7 limb ending position information included in the group of image frames.
[0085] For example, the starting position information of the head is represented by S_L. The angle is 30 degrees to the left, which is recorded as -30 degrees, and the angle is 30 degrees to the right, which is 30 degrees. Then the average value of the m starting position information of the head is It can be expressed by the following formula:
[0086]
[0087] Among them, S_L i Indicates the starting position information of the i-th head, |S_L i | represents the absolute value of the starting position information of the i-th head, 1≤i≤m, and m and i are both positive integers.
[0088] It should be noted that the above calculation method is only an example, and different limb parts may have different statistical standards and reference systems.
[0089] Optionally, determining the change speed between the target limb starting position and the target limb ending position according to the duration of the target audio file includes:
[0090] Determine the duration of the limb position change corresponding to each group of image frames according to the duration of the target audio file corresponding to the target image frame in each group of image frames;
[0091] Determine the limb position change speed corresponding to each group of image frames according to the limb starting position information, limb ending position information and the limb position change duration corresponding to each group of image frames;
[0092] The change speed between the target limb starting position and the target limb ending position is determined according to the change speed of the limb position corresponding to the target group image frame.
[0093] As an implementation method, assuming that a certain group of image frames includes the target image frames in the m groups of data, the duration of the limb position change corresponding to the group of image frames is obtained based on the average duration of the target audio file corresponding to the target image frames in the m groups of data. Based on the ratio of the position change value between the limb end position information and the limb start position information corresponding to each group of image frames and the above change duration, the limb position change speed corresponding to the group of image frames is obtained.
[0094] After obtaining the position and change speed of each body part before and after each target keyword at different speaking speeds based on the above implementation method, the speaking speed level with the largest number of occurrences is selected as the above target speaking speed level according to the speaking speed distribution of each target keyword, and based on the limb starting position information and limb ending position information of the target group image frame corresponding to the target speaking speed level, the target limb starting position, target limb ending position and change speed corresponding to the first comment content are determined to obtain the first limb movement.
[0095] Optionally, the method of the embodiment of the present application further includes:
[0096] Converting the first comment content into a phoneme sequence, wherein the phoneme sequence includes at least one phoneme, and each phoneme corresponds to a facial expression;
[0097] Determining the facial expression of the first digital person according to the facial expression corresponding to each phoneme in the phoneme sequence;
[0098] The first digital human is controlled to show the facial expression.
[0099] In the embodiment of the present application, the facial expression includes lip shape. When controlling the first digital human to display the first body movement corresponding to the first comment content, the first digital human is also controlled to display the facial expression, so that the first digital human is more consistent with the normal human posture, thereby improving the appreciation and fun of the user's comment.
[0100] Optionally, determining the facial expression of the first digital person according to the facial expression corresponding to each phoneme in the phoneme sequence includes:
[0101] Acquire at least one image frame matching the image information of the first digital human;
[0102] The facial expression corresponding to each phoneme is adjusted according to the facial expression in the image frame to obtain the facial expression of the first digital person.
[0103] The rules may be different for languages of different countries. Because there are different phonemes, the changes in mouth shapes are different. For example, Chinese phonemes: a, o, e, b, p, m, f, etc., that is, these Chinese phonemes corresponding to a comment are mapped to the corresponding mouth shapes. However, because people are different when mapped to digital humans, the mouth shapes cannot be the same for all digital humans. Therefore, here, the facial expressions corresponding to the phonemes are adjusted in combination with at least one image frame that matches the image information of the first digital human to obtain facial expressions that match the image of the digital human.
[0104] The image of the digital human can be determined based on the user's preferences, personality, etc., and the intelligent labeling capability can be used to set labels in the image frames of the reference data set, so that the label can be used to extract image frames that match the image information of the digital human in the reference data set, and generate specific facial expressions of specific comment content of a specific user based on the facial expressions in the image frame and the facial expressions corresponding to each of the above phonemes, so that the digital human can truly have thousands of faces.
[0105] Among them, when one mouth shape changes to another, that is, when the fingerprint sequence of one mouth shape changes to the fingerprint sequence of another mouth shape, in the process of change, the random fluctuations in the current sequence are eliminated each time to make the sequence smoother to reflect its basic trajectory, and combined with a certain model for prediction, such smoothing operations are performed several times in the entire fingerprint sequence to make the fingerprint sequence of the current mouth shape smoothly transition to the fingerprint sequence of the new mouth shape, thereby achieving a smooth transition of the mouth shape. Digital human facial expressions also refer to the mouth shape method.
[0106] Based on the first body movement and facial expression, a model is built to obtain a comment digital person customized by the user with personalized lip shape and facial expression, namely the first digital person. The change speed corresponding to the first body movement can be understood as the action style of the first comment content. The action style corresponding to a fast speech speed is concise and brisk, and the action style corresponding to a slow speech speed is soft and slow.
[0107] Optionally, the method of the embodiment of the present application further includes:
[0108] When receiving a first input from a user to the first digital human, the first digital human is controlled to voice broadcast the first comment content.
[0109] As an implementation method, the corresponding commentary voice is generated by using Text To Speech (TTS) technology.
[0110] In the embodiment of the present application, when the user enters the comment interface, the first digital human only displays the corresponding body movements and facial labels. When the user clicks on one of the digital humans, the video volume is automatically turned down, and the volume of the digital human's voice broadcast is higher than the video volume, thereby realizing the digital human's voice broadcast of the comment content.
[0111] Optionally, the method of the embodiment of the present application further includes:
[0112] Controlling the first digital human to display the first body movement in a first display area of the comment interface, wherein the first display area is a display area corresponding to the first comment content, and different comment contents correspond to different display areas;
[0113] Alternatively, the first digital human is controlled to display the first body movement in a second display area of the comment interface, wherein the second display area is a display area corresponding to at least two comment contents in the comment interface.
[0114] In the present application embodiment, Figure 2 As shown, a digital person can be displayed independently in the display area corresponding to each comment content, or Figure 3 As shown, one display area displays a mixed display of digital people corresponding to multiple comment contents.
[0115] Optionally, the method of the embodiment of the present application further includes:
[0116] When receiving a first input from a user to a first digital person, a second digital person is controlled to display a preset expression and / or a preset action, and the second digital person is a digital person corresponding to the second comment content.
[0117] For example, when users post comments on ball games, they will be asked to choose which team to support. When the user clicks on one of the digital humans to broadcast the comment, other digital humans will show supportive or opposing actions and expressions based on the selected team. The actions and expressions can be set to be unified or set by the user according to personal preference.
[0118] In the embodiment of the present application, when the first digital human receives the first input, the second digital human is controlled to display a preset expression and / or a preset action, thereby achieving the purpose of one digital human broadcasting a comment and the other digital humans performing corresponding interactions.
[0119] The method of the embodiment of the present application can present a digital person with a specific speech speed, specific actions, specific spoken language and expressions, and display it on the client. If it is a follow-up comment under a comment, it can also be presented in the form of face-to-face communication between multiple digital people, which increases the fun of users watching comments.
[0120] like Figure 4As shown, the embodiment of the present application also provides a device for processing comment content, including:
[0121] A first processing module 401 is used to split the first comment content into at least two target keywords;
[0122] A first acquisition module 402 is used to acquire at least one set of data matching each of the target keywords, each set of data includes a target image frame, and the target image frame includes limb position information;
[0123] A first determination module 403 is used to determine a first limb action corresponding to the first comment content according to limb position information of at least two target image frames;
[0124] The second processing module 404 is used to control the first digital human to show the first body movement.
[0125] Optionally, each set of data further includes a target audio file corresponding to the target image frame; and the first determining module includes:
[0126] A first determination submodule is used to determine the target limb starting position and the target limb ending position corresponding to the first comment content according to the limb position information of at least two target image frames;
[0127] A second determination submodule is used to determine the change speed between the starting position of the target limb and the ending position of the target limb according to the duration of the target audio file;
[0128] The third determination submodule is used to obtain a first limb movement corresponding to the first comment content according to the target limb starting position, the target limb ending position and the change speed.
[0129] Optionally, the first determining submodule includes:
[0130] A first acquisition unit is configured to group target image frames in the at least one group of data according to the speech rate level corresponding to the target audio file of the at least one group of data to obtain at least one group of image frames, wherein the speech rate level of the target audio file corresponding to each target image frame in the same group of image frames is the same, and the speech rate levels of the target audio files corresponding to target image frames in different groups of image frames are different;
[0131] A second acquisition unit is used to acquire the limb starting position information and limb ending position information corresponding to each group of image frames according to the limb position information of the target image frame in each group of image frames;
[0132] A first determination unit is used to determine the target limb starting position and target limb ending position corresponding to the first comment content according to the limb starting position information and limb ending position information corresponding to the target group image frame; wherein the speaking speed level corresponding to the target group image frame is the target speaking speed level, and the target speaking speed level is the speaking speed level that appears most times among the speaking speed levels of the target audio file of the at least one group of data.
[0133] Optionally, the second determining submodule includes:
[0134] A second determining unit is used to determine the duration of the limb position change corresponding to each group of image frames according to the duration of the target audio file corresponding to the target image frame in each group of image frames;
[0135] A third determining unit is used to determine the limb position change speed corresponding to each group of image frames according to the limb starting position information, the limb ending position information and the limb position change duration corresponding to each group of image frames;
[0136] The fourth determining unit is used to determine the change speed between the starting position of the target limb and the ending position of the target limb according to the change speed of the limb position corresponding to the target group image frames.
[0137] Optionally, the device of the embodiment of the present application further includes:
[0138] A third processing module, configured to convert the first comment content into a phoneme sequence, wherein the phoneme sequence includes at least one phoneme, and each phoneme corresponds to a facial expression;
[0139] A second determination module, configured to determine the facial expression of the first digital person according to the facial expression corresponding to each phoneme in the phoneme sequence;
[0140] The fourth processing module is used to control the first digital human to display the facial expression.
[0141] Optionally, the second determining module includes:
[0142] A first acquisition submodule, used to acquire at least one image frame matching the image information of the first digital human;
[0143] The adjustment submodule is used to adjust the facial expression corresponding to each phoneme according to the facial expression in the image frame to obtain the facial expression of the first digital person.
[0144] Optionally, the device of the embodiment of the present application further includes:
[0145] The fifth processing module is used to control the first digital human to voice broadcast the first comment content when receiving the first input from the user to the first digital human.
[0146] Optionally, the second processing module is used to:
[0147] Controlling the first digital human to display the first body movement in a first display area of the comment interface, wherein the first display area is a display area corresponding to the first comment content, and different comment contents correspond to different display areas;
[0148] Alternatively, the first digital human is controlled to display the first body movement in a second display area of the comment interface, wherein the second display area is a display area corresponding to at least two comment contents in the comment interface.
[0149] The device of the embodiment of the present application splits the first comment content into at least two target keywords; obtains at least one set of data matching each of the target keywords, each set of data includes a target image frame, and the target image frame includes body position information; determines the first body movement corresponding to the first comment content according to the body position information of at least two target image frames; and controls the first digital human to display the first body movement. Thus, the purpose of presenting user comments through the body movements of the digital human is achieved, and the user's comment content can be presented more vividly through the body movements of the digital human.
[0150] It should be noted that the device is a device corresponding to the above method, and all implementation methods in the above method embodiments are applicable to the embodiments of the device and can achieve the same technical effects.
[0151] like Figure 5 As shown, an embodiment of the present application also provides an electronic device, including: a processor 501, a transceiver 504, a memory 503, and a computer program stored in the memory 503 and executable on the processor 501, and the processor 501 implements the steps of the above-mentioned method for processing comment content when executing the computer program.
[0152] Specifically, the processor 501 is used to split the first comment content into at least two target keywords; obtain at least one set of data matching each of the target keywords, each set of data including a target image frame, and the target image frame including limb position information; determine the first limb movement corresponding to the first comment content based on the limb position information of at least two of the target image frames; and control the first digital human to display the first limb movement.
[0153] Optionally, each set of data further includes a target audio file corresponding to the target image frame; the processor 501 is further configured to:
[0154] Determine the target limb starting position and the target limb ending position corresponding to the first comment content according to the limb position information of at least two of the target image frames;
[0155] Determining a change speed between a starting position of the target limb and an ending position of the target limb according to a duration of the target audio file;
[0156] A first limb movement corresponding to the first comment content is obtained according to the target limb starting position, the target limb ending position and the change speed.
[0157] Optionally, the processor 501 is further configured to:
[0158] According to the speech rate level corresponding to the target audio file of the at least one group of data, the target image frames in the at least one group of data are grouped to obtain at least one group of image frames, wherein the speech rate level of the target audio files corresponding to the target image frames in the same group of image frames is the same, and the speech rate levels of the target audio files corresponding to the target image frames in different groups of image frames are different;
[0159] According to the limb position information of the target image frame in each group of image frames, the limb starting position information and the limb ending position information corresponding to each group of image frames are obtained;
[0160] According to the limb starting position information and limb ending position information corresponding to the target group image frames, the target limb starting position and target limb ending position corresponding to the first comment content are determined; wherein the speaking speed level corresponding to the target group image frames is the target speaking speed level, and the target speaking speed level is the speaking speed level that appears most times among the speaking speed levels of the target audio file of the at least one group of data.
[0161] Optionally, the processor 501 is further configured to:
[0162] Determine the duration of the limb position change corresponding to each group of image frames according to the duration of the target audio file corresponding to the target image frame in each group of image frames;
[0163] Determine the limb position change speed corresponding to each group of image frames according to the limb starting position information, the limb ending position information and the limb position change duration corresponding to each group of image frames;
[0164] The change speed between the target limb starting position and the target limb ending position is determined according to the change speed of the limb position corresponding to the target group image frames.
[0165] Optionally, the processor 501 is further configured to:
[0166] Converting the first comment content into a phoneme sequence, wherein the phoneme sequence includes at least one phoneme, and each phoneme corresponds to a facial expression;
[0167] Determining the facial expression of the first digital person according to the facial expression corresponding to each phoneme in the phoneme sequence;
[0168] The first digital human is controlled to show the facial expression.
[0169] Optionally, the processor 501 is further configured to:
[0170] Acquire at least one image frame matching the image information of the first digital human;
[0171] The facial expression corresponding to each phoneme is adjusted according to the facial expression in the image frame to obtain the facial expression of the first digital person.
[0172] Optionally, the processor 501 is further configured to:
[0173] When receiving a first input from a user to the first digital human, the first digital human is controlled to voice broadcast the first comment content.
[0174] Optionally, the processor 501 is further configured to:
[0175] Controlling the first digital human to display the first body movement in a first display area of the comment interface, wherein the first display area is a display area corresponding to the first comment content, and different comment contents correspond to different display areas;
[0176] Alternatively, the first digital human is controlled to display the first body movement in a second display area of the comment interface, wherein the second display area is a display area corresponding to at least two comment contents in the comment interface.
[0177] It should be noted that in Figure 5In the embodiment, the bus architecture may include any number of interconnected buses and bridges, specifically one or more processors represented by processor 501 and various circuits of memory represented by memory 503 are linked together. The bus architecture may also link together various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. The bus interface 502 provides an interface. The transceiver 504 may be a plurality of components, namely, a transmitter and a transceiver, providing a unit for communicating with various other devices on a transmission medium. For different terminals, the user interface 505 may also be an interface capable of externally connecting or internally connecting required devices, and the connected devices include but are not limited to a keypad, a display, a speaker, a microphone, a joystick, and the like. The processor 501 is responsible for managing the bus architecture and general processing, and the memory 503 may store data used by the processor 501 when performing operations.
[0178] Those skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or may be accomplished by instructing related hardware through a computer program, wherein the computer program includes instructions for executing part or all of the steps of the above method; and the computer program may be stored in a readable storage medium, and the storage medium may be any form of storage medium.
[0179] In addition, the specific embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps of the above-mentioned comment content processing method are implemented and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.
[0180] The embodiment of the present application also provides a computer program product, including computer instructions, which, when executed by a processor, implement the various processes of the above-mentioned comment content processing method embodiment and can achieve the same technical effect. To avoid repetition, they are not described here.
[0181] In the several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
[0182] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may be physically included separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional units.
[0183] The above-mentioned integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform some steps of the sending and receiving methods described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store program codes.
[0184] The above is a preferred embodiment of the present application. It should be pointed out that for ordinary personnel in this technical field, several improvements and modifications can be made without departing from the principles described in the present application. These improvements and modifications are also within the scope of protection of the present application.
Claims
1. A method for processing comment content, characterized in that: include: Split the first comment content into at least two target keywords; Acquire at least one set of data matching each of the target keywords, each set of data comprising a target image frame, wherein the target image frame comprises limb position information; Determining a first limb movement corresponding to the first comment content according to limb position information of at least two target image frames; The first digital human is controlled to show the first body movement.
2. The method according to claim 1, characterized in that Each set of data also includes a target audio file corresponding to the target image frame; determining the first body movement corresponding to the first comment content according to the body position information of at least two target image frames includes: Determine the target limb starting position and the target limb ending position corresponding to the first comment content according to the limb position information of at least two of the target image frames; Determining a change speed between a starting position of the target limb and an ending position of the target limb according to a duration of the target audio file; A first limb movement corresponding to the first comment content is obtained according to the target limb starting position, the target limb ending position and the change speed.
3. The method according to claim 2, characterized in that The step of determining the target limb starting position and the target limb ending position corresponding to the first comment content according to the limb position information of at least two target image frames includes: According to the speech rate level corresponding to the target audio file of the at least one group of data, the target image frames in the at least one group of data are grouped to obtain at least one group of image frames, wherein the speech rate level of the target audio files corresponding to the target image frames in the same group of image frames is the same, and the speech rate levels of the target audio files corresponding to the target image frames in different groups of image frames are different; According to the limb position information of the target image frame in each group of image frames, the limb starting position information and the limb ending position information corresponding to each group of image frames are obtained; According to the limb starting position information and limb ending position information corresponding to the target group image frames, the target limb starting position and target limb ending position corresponding to the first comment content are determined; wherein the speaking speed level corresponding to the target group image frames is the target speaking speed level, and the target speaking speed level is the speaking speed level that appears most times among the speaking speed levels of the target audio file of the at least one group of data.
4. The method according to claim 3, characterized in that: Determining the change speed between the target limb starting position and the target limb ending position according to the duration of the target audio file includes: Determine the duration of the limb position change corresponding to each group of image frames according to the duration of the target audio file corresponding to the target image frame in each group of image frames; Determine the limb position change speed corresponding to each group of image frames according to the limb starting position information, the limb ending position information and the limb position change duration corresponding to each group of image frames; The change speed between the target limb starting position and the target limb ending position is determined according to the change speed of the limb position corresponding to the target group image frames.
5. The method according to claim 1, characterized in that: Also includes: Converting the first comment content into a phoneme sequence, wherein the phoneme sequence includes at least one phoneme, and each phoneme corresponds to a facial expression; Determining the facial expression of the first digital person according to the facial expression corresponding to each phoneme in the phoneme sequence; The first digital human is controlled to show the facial expression.
6. The method according to claim 5, characterized in that The step of determining the facial expression of the first digital person according to the facial expression corresponding to each phoneme in the phoneme sequence includes: Acquire at least one image frame matching the image information of the first digital human; The facial expression corresponding to each phoneme is adjusted according to the facial expression in the image frame to obtain the facial expression of the first digital person.
7. The method according to claim 1, characterized in that Also includes: When receiving a first input from a user to the first digital human, the first digital human is controlled to voice broadcast the first comment content.
8. The method according to claim 1, characterized in that Controlling the first digital human to display the first body movement includes: Controlling the first digital human to display the first body movement in a first display area of the comment interface, wherein the first display area is a display area corresponding to the first comment content, and different comment contents correspond to different display areas; Alternatively, the first digital human is controlled to display the first body movement in a second display area of the comment interface, wherein the second display area is a display area corresponding to at least two comment contents in the comment interface.
9. A device for processing comment content, characterized in that: include: A first processing module, used to split the first comment content into at least two target keywords; A first acquisition module, used to acquire at least one set of data matching each of the target keywords, each set of data including a target image frame, wherein the target image frame includes limb position information; A first determination module, configured to determine a first limb movement corresponding to the first comment content according to limb position information of at least two target image frames; The second processing module is used to control the first digital human to show the first body movement.
10. An electronic device, characterized in that: include: A processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the method for processing comment content as claimed in any one of claims 1 to 8.
11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method for processing comment content according to any one of claims 1 to 8 are implemented.
12. A computer program product, characterized in that The method comprises computer instructions, which, when executed by a processor, implement the steps of the method for processing comment content as claimed in any one of claims 1 to 8.
