Image generation method and device, electronic device and storage medium

Abstract

The present application discloses an image generation method and generation device, electronic device and storage medium, belonging to the field of image processing technology. The image generation method includes: receiving a first input in the case of playing a video; displaying a target interface in response to the first input, wherein the target interface includes moment information and an initial image frame corresponding to the moment information; determining target time period information according to the moment information, wherein the target time period information includes the moment information; determining the line information of the video in the target time period and displaying the line information; determining a key frame in the video according to the initial image frame; and generating a corresponding image file according to the line information and the key frame of the video.

Description

Technical Field

[0001] The present application belongs to the field of image processing technology, and specifically relates to an image generation method and generation device, an electronic device and a storage medium. Background Art

[0002] In the related art, when playing a video, such as watching a movie, a user may be interested in certain scenes and want to record the scene. The current video screenshot can only capture the current video frame, and the user needs to use professional software to edit the video frame to manually add the dialogue in the scene, which is not a good experience. Summary of the invention

[0003] The purpose of the embodiments of the present application is to provide an image generation method and generation device, an electronic device and a storage medium, which can solve the problem that users need to manually record scenes through professional software.

[0004] In a first aspect, an embodiment of the present application provides an image generation method, comprising:

[0005] In the case of playing a video, receiving a first input;

[0006] In response to the first input, display a target interface, wherein the target interface includes time information and an initial image frame corresponding to the time information, and the time information is the time when the first input is received;

[0007] Determine target time period information according to the time information, wherein the target time period information includes the time information;

[0008] Determine the dialogue information of the video within the target time period and display the dialogue information;

[0009] Determining key frames in the video based on the initial image frame;

[0010] Generate corresponding image files based on the dialogue information and the key frames of the video.

[0011] In a second aspect, an embodiment of the present application provides an image generating device, including:

[0012] A receiving module, used for receiving a first input when playing a video;

[0013] A display module, configured to display a target interface in response to the first input, wherein the target interface includes time information and an initial image frame corresponding to the time information, and the time information is the time when the first input is received;

[0014] The receiving module is further used to receive a second input for the time information;

[0015] A determination module, used to determine target time period information according to the time information, wherein the target time period information includes the time information;

[0016] The display module is also used to determine the line information of the video within the target time period and display the line information;

[0017] The determination module is further used to determine a key frame in the video based on the initial image frame;

[0018] The generation module is used to generate corresponding image files according to the dialogue information and the key frames of the video.

[0019] In a third aspect, an embodiment of the present application provides an electronic device, comprising a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the method of the first aspect.

[0020] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method of the first aspect are implemented.

[0021] In a fifth aspect, an embodiment of the present application provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the steps of the method of the first aspect.

[0022] In an embodiment of the present application, when a user plays a video, if a certain scene needs to be recorded, the target interface can be called out through the first input. The target interface displays the scene of the current video, and simultaneously records the moment information when the first input is received, and automatically obtains the dialogue information in the video content near the moment information. The target image is generated through the key frames of the video and the dialogue information, so that the target image includes the pictures of the important scenes in the video, and also records the dialogue content in the scene. There is no need for the user to manually take screenshots and edit them, nor is there any need to use professional software, thereby improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 A flowchart of an image generating method according to an embodiment of the present application is shown;

[0024] Figure 2 One of the interface schematic diagrams of the image generating method according to an embodiment of the present application is shown;

[0025] Figure 3 A second interface schematic diagram of the image generating method according to an embodiment of the present application is shown;

[0026] Figure 4A third interface schematic diagram of the image generating method according to an embodiment of the present application is shown;

[0027] Figure 5 A fourth interface schematic diagram of the image generating method according to an embodiment of the present application is shown;

[0028] Figure 6 A fifth interface schematic diagram of the image generating method according to an embodiment of the present application is shown;

[0029] Figure 7 A structural block diagram of an image generating device according to an embodiment of the present application is shown;

[0030] Figure 8 A structural block diagram of an electronic device according to an embodiment of the present application is shown;

[0031] Fig. 9 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of the present application. DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0033] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0034] The image generation method and generation device, electronic device and storage medium provided in the embodiments of the present application are described in detail below with reference to the accompanying drawings through specific embodiments and their application scenarios.

[0035] In some embodiments of the present application, a method for generating an image is provided. Figure 1 A flow chart of an image generation method according to an embodiment of the present application is shown. Figure 1 As shown, the method includes:

[0036] Step 102, receiving a first input while playing a video;

[0037] Step 104, displaying a target interface in response to the first input;

[0038] In step 104, the target interface includes the time information and the initial image frame of the video under the time information, and the time information is the time when the first input is received;

[0039] Step 106, determining the target time period information according to the time information;

[0040] In step 106, the target time period information includes time information;

[0041] Step 108, determining the speech information of the video within the target time period, and displaying the speech information;

[0042] Step 110, determining a key frame in the video according to the initial image frame;

[0043] Step 112: Generate a corresponding image file according to the dialogue information and the key frames of the video.

[0044] In the embodiment of the present application, when the user plays a video, such as a movie, if there is a scene that he wants to save or share, he can call out the target interface through the first input. The first input can be a trigger input of a screenshot mark in the video playback interface, or a gesture input or an input of pressing a physical button of an electronic device, and the embodiment of the present application does not limit this.

[0045] The target interface can be a "floating window" that covers at least part of the video playback interface, or it can be a completely new interface that replaces the video playback interface. In the target interface, the image frame of the video at the moment the first input is received is displayed and marked as the initial image frame.

[0046] At the same time, when determining the target time period, the user can select a period of time based on the moment information of receiving the first input as the starting point, such as taking the moment information as the starting point, going back X seconds to reach the first moment, and taking the moment information as the starting point, fast forward Y seconds to reach the second moment, and taking the first moment and the second moment as endpoints to determine a period of time, which is the target time period.

[0047] In other implementations, a period of time may be preset, with the time information as the starting point or the end point, and the target time period may be determined based on the time information and the preset time period.

[0048] Furthermore, the dialogue information generated by the played video within the target time period is collected. Specifically, Figure 2 One of the interface schematic diagrams of the image generation method according to an embodiment of the present application is shown, such as Figure 2As shown, the target interface 200 displays a video frame 202, which is specifically the initial image frame, and also displays the line information 204 within the target time period. At the same time, the target interface 200 also displays a time mark 206 corresponding to the moment information. With the time mark 206 as the starting point, a period of time can be selected as the target time period. Figure 2 For example, the current time information is 30 minutes, and the user selects the interval from (30 minutes-15 seconds) to (30 minutes+10 seconds) as the target time period through the time identifier 206 .

[0049] The dialogue information can be obtained by identifying a subtitle file or performing voice recognition on the audio within the target time period. Based on the dialogue information and the key frames of the video, an image file corresponding to the scene that the user wants to keep or share is generated. The key frame of the video can be an initial image frame, or it can receive user input. According to the user input, any image frame is selected as a key frame within the target time period of the played video. Therefore, the final target image includes the screen of the important scene in the video, and also records the conversation content in the scene. There is no need for the user to manually take screenshots and edit, nor is there any need to use professional software, which improves the user experience.

[0050] In some embodiments of the present application, determining the line information of a video within a target time period includes:

[0051] Get N image frames of the video within the target time period, where N is a positive integer;

[0052] Obtain the dialogue texts contained in N image frames respectively;

[0053] Determine the line information based on the dialogue text.

[0054] In the embodiment of the present application, the corresponding dialogue information can be determined by the N image frames of the played video in the target time period. Specifically, the video file is composed of a plurality of continuous frame images, so in the target time period, there are N continuous image frames, and these image frames are played continuously at a set frame rate to obtain the video screen in the target time period.

[0055] Among them, each video frame corresponds to a picture. The dialogue text of N image frames can be determined by collecting the subtitle information in the picture of each video frame. The audio part in the video file can also be segmented according to the length of each video frame in the video, so as to identify the dialogue text corresponding to one or more video frames.

[0056] After identifying these dialogue texts, these dialogue texts are sorted and displayed according to the time sequence in which each dialogue text appears in the video, such as Figure 2As shown, the corresponding lines information 204 displayed in the target interface 200 is obtained.

[0057] This application determines the dialogue text based on N image frames within a target time period, thereby obtaining the dialogue information within a period of time, thereby ensuring that the dialogue information displayed in the image file is complete and coherent, thereby improving the user experience.

[0058] In some embodiments of the present application, determining key frame dialogue information in a video according to an initial image frame includes:

[0059] In case of receiving a second input, determining a key frame according to the second input, wherein the second input is an input for selecting a key frame from the N image frames;

[0060] In case the second input is not received, the initial image frame is determined as a key frame.

[0061] In the embodiment of the present application, the user can manually select the key frame of the video from the N image frames within the target time period and use it as the image content of the generated image file. Specifically, for example, Figure 3 The second interface schematic diagram of the image generation method according to the embodiment of the present application is shown as follows: Figure 3 As shown, a drag bar 302 is displayed in the target interface 300 , and the user can switch the key frames displayed in the current target interface among N video frames by dragging the slider 304 .

[0062] If the user does not manually select a key frame, the initial image frame 306 displayed in the current target interface is used as the key frame by default, thereby generating an image file.

[0063] The embodiment of the present application uses the target frame selected by the user as the key frame, so that the generated image file can better express the scene that the user wants to record, thereby improving the user experience.

[0064] In some embodiments of the present application, the image file is a moving picture file;

[0065] Generate corresponding image files based on the dialogue information and the key frames of the video, including:

[0066] According to the dialogue text, M target frames containing the dialogue text are determined in N image frames, where M is a positive integer less than or equal to N;

[0067] Generate corresponding animated image files according to M target frames;

[0068] Generate image files based on the animated image files and dialogue information.

[0069] In an embodiment of the present application, the image file may be an animated image file, i.e., a dynamic image, such as an image in .GIF format, wherein the dynamic image includes a plurality of frame images, and these frame images are played continuously in sequence to obtain a "dynamic" effect.

[0070] When the user wishes to generate a dynamic image, M target frames containing dialogue text can be determined among N image frames. When all N image frames include dialogue text, that is, M=N, the N image frames can be directly combined into a dynamic image file in time sequence.

[0071] Among them, N image frames can also be extracted or added according to the properties of the dynamic image file size, dynamic image frame rate, etc. set by the user. In the case of extracting frames, if the dialogue texts in two consecutive image frames are the same and the image contents are similar, one of the two similar image frames can be deleted and only one of the two image frames is retained, thereby reducing the volume of the dynamic image finally generated.

[0072] If there are several frames without dialogue text in N image frames, the M image frames containing dialogue text are integrated to obtain a set of image frames containing only dialogue text, and a dynamic image is generated. If the dialogue text in the N image frames is discontinuous, such as dialogue text in the 1st, 2nd and 3rd frames, dialogue text in the 8th, 9th and 10th frames, and no dialogue text in the 4th, 5th, 6th and 7th frames among 10 image frames, the 1st, 2nd and 3rd frames can be combined into one dynamic image, and the 8th, 9th and 10th frames can be combined into a second dynamic image, that is, two dynamic images are generated.

[0073] After generating two dynamic images, the two dynamic images can be spliced ​​end to end in chronological order, and a simple transition effect can be formed at the position of the missing frames in the middle to improve the smoothness of the image.

[0074] The embodiment of the present application generates dynamic images to make image files more expressive, which can better display the scene content that the user wants to save or share, thereby improving the user experience.

[0075] In some embodiments of the present application, generating a corresponding image file according to the dialogue information and the key frames of the video includes:

[0076] receiving adjustment input for line information;

[0077] In response to the adjustment input, the speech information is adjusted to adjust the display mode of the speech information and the position of the speech information in the key frame;

[0078] Generate an image file according to the adjusted dialogue information and key frames;

[0079] The display mode of the dialogue information includes at least one of the following: the font of the dialogue information, the color of the dialogue information, the transparency of the dialogue information, and the size of the dialogue information.

[0080] In the embodiment of the present application, when generating an image file, the recognized dialogue information can also be adjusted through the fourth input, specifically including adjusting the display mode of the dialogue information and adjusting the position of the dialogue information.

[0081] For example, Figure 4 The third interface diagram of the image generation method according to the embodiment of the present application is shown as follows: Figure 4 As shown, in the target interface 400, a key frame 402 is displayed, including dialogue information 404. At the same time, the target interface 400 also displays an adjustment menu 406, through which the display mode of the dialogue information, specifically the font, color, transparency and size of the dialogue information can be adjusted.

[0082] Furthermore, the dialogue information can be displayed in the image file in the form of a "chat bubble". The user can change the display position of the corresponding dialogue information in the key frame, that is, in the final image file, by dragging the chat bubble, thereby achieving flexible setting of the image file and improving the user experience.

[0083] In some embodiments of the present application, generating an image file according to the animated image file and the line information includes:

[0084] Perform image recognition on the key frames to determine the face images contained in the key frames;

[0085] According to the face image, determine the corresponding character name;

[0086] According to the video, identify the character name corresponding to each line in the line information;

[0087] An image file is generated according to the character name, lines and key frames, wherein the lines and the character name are displayed correspondingly in the image file.

[0088] In the embodiment of the present application, if the scene includes characters, that is, the characters having a conversation appear in the picture, the corresponding character names, such as "Little A", "Little B", etc., can be determined by performing face recognition on the facial images of the characters. At the same time, the corresponding clips in the video are further identified to determine which character "said" each sentence of the conversation text.

[0089] Specifically, when identifying a person, facial data can be obtained and then combined with the video information of the video being played. Assuming that the video is a movie, the actor information and role information of the movie can be combined to obtain a corresponding list between the actors and roles. Then, the actors in the scene can be recognized through facial recognition to determine the name of the role played by the actor, which is the name of the character.

[0090] Furthermore, by performing voiceprint recognition or timbre recognition on the audio information corresponding to the video clip, it is possible to identify which character specifically uttered each line of dialogue. It is understandable that the characters corresponding to the dialogue text can also be identified by identifying the facial movements (mouth movements) of each character when the dialogue text is played. This application does not limit this.

[0091] After determining the character names and the lines corresponding to each character name, the user can choose whether to display the character name in the generated image file. If the user does not choose to generate the character name, the image file is directly generated based on the lines and key frames. At this time, the dialogue in the image file can be displayed as follows:

[0092] "Are you on duty today?"

[0093] "I'm not on duty today. Today is Little C's turn."

[0094] If the user chooses to generate a character name, the name of the character who said the line will be displayed before each line in the image file. At this time, the dialogue in the image file can be displayed as follows:

[0095] "Xiao A: Are you on duty today?"

[0096] "Little B: I am not on duty today. Today is Little C's turn."

[0097] This application identifies the names of characters in the scene and displays the character names and lines in correspondence, so that when a user shares an image file, the person being shared can quickly understand the scene content and the logic of the conversation after seeing the image file, thereby improving the user experience.

[0098] In some embodiments of the present application, the number of dialogue statements is P, where P is a positive integer;

[0099] Generate image files based on character names, lines and key frames, including:

[0100] Receiving a third input for a line sentence, wherein the third input is an input for selecting a target line sentence from among the P line sentences;

[0101] In response to the third input, determining Q target dialogue sentences, where Q is a positive integer less than or equal to P;

[0102] Generate an image file based on the character name, Q target lines and key frames.

[0103] In an embodiment of the present application, if there are multiple recognized dialogue sentences, but not all of them are what the user wants to keep, then the user can use a third input to select Q target dialogue sentences that the user wants to keep from all P dialogue sentences, so that the final generated image file only includes the target dialogue sentences that the user wants to keep, thereby improving the user experience.

[0104] Specifically, Figure 5 A fourth interface diagram of the image generation method according to an embodiment of the present application is shown, Figure 5 As shown, in the target interface 500, four lines are displayed, wherein the user can select several of the lines as target lines through touch input, and the box mark next to the selected target line is filled with black, thereby indicating that the line is the target line.

[0105] The embodiment of the present application allows the user to select a target dialogue sentence that the user wants to retain from multiple identified dialogue sentences, so that the generated image file can better express the scene that the user wants to save or share, thereby improving the user experience.

[0106] In some embodiments of the present application, before generating an image file according to the character name, the lines and the key frames, the method further includes:

[0107] When the number of character names is X, X character names are displayed, where X is an integer greater than 1;

[0108] receiving a fourth input of the X person names, wherein the fourth input is an input for selecting a target person name from the X person names;

[0109] In response to the fourth input, determining Y target person names, where Y is a positive integer;

[0110] Generate image files based on character names, lines and key frames, including:

[0111] Generate image files based on the target character name, lines and key frames.

[0112] In the embodiment of the present application, if the number of recognized character names is multiple, specifically, X character names are recognized, then the X character names are displayed in the form of a list. At the same time, a fourth input from the user is received, and according to the fourth input from the user, Y target character names that the user is concerned about are selected from the X character names.

[0113] It can be understood that the number of target person names can be one or more.

[0114] After the user selects the target character's name, the lines corresponding to the target character's name are selected from all the lines, and the final image file is generated based on these lines, the target character corresponding to the target character name, and the key frames. Therefore, the final image file only contains the characters that the user is concerned about and the lines of the characters that the user is concerned about, making the image file more in line with the scene that the user wants to keep or share.

[0115] In some embodiments of the present application, an image file is generated according to a character name, a line sentence, and a key frame, including:

[0116] According to the face image, determine the number of people included in the key frame;

[0117] In the case where there are multiple persons, intercepting the person image corresponding to the face image;

[0118] Add the character image to the key frame to obtain the character frame;

[0119] An image file is generated according to the character name, dialogue sentences and character frames, wherein in the image file, the character name and the character image are displayed correspondingly, and the dialogue sentences and the character image are displayed correspondingly.

[0120] In the embodiment of the present application, if there are multiple face images, it means that there are multiple characters in the scene currently selected by the user. However, these characters participating in the conversation may not appear in the same image frame at the same time. At this time, you can enter the "Fun Multi-person Screenshot" mode.

[0121] Specifically, first, character images corresponding to multiple face images are captured, wherein if a full-body image of a character appears in the video, the full-body image is preferentially selected, and if no full-body image appears, a half-body image or a face image is captured.

[0122] After capturing the character images, the character images are merged into the same image according to the dialogue object, that is, the characters are synthesized into key frames. For example, Figure 6 A fifth interface diagram of the image generation method according to an embodiment of the present application is shown, Figure 6 As shown, there are two characters, namely Little A and Little B. The character images of Little A and Little B are cut out from different image frames and added to the key frame 602, creating a feeling that Little A and Little B are "face to face".

[0123] Furthermore, in the generated image file, the character images of Xiao A and Xiao B are marked with their names, and the lines of Xiao A and Xiao B are displayed in the form of dialogue bubbles, so that readers can directly recognize the lines of Xiao A and Xiao B respectively, thereby improving the user experience.

[0124] In some embodiments of the present application, an image generating device is provided. Figure 7 FIG. 4 shows a structural block diagram of an image generating device according to an embodiment of the present application. Figure 7 As shown, the image generating device 700 includes:

[0125] The receiving module 702 is used to receive a first input when playing a video;

[0126] A display module 704 is used to display a target interface in response to the first input, wherein the target interface includes time information and an initial image frame corresponding to the time information, and the time information is the time when the first input is received;

[0127] A determination module 706, configured to determine target time period information according to the time information, wherein the target time period information includes the time information;

[0128] The display module 704 is also used to determine the speech information of the video within the target time period and display the speech information;

[0129] The determination module 706 is further configured to determine a key frame in the video based on the initial image frame;

[0130] The generation module 708 is used to generate corresponding image files according to the dialogue information and the key frames of the video.

[0131] In an embodiment of the present application, when a user plays a video, if a certain scene needs to be recorded, the target interface can be called out through the first input. The target interface displays the scene of the current video, and simultaneously records the moment information when the first input is received, and automatically obtains the dialogue information in the video content near the moment information. The target image is generated through the key frames of the video and the dialogue information, so that the target image includes the pictures of the important scenes in the video, and also records the dialogue content in the scene. There is no need for the user to manually take screenshots and edit them, nor is there any need to use professional software, thereby improving the user experience.

[0132] In some embodiments of the present application, the image generating device further includes:

[0133] An acquisition module is used to acquire N image frames of a video within a target time period, where N is a positive integer; and acquire the dialogue texts contained in the N image frames respectively;

[0134] The generation module is also used to determine the line information based on the dialogue text.

[0135] This application determines the dialogue text based on N image frames within a target time period, thereby obtaining the dialogue information within a period of time, thereby ensuring that the dialogue information displayed in the image file is complete and coherent, thereby improving the user experience.

[0136] In some embodiments of the present application, the determination module is further used to determine the key frame based on the second input when a second input is received, wherein the second input is an input for selecting a key frame from N image frames; when the second input is not received, the initial image frame is determined as the key frame.

[0137] The embodiment of the present application uses the target frame selected by the user as the key frame, so that the generated image file can better express the scene that the user wants to record, thereby improving the user experience.

[0138] In some embodiments of the present application, the image file is a moving picture file;

[0139] The determination module is further used to determine M target frames containing the dialogue text among the N image frames according to the dialogue text, where M is a positive integer less than or equal to N;

[0140] The generation module is also used to generate corresponding animated image files according to the M target frames; and to generate image files according to the animated image files and dialogue information.

[0141] The embodiment of the present application generates dynamic images to make image files more expressive, which can better display the scene content that the user wants to save or share, thereby improving the user experience.

[0142] In some embodiments of the present application, the receiving module is further used to receive adjustment input for the line information;

[0143] The image generating device also includes:

[0144] An adjustment module, for adjusting the line information in response to the adjustment input, so as to adjust the display mode of the line information and adjust the position of the line information in the key frame;

[0145] The generation module is also used to generate an image file according to the adjusted dialogue information and key frames;

[0146] The display mode of the dialogue information includes at least one of the following: the font of the dialogue information, the color of the dialogue information, the transparency of the dialogue information, and the size of the dialogue information.

[0147] The dialogue information of the embodiment of the present application can be displayed in the image file in the form of a "chat bubble". The user can change the display position of the corresponding dialogue information in the key frame, that is, in the final image file, by dragging the chat bubble, thereby achieving flexible setting of the image file and improving the user experience.

[0148] In some embodiments of the present application, the image generating device further includes:

[0149] A recognition module is used to perform image recognition on the key frame and determine the face image contained in the key frame;

[0150] The determination module is also used to determine the corresponding character name based on the face image;

[0151] The recognition module is also used to recognize the character name corresponding to each line sentence in the line information according to the video;

[0152] The generation module is also used to generate an image file according to the character name, lines and key frames, wherein the lines and character names are displayed correspondingly in the image file.

[0153] This application identifies the names of characters in the scene and displays the character names and lines in correspondence, so that when a user shares an image file, the person being shared can quickly understand the scene content and the logic of the conversation after seeing the image file, thereby improving the user experience.

[0154] In some embodiments of the present application, the number of dialogue statements is P, where P is a positive integer;

[0155] The receiving module is further used to receive a third input for a line sentence, wherein the third input is an input for selecting a target line sentence from the P line sentences;

[0156] The determination module is further configured to determine Q target dialogue sentences in response to a third input, wherein Q is a positive integer less than or equal to P;

[0157] The generation module is also used to generate image files according to the character name, Q target lines and key frames.

[0158] In an embodiment of the present application, if there are multiple recognized dialogue sentences, but not all of them are what the user wants to keep, then the user can use a third input to select Q target dialogue sentences that the user wants to keep from all P dialogue sentences, so that the final generated image file only includes the target dialogue sentences that the user wants to keep, thereby improving the user experience.

[0159] In some embodiments of the present application, the display module is further configured to display X names of characters when the number of the names of characters is X, where X is an integer greater than 1;

[0160] The receiving module is further used to receive a fourth input of the X person names, wherein the fourth input is an input for selecting a target person name from the X person names;

[0161] The determination module is further used to determine Y target person names in response to the fourth input, where Y is a positive integer;

[0162] The generation module is also used to generate image files according to the target character name, lines and key frames.

[0163] In the embodiment of the present application, when the number of character names is X, X character names are displayed, where X is an integer greater than 1;

[0164] receiving a fourth input of the X person names, wherein the fourth input is an input for selecting a target person name from the X person names;

[0165] In response to the fourth input, determining Y target person names, where Y is a positive integer;

[0166] Generate image files based on character names, lines and key frames, including:

[0167] Generate image files based on the target character name, lines and key frames.

[0168] In some embodiments of the present application, the determination module is further used to determine the number of characters included in the key frame based on the face image;

[0169] The image generating device further comprises:

[0170] A capture module, used for capturing the person image corresponding to the face image when there are multiple persons;

[0171] A synthesis module, used for adding a character image to a key frame to obtain a character frame;

[0172] The generation module is also used to generate an image file according to the character name, dialogue sentences and character frames, wherein in the image file, the character name and the character image are displayed correspondingly, and the dialogue sentences and the character image are displayed correspondingly.

[0173] After capturing the character images, the character images are merged into the same image according to the dialogue object. For example, if there are two characters, namely Xiao A and Xiao B, the character images of Xiao A and Xiao B are captured in different image frames and added to the key frame to create a feeling of Xiao A and Xiao B "face to face". Furthermore, in the generated image file, the character images of Xiao A and Xiao B are marked with their names, and the lines of Xiao A and Xiao B are displayed in the form of dialogue bubbles, so that readers can directly recognize the lines of Xiao A and Xiao B, respectively, to improve the user experience.

[0174] The image processing device in the embodiment of the present application can be a device, or a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. Exemplarily, the mobile electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, an in-vehicle electronic device, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), etc. The non-mobile electronic device can be a server, a network attached storage (NAS), a personal computer (PC), a television (TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0175] The image processing device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0176] The image processing device provided in the embodiment of the present application can implement each process implemented in the above method embodiment, and will not be described again here to avoid repetition.

[0177] Optionally, the embodiment of the present application further provides an electronic device 800, Figure 8 A structural block diagram of an electronic device according to an embodiment of the present application is shown. Figure 8 As shown, it includes a processor 802, a memory 804, and a program or instruction stored in the memory 804 and executable on the processor 802. When the program or instruction is executed by the processor 802, each process of the above-mentioned method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0178] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0179] Fig. 9 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of the present application.

[0180] The electronic device 900 includes but is not limited to: a radio frequency unit 901, a network module 902, an audio output unit 903, an input unit 904, a sensor 905, a display unit 906, a user input unit 907, an interface unit 908, a memory 909, and a processor 910 and other components.

[0181] Those skilled in the art will appreciate that the electronic device 900 may also include a power source (such as a battery) for supplying power to each component, and the power source may be logically connected to the processor 910 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system. Fig. 9 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.

[0182] The user input unit 907 is used to receive a first input when playing a video;

[0183] The processor 910 is configured to display a target interface in response to the first input, wherein the target interface includes time information and an initial image frame corresponding to the time information, and the time information is the time when the first input is received;

[0184] Processor 910 is also used to determine target time period information based on moment information, wherein the target time period information includes moment information; determine the line information of the video within the target time period and display the line information; determine the key frame in the video based on the initial image frame; and generate a corresponding image file based on the line information and the key frame of the video.

[0185] Optionally, the processor 910 is further configured to obtain N image frames of the video within a target time period, where N is a positive integer; obtain dialogue texts contained in the N image frames respectively; and determine the dialogue information according to the dialogue texts.

[0186] Optionally, the processor 910 is further used to determine the key frame according to the second input when a second input is received, wherein the second input is an input for selecting a key frame from N image frames; when the second input is not received, the initial image frame is determined as the key frame.

[0187] Optionally, the image file is a dynamic image file; the processor 910 is also used to determine M target frames containing dialogue text in N image frames based on the dialogue text, where M is a positive integer less than or equal to N; generate corresponding dynamic image files based on the M target frames; and generate an image file based on the dynamic image file and dialogue information.

[0188] An image file is generated according to the adjusted dialogue information and key frames; wherein the display mode of the dialogue information includes at least one of the following: the font of the dialogue information, the color of the dialogue information, the transparency of the dialogue information, and the size of the dialogue information.

[0189] Optionally, processor 910 is also used to perform image recognition on key frames to determine facial images contained in the key frames; determine the corresponding character names based on the facial images; identify the character names corresponding to each dialogue sentence in the dialogue information based on the video; generate an image file based on the character name, dialogue sentence and key frames, wherein in the image file, the dialogue sentence and the character name are displayed correspondingly.

[0190] Optionally, the number of lines is P, where P is a positive integer; the user input unit 907 is further used to receive a third input for the lines, where the third input is an input for selecting a target line from the P lines;

[0191] Processor 910 is further configured to determine Q target dialogue sentences in response to a third input, where Q is a positive integer less than or equal to P; and generate an image file based on the character name, the Q target dialogue sentences and the key frames.

[0192] Optionally, the display unit 906 is configured to display X names of characters when the number of the names of characters is X, where X is an integer greater than 1;

[0193] The user input unit 907 is further used to receive a fourth input of the X person names, wherein the fourth input is an input for selecting a target person name from the X person names;

[0194] The processor 910 is further configured to determine Y target character names in response to a fourth input, where Y is a positive integer; and generate an image file according to the target character names, lines and key frames.

[0195] Optionally, processor 910 is also used to determine the number of characters included in the key frame based on the facial image; when there are multiple characters, capture the character image corresponding to the facial image; add the character image to the key frame to obtain a character frame; generate an image file based on the character name, dialogue sentence and character frame, wherein in the image file, the character name and the character image are displayed correspondingly, and the dialogue sentence and the character image are displayed correspondingly.

[0196] In an embodiment of the present application, when a user plays a video, if a certain scene needs to be recorded, the target interface can be called out through the first input. The target interface displays the scene of the current video, and simultaneously records the moment information when the first input is received, and automatically obtains the dialogue information in the video content near the moment information. The target image is generated through the key frames of the video and the dialogue information, so that the target image includes the pictures of the important scenes in the video, and also records the dialogue content in the scene. There is no need for the user to manually take screenshots and edit them, nor is there any need to use professional software, thereby improving the user experience.

[0197] It should be understood that in an embodiment of the present application, the input unit 904 may include a graphics processor (GPU) 9041 and a microphone 9042, and the graphics processor 9041 processes image data of a static picture or video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode.

[0198] The display unit 906 may include a display panel 9061, which may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 907 includes a touch panel 9071 and other input devices 9072. The touch panel 9071 is also called a touch screen. The touch panel 9071 may include two parts: a touch detection device and a touch controller. Other input devices 9072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which are not repeated here. The memory 909 may be used to store software programs and various data, including but not limited to applications and operating systems. The processor 910 may integrate an application processor and a modem processor, wherein the application processor mainly processes an operating system, a user interface, and applications, etc., and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 910.

[0199] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the above-mentioned method embodiment are implemented and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0200] The processor is a processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0201] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0202] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0203] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0204] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0205] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. An image generation method, characterized in that: include: In the case of playing a video, receiving a first input; In response to the first input, display a target interface, wherein the target interface includes time information and an initial image frame corresponding to the time information, and the time information is the time when the first input is received; determining target time period information according to the time information, wherein the target time period information includes the time information; Determining the speech information of the video within the target time period, and displaying the speech information; Determining a key frame in the video according to the initial image frame; Generate a corresponding image file according to the dialogue information and the key frame; The image file is a dynamic image file, including N image frames within the target time period, the N image frames including dialogue texts, wherein N is a positive integer, and the dialogue information is determined according to the dialogue texts; The generating a corresponding image file according to the dialogue information and the key frame of the video includes: According to the dialogue text, among the N image frames, M target frames containing the dialogue text are determined as key frames of the video, where M is a positive integer less than or equal to N; Generate corresponding animated image files according to the M target frames; Generate the image file according to the animated image file and the dialogue information; The step of generating the image file according to the animated image file and the dialogue information includes: Performing image recognition on the key frame to determine a face image contained in the key frame; Determine the corresponding character name according to the face image; Performing voiceprint recognition or timbre recognition on the audio information corresponding to the video clip according to the video, and identifying the character name corresponding to each line sentence in the line information; The image file is generated according to the character name, the dialogue sentence and the key frame, wherein in the image file, the dialogue sentence and the character name are displayed correspondingly.

2. The image generation method according to claim 1, characterized in that: The determining the dialogue information of the video within the target time period includes: Acquire N image frames of the video within the target time period; Respectively obtaining the dialogue texts contained in the N image frames; The dialogue information is determined according to the dialogue text.

3. The image generation method according to claim 2, characterized in that: Determining a key frame in the video according to the initial image frame includes: determining the key frame according to the second input when a second input is received, wherein the second input is an input for selecting the key frame among the N image frames; In case the second input is not received, the initial image frame is determined as the key frame.

4. The image generation method according to claim 1, characterized in that: The number of the lines is P, where P is a positive integer; The step of generating the image file according to the character name, the dialogue sentence and the key frame comprises: Receiving a third input of the speech sentence, wherein the third input is an input for selecting a target speech sentence from among the P speech sentences; In response to the third input, determining Q target dialogue sentences, where Q is a positive integer less than or equal to P; The image file is generated according to the character name, the Q target dialogue sentences and the key frames.

5. The image generation method according to claim 1, characterized in that: Before generating the image file according to the character name, the dialogue sentence and the key frame, the method further includes: When the number of the character names is X, display X character names, where X is an integer greater than 1; receiving a fourth input of the X person names, wherein the fourth input is an input for selecting a target person name from the X person names; In response to the fourth input, determining Y target person names, where Y is a positive integer; The step of generating the image file according to the character name, the dialogue sentence and the key frame comprises: The image file is generated according to the target character name, the dialogue sentence and the key frame.

6. The image generation method according to claim 1, characterized in that: The step of generating the image file according to the character name, the dialogue sentence and the key frame comprises: Determining the number of people included in the key frame according to the face image; In the case where there are multiple persons, intercepting person images corresponding to the face images; Adding the character image to the key frame to obtain a character frame; The image file is generated according to the character name, the dialogue sentence and the character frame, wherein in the image file, the character name and the character image are displayed correspondingly, and the dialogue sentence and the character image are displayed correspondingly.

7. An image generating device, characterized in that: include: A receiving module, used for receiving a first input when playing a video; A display module, configured to display a target interface in response to the first input, wherein the target interface includes time information and an initial image frame corresponding to the time information, and the time information is the time when the first input is received; a determination module, configured to determine target time period information according to the time information, wherein the target time period information includes the time information; The display module is further used to determine the speech information of the video within the target time period and display the speech information; The determination module is further used to determine a key frame in the video according to the initial image frame; A generating module, used for generating a corresponding image file according to the line information and the key frame; The image file is a dynamic image file, including N image frames within the target time period, the N image frames including dialogue texts, wherein N is a positive integer, and the dialogue information is determined according to the dialogue texts; The determination module is further used to determine, according to the dialogue text, M target frames containing the dialogue text among the N image frames as key frames of the video, where M is a positive integer less than or equal to N; The generating module is further used to generate corresponding animated image files according to the M target frames; Generate the image file according to the animated image file and the dialogue information; The image generating device further comprises: A recognition module, used to perform image recognition on the key frame to determine the face image contained in the key frame; The determination module is further used to determine the corresponding character name according to the face image; The recognition module is further used to perform voiceprint recognition or timbre recognition on the audio information corresponding to the video clip according to the video, and recognize the character name corresponding to each line sentence in the line information; The generating module is further used to generate the image file according to the character name, the dialogue sentence and the key frame, wherein in the image file, the dialogue sentence and the character name are displayed correspondingly.

8. An electronic device, characterized in that: The method comprises a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the method according to any one of claims 1 to 6.

9. A readable storage medium, characterized in that: The readable storage medium stores a program or instruction, and when the program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images