Video cover generation method and device, electronic equipment and computer readable medium

Abstract

The invention relates to a video cover generation method and device, electronic equipment and a computer readable storage medium. The method comprises the following steps: reading a current video frame from a video file; performing face image detection on the current video frame, and setting a face image as a figure subject picture of a cover; recognizing face key points of the character main body picture, calculating the character main body picture of which the side face angle is smaller than an angle threshold value according to the face key points, and setting the character main body picture as a cover main body picture; identifying a text of the cover main body picture and a rectangular surrounding frame of the text, and setting the cover main body picture, in which the rectangular surrounding frame is not overlapped with the character main body picture, as a video cover picture; outputting the video cover picture as a cover picture of the video file; according to the technical scheme, intelligent judgment of the high-quality video frame can be realized, so that the available video cover picture is quickly output, and the efficiency of generating the video cover picture is greatly improved.

Description

Technical Field

[0001] The present application relates to the field of video processing technology, and in particular to a method, device, electronic device and computer-readable storage medium for generating a video cover. Background Art

[0002] With the rapid development of video production and dissemination technology, short videos are being accepted by more and more users; short videos are low-cost and fast to produce, and a large number of short videos can be produced in a short period of time. For example, short dramas as a new form of video entertainment have been rapidly developed in short video promotion platforms. For example, in video live broadcast platforms, in order to attract users to watch the live broadcast of the live broadcast room, the live broadcast highlights of the live broadcast room are usually displayed on the homepage, thereby attracting users to click in to watch the live broadcast.

[0003] At present, the method of making cover pictures of short videos generally selects a certain frame in the short video as the video cover to attract viewers to click and watch. Usually, the video frames are watched frame by frame, and then a certain video frame is captured as the cover of the short video. However, for the production needs of a large number of short drama videos, live broadcast highlights, etc., the method of watching frame by frame and then generating the cover is inefficient and difficult to adapt to actual usage needs. Summary of the invention

[0004] Based on this, it is necessary to provide a video cover generation method, device, electronic device and computer-readable storage medium, which can quickly generate a cover picture of a short video.

[0005] A video cover generation method, comprising:

[0006] Read the current video frame from the video file;

[0007] Performing face image detection on the current video frame, and setting the face image as the main character image of the cover;

[0008] Identify key points of the face of the main character picture, and calculate the main character picture with a side face angle less than an angle threshold according to the key points of the face, and set it as the main character picture of the cover;

[0009] Identify the text of the cover main body picture and its rectangular enclosing frame, and set the cover main body picture that does not overlap with the rectangular enclosing frame and the character main body picture as the video cover picture;

[0010] The video cover picture is output as the cover picture of the video file.

[0011] In one embodiment, performing face image detection on the current video frame and setting the face image as the main subject image of the cover includes:

[0012] Use a face detection algorithm to detect whether there is a face image in the current video frame;

[0013] If it does not exist, read the next frame from the video file as the current video frame and perform face image detection until a face image is detected.

[0014] In one embodiment, identifying key points of the face of the main character picture, and calculating the main character picture whose side face angle is less than an angle threshold according to the key points of the face as the cover main character picture, includes:

[0015] Using a facial key point positioning algorithm to identify the facial key points of the main character image, and calculating the side face angle of the main character image according to a facial key point model;

[0016] If the side face angle of the person's face is less than the angle threshold, the person's main body picture is set as the cover main body picture.

[0017] In one embodiment, calculating the side face angle of a person's face in a subject image according to a face key point model includes:

[0018] According to the facial key point model, the left mouth corner point (x l ,y l ), midpoint of lips (x m ,y m ), right corner of mouth (x r ,y r );

[0019] Calculate the distance between the left corner of the mouth and the midpoint of the lip and the distance between the right corner of the mouth and the midpoint of the lip

[0020] Calculate the distance ratio between the left and right corners of the mouth and the center of the lips (ratio = dl / dr);

[0021] If the side face angle of the person's face is less than the angle threshold, the main body picture of the person is set as the cover main body picture, including: if the distance ratio ratio is within a preset ratio range, it is determined that the main body picture of the person is a small-angle side face, and the main body picture of the person is set as the cover main body picture.

[0022] In one embodiment, identifying the text of the cover main body picture and its rectangular enclosing frame, and setting the cover main body picture that does not overlap the rectangular enclosing frame with the character main body picture as the video cover picture, includes:

[0023] Use the OCR text detection algorithm to detect whether there is text in the current video frame; if there is no text, output the cover main body image as the video cover image;

[0024] If there is text, further detect whether the rectangular enclosing frame of each text overlaps with the main image of the character; if there is no overlap, set the cover main image as the video cover image.

[0025] In one embodiment, the video cover generation method further includes:

[0026] If the side face angle of the person's face is greater than or equal to the angle threshold, the main image of the person is added to the candidate image set, and the next frame is read from the video file as the current video frame;

[0027] If the rectangular enclosing frame overlaps with the main image of the person, the cover main image is added to the candidate image set, and the next frame is read from the video file as the current video frame.

[0028] In one embodiment, the video cover generation method further includes:

[0029] Reading video frames in the candidate image set;

[0030] Calculate the ratio of the distances between the left corner of the mouth and the right corner of the mouth of the face key point model of each video frame and the center point of the lips, and convert the ratio into a first quality score;

[0031] Calculate the overlapping area between all text enclosing rectangular boxes and the subject face rectangular box of each video frame, record the maximum overlapping area value, and convert the maximum overlapping area value into a second quality score;

[0032] A comprehensive score of each video frame is calculated according to the first quality score and the second quality score, and a video frame with the largest comprehensive score is selected from all candidate image sets as the video cover picture.

[0033] In one embodiment, the calculation formula for the first quality score is:

[0034]

[0035] Among them, score1 is the first quality score, and ratio is the ratio of the distance between the left corner of the mouth and the right corner of the mouth to the center of the lips;

[0036] The calculation formula of the second quality score is:

[0037] score2=1-iou_max

[0038] Among them, score2 is the second quality score, and iou_max is the maximum overlapping area value;

[0039] The calculation formula of the comprehensive score is:

[0040] score0=0.5×score1+0.5×score2

[0041] Among them, score0 is the comprehensive score.

[0042] A video cover generating device, comprising:

[0043] Video frame reading module, used to read the current video frame from the video file;

[0044] A face detection module is used to detect a face image of the current video frame and set the face image as the main body picture of the person on the cover;

[0045] A side face angle recognition module is used to recognize key points of the face of the main character picture, and calculate the main character picture with a side face angle less than an angle threshold according to the key points of the face, and set it as the main character picture of the cover;

[0046] A text occlusion recognition module, used to recognize the text and its rectangular enclosing frame of the cover main body picture, and set the cover main body picture where the rectangular enclosing frame does not overlap with the character main body picture as the video cover picture;

[0047] The cover output module is used to output the video cover picture as the cover picture of the video file.

[0048] An electronic device, comprising:

[0049] one or more processors;

[0050] Memory;

[0051] One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the steps of the video cover generation method.

[0052] A computer-readable storage medium stores at least one instruction, at least one program, a code set or an instruction set, wherein the at least one instruction, the at least one program, the code set or the instruction set is loaded by the processor and executes the steps of the video cover generation method.

[0053] The technical solution of the present application reads video frames from a video file to perform face image detection to obtain a main body picture of a person, identifies key points of the face and calculates the main body picture of the person whose side face angle is less than an angle threshold to obtain a cover main body picture, identifies the rectangular bounding box of the text, and compares it with the main body picture of the person to obtain a non-overlapping cover main body picture as a video cover picture for output; the technical solution processes the video frames of the video file frame by frame, and successively performs logical processing such as face detection, key point positioning, and text occlusion recognition. When all the above conditions are met, the corresponding video cover picture is output and associated with the cover picture of the video file; thereby, intelligent judgment of high-quality video frames can be achieved, and then the available video cover picture can be quickly output, which greatly improves the efficiency of generating video cover pictures.

[0054] Furthermore, when processing video frames one by one, logical judgments of face detection, key point positioning, and text occlusion recognition are performed in sequence for each video frame. When any video frame meets the conditions after logical judgment, the corresponding video cover picture is output. The video cover picture can be output quickly and efficiently, which is particularly suitable for scenarios such as short drama video promotion, live broadcast room cover display, etc. that need to process a large number of video files and quickly generate cover pictures, so that the cover picture of the short video can be obtained quickly and efficiently.

[0055] Furthermore, based on the candidate image set, after traversing and reading the video file and making a judgment, when the video cover picture cannot be obtained, the video frame can be read from the candidate image set, and the side face angle size and the overlap size of the face obstructed by the text can be converted into corresponding quality scores, and then the video cover picture is calculated based on the comprehensive quality score. This ensures that a suitable video cover picture can still be generated after a frame-by-frame processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 is a schematic diagram of an example short video platform;

[0057] Figure 2 is a flow chart of a method for generating a video cover in one embodiment;

[0058] Figure 3 is a flow chart of a method for generating a video cover according to another embodiment;

[0059] Figure 4 is a flow chart of generating a video cover based on a candidate image set according to an embodiment;

[0060] Figure 5 This is a schematic diagram of an example video cover screen;

[0061] Figure 6 is a schematic diagram of the structure of a video cover generating device according to an embodiment;

[0062] Figure 7 is a schematic diagram of an example network video system;

[0063] Figure 8 is a block diagram of an example electronic device. DETAILED DESCRIPTION

[0064] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0065] In the embodiments of the present application, "at least one" means one or more, and "multiple" means two or more. For example, multiple objects refer to two or more objects. "Include" or "comprising" and other similar words mean that the information appearing before "include" or "comprising" covers the information listed after "include" or "comprising" and its equivalents, and does not exclude other information. The "and / or" mentioned in the embodiments of the present application indicates that there can be three kinds of relationships. The character " / " generally indicates that the objects associated before and after are in an "or" relationship.

[0066] The technical solution provided in the embodiments of the present application can be applied to Figure 1 In the application scenario of the method related to this application shown in FIG. Figure 1 It is a schematic diagram of an example short video platform, which may include a server and a client. The client communicates data with the server through a network, so that the short video generated by the server can be played on the client, and the video cover screen generated based on the technical solution of the present application can be played on the client; wherein, for the client, its terminal device can be but not limited to various personal computers, laptops, smart phones and tablets, and the live broadcast server can be implemented with an independent server or a server cluster consisting of multiple servers.

[0067] In order to facilitate users to quickly and efficiently obtain the cover image of short videos, the technical solution of the embodiment of the present application performs logical processing through algorithms such as face detection, key point positioning, and text occlusion recognition, to achieve intelligent judgment of high-quality video frames, and then quickly output available video cover images, which is suitable for scenarios such as short drama video promotion and live broadcast room cover display.

[0068] refer to Figure 2 As shown, Figure 2 The flowchart of a method for generating a video cover in an embodiment includes the following steps:

[0069] Step S101, reading the current video frame from the video file.

[0070] In this step, for the video file of the short video, read one video frame frame by frame as the current video frame, and then perform the cover generation process on it.

[0071] Step S102, perform face image detection on the current video frame, and set the face image as the main body picture of the cover.

[0072] In this step, a face detection algorithm can be used to perform face image detection on the current video frame. When a face image exists in the current video frame, set the detected face image as the main body picture of the cover and perform the processing of the next step.

[0073] Exemplarily, after detecting the face image, if there is only one face image, then this face image is the main body of the picture; if multiple face images are detected, then sort according to the area of the face detection bounding rectangle and select the face image with the largest area as the main body picture of the person. The bounding box of the main body picture of the person is denoted as

[0074] Step S103, identify the face key points of the main body picture of the person, and set the main body picture of the person with a face side face angle less than the angle threshold calculated according to the face key points as the main cover picture.

[0075] In this step, a face key point localization algorithm can be used to identify the face key points of the main body picture of the person. Through the detection of the face key points, the face side face angle of the main body picture of the person can be calculated in combination with the face key point model. If the face side face angle is less than the angle threshold, then this main body picture of the person can be set as the main cover picture and perform the processing of the next step.

[0076] In one embodiment, for the face key point model, taking the mainstream 68-point face key points as an example, the face side face angle can be calculated according to the left mouth corner point (the 49th point), the midpoint of the lips (the 52nd point), and the right mouth corner point (the 55th point) of the face key point model.

[0077] Specifically, when calculating the face side face angle of the main body picture of the person according to the face key point model, calculate the distance between the left mouth corner point and the midpoint of the lips and the distance between the right mouth corner point and the midpoint of the lips Then calculate the distance ratio ratio = dl / dr between the left and right mouth corner points and the center point of the lips; compare the size relationship between the face side face angle and the angle threshold through the distance ratio ratio. If the distance ratio ratio is within the preset ratio range, such as 0.33 < ratio < 3, then it can be determined that the main body picture of the person is a small-angle side face, and thus this main body picture of the person is set as the main cover picture.

[0078] S104, identifying the text of the cover main body picture and its rectangular enclosing frame, and setting the cover main body picture that does not overlap the rectangular enclosing frame with the character main body picture as the video cover picture.

[0079] In this step, the OCR (Optical Character Recognition) text detection algorithm can be used to detect the text in the current video frame, and by comparing the rectangular bounding box of the text, if it does not overlap with the main body of the character, the cover main body picture can be set as the video cover picture, and the next step of processing is executed.

[0080] Specifically, OCR is used to identify whether there is text on the cover body screen. If there is text, all the bounding rectangles of the detected text are recorded. For example, the i-th text bounding rectangle is recorded as Then determine the bounding rectangle of each text Whether it can be used as the video cover image is determined by whether there is an overlapping area with the bounding box bbox of the main image of the character (i.e. IOU>0).

[0081] S105: output the video cover picture as the cover picture of the video file.

[0082] In this step, the video cover screen obtained after processing in the previous step is used to associate it with the video file of the short video and use it as its cover screen for output, so that a cover screen can be quickly generated for the video file.

[0083] As in the technical solution of the above-mentioned embodiment, the video frames of the video file are read frame by frame for processing, and logical processing such as face detection, key point positioning, and text occlusion recognition are successively carried out. When all the above conditions are met, the corresponding video cover picture is output and associated with the cover picture of the video file; thereby, intelligent judgment of high-quality video frames can be achieved, and then the available video cover pictures can be quickly output, which greatly improves the efficiency of generating video cover pictures.

[0084] In order to make the video cover generation method of the present application more clear, more embodiments are described below.

[0085] refer to Figure 3 As shown, Figure 3 is a flow chart of a method for generating a video cover according to another embodiment, comprising the following steps:

[0086] Step S201, reading the next frame from the video file as the current video frame;

[0087] Step S202, using a face detection algorithm to detect whether there is a face image in the current video frame; if not, return to step S201, if so, set the detected face image with the largest area as the main character image of the cover, and execute step S203.

[0088] Step S203, using a facial key point positioning algorithm to identify facial key points of the main character image, and calculating the side face angle of the main character image according to a facial key point model;

[0089] Step S204, determine whether the side face angle of the person's face is less than the angle threshold; if so, set the person's main body picture as the cover main body picture, and execute step S205, otherwise return to step S201.

[0090] Step S205, using an OCR text detection algorithm to detect whether there is text in the current video frame; if there is no text, go to step S207; if there is text, execute step S206.

[0091] Step S206, determine whether the rectangular enclosing frame of each text overlaps with the main body picture of the character; if there is overlap, return to step S201; if there is no overlap, set the cover main body picture as the video cover picture, and go to step S207.

[0092] Step S207, outputting the video cover picture of the video file.

[0093] As in the technical solution of the above-mentioned embodiment, when processing video frames frame by frame, logical judgments of face detection, key point positioning, and text occlusion recognition are performed in sequence for each video frame. When any video frame meets the conditions after logical judgment, the corresponding video cover picture is output; the video cover picture can be output quickly and efficiently, which is particularly suitable for scenarios such as short drama video promotion, live broadcast cover display, etc. that need to process a large number of video files and quickly generate cover pictures, so as to quickly and efficiently obtain the cover picture of the short video.

[0094] In one embodiment, it is considered that there may not be any video frame that meets the conditions to be used as the video cover image after traversing the video frames of the entire video file; accordingly, the technical solution of the present application also sets up a candidate image set, and saves the video frames that meet the current conditions during the judgment process for use in the subsequent cover generation process.

[0095] Accordingly, when it is determined in step S204 that the side face angle of the person is greater than or equal to the angle threshold, the main body picture of the person is added to the candidate image set, and then the process returns to step S201; when it is detected in step S206 that the rectangular enclosing box of the text overlaps with the main body picture of the person, the main body picture of the cover is added to the candidate image set, and then the process returns to step S201.

[0096] In one embodiment, the video cover generation method of the present application, after traversing and reading the video file and making a judgment, when the video cover picture cannot be obtained, refers to Figure 4 As shown, Figure 4 The flowchart of generating a video cover based on a candidate image set according to an embodiment includes the following steps:

[0097] Step S301, reading the character main image and cover main image in the candidate image set.

[0098] Step S302, calculating the ratio of the distances between the left corner of the mouth and the right corner of the mouth of the facial key point model of each video frame and the center point of the lips, and converting the ratio into a first quality score.

[0099] Exemplarily, the calculation formula of the first quality score may be as follows:

[0100]

[0101] Among them, score1 is the first quality score, and ratio is the ratio of the distance between the left corner of the mouth and the right corner of the mouth and the center point of the lips.

[0102] Step S303, calculating the overlapping areas between all text enclosing rectangular frames and the subject face rectangular frame of each video frame, recording the maximum overlapping area value, and converting the maximum overlapping area value into a second quality score.

[0103] Exemplarily, the calculation formula of the second quality score may be as follows:

[0104] score2=1-iou_max

[0105] Among them, score2 is the second quality score, and iou_max is the maximum overlapping area value;

[0106] Step S304: Calculate the comprehensive score of each video frame according to the first quality score and the second quality score, and select the video frame with the largest comprehensive score from all candidate image sets as the video cover picture.

[0107] Exemplarily, the calculation formula of the comprehensive score value can be as follows:

[0108] score0=0.5×score1+0.5×score2

[0109] Among them, score0 is a comprehensive score value, and the video frame with the largest comprehensive score score0 is used as the video cover picture.

[0110] As in the technical solution of the above-mentioned embodiment, based on the designed candidate image set, after traversing and reading the video file and making a judgment, when the video cover picture cannot be obtained, the video frame can be read from the candidate image set, and the side face angle size and the overlap size of the face blocked by the text in each video frame can be converted into a corresponding quality score, and then the video cover picture is calculated based on the comprehensive quality score, thereby ensuring that a suitable video cover picture can still be generated after a frame-by-frame processing.

[0111] like Figure 5 As shown, Figure 5 This is a schematic diagram of an example video cover screen, which shows two examples of cover screens. The technical solution of the present application can identify video frames containing face images, small side faces and faces without text obstructing them as video cover screens, thereby achieving the purpose of quickly generating cover screens for short dramas or live broadcast highlights, greatly improving the efficiency of generating video cover screens.

[0112] An embodiment of a video cover generation device is described below.

[0113] like Figure 6 As shown, Figure 6 The structure diagram of a video cover generating device according to an embodiment of the present invention includes:

[0114] The video frame reading module 101 is used to read the current video frame from the video file;

[0115] A face detection module 102 is used to detect a face image of the current video frame and set the face image as the main subject image of the cover;

[0116] The side face angle recognition module 103 is used to recognize the key points of the face of the main character picture, and calculate the main character picture with the side face angle less than the angle threshold according to the key points of the face as the cover main character picture;

[0117] The text occlusion recognition module 104 is used to recognize the text and its rectangular enclosing frame of the cover main body picture, and set the cover main body picture where the rectangular enclosing frame does not overlap with the character main body picture as the video cover picture;

[0118] The cover output module 105 is used to output the video cover picture as the cover picture of the video file.

[0119] The video cover generating device of this embodiment can execute a video cover generating method provided by the embodiment of the present application, and the implementation principle is similar. The actions performed by each module in the video cover generating device in each embodiment of the present application correspond to the steps in the video cover generating method in each embodiment of the present application. For the detailed functional description of each module of the video cover generating device, please refer to the description in the corresponding video cover generating method shown in the previous text, which will not be repeated here.

[0120] The following describes an example of the application of the video cover generation method of the present application in a network video system. Figure 7 As shown, Figure 7 This is a schematic diagram of an exemplary network video system, the system framework includes a server and a client, and the client runs on a terminal device, such as a PDA, smart phone, tablet computer, desktop computer or laptop computer. The server can provide video content such as short videos and short dramas. The server can use the video cover generation method provided by this application to quickly generate video cover images of short videos and short dramas, and then send them to the client for playback. The technical solution of this application processes the video frames of the video file frame by frame, and successively performs logical processing such as face detection, key point positioning, and text occlusion recognition. When all the above conditions are met, the corresponding video cover image is output as the cover image of the video file; thereby, intelligent judgment of high-quality video frames can be achieved, and then the available video cover image can be quickly output. When processing the video frames frame by frame, logical judgments of face detection, key point positioning, and text occlusion recognition are performed in sequence for each video frame. When any video frame meets the conditions after logical judgment, the corresponding video cover image is output; the video cover image can be output quickly and efficiently, especially suitable for It is used in scenarios such as short drama video promotion and live broadcast cover display that need to process a large number of video files and quickly generate cover images, to quickly and efficiently obtain the cover image of the short video. Based on the designed candidate image set, after traversing and reading the video files and making judgments, when the video cover image cannot be obtained, the video frame can be read from the candidate image set, and the side face angle size and the overlap size of the face blocked by text can be converted into corresponding quality scores, and then the video cover image is calculated based on the comprehensive quality scores. This ensures that after a frame-by-frame processing, a suitable video cover image can still be generated, which greatly improves the efficiency of generating video cover images.

[0121] Embodiments of electronic devices and computer-readable storage media are described below.

[0122] The present application provides a technical solution for an electronic device to implement functions related to a video cover generation method. The electronic device of this embodiment includes one or more processors, a memory; one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by one or more processors, and the one or more programs are configured for the steps of the video cover generation method of any embodiment.

[0123] like Figure 8 As shown, Figure 8 1 is a block diagram of an exemplary electronic device; the electronic device may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc. The electronic device 100 may include one or more of the following components: a processing component 102, a memory 104, a power component 106, a multimedia component 108, an audio component 109, an input / output (I / O) interface 112, a sensor component 114, and a communication component 116.

[0124] The processing component 102 generally controls the overall operation of the electronic device 100, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations.

[0125] The memory 104 is configured to store various types of data to support operations in the electronic device 100, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0126] The power supply component 106 provides power to various components of the electronic device 100 .

[0127] The multimedia component 109 includes a screen that provides an output interface between the electronic device 100 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). In some embodiments, the multimedia component 108 includes a front camera and / or a rear camera.

[0128] The audio component 109 is configured to output and / or input audio signals.

[0129] I / O interface 112 provides an interface between processing component 102 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, a home button, a volume button, a start button, and a lock button.

[0130] The sensor component 114 includes one or more sensors for providing various aspects of status assessment for the electronic device 100. The sensor component 114 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact.

[0131] The communication component 116 is configured to facilitate wired or wireless communication between the electronic device 100 and other devices. The electronic device 100 can access a wireless network based on a communication standard, such as WiFi, an operator network (such as 2G, 3G, 4G or 5G), or a combination thereof.

[0132] The present application provides a technical solution of a computer-readable storage medium, which is used to implement functions related to a video cover generation method. The computer-readable storage medium stores at least one instruction, at least one program, code set or instruction set, and the at least one instruction, at least one program, code set or instruction set is loaded by a processor and executes the video cover generation method of any embodiment.

[0133] In an exemplary embodiment, the computer-readable storage medium may be a non-temporary computer-readable storage medium including instructions, such as a memory including instructions. For example, the non-temporary computer-readable storage medium may be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

[0134] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A video cover generation method, characterized in that: include: Read the current video frame from the video file; Performing face image detection on the current video frame, and setting the face image as the main character image of the cover; Identify key points of the face of the main character picture, and calculate the main character picture with a side face angle less than an angle threshold according to the key points of the face, and set it as the main character picture of the cover; Identify the text of the cover main body picture and its rectangular enclosing frame, and set the cover main body picture that does not overlap with the rectangular enclosing frame and the character main body picture as the video cover picture; The video cover picture is output as the cover picture of the video file.

2. The video cover generation method according to claim 1, characterized in that: Performing face image detection on the current video frame and setting the face image as the main subject image of the cover, including: Use a face detection algorithm to detect whether there is a face image in the current video frame; If it does not exist, read the next frame from the video file as the current video frame and perform face image detection until a face image is detected.

3. The video cover generation method according to claim 1, characterized in that: Identifying key points of the face of the main character picture, and calculating the main character picture whose side face angle is less than an angle threshold according to the key points of the face as the main character picture of the cover, including: Using a facial key point positioning algorithm to identify the facial key points of the main character image, and calculating the side face angle of the main character image according to a facial key point model; If the side face angle of the person's face is less than the angle threshold, the person's main body picture is set as the cover main body picture.

4. The video cover generation method according to claim 3, characterized in that: Calculate the face profile angle of the main body of the image based on the face key point model, including: According to the facial key point model, the left mouth corner point (x l ,y l ), midpoint of lips (x m ,y m ), right corner of mouth (x r ,y r ); Calculate the distance between the left corner of the mouth and the midpoint of the lip and the distance between the right corner of the mouth and the midpoint of the lip Calculate the distance ratio between the left and right corners of the mouth and the center of the lips (ratio = dl / dr); If the side face angle of the person's face is less than the angle threshold, the main body picture of the person is set as the cover main body picture, including: if the distance ratio ratio is within a preset ratio range, it is determined that the main body picture of the person is a small-angle side face, and the main body picture of the person is set as the cover main body picture.

5. The video cover generation method according to claim 1, characterized in that: Identifying the text of the cover main body picture and its rectangular enclosing frame, and setting the cover main body picture that does not overlap the rectangular enclosing frame with the character main body picture as the video cover picture, including: Use the OCR text detection algorithm to detect whether there is text in the current video frame; if there is no text, output the cover main body image as the video cover image; If there is text, further detect whether the rectangular enclosing frame of each text overlaps with the main image of the character; if there is no overlap, set the cover main image as the video cover image.

6. The video cover generation method according to any one of claims 3 to 5, characterized in that: Also includes: If the side face angle of the person's face is greater than or equal to the angle threshold, the main image of the person is added to the candidate image set, and the next frame is read from the video file as the current video frame; If the rectangular enclosing frame overlaps with the main image of the person, the cover main image is added to the candidate image set, and the next frame is read from the video file as the current video frame.

7. The video cover generation method according to claim 5, characterized in that: Also includes: Reading video frames in the candidate image set; Calculate the ratio of the distances between the left corner of the mouth and the right corner of the mouth of the face key point model of each video frame and the center point of the lips, and convert the ratio into a first quality score; Calculate the overlapping area between all text enclosing rectangular boxes and the subject face rectangular box of each video frame, record the maximum overlapping area value, and convert the maximum overlapping area value into a second quality score; A comprehensive score of each video frame is calculated according to the first quality score and the second quality score, and a video frame with the largest comprehensive score is selected from all candidate image sets as the video cover picture.

8. The video cover generation method according to claim 7, characterized in that: The calculation formula of the first quality score is: Among them, score1 is the first quality score, and ratio is the ratio of the distance between the left corner of the mouth and the right corner of the mouth to the center of the lips; The calculation formula of the second quality score is: score2=1-iou_max Among them, score2 is the second quality score, and iou_max is the maximum overlapping area value; The calculation formula of the comprehensive score is: score0=0.5×score1+0.5×score2 Among them, score0 is the comprehensive score.

9. A video cover generation device, characterized in that: include: Video frame reading module, used to read the current video frame from the video file; A face detection module is used to detect a face image of the current video frame and set the face image as the main body picture of the person on the cover; A side face angle recognition module is used to recognize key points of the face of the main character picture, and calculate the main character picture with a side face angle less than an angle threshold according to the key points of the face, and set it as the main character picture of the cover; A text occlusion recognition module, used to recognize the text and its rectangular enclosing frame of the cover main body picture, and set the cover main body picture where the rectangular enclosing frame does not overlap with the character main body picture as the video cover picture; The cover output module is used to output the video cover picture as the cover picture of the video file.

10. An electronic device, characterized in that: The electronic device comprises: one or more processors; Memory; One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the steps of the video cover generation method described in any one of claims 1-8.

11. A computer-readable storage medium, characterized in that: The storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded by the processor and executes the steps of the video cover generation method described in any one of claims 1-8.

Images