Video cover determination method, device, equipment and storage medium

By receiving user-entered video cover generation instructions, determining the target object location and face area, adjusting the cropping box to match the setting parameters, solving the problem of low efficiency in determining video covers in the existing technology, and achieving efficient and accurate video covers generate.

CN118474482BActive Publication Date: 2024-12-03GALAXY INTERNET TELEVISION (ZHEJIANG) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410933894.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-07-12
Publication Date
2024-12-03
Estimated Expiration
2044-07-12

AI Technical Summary

Technical Problem

In the prior art, the determination of video covers is low, and it is necessary to manually select video frames and crop boxes, resulting in inefficiency.

Method used

By receiving the video cover generation instructions input by the user, we obtain the setting parameters of the target object in the video frame, determine the location of the target object and the area occupied by the face area, adjust the size and position of the cropping box, match the setting parameters to crop the video frame, and generate the video Cover.

Benefits of technology

Improve the efficiency of video cover determination, avoid the inefficiency problem of manual selection, and do not rely on labeled training data or deep learning models, improving the accuracy of different types of video covers.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118474482B_ABST
    Figure CN118474482B_ABST
Patent Text Reader

Abstract

The present invention provides a video cover determination method, device, equipment and storage medium, which are applied to the field of image processing technology. The method includes: receiving a video cover generation instruction for a target video input by a user; determining a target position of a target object in a target video frame for each target video frame in the target video; adjusting the size of an initial cropping frame in the target video frame based on the area occupied by the face region of the target object in the target video frame to obtain a target cropping frame; adjusting the position of the target cropping frame based on the target position to obtain an adjusted position; when the adjusted position matches a set parameter, cropping the target video frame based on the adjusted position to obtain a target image in the target cropping frame; and determining a video cover of the target video based on the target image. The video cover determination method, device, equipment and storage medium provided by the embodiments of the present invention can improve the efficiency of determining a video cover.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a video cover determination method, device, equipment and storage medium. Background Art

[0002] In the field of video processing and editing, the determination of video cover is crucial to the attractiveness and recognizability of the video. Video cover is usually used in online video platforms, social media applications or any occasion where a quick preview of video content is required.

[0003] In the prior art, when determining a video cover, the user usually manually selects a frame of an image in the video and selects a specific area of ​​the image as the cover. For example, the user can select a certain area by dragging a cropping box or other user interface elements, and crop the image corresponding to the area as the video cover.

[0004] However, in the above method, manually selecting the video cover will make the efficiency of determining the video cover low. Summary of the invention

[0005] The present invention provides a video cover determination method, device, equipment and storage medium, which are used to solve the defect of low efficiency in determining video covers in the prior art and achieve the purpose of improving the efficiency in determining video covers.

[0006] The present invention provides a method for determining a video cover, comprising:

[0007] Receiving a video cover generation instruction for a target video input by a user, and in response to the video cover generation instruction, acquiring setting parameters of the target object in the video cover;

[0008] For each target video frame in the target video, determining a target position of the target object in the target video frame;

[0009] Based on the area occupied by the face region of the target object in the target video frame, adjusting the size of the initial cropping frame in the target video frame to obtain a target cropping frame;

[0010] Based on the target position, adjusting the position of the target cropping frame to obtain an adjusted position;

[0011] In a case where the adjusted position matches the setting parameters, cropping the target video frame based on the adjusted position to obtain a target image within the target cropping frame;

[0012] A video cover of the target video is determined based on the target image.

[0013] According to a video cover determination method provided by the present invention, adjusting the size of the initial cropping frame in the target video frame based on the area occupied by the face region of the target object in the target video frame to obtain the target cropping frame includes:

[0014] determining a cropping frame adjustment ratio based on the area, where the cropping frame adjustment ratio is 1 when the area is greater than a preset minimum area and less than a preset maximum area, and the cropping frame adjustment ratio is a ratio of the area to the preset maximum area when the area is greater than the preset maximum area;

[0015] Based on the cropping frame adjustment ratio, the size of the initial cropping frame in the target video frame is adjusted to obtain a target cropping frame.

[0016] According to a video cover determination method provided by the present invention, the target position includes the top position of the target object in the target video frame, and the leftmost position or the rightmost position of the target object in the target video frame;

[0017] The step of adjusting the position of the target cropping frame based on the target position to obtain the adjusted position includes:

[0018] In a case where the target video frame includes other objects except the target object, cropping the other objects to obtain a cropped target video frame;

[0019] Adjusting the top edge of the target cropping frame in the cropped target video frame to be aligned with the top position, to obtain the adjusted position of the target cropping frame in the vertical direction;

[0020] The left side of the target cropping frame is adjusted to the leftmost position, or the right side of the target cropping frame is adjusted to the rightmost position to obtain an adjusted position.

[0021] According to a method for determining a video cover provided by the present invention, the setting parameters include a top reference distance range, a bottom reference distance range, a left reference distance range, and a right reference distance range; the top reference distance range is used to characterize the distance of the face area relative to the top edge of the video cover, the bottom reference distance range is used to characterize the distance of the face area relative to the bottom edge of the video cover, the left reference distance range is used to characterize the distance of the face area relative to the left edge of the video cover, and the right reference distance range is used to characterize the distance of the face area relative to the right edge of the video cover;

[0022] The adjusted position matches the setting parameter, including:

[0023] Based on the adjusted position, determining a top distance between a top edge of a face detection frame corresponding to the face area and a top edge of the target cropping frame;

[0024] Based on the adjusted position, determining a bottom distance between a bottom edge of a face detection frame corresponding to the face area and a bottom edge of the target cropping frame;

[0025] Based on the adjusted position, determining a left distance between a left side edge of a face detection frame corresponding to the face area and a left side edge of the target cropping frame;

[0026] Based on the adjusted position, determining a right side distance between a right side edge of a face detection frame corresponding to the face area and a right side edge of the target cropping frame;

[0027] When the top distance matches the top reference distance range, the bottom distance matches the bottom reference distance range, the left distance matches the left reference distance range, and the right distance matches the right reference distance range, it is determined that the adjusted position matches the setting parameters.

[0028] According to a video cover determination method provided by the present invention, the method further includes:

[0029] For each video frame in the target video, when the target object is included in the video frame, converting an image of a face region of the target object into a grayscale image, and filtering the grayscale image to obtain a filtered image;

[0030] determining a variance of the filtered image;

[0031] When the variance is greater than or equal to a preset threshold, the video frame is determined as the target video frame.

[0032] According to a video cover determination method provided by the present invention, determining the video cover of the target video based on the target image includes:

[0033] Determining a text area based on a position of a face area in the target image;

[0034] Determine a target pixel color average value in the text area, and determine a target text color corresponding to the target pixel color average value based on a correspondence between the pixel color average value and the text color;

[0035] The recommendation corresponding to the target video is displayed in the text area using the target text color to obtain a video cover of the target video.

[0036] According to a video cover determination method provided by the present invention, the recommendation corresponding to the target video is displayed in the text area using the target text color to obtain the video cover of the target video, including:

[0037] When the number of characters in the recommendation is greater than a preset number of characters, semantic segmentation is performed on the recommendation to obtain at least two segmented words;

[0038] Determine the number of segmented words displayed in each line based on the total number of segmented words and the preset number of lines;

[0039] Based on the number of word segments displayed in each line, the target text color is adopted to display the recommendation in the text area to obtain the video cover of the target video.

[0040] The present invention also provides a video cover determination device, comprising:

[0041] A receiving module, configured to receive a video cover generation instruction for a target video input by a user, and obtain setting parameters of the target object in the video cover in response to the video cover generation instruction;

[0042] A determination module, configured to determine, for each target video frame in the target video, a target position of the target object in the target video frame;

[0043] An adjustment module, configured to adjust the size of an initial cropping frame in the target video frame based on the area occupied by the face region of the target object in the target video frame to obtain a target cropping frame;

[0044] The adjustment module is further used to adjust the position of the target cropping frame based on the target position to obtain an adjusted position;

[0045] A cropping module, configured to crop the target video frame based on the adjusted position to obtain a target image within the target cropping frame when the adjusted position matches the setting parameters;

[0046] The determination module is further used to determine the video cover of the target video based on the target image.

[0047] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any of the above-mentioned methods for determining a video cover is implemented.

[0048] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the video cover determination method as described in any one of the above is implemented.

[0049] The present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements any of the above-mentioned methods for determining a video cover.

[0050] The video cover determination method, device, equipment and storage medium provided by the present invention receive a video cover generation instruction for a target video input by a user, and in response to the video cover generation instruction, obtain setting parameters of a target object in the video cover, determine the target position of the target object in the target video frame for each target video frame in the target video, adjust the size of an initial cropping frame in the target video frame based on the area occupied by the face region of the target object in the target video frame to obtain a target cropping frame, and adjust the position of the target cropping frame based on the target position to obtain an adjusted position, and when the adjusted position matches the setting parameters, crop the target video frame based on the adjusted position to obtain a target image within the target cropping frame, thereby determining the video cover of the target video based on the target image. Since a video cover that meets user requirements can be determined based only on the position of the target object in the target video frame and the area occupied by the target object's face area in the target video frame, the problem of low efficiency of video cover caused by manual selection of video cover in the prior art is avoided. Moreover, the above scheme does not require labeled training data or training of deep learning models, which can avoid the problem of poor generalization performance of deep learning models in different fields, resulting in low accuracy when determining video covers of different types of videos through the deep learning model, due to the fact that the performance of the deep learning model is highly dependent on the quality and diversity of the training data. This can thereby improve the accuracy of video covers of different types of videos. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0052] Figure 1 A schematic diagram of a flow chart of a method for determining a video cover according to an embodiment of the present invention.

[0053] Figure 2 A schematic diagram of a target video frame provided by an embodiment of the present invention.

[0054] Figure 3 A schematic diagram of performing portrait instance segmentation on a target video frame according to an embodiment of the present invention.

[0055] Figure 4A schematic diagram of an initial cropping frame provided by an embodiment of the present invention.

[0056] Figure 5 A schematic diagram of a target cropping frame provided by an embodiment of the present invention.

[0057] Figure 6 This is a schematic diagram of a target cropping frame after adjusting its position in the vertical direction provided by an embodiment of the present invention.

[0058] Figure 7 This is a schematic diagram after other objects are cropped according to an embodiment of the present invention.

[0059] Figure 8 This is a schematic diagram of a target cropping frame after adjusting its position in the horizontal direction provided by an embodiment of the present invention.

[0060] Fig. 9 A schematic diagram of setting parameters provided by an embodiment of the present invention.

[0061] Fig.10 A schematic diagram of the structure of a video cover determination device provided in an embodiment of the present invention.

[0062] Fig.11 An example physical structure diagram of an electronic device is provided. DETAILED DESCRIPTION

[0063] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0064] At present, when recommending film and television programs to users, it is usually necessary to produce multiple video covers of various sizes, and there are different requirements for the size and position of the characters in the video cover content, which leads to the production of character video covers requiring more manpower, material resources and time costs. In the prior art, video covers are usually determined through user interaction or deep learning. For user interaction, users usually select an area in the video frame through a draggable cropping box or other user interface elements, and then crop the picture corresponding to the area as the video cover. However, the efficiency of determining the video cover in this method is low.

[0065] In consideration of the above problems, an embodiment of the present invention proposes a method for determining a video cover. In the method, the size of the initial cropping frame can be adjusted based on the area occupied by the face region of the target object in each target video frame of the target video to obtain a target cropping frame, and then the position of the target cropping frame can be adjusted based on the position of the target object in the target video frame. After the size and position of the cropping frame are adjusted, it is determined whether the position of the target cropping frame matches the setting parameters input by the user. If they match, the image in the target cropping frame is determined as the video cover of the target video. Since a video cover that meets the user's requirements can be determined based only on the position of the target object in the target video frame and the area occupied by the face region of the target object in the target video frame, the problem of low efficiency of the video cover caused by manually selecting the video cover in the prior art is avoided, and the efficiency of determining the video cover is improved.

[0066] Combine the following Figures 1 to 9 The video cover determination method provided by the embodiment of the present invention is described. The embodiment of the present invention can be applicable to video cover determination scenarios of different types of videos. A universal interface service is designed in the present invention, which can be applied to video editing software, movie production, automatic cropping of characters or specific scenes in the video for post-production or analysis, and can also be applied to surveillance image processing, etc. to analyze surveillance videos and automatically crop out potentially meaningful scenes or behaviors. The executor of the method can be an electronic device such as a terminal device, a computer, a server, a server cluster, or a specially designed video cover determination device, or a video cover determination device set in the electronic device. The video cover determination device can be implemented by software, hardware, or a combination of both.

[0067] Figure 1 A schematic diagram of a flow chart of a method for determining a video cover provided by an embodiment of the present invention, such as Figure 1 As shown, the method includes:

[0068] Step 101: receiving a video cover generation instruction for a target video input by a user, and in response to the video cover generation instruction, obtaining setting parameters of the target object in the video cover.

[0069] In this embodiment, the target object may include a person or a robot, etc. In the embodiments of the present invention, the target object is taken as an example of a person.

[0070] If it is necessary to generate a video cover for a target video, the user can input a video cover generation instruction to the electronic device through voice, text, or touch. In addition, due to different target videos, the position specifications of the target object in the video cover may be different. In addition, for the same target video, in different application scenarios, it is also possible to generate video covers with different position specifications of the target object. Therefore, the video cover instruction input by the user may include the setting parameters of the target object in the video cover, that is, the user can input the setting parameters of the target object in the video cover through the video cover generation instruction. In addition, the setting parameters can also be parameters that are pre-set and stored in the electronic device. Among them, the setting parameters can be understood as the position specifications of the target object, that is, the position of the target object in the video cover that the user wants or requires and the face area size of the target object. For example, the position of the target object in the video cover can include the distance range between each frame of the face detection frame and the corresponding edge of the video cover, such as the distance between the left frame of the face detection frame and the left edge of the video cover, the distance between the right frame of the face detection frame and the right edge of the video cover, etc. In addition, the video cover generation instruction can also include the size of the video cover, such as the required height and width of the video cover.

[0071] The setting parameters may be learned from the positions of the target objects and the face areas in the multiple manually designed video covers. For example, the position information of the target objects and the face areas in the manually designed video covers are detected by a face detection algorithm, and a standardized reference standard is obtained through statistics and analysis to obtain the setting parameters. In addition, the setting parameters may also be filled in manually. For example, the user may input the length range of each frame of the face detection frame and the top, bottom, left and right of the video cover, as well as the face area, etc. according to their own needs.

[0072] It should be understood that the setting parameters can be modified according to actual needs or application scenarios. After modifying the setting parameters, a new video cover corresponding to the modified setting parameters can be obtained. For example, modifying the face area ratio can obtain close-up, near-shot, mid-shot and long-shot effects, modifying the length range of each frame of the face detection frame and the top, bottom, left and right of the video cover can control the face within a certain area, and modifying the height and width of the video cover can set the size of the video cover.

[0073] By including the setting parameters of the target object in the video cover in the video cover generation instruction, a video cover that meets the user's needs can be generated, thereby improving the accuracy of the video cover. In addition, the setting parameters in the video cover generation instruction can be modified and extended. By modifying different values, different video covers can be generated to achieve different cover effects, thereby improving the flexibility of video cover generation.

[0074] Step 102: for each target video frame in the target video, determine the target position of the target object in the target video frame.

[0075] Specifically, the target video frame can be a video frame obtained after preprocessing and quality screening of all video frames in the target video. The target video frame is an image that includes the target object, and the target object's face is clear and the eyes are not closed. The number of target video frames can be one or more.

[0076] In addition, in the target video frame, there are often difficult-to-identify group actors and back portraits. In order to avoid these portraits appearing in the cover image, for each target video frame, the target video frame can be segmented by a pre-trained portrait segmentation network model based on a mask region-based convolutional neural network (Mask R-CNN). Through portrait instance segmentation, each user in the target video frame can be identified, and a corresponding instance mask can be generated for each user. In addition, the user in each target video frame can be identified based on a pre-trained user recognition model to obtain a recognition result, wherein the recognition result includes the user's identifier and the position coordinates of the user in the target video frame. The user's identifier can be, for example, the user's name or a corresponding number. When the user inputs a video cover generation instruction, the video cover instruction also includes the identifier of the target object to be included in the video cover. Therefore, based on the identifier of the target object in the video cover instruction and the user identifier and position coordinates in the image recognition result, it is possible to distinguish which part of the instance mask in the target video frame corresponds to the target object, and at the same time distinguish the instance mask related to the non-target object. Therefore, the non-target object can be removed based on the instance mask related to the non-target object, ensuring that the video cover does not contain the non-target object.

[0077] Figure 2 A schematic diagram of a target video frame provided by an embodiment of the present invention, Figure 3 A schematic diagram of performing portrait instance segmentation on a target video frame according to an embodiment of the present invention is shown in FIG. Figure 2 and Figure 3 As shown in Figure 2, after inputting the target video frame into Mask R-CNN, we can get Figure 3 The portrait instance segmentation results shown in , where Figure 3 The black part represents the background area, the red part represents the area where the target object is located, and the blue part represents the area where the non-target object is located.

[0078] In addition, after the instance mask of the target object in the target video frame is determined, the target position of the target object in the target video frame may be determined based on the instance mask.

[0079] Of course, the target position of the target object in the target video frame may also be determined through a pre-trained object recognition model.

[0080] Among them, the target position of the target object in the target video frame may, for example, include the key position coordinates of the target object in the target video frame, such as the top position y1 of the target object in the target video frame, and the leftmost position x1 or the rightmost position x2 of the target object in the target video frame, and of course, may also include the bottommost position y2 of the target object in the target video frame.

[0081] Step 103: Based on the area occupied by the face region of the target object in the target video frame, the size of the initial cropping frame in the target video frame is adjusted to obtain a target cropping frame.

[0082] In this step, the face region of the target object can be determined by the face detection algorithm, wherein the area occupied by the face region in the target video frame can be understood as the area occupied by the face detection frame detected by the face detection algorithm in the target video frame. In addition, after the face detection frame is determined, an initial cropping frame can be generated in the target video frame based on the position and size of the face detection frame, such as using the face detection frame as the initial cropping frame, or scaling the face detection frame by a preset ratio to obtain the initial cropping frame.

[0083] Furthermore, based on the area occupied by the face detection frame in the target video frame, the size of the initial cropping frame in the target video frame is adjusted to obtain a target cropping frame, wherein the target cropping frame contains a complete face area. This method can obtain video covers with different scene effects by adjusting the area of ​​the face area of ​​the target object in the video cover.

[0084] Step 104: Based on the target position, adjust the position of the target cropping frame to obtain an adjusted position.

[0085] In this step, since the target object needs to be included in the video cover, after determining the target position of the target object in the target video frame, the position of the target cropping frame will be adjusted based on the target position of the target object so that the target cropping frame contains the target object and does not contain other objects.

[0086] Step 105: When the adjusted position matches the set parameters, the target video frame is cropped based on the adjusted position to obtain a target image within a target cropping frame.

[0087] In this step, after adjusting the position of the target cropping frame, it is necessary to further check whether the adjusted position matches the setting parameters desired by the user. For example, if the distance between the left side of the face detection frame and the left side of the video cover in the setting parameters input by the user is 5cm-10cm, then it is necessary to check whether the distance between the left side of the target object's face detection frame and the left side of the target cropping frame is within 5cm-10cm in the target cropping frame after the position is adjusted, and so on. The distance between the top edge of the face detection frame and the top edge of the video cover, the distance between the bottom edge of the face detection frame and the bottom edge of the video cover, and the distance between the right side of the face detection frame and the right side of the video cover in the setting parameters are checked in a similar manner to the left side, and will not be repeated here.

[0088] If it is determined that the adjusted position of the target cropping box matches the setting parameters input by the user, it means that the image in the target cropping box is the video cover that the user wants. Therefore, the image currently selected by the target cropping box can be cropped from the target video frame to obtain the target image in the target cropping box.

[0089] If it is determined that the adjusted position of the target cropping frame does not match the setting parameters input by the user, it means that the target video frame cannot be cropped to obtain the video cover desired by the user, and the target video frame will be discarded. For each target video frame, it will be processed according to the method in the embodiment of the present invention to determine whether the video cover desired by the user can be cropped from the target video frame.

[0090] Step 106: Determine a video cover of the target video based on the target image.

[0091] In this step, after the target image within the target cropping frame is cropped, the target image can be directly used as the video cover of the target video, or special effects or text can be added to the target image to generate a video cover.

[0092] It should be noted that, if the video covers corresponding to multiple target video frames are determined through the above method, the user can manually select the most suitable video cover, or select the most suitable cover through machine learning and use the selected video cover as the video cover of the final target video.

[0093] It should be understood that when multiple target video frames are included, the video cover corresponding to each target video frame can also be determined in sequence according to the order of the target video frames. Once a suitable video cover is determined, the determined video cover can be used as the video cover of the final target video, and no corresponding processing will be performed on subsequent target video frames.

[0094] The video cover determination method provided by the embodiment of the present invention receives a video cover generation instruction for a target video input by a user, obtains setting parameters of a target object in the video cover in response to the video cover generation instruction, determines the target position of the target object in the target video frame for each target video frame in the target video, adjusts the size of an initial cropping frame in the target video frame based on the area occupied by a face region of the target object in the target video frame to obtain a target cropping frame, and adjusts the position of the target cropping frame based on the target position to obtain an adjusted position, and when the adjusted position matches the setting parameters, crops the target video frame based on the adjusted position to obtain a target image within the target cropping frame, thereby determining the video cover of the target video based on the target image. Since a video cover that meets user requirements can be determined based only on the position of the target object in the target video frame and the area occupied by the target object's face area in the target video frame, the problem of low efficiency of video cover caused by manual selection of video cover in the prior art is avoided. Moreover, the above scheme does not require labeled training data or training of deep learning models, which can avoid the problem of poor generalization performance of deep learning models in different fields, resulting in low accuracy when determining video covers of different types of videos through the deep learning model, due to the fact that the performance of the deep learning model is highly dependent on the quality and diversity of the training data. This can thereby improve the accuracy of video covers of different types of videos.

[0095] In addition, since users can input different setting parameters according to their needs through the video cover command, they can obtain covers with different effects, which improves the scalability of video cover generation.

[0096] Exemplarily, on the basis of the above-mentioned embodiment, based on the area occupied by the face area of ​​the target object in the target video frame, the size of the initial cropping frame in the target video frame is adjusted, and when the target cropping frame is obtained, it can be adjusted in the following manner: the cropping frame adjustment ratio is determined based on the area, and when the area is larger than the preset minimum area and smaller than the preset maximum area, the cropping frame adjustment ratio is 1; when the area is larger than the preset maximum area, the cropping frame adjustment ratio is the ratio of the area to the preset maximum area; based on the cropping frame adjustment ratio, the size of the initial cropping frame in the target video frame is adjusted to obtain the target cropping frame.

[0097] Specifically, the cropping frame adjustment ratio can be determined based on the area occupied by the face region of the target object in the target video frame by the following formula (1):

[0098] (1)

[0099] in, Indicates the cropping frame adjustment ratio. Indicates the area occupied by the target object’s face region in the target video frame. Indicates the preset minimum area, that is, the minimum area of ​​the face in the preset target object specification. Indicates the preset maximum area, that is, the maximum area of ​​the face in the preset target object specification. and It can be a user input value included in the video cover generation instruction, or it can be a pre-defined value when generating the video cover. and The size of the video cover can achieve different effects such as close-up, near shot, mid shot and long shot of the target object.

[0100] Figure 4 A schematic diagram of an initial cropping frame provided by an embodiment of the present invention, Figure 5 A schematic diagram of a target cropping frame provided by an embodiment of the present invention, such as Figure 4 As shown, after detecting the face in the target image frame and determining the face detection frame, an initial cropping frame can be generated based on the face detection frame. Figure 5 As shown, after determining the cropping frame adjustment ratio based on the area occupied by the face region of the target object in the target video frame, the size of the initial cropping frame can be adjusted based on the cropping frame adjustment ratio to obtain the target cropping frame.

[0101] It should be noted that if the size of the target cropping frame after adjustment is larger than the size of the video cover required by the user, or Less than , it means that the target video frame cannot determine a video cover that meets the requirements, and the target video frame will be discarded.

[0102] In this embodiment, after determining the cropping frame adjustment ratio by the area occupied by the face area of ​​the target object in the target video frame, the size of the initial cropping frame is adjusted based on the cropping frame adjustment ratio, so that the size of the obtained target cropping frame meets user needs, thereby improving the accuracy and precision of the final video cover.

[0103] Exemplarily, based on the above embodiments, the target position of the target object includes the top position of the target object in the target video frame, and the leftmost position or the rightmost position of the target object in the target video frame; when adjusting the position of the target cropping frame based on the target position to obtain the adjusted position, it can be that when the target video frame includes other objects except the target object, the other objects are cropped to obtain the cropped target video frame, the top edge of the target cropping frame in the cropped target video frame is adjusted to be aligned with the top position to obtain the adjusted position of the target cropping frame in the vertical direction, the left edge of the target cropping frame is adjusted to the leftmost position, or the right edge of the target cropping frame is adjusted to the rightmost position to obtain the adjusted position.

[0104] Specifically, Figure 3 As shown, after the coordinate system is constructed with the upper left corner vertex of the target video frame as the origin and the length and width of the target video frame as the x-axis and y-axis, the top position y1 of the target object in the target video frame is the vertical coordinate of the topmost pixel point of the target object in the target video frame in the coordinate system, the leftmost position x1 of the target object in the target video frame is the horizontal coordinate of the leftmost pixel point of the target object in the target video frame in the coordinate system, and the rightmost position x2 of the target object in the target video frame is the horizontal coordinate of the rightmost pixel point of the target object in the target video frame in the coordinate system.

[0105] When adjusting the position of the target cropping frame based on the target position of the target object in the target video frame, when the target video frame includes other objects except the target object, in order to avoid including other objects in the video cover, other objects can be cropped first to obtain a cropped target video frame, wherein the cropped target video frame does not include other objects. In the specific implementation process, Figure 3 As shown, after performing portrait instance segmentation on the target video frame and obtaining instance masks of the target object and other objects, the mask coordinate information of the other objects can be accurately determined, and then the other objects can be cropped based on the mask coordinate information to obtain the cropped target video frame.

[0106] Taking the vertical region in the target video frame as an example, it can be determined whether there are instance masks of other objects in the vertical region in the target video frame according to the instance segmentation result. If there are, the cropping operation is first performed according to the position of the target object and the position of other objects: if there are other objects on the left side of the target object, the rightmost horizontal coordinates of the other objects can be used as the starting cropping position of the video cover; if there are no other objects on the left side of the target object, the leftmost boundary of the target video frame can be used as the starting cropping position of the video cover; if there are other objects on the right side of the target object, the leftmost horizontal coordinates of the other objects can be used as the ending cropping position of the video cover, that is, cropped to the leftmost horizontal coordinates of other objects; if there are no other objects on the right side of the target object, the rightmost boundary of the target video frame can be used as the ending cropping position of the video cover, that is, cropped to the rightmost horizontal coordinates of the target video. In the above manner, it can be ensured that the cropped video cover does not contain other objects.

[0107] After the cropped target video frame is obtained, the position of the target cropping frame in the cropped target video frame will be adjusted. Figure 6 This is a schematic diagram of a target cropping frame after adjusting its position in the vertical direction according to an embodiment of the present invention. Figure 6 As shown, for the position in the vertical direction, in order to ensure the integrity of the target object, the top edge of the target cropping frame can be aligned with the top position y1 of the target object in the target video frame, so that the adjusted position of the target cropping frame in the vertical direction can be obtained, as shown in FIG. Figure 6 As shown, by adjusting the yellow target cropping frame in the left figure up and down and cropping the image, the image in the yellow target cropping frame in the right figure can be obtained.

[0108] Among them, the above-mentioned alignment may include adjusting the top edge of the target cropping frame to the position of the top position y1, or adjusting the top edge of the target cropping frame to a position above the top position y1 and at a preset distance from the top position y1, so that there is a preset distance between the top edge of the target cropping frame and the top position y1.

[0109] In addition, it is also necessary to adjust the horizontal position of the target cropping frame. Specifically, the left side of the target cropping frame can be adjusted to the leftmost position x1 of the target object in the target video frame, or the right side of the target cropping frame can be adjusted to the rightmost position x2 of the target object in the target video frame.

[0110] Figure 7 This is a schematic diagram of cutting out other objects provided by an embodiment of the present invention, such as Figure 7 As shown, after determining the position of the target object and other objects by analyzing the example mask image, other objects can be cut out, such as by following Figure 7 The dashed lines shown clip out other objects. Figure 8 This is a schematic diagram of a target cropping frame after adjusting its position in the horizontal direction according to an embodiment of the present invention. Figure 8 As shown in FIG. 1 , after other objects are cropped, the left side of the target cropping frame can be adjusted to the leftmost position x1 of the target object in the target video frame, thereby further cropping the target video frame in the horizontal direction to obtain Figure 8 Image within the target crop box in the lower center.

[0111] In this embodiment, when the target video includes other objects besides the target object, the other objects can be cropped, so that the final video cover does not contain other objects. In addition, by adjusting the target cropping frame in the horizontal and vertical directions to obtain the image within the target cropping frame, the target cropping frame is adjusted in a relatively simple manner, and the video cover can be obtained quickly, thereby improving the efficiency of determining the video cover.

[0112] Exemplarily, based on the above embodiment, the setting parameters include a top reference distance range, a bottom reference distance range, a left reference distance range and a right reference distance range; the top reference distance range is used to characterize the distance of the face area relative to the top edge of the video cover, the bottom reference distance range is used to characterize the distance of the face area relative to the bottom edge of the video cover, the left reference distance range is used to characterize the distance of the face area relative to the left edge of the video cover, and the right reference distance range is used to characterize the distance of the face area relative to the right edge of the video cover. When determining that the adjusted position matches the setting parameters, it includes determining, based on the adjusted position, the top edge of the face detection frame corresponding to the face area and the target cropping The top distance between the top edge of the face detection frame corresponding to the face area and the top edge of the target cropping frame is determined based on the adjusted position; based on the adjusted position, the bottom distance between the bottom edge of the face detection frame corresponding to the face area and the bottom edge of the target cropping frame is determined; based on the adjusted position, the left distance between the left edge of the face detection frame corresponding to the face area and the left edge of the target cropping frame is determined; based on the adjusted position, the right distance between the right edge of the face detection frame corresponding to the face area and the right edge of the target cropping frame is determined; when the top distance matches the top reference distance range, the bottom distance matches the bottom reference distance range, the left distance matches the left reference distance range, and the right distance matches the right reference distance range, it is determined that the adjusted position matches the setting parameters.

[0113] Specifically, Fig. 9 A schematic diagram of setting parameters provided by an embodiment of the present invention, such as Fig. 9 As shown, the setting parameters include a top reference distance range, a bottom reference distance range, a left reference distance range, and a right reference distance range.

[0114] After adjusting the position of the target cropping frame, the top distance between the top edge of the face detection frame corresponding to the face area in the target cropping frame and the top edge of the target cropping frame can be determined by the face detection algorithm, and whether the top distance matches the top reference distance range. The top distance matches the top reference distance range, which can be understood as the top distance is within the top reference distance range.

[0115] Similarly, the bottom distance between the bottom edge of the face detection frame corresponding to the face area in the target cropping frame and the bottom edge of the target cropping frame can be determined, and it is determined whether the bottom distance matches the bottom reference distance range. The left distance between the left edge of the face detection frame corresponding to the face area in the target cropping frame and the left edge of the target cropping frame is determined, and it is determined whether the left distance matches the left reference distance range. The right distance between the right edge of the face detection frame corresponding to the face area in the target cropping frame and the right edge of the target cropping frame is determined, and it is determined whether the right distance matches the right reference distance range.

[0116] When the top distance matches the top reference distance range, the bottom distance matches the bottom reference distance range, the left distance matches the left reference distance range, and the right distance matches the right reference distance range, it is considered that the image in the target cropping frame matches the video cover required by the user.

[0117] In addition, if Fig. 9 As shown, in actual applications, there are usually requirements for the height and width of the video cover. Therefore, after cropping the target image that meets the set parameter requirements, the target image can also be scaled to obtain a video cover that meets the height and width standards.

[0118] In addition, if Fig. 9 As shown, in practical applications, there are usually requirements for the face area. By enlarging or reducing the face area, different video covers can be obtained.

[0119] In this embodiment, by matching the position of the target object in the image within the target cropping frame with the setting parameters, it is determined whether the image within the target cropping frame meets the preset video cover requirements, thereby improving the accuracy of the determined video cover.

[0120] In addition, through the cropping method in the embodiment of the present invention, the integrity of the target object in the target image can be guaranteed, the proportion and alignment of the target object can be maintained, and the above-mentioned cropping method takes into account the picture composition, visual reference and adjustment, so that the quality of the final video cover is relatively high.

[0121] Further, based on the above embodiments, the face of the target object in the target video may be blurred due to movement. In order to further improve the quality of the determined video cover, before determining the video cover, blurred video frames can be filtered out from the video frames of the target video. Exemplarily, for each video frame in the target video, when the target object is included in the video frame, the image of the face area of ​​the target object is converted into a grayscale image, and the grayscale image is filtered to obtain a filtered image; the variance of the filtered image is determined; when the variance is greater than or equal to a preset threshold, the video frame is determined as a target video frame.

[0122] Specifically, after acquiring the target video, the target video will be preprocessed, that is, each video frame in the target video will be filtered to discard images that do not meet expectations or are of low quality. In the specific implementation process, for each video frame in the target video, when the video frame includes the target object, the face area of ​​the detected target object can be normalized to 100. 100, and then convert the normalized face image into a grayscale image, and filter the grayscale image to obtain a filtered image, such as Laplace filtering. Further, the variance of the filtered image can be calculated. If the variance is less than a preset threshold, it means that the face area is relatively blurred, and the video frame will be discarded. If the variance is greater than or equal to the preset threshold, it means that the face area is relatively clear, and the video frame is determined as the target video frame as the video frame for cropping the video cover.

[0123] In addition, in order to avoid the final video cover containing text, the pre-trained optical character detection network model PaddleOCR can also be used to detect the position of text in the video frame. If the text is within the preset threshold range, the video frame will be discarded. If the text is within the target cropping frame, the text erasure algorithm will be used to erase the text.

[0124] In the present invention, it is also possible to detect whether the target object in the video frame has closed eyes based on the eye closing detection algorithm, and discard the video frames in which the target object has closed eyes, thereby avoiding the target object having closed eyes in the video cover determined later.

[0125] In this embodiment, the image of the face area of ​​the target object is converted into a grayscale image, and the variance of the filtered image is calculated after filtering the grayscale image. Based on the variance, the blurred image of the face area can be screened out, thereby improving the quality of the video cover determined subsequently.

[0126] Exemplarily, based on the above embodiments, when determining the video cover of the target video based on the target image, the text area can be determined based on the position of the face area in the target image, the target pixel color average value in the text area can be determined, and the target text color corresponding to the target pixel color average value can be determined based on the correspondence between the pixel average value and the text color. The recommendation corresponding to the target video is displayed in the text area using the target text color to obtain the video cover of the target video.

[0127] Specifically, for the target video, the user may enter a recommendation, or summarize a recommendation through the learning model, and add the recommendation to the appropriate position of the video cover. The recommendation can be a summary of the video content. When adding a recommendation, you first need to determine the adding position. For example, the text area can be determined based on the position of the face area in the target image cropped from the target cropping frame. For example, in order to avoid the recommendation blocking the face, if the face is on one side of the target image, the text area is determined to be on the opposite side of the face position in the target image. If the face is centered, the text area is determined to be on the lower side of the face in the target image, etc.

[0128] In addition, in order to make the recommendation words have a greater contrast with the target image when they are presented in the target image, that is, dark text is used for light target images, and light text is used for dark target images, etc., the average color of the target pixels in the text area can be determined by formula (2):

[0129] (2)

[0130] in, Represents the average color of the target pixels within the text area. Represents the average pixel value of the image in the text area in the R color channel. Represents the average pixel value of the image in the text area in the G color channel, Represents the average pixel value of the image in the text area in the B color channel.

[0131] The correspondence between the pixel average value and the text color is stored in the electronic device. By querying the correspondence, the target text color corresponding to the target pixel color average value can be determined. When it is greater than the preset value, it means that the background color in the text area is darker. At this time, you can choose a light font. Otherwise, you can use a dark font.

[0132] In summary, the recommendation corresponding to the target video can be displayed in the text area using the determined target text color, so that the recommendation can be added to the target image to obtain the final video cover.

[0133] In this embodiment, the text color with a large contrast to the target image color can be determined based on the color of the target image, and the determined recommendation can be displayed in the text area using the text color. This not only ensures that the text and background colors have a high contrast, but also ensures that the recommendation will not obstruct the face area of ​​the target object, thereby improving the quality of the final video cover.

[0134] Exemplarily, on the basis of the above-mentioned embodiment, when the recommendation corresponding to the target video is displayed in the text area using the target text color to obtain the video cover of the target video, when the number of words in the recommendation is greater than the preset number of words, the recommendation is semantically segmented to obtain at least two participles; based on the total number of participles and the preset number of lines, the number of participles displayed in each line is determined; based on the number of participles displayed in each line, the recommendation is displayed in the text area using the target text color to obtain the video cover of the target video.

[0135] Specifically, when the recommendation is too long, displaying the recommendation on the same line will affect the visual effect. Therefore, when the number of words in the recommendation is greater than the preset number of words, the recommendation needs to be divided into lines, and when displaying in lines, it is necessary to avoid the situation where the number of words in each line is greatly different and a word is displayed in different lines. Therefore, in an embodiment of the present invention, the recommendation can be semantically segmented to obtain at least two participles. For example, if the recommendation is "This is a good movie", after semantic segmentation of the recommendation, "This is / a / good / movie" can be obtained. Afterwards, the number of participles that need to be displayed in each line will be determined based on the total number of participles and the preset number of lines. Assuming that the total number of participles determined is n and the preset number of lines is 2, the number of participles that need to be displayed in the first line is , the number of segmented words that need to be displayed in the second line is .

[0136] After determining the number of words to be displayed in each line, the target text color is used based on the number of words to be displayed in each line, and the recommendation is displayed in the text area, so that the video cover of the target video can be obtained. Continuing with the above example, if the target text color is determined to be black, black is used to display "This is a movie" in the first line and "A good movie" in the second line.

[0137] In this embodiment, when the recommendation is too long, it can be semantically segmented and displayed in lines based on the total number of segmented words and the preset number of lines. This can avoid a large difference in the number of characters in each line and a word being displayed in different lines, thereby improving the user's visual effect and the quality of the video cover.

[0138] The video cover determination method provided by the embodiment of the present invention can not only improve the efficiency of video cover determination, but also, compared with the method of determining the video cover through deep learning, does not require labeled training data or training of a deep learning model. This can avoid the problem of low accuracy when determining video covers of different types of videos through the deep learning model, due to the fact that the performance of the deep learning model is highly dependent on the quality and diversity of the training data, resulting in poor generalization performance of the deep learning model in different fields. This can thereby improve the accuracy of video covers of different types of videos.

[0139] The video cover determination device provided by the present invention is described below. The video cover determination device described below and the video cover determination method described above can be referenced to each other.

[0140] Fig.10 A schematic diagram of the structure of a video cover determination device provided by an embodiment of the present invention, referring to Fig.10 As shown, the posture estimation device 1000 includes:

[0141] The receiving module 11 is used to receive a video cover generation instruction for a target video input by a user, and obtain setting parameters of the target object in the video cover in response to the video cover generation instruction;

[0142] A determination module 12, configured to determine, for each target video frame in the target video, a target position of the target object in the target video frame;

[0143] An adjustment module 13, configured to adjust the size of an initial cropping frame in the target video frame based on the area occupied by the face region of the target object in the target video frame to obtain a target cropping frame;

[0144] The adjustment module 13 is further used to adjust the position of the target cropping frame based on the target position to obtain an adjusted position;

[0145] A cropping module 14 is used to crop the target video frame based on the adjusted position to obtain a target image within the target cropping frame when the adjusted position matches the setting parameters;

[0146] The determination module 12 is further configured to determine a video cover of the target video based on the target image.

[0147] In an exemplary embodiment, the adjustment module 13 is specifically configured to:

[0148] determining a cropping frame adjustment ratio based on the area, where the cropping frame adjustment ratio is 1 when the area is greater than a preset minimum area and less than a preset maximum area, and the cropping frame adjustment ratio is a ratio of the area to the preset maximum area when the area is greater than the preset maximum area;

[0149] Based on the cropping frame adjustment ratio, the size of the initial cropping frame in the target video frame is adjusted to obtain a target cropping frame.

[0150] In an exemplary embodiment, the target position includes a top position of the target object in the target video frame, and a leftmost position or a rightmost position of the target object in the target video frame;

[0151] The adjustment module 13 is specifically used for:

[0152] In a case where the target video frame includes other objects except the target object, cropping the other objects to obtain a cropped target video frame;

[0153] Adjusting the top edge of the target cropping frame in the cropped target video frame to be aligned with the top position, to obtain the adjusted position of the target cropping frame in the vertical direction;

[0154] The left side of the target cropping frame is adjusted to the leftmost position, or the right side of the target cropping frame is adjusted to the rightmost position to obtain an adjusted position.

[0155] In an example embodiment, the setting parameters include a top reference distance range, a bottom reference distance range, a left reference distance range, and a right reference distance range; the top reference distance range is used to characterize the distance of the face area relative to the top edge of the video cover, the bottom reference distance range is used to characterize the distance of the face area relative to the bottom edge of the video cover, the left reference distance range is used to characterize the distance of the face area relative to the left edge of the video cover, and the right reference distance range is used to characterize the distance of the face area relative to the right edge of the video cover;

[0156] The device further comprises: a matching module, wherein:

[0157] The matching module is also used to:

[0158] Based on the adjusted position, determining a top distance between a top edge of a face detection frame corresponding to the face area and a top edge of the target cropping frame;

[0159] Based on the adjusted position, determining a bottom distance between a bottom edge of a face detection frame corresponding to the face area and a bottom edge of the target cropping frame;

[0160] Based on the adjusted position, determining a left distance between a left side edge of a face detection frame corresponding to the face area and a left side edge of the target cropping frame;

[0161] Based on the adjusted position, determining a right side distance between a right side edge of a face detection frame corresponding to the face area and a right side edge of the target cropping frame;

[0162] When the top distance matches the top reference distance range, the bottom distance matches the bottom reference distance range, the left distance matches the left reference distance range, and the right distance matches the right reference distance range, it is determined that the adjusted position matches the setting parameters.

[0163] In an example embodiment, the apparatus further comprises a processing module, wherein:

[0164] a processing module, for converting, for each video frame in the target video, an image of a face region of the target object into a grayscale image, and filtering the grayscale image to obtain a filtered image when the target object is included in the video frame;

[0165] The determination module 12 is further used to determine the variance of the filtered image;

[0166] The determination module 12 is further configured to determine the video frame as the target video frame when the variance is greater than or equal to a preset threshold.

[0167] In an exemplary embodiment, the determination module 12 is specifically configured to:

[0168] Determining a text area based on a position of a face area in the target image;

[0169] Determine a target pixel color average value in the text area, and determine a target text color corresponding to the target pixel color average value based on a correspondence between the pixel color average value and the text color;

[0170] The recommendation corresponding to the target video is displayed in the text area using the target text color to obtain a video cover of the target video.

[0171] In an exemplary embodiment, the determination module 12 is specifically configured to:

[0172] When the number of characters in the recommendation is greater than a preset number of characters, semantic segmentation is performed on the recommendation to obtain at least two segmented words;

[0173] Determine the number of segmented words displayed in each line based on the total number of segmented words and the preset number of lines;

[0174] Based on the number of word segments displayed in each line, the target text color is adopted to display the recommendation in the text area to obtain the video cover of the target video.

[0175] The device of this embodiment can be used to execute the method of any embodiment in the embodiment of the video cover determination method side. Its specific implementation process and technical effects are similar to those in the embodiment of the video cover determination method side. For details, please refer to the detailed introduction in the embodiment of the video cover determination method side, which will not be repeated here.

[0176] Fig.11 An example of a physical structure diagram of an electronic device is shown in FIG. Fig.11 As shown, the electronic device may include: a processor (processor) 1110 , a communication interface (Communications Interface) 1120 , a memory (memory) 1130 and a communication bus 1140 , wherein the processor 1110 , the communication interface 1120 , and the memory 1130 communicate with each other via the communication bus 1140 . The processor 1110 can call the logic instructions in the memory 1130 to execute the video cover determination method, which includes: receiving a video cover generation instruction for a target video input by a user, and in response to the video cover generation instruction, obtaining setting parameters of the target object in the video cover; for each target video frame in the target video, determining the target position of the target object in the target video frame; based on the area occupied by the face area of ​​the target object in the target video frame, adjusting the size of the initial cropping frame in the target video frame to obtain the target cropping frame; based on the target position, adjusting the position of the target cropping frame to obtain the adjusted position; when the adjusted position matches the setting parameters, cropping the target video frame based on the adjusted position to obtain the target image within the target cropping frame; and determining the video cover of the target video based on the target image.

[0177] In addition, the logic instructions in the above-mentioned memory 1130 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0178] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the video cover determination method provided by the above-mentioned methods, which includes: receiving a video cover generation instruction for a target video input by a user, and in response to the video cover generation instruction, obtaining setting parameters of a target object in the video cover; for each target video frame in the target video, determining a target position of the target object in the target video frame; based on the area occupied by the face area of ​​the target object in the target video frame, adjusting the size of an initial cropping frame in the target video frame to obtain a target cropping frame; based on the target position, adjusting the position of the target cropping frame to obtain an adjusted position; when the adjusted position matches the setting parameters, cropping the target video frame based on the adjusted position to obtain a target image within the target cropping frame; and determining the video cover of the target video based on the target image.

[0179] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the video cover determination method provided by the above-mentioned methods, the method comprising: receiving a video cover generation instruction for a target video input by a user, and in response to the video cover generation instruction, obtaining setting parameters of a target object in the video cover; determining a target position of the target object in the target video frame for each target video frame in the target video; adjusting the size of an initial cropping frame in the target video frame based on the area occupied by a face region of the target object in the target video frame to obtain a target cropping frame; adjusting the position of the target cropping frame based on the target position to obtain an adjusted position; when the adjusted position matches the setting parameters, cropping the target video frame based on the adjusted position to obtain a target image within the target cropping frame; and determining the video cover of the target video based on the target image.

[0180] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0181] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0182] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for determining a video cover, characterized in that: include: Receiving a video cover generation instruction for a target video input by a user, and in response to the video cover generation instruction, acquiring setting parameters of the target object in the video cover; The setting parameters include a top reference distance range, a bottom reference distance range, a left reference distance range and a right reference distance range; the top reference distance range is used to characterize the distance of the face area relative to the top edge of the video cover, the bottom reference distance range is used to characterize the distance of the face area relative to the bottom edge of the video cover, the left reference distance range is used to characterize the distance of the face area relative to the left edge of the video cover, and the right reference distance range is used to characterize the distance of the face area relative to the right edge of the video cover; For each target video frame in the target video, determining a target position of the target object in the target video frame based on a user recognition model and a portrait segmentation network model; Based on the area occupied by the face region of the target object in the target video frame, the size of the initial cropping frame in the target video frame is adjusted to obtain a target cropping frame; wherein the adjustment ratio of the initial cropping frame is: ,in, Indicates the cropping frame adjustment ratio. represents the area occupied by the face region of the target object in the target video frame, Indicates the preset minimum area. Indicates the preset maximum area. and A user input value included in the video cover generation instruction; Based on the target position, adjusting the position of the target cropping frame to obtain an adjusted position; In a case where the adjusted position matches the setting parameters, cropping the target video frame based on the adjusted position to obtain a target image within the target cropping frame; Determine a video cover of the target video based on the target image; The target position includes a top position of the target object in the target video frame, and a leftmost position or a rightmost position of the target object in the target video frame; The step of adjusting the position of the target cropping frame based on the target position to obtain the adjusted position includes: In a case where the target video frame includes other objects except the target object, cropping the other objects to obtain a cropped target video frame; Adjusting the top edge of the target cropping frame in the cropped target video frame to be aligned with the top position, to obtain the adjusted position of the target cropping frame in the vertical direction; Adjusting the left side of the target cropping frame to the leftmost position, or adjusting the right side of the target cropping frame to the rightmost position, to obtain an adjusted position; The method further comprises: In the case where each of the video frames includes the target object, normalizing the size of the face region of the target object in each of the video frames to the same size, converting the normalized face image into a grayscale image, and filtering the grayscale image to obtain a filtered image; For each of the filtered images, determining a variance of the filtered image; When the variance is greater than or equal to a preset threshold, determining the video frame corresponding to the filtered image as the target video frame; The determining the video cover of the target video based on the target image includes: Determining a text area based on a position of a face area in the target image; Determine a target pixel color average value in the text area, and determine a target text color corresponding to the target pixel color average value based on a correspondence between the pixel color average value and the text color; Displaying the recommendation corresponding to the target video in the text area using the target text color to obtain a video cover of the target video; The step of displaying the recommendation corresponding to the target video in the text area using the target text color to obtain a video cover of the target video includes: When the number of characters in the recommendation is greater than a preset number of characters, semantic segmentation is performed on the recommendation to obtain at least two segmented words; Determine the number of segmented words displayed in each line based on the total number of segmented words and the preset number of lines; Based on the number of word segments displayed in each line, the recommendation is displayed in the text area using the target text color to obtain a video cover of the target video; The step of adjusting the size of the initial cropping frame in the target video frame based on the area occupied by the face region of the target object in the target video frame to obtain the target cropping frame includes: determining a cropping frame adjustment ratio based on the area, where the cropping frame adjustment ratio is 1 when the area is greater than a preset minimum area and less than a preset maximum area, and where the area is greater than the preset maximum area, the cropping frame adjustment ratio is a ratio of the area to the preset maximum area; both the preset minimum area and the preset maximum area are adjustable; Based on the cropping frame adjustment ratio, the size of the initial cropping frame in the target video frame is adjusted to obtain a target cropping frame.

2. The video cover determination method according to claim 1, characterized in that: The adjusted position matches the setting parameter, including: Based on the adjusted position, determining a top distance between a top edge of a face detection frame corresponding to the face area and a top edge of the target cropping frame; Based on the adjusted position, determining a bottom distance between a bottom edge of a face detection frame corresponding to the face area and a bottom edge of the target cropping frame; Based on the adjusted position, determining a left distance between a left side edge of a face detection frame corresponding to the face area and a left side edge of the target cropping frame; Based on the adjusted position, determining a right side distance between a right side edge of a face detection frame corresponding to the face area and a right side edge of the target cropping frame; When the top distance matches the top reference distance range, the bottom distance matches the bottom reference distance range, the left distance matches the left reference distance range, and the right distance matches the right reference distance range, it is determined that the adjusted position matches the setting parameters.

3. A video cover determination device, characterized in that: include: A receiving module, used to receive a video cover generation instruction for a target video input by a user, and in response to the video cover generation instruction, obtain setting parameters of the target object in the video cover; the setting parameters include a top reference distance range, a bottom reference distance range, a left reference distance range and a right reference distance range; the top reference distance range is used to characterize the distance of the face area relative to the top edge of the video cover, the bottom reference distance range is used to characterize the distance of the face area relative to the bottom edge of the video cover, the left reference distance range is used to characterize the distance of the face area relative to the left edge of the video cover, and the right reference distance range is used to characterize the distance of the face area relative to the right edge of the video cover; A determination module, configured to determine, for each target video frame in the target video, a target position of the target object in the target video frame based on a user recognition model and a portrait segmentation network model; The adjustment module is used to adjust the size of the initial cropping frame in the target video frame based on the area occupied by the face area of ​​the target object in the target video frame to obtain a target cropping frame; wherein the adjustment ratio of the initial cropping frame is: ,in, Indicates the cropping frame adjustment ratio. represents the area occupied by the face region of the target object in the target video frame, Indicates the preset minimum area. Indicates the preset maximum area. and A user input value included in the video cover generation instruction; The adjustment module is further used to adjust the position of the target cropping frame based on the target position to obtain an adjusted position; A cropping module, configured to crop the target video frame based on the adjusted position to obtain a target image within the target cropping frame when the adjusted position matches the setting parameters; The determination module is further used to determine the video cover of the target video based on the target image; The target position includes a top position of the target object in the target video frame, and a leftmost position or a rightmost position of the target object in the target video frame; The adjustment module is specifically used for: In a case where the target video frame includes other objects except the target object, cropping the other objects to obtain a cropped target video frame; Adjusting the top edge of the target cropping frame in the cropped target video frame to be aligned with the top position, to obtain the adjusted position of the target cropping frame in the vertical direction; Adjusting the left side of the target cropping frame to the leftmost position, or adjusting the right side of the target cropping frame to the rightmost position, to obtain an adjusted position; The device also includes a processing module, wherein: The processing module is used for normalizing the size of the face area of ​​the target object in each of the video frames to the same size when the target object is included in each of the video frames, converting the normalized face image into a grayscale image, and filtering the grayscale image to obtain a filtered image; The determination module is further used to determine the variance of the filtered image for each of the filtered images; The determination module is further configured to determine, when the variance is greater than or equal to a preset threshold, a video frame corresponding to the filtered image as the target video frame; The determination module is specifically used for: Determining a text area based on a position of a face area in the target image; Determine a target pixel color average value in the text area, and determine a target text color corresponding to the target pixel color average value based on a correspondence between the pixel color average value and the text color; Displaying the recommendation corresponding to the target video in the text area using the target text color to obtain a video cover of the target video; The determining module is specifically used for: When the number of characters in the recommendation is greater than a preset number of characters, semantic segmentation is performed on the recommendation to obtain at least two segmented words; Determine the number of segmented words displayed in each line based on the total number of segmented words and the preset number of lines; Based on the number of word segments displayed in each line, the recommendation is displayed in the text area using the target text color to obtain a video cover of the target video; The adjustment module is specifically used for: determining a cropping frame adjustment ratio based on the area, where the cropping frame adjustment ratio is 1 when the area is greater than a preset minimum area and less than a preset maximum area, and where the area is greater than the preset maximum area, the cropping frame adjustment ratio is a ratio of the area to the preset maximum area; both the preset minimum area and the preset maximum area are adjustable; Based on the cropping frame adjustment ratio, the size of the initial cropping frame in the target video frame is adjusted to obtain a target cropping frame.

4. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the video cover determination method as described in claim 1 or 2 is implemented.

5. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the video cover determination method as claimed in claim 1 or 2 is implemented.