Methods for generating images for video thumbnails and apparatus thereof

Abstract

The present disclosure relates to a method for generating an image for a video thumbnail and an apparatus thereof. The method may include: receiving a selection of a first video that is a thumbnail generation target from a user terminal, detecting a scene change in the first video and dividing the first video into a plurality of sections for each scene based on the scene change, extracting one or more first images including a face of a character from each of the plurality of sections for each scene, calculating a quality score of a face region included in the first image using a first artificial intelligence model, generating a second image by changing the first image into a preset format when the quality score is a threshold or more, and transmitting the second image to the user terminal.

Description

Technical Field

[0001] This disclosure relates to a method and apparatus for generating images for video thumbnails. More specifically, it relates to a method and apparatus for automatically generating thumbnails or automatically generating images for creating thumbnails simply by selecting a video. Prior Technology

[0002] A video thumbnail is a still image representing the content of a video, serving as a preview before the user watches it. Primarily used on streaming platforms and social media, it visually summarizes the content's theme to attract viewers' attention. Effectively crafted thumbnails can increase click-through rates (CTR) and help expand the content's exposure and reach.

[0003] A common method for creating video thumbnails is for designers to select the scene that best represents the core content of the video, and then use image editing software to add appropriate text and graphic elements.

[0004] However, creating video thumbnails for video content on streaming platforms using the methods described above can be inefficient. Streaming platforms may contain thousands of video content items, and manually creating thousands of video thumbnails for each item would be highly inefficient. Therefore, a technology is needed to improve the efficiency of video thumbnail creation.

[0005] Prior technology documents

[0006] Patent documents

[0007] (Patent Document 0001) Korean Patent Publication No. 10-2024-0104384 (Published on July 5, 2024) Summary of the Invention

[0008] Problem to be solved:

[0009] The technical problem to be solved by some embodiments disclosed herein is to provide a method and apparatus for improving the efficiency of video thumbnail creation.

[0010] Another technical problem to be solved through some embodiments disclosed herein is to provide a method and apparatus for automatically generating multiple images for video thumbnails of video content.

[0011] Another technical problem to be solved through some embodiments disclosed herein is to provide a method and apparatus for creating video thumbnails using artificial intelligence.

[0012] It should be understood that the technical problems disclosed herein are not limited to those described above, and those skilled in the art can clearly understand other unmentioned technical problems through the following explanation.

[0013] The technical solution of the present invention is as follows:

[0014] According to an embodiment of this disclosure, an image generation method and apparatus for video thumbnails can be provided. The method includes the following steps: receiving a first video selected as the target for thumbnail generation from a user terminal; detecting scene switching in the first video and segmenting the first video into multiple scene intervals based on the scene switching; extracting at least one or more first images containing the faces of characters appearing in each of the multiple scene intervals; inputting the first images into a first artificial intelligence model and outputting a quality score of the facial regions contained in the first image; if the quality score is greater than or equal to a preset first threshold, converting the first image into a preset format to generate a second image; and sending the second image to the user terminal.

[0015] In one embodiment, the step of extracting at least one or more first images containing the face of a character from each of the plurality of scene intervals may include the following steps: generating an embedding vector corresponding to the facial region contained in each of the extracted first images; performing clustering based on the similarity of the embedding vectors to generate a first character image cluster and a second character image cluster; if the number of images included in the first character image cluster is greater than or equal to a second threshold, then identifying the first character as the main character; and deleting first images from the at least one or more first images that do not contain the main character.

[0016] In one embodiment, the following steps may also be included: arranging the second image based on the cluster of images of the characters appearing and displaying it on the user terminal.

[0017] In one embodiment, the step of extracting at least one or more first images containing the faces of characters appearing in each of the plurality of scene intervals includes the following step: deleting blank areas from each of the first images to maintain the aspect ratio.

[0018] In one embodiment, the step of extracting at least one or more first images containing the faces of characters appearing in each of the plurality of scene intervals may include the following steps: inputting the extracted first image into a second artificial intelligence model to detect the first image that does not meet the review criteria; and deleting the first image detected in the at least one or more first images.

[0019] In one embodiment, the step of extracting at least one or more first images containing the faces of characters appearing in each of the plurality of scene intervals includes the following steps: inputting multiple frames constituting the plurality of scene intervals into a second artificial intelligence model to output at least one or more first images containing the faces of characters appearing in each of the plurality of scene intervals. The second artificial intelligence model may be an artificial intelligence model trained to use viewing records of a second video to extract images from the first video whose similarity to scenes in the second video that have been viewed more than or equal to a preset number of times is greater than or equal to a third threshold.

[0020] In one embodiment, the step of inputting the first image into the first artificial intelligence model and outputting the quality score of the facial region contained in the first image may include the following steps: inputting the difference between the position of the facial region and a preset position in the first image into the first artificial intelligence model and outputting the quality score of the first image.

[0021] In one embodiment, the step of inputting the first image into the first artificial intelligence model and outputting the quality score of the facial region contained in the first image may include the following steps: inputting the difference between the size of the facial region and a preset size into the first artificial intelligence model and outputting the quality score of the first image.

[0022] In one embodiment, the step of inputting the first image into the first artificial intelligence model and outputting the quality score of the facial region contained in the first image may include the following steps: detecting the region corresponding to the eye in the facial region; inputting the difference between the size of the region corresponding to the eye and a preset size into the first artificial intelligence model and outputting the quality score of the first image.

[0023] In one embodiment, the step of inputting the first image into the first artificial intelligence model and outputting the quality score of the facial region contained in the first image may include the following steps: inputting the first image into the first artificial intelligence model, calculating the expression score of the facial region; and using the expression score to calculate the quality score.

[0024] In one embodiment, the step of generating the second image may include the following steps: if the quality score is greater than or equal to the first threshold and the expression score is less than the second threshold, then the first image is converted into a preset format to generate the second image.

[0025] In one embodiment, the first video is a video whose resolution is reduced through image downscaling, and the step of generating the second image may further include the following step: increasing the resolution of the second image.

[0026] In one embodiment, the step of generating the second image may include the following steps: if the size of the first image is smaller than the size of a preset format, then place the first image within the border of the preset format; and input the first image into a generative artificial intelligence model to generate a background image for the border that does not contain the first image; and synthesize the first image and the background image to generate the second image.

[0027] In one embodiment, the step of generating the second image may include the following steps: considering the position of the facial region of the first image and the blank area of ​​the first image, selecting a first format from a plurality of preset formats; and converting the first image to the first format to generate the second image.

[0028] In one embodiment, the following steps may also be included: arranging the second image based on the quality score and displaying it on the user terminal.

[0029] In one embodiment, the method may further include the following steps: a user receiving the user terminal selects a portion of the image from the second image; and using the selected second image, trains a third artificial intelligence model to predict the likelihood of the user's selection.

[0030] In one embodiment, the method may further include the following steps: inputting a second video into a third artificial intelligence model that has completed the training, and extracting images for thumbnails of the second video, wherein the second video is a different video from the first video.

[0031] In one embodiment, the method may further include the following steps: receiving a third image modified based on the second image from the user terminal; using the difference between the second image and the third image to train a fourth artificial intelligence model for extracting images from the video whose similarity to the third image is greater than or equal to a preset second threshold.

[0032] In one embodiment, the method may further include the following steps: inputting a second video into a fourth artificial intelligence model that has completed the training, and extracting images for thumbnails of the second video, wherein the second video is a different video from the first video. Simple Explanation of the Diagram

[0033] Figure 1 is a configuration diagram showing the overall configuration of an image generation apparatus for video thumbnails according to an embodiment of the present disclosure.

[0034] Figure 2 is a flowchart of an image generation method for video thumbnails according to another embodiment of the present disclosure.

[0035] Figure 3 is a detailed flowchart illustrating part of the operation of the method described with reference to Figure 2.

[0036] Figure 4 is another detailed flowchart illustrating part of the operation of the method described with reference to Figure 2.

[0037] Figure 5 is a further detailed flowchart illustrating part of the operation of the method described with reference to Figure 2.

[0038] Figure 6 is a detailed flowchart illustrating the other operations following the method described in Figure 2.

[0039] Figure 7 is a detailed flowchart illustrating another operation following the method described in Figure 2.

[0040] Figures 8 to 10 show screens displayed during partial operation of the method described with reference to Figure 2.

[0041] Figures 11 and 12 are example diagrams used to illustrate the method described with reference to Figure 2.

[0042] Figure 13 is an example diagram used to illustrate the method described with reference to Figure 3.

[0043] Figures 14 and 15 are another example diagrams used to illustrate the method described with reference to Figure 2.

[0044] Figure 16 is an example diagram used to illustrate the method described with reference to Figure 5.

[0045] Figures 17 and 18 are further example diagrams used to illustrate the method described with reference to Figure 2.

[0046] Figure 19 is a hardware structure diagram of a computing device used in some embodiments of this disclosure. Implementation

[0047] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings. The advantages and features of this disclosure, as well as the methods for achieving these advantages and features, will be clearly understood by referring to the detailed embodiments and drawings described below. However, the technical concept of this disclosure is not limited to the following embodiments, but can be implemented in many forms. The purpose of the following embodiments is only to make the technical concept of this disclosure more complete and to fully inform those skilled in the art of the subject matter of this disclosure of the scope of the disclosure. The technical concept of this disclosure is defined only within the scope of the claims.

[0048] When assigning element symbols to the technical features in the various figures, it should be noted that for the same technical feature, even if it is indicated in different figures, the same technical feature should have the same element symbol as much as possible. Furthermore, when describing the embodiments disclosed herein, detailed descriptions of related well-known structures or functions will be omitted if it is deemed that such detailed descriptions would obscure the main points.

[0049] Unless otherwise defined, all terms used in this disclosure (including technical and scientific terms) may be used in the sense commonly understood by those skilled in the art to which the embodiments of this disclosure pertain. Furthermore, terms defined in commonly used dictionaries should not be interpreted ideally or excessively unless explicitly defined otherwise. The terminology used in this disclosure is for the purpose of describing embodiments and not for limiting the embodiments of this disclosure. In this disclosure, unless specifically mentioned otherwise, the singular includes the plural.

[0050] Furthermore, when describing the technical features of the embodiments disclosed herein, terms such as first, second, A, B, (a), and (b) may be used. These terms are used only to distinguish technical features from other technical features, and the nature, order, or sequence of the corresponding technical features are not limited by these terms. When a technical feature is described as "connected," "coupled," or "connected" to another element, that technical feature may be directly connected or connected to that other technical feature, and each technical feature may also be "connected," "coupled," or "connected" to another technical feature.

[0051] First, before describing the embodiments of this disclosure, the application objects of this disclosure will be described. This disclosure relates to a method and apparatus for generating images for video thumbnails.

[0052] In this context, a video thumbnail can refer to an image on a streaming platform that represents video content, enabling users of the platform to identify it. Specifically, a video thumbnail may include posters or preview images of video content on a page where video content can be selected. Furthermore, a video thumbnail may also include preview images of each episode on a page where episode numbers can be selected, as well as images used to introduce the video content. In other words, the video thumbnails disclosed herein can include all forms of images displayed on a streaming platform to represent video content.

[0053] Furthermore, images used for thumbnails can include images that are themselves thumbnails, as well as images that can be used to create thumbnails. Images that can be used to create thumbnails may be manually edited or modified to create thumbnails. Going forward, "thumbnail" and "image used for thumbnails" may be used interchangeably, and a thumbnail may refer to "image used for thumbnails" unless otherwise stated.

[0054] The configuration and operation of an image generation apparatus for video thumbnails according to an embodiment of this disclosure will now be described with reference to FIG1. ​​Hereinafter, the image generation apparatus for video thumbnails will be simply referred to as a thumbnail generation apparatus.

[0055] The thumbnail generation device 2 disclosed herein can extract video thumbnails 30 from a video 10 corresponding to video content. Specifically, the thumbnail generation device 2 can receive a selection of videos for thumbnail generation from a user terminal 1. Furthermore, the thumbnail generation device 2 can extract multiple images from the video 10. In addition, the thumbnail generation device 2 can use an artificial intelligence model 3 to select images from the extracted multiple images whose quality scores are greater than or equal to a preset threshold for use as video thumbnails 30. Here, quality score can refer to the standard for measuring whether a specific image is suitable for use as a thumbnail or poster. Quality score can be calculated by comprehensively considering various factors, such as brightness, sharpness, resolution, color accuracy, visual balance, characteristics of objects in the image (e.g., object position, number of objects, object size), and whether it is a main character. Images with high quality scores have outstanding visual effects; therefore, when using the corresponding image as a thumbnail on a streaming media platform, users of the streaming media platform can clearly identify the video content corresponding to the thumbnail. Therefore, if thumbnails of video content are created based on images with high quality scores, the convenience for users of the streaming media platform can be improved, and users can be guided to watch the corresponding video content.

[0056] In addition, the thumbnail generation device 2 can send the image used for the video thumbnail 30 to the user terminal 1 so that the user of the user terminal can edit the image.

[0057] In some embodiments, at least two of the user terminal 1, thumbnail generation device 2, and artificial intelligence model 3 may be the same computing device. For example, user terminal 1 and thumbnail generation device 2 may be a first computing device, and the computing device containing artificial intelligence model 3 may be a second computing device.

[0058] In some embodiments, at least one of the user terminal 1, the thumbnail generation device 2, and the artificial intelligence model 3 may be composed of multiple computing devices. For example, the user terminal 1 may be composed of a first computing device (e.g., a laptop computer) and a second computing device (e.g., a desktop computer), and the thumbnail generation device 2 may be composed of a third computing device (e.g., a first server) and a fourth computing device (e.g., a second server).

[0059] It should be noted that the operation of the thumbnail generation apparatus 2 disclosed herein is not limited to the examples described above, and may include operations related to the image generation method for video thumbnails (hereinafter referred to as the thumbnail generation method) described below in some embodiments of this disclosure. Furthermore, even without further description, the technical concepts that can be understood through the embodiments of this disclosure described below can also be applied to the above-described thumbnail generation apparatus.

[0060] So far, a thumbnail generation apparatus according to an embodiment of the present disclosure has been described with reference to FIG1. ​​Next, a thumbnail generation method according to some embodiments of the present disclosure will be described with reference to FIGS. 2 to 18. In the following description, the execution entity of a specific step / operation may be omitted, and it can be understood that the step / operation is performed by the computing device constituting the thumbnail generation apparatus described above.

[0061] First, referring to FIG2, a thumbnail generation method according to an embodiment of the present disclosure will be described.

[0062] First, as shown in Figure 2, the user terminal can receive a selection of videos as targets for thumbnail generation (S10). It should be noted that receiving video selection includes not only receiving the video data itself, but also receiving information that a particular video data item has already been selected.

[0063] For example, as shown in Figure 8, a webpage 100 managing video content on a streaming media platform can be displayed on a user's terminal screen. When the button 101 for generating an image for a thumbnail is clicked, the video content matching the corresponding webpage 100 is selected as the thumbnail generation target. Information indicating the selection of the corresponding video content can then be received from the user's terminal.

[0064] Secondly, scene transitions can be detected in the video, and the video can be segmented into multiple scene intervals based on these transitions (S20). A scene transition refers to a visual change between frames in the video due to changes in the information or plot of the video content. Specifically, videos matching the selected video content for thumbnail generation can be retrieved from a database. Then, the retrieved videos can be analyzed frame-by-frame. If visual and auditory changes are detected as a result of the analysis, a scene transition can be determined. For example, differences between frames (e.g., brightness differences between frames) can be compared based on factors such as image brightness, color distribution, and edge changes, and a scene transition can be detected when these differences exceed a specific threshold. Furthermore, objects in the video can be sensed and tracked, and scene transitions can be detected when objects change. Additionally, changes in background music or dialogue can be sensed to detect scene transitions. When a scene transition is detected, the interval between adjacent time points at the time of detection can be defined as a scene interval. Through this process, the video can be segmented into multiple scene intervals.

[0065] As a specific example, as shown in Figure 11, video 10 can be acquired. For ease of explanation, assume the video consists of 25 frames. First, video 10 can be analyzed frame by frame. As a result of the analysis, visual changes can be detected between frames F3 and F4, between frames F9 and F10, between frames F16 and F17, and between frames F20 and F21. Therefore, based on the detected visual changes, scene intervals can be determined and the video segmented. Specifically, frames F1 to F3 can be defined as the first interval, frames F4 to F9 as the second interval, frames F10 to F16 as the third interval, frames F17 to F20 as the fourth interval, and frames F21 to F25 as the fifth interval. That is, video 10 can be segmented into five scene intervals.

[0066] In one embodiment, the video used to detect scene transitions and segment into multiple scene intervals can be a video whose resolution has been reduced by downscaling the original video. The original video matching the video content selected by the user is stored as is in a separate database, and only the downscaled version of the original video can be retrieved. Further processing can be performed to generate images for video thumbnails from the retrieved downscaled video. Thus, in the operations described below, when using an artificial intelligence model, low-resolution video can be used to improve the processing speed of the artificial intelligence model and reduce processing costs.

[0067] Returning to Figure 2, at least one or more images containing the faces of characters can be extracted from each of the multiple scene intervals (S30). Specifically, only images of higher quality can be extracted from the images representing each of the multiple scene intervals. Image quality can be determined based on the image's brightness value and its standard deviation. Furthermore, image quality can be determined based on the sharpness determined by calculating the image's gradient using various screening programs (e.g., Sobel screening program, Laplacian screening program). Additionally, only images containing the faces of characters can be extracted. This can be determined by analyzing facial regions in the images using a deep learning-based detection model.

[0068] For example, as shown in Figure 12, images 20 containing the faces of characters appearing in the scene can be extracted from the first interval S1 to the fifth interval S5. Image extraction can include extracting individual frames within a scene interval and synthesizing multiple frames to extract a single image.

[0069] In one embodiment, after extracting at least one or more images containing the faces of characters from each of multiple scene segments, blank areas used to maintain aspect ratio can be removed from the extracted images. These blank areas can refer to letterboxes or pillars used to maintain aspect ratio. A letterbox can refer to blank areas located above and below the video to match the aspect ratio. Similarly, a pillar can refer to blank areas located on the left and right sides of the video to match the aspect ratio.

[0070] For example, as shown in Figure 14, an image 26a containing mailbox 26b can be extracted from a segment of the video scene. After extraction, the extracted image 26a can automatically remove mailbox 26b. The image 26c with mailbox 26b removed can be further processed to generate an image for the video thumbnail.

[0071] For example, as shown in Figure 14, an image 27a containing the microphone 27b can be extracted from a segment of the video scene. After extraction, the extracted image 27a can automatically remove the microphone 27b. The image 27c with the microphone 27b removed can be further processed to generate an image for the video thumbnail.

[0072] In one embodiment, an artificial intelligence model can be used to extract images from each of multiple scene intervals. Here, the artificial intelligence model can refer to an artificial intelligence model trained using other videos and viewing records of those videos.

[0073] Specifically, artificial intelligence models can be trained to predict which scenes viewers are likely to watch more often by analyzing viewing patterns in specific scenarios. During training, these models can receive various video viewing data (e.g., playback time, pause frequency, repeated viewing intervals, cumulative view counts) and extract viewing patterns for specific scenes as features. The trained model can then predict viewer preferences based on the video's visual and auditory features, content flow, and user behavior data.

[0074] For example, as shown in Figure 15, the scene with the highest cumulative number of views can be input into the artificial intelligence model. The scene with the highest cumulative number of views can refer to the scene corresponding to the video time with the highest cumulative number of views within a preset video time interval. Specifically, frame f1 at video time t1 and frame f2 at video time t2 are local maximums and can therefore be considered the scene with the highest cumulative number of views. By inputting frame f1 at video time t1 and frame f2 at video time t2 into the artificial intelligence model, the model can learn viewing patterns.

[0075] Subsequently, when this AI model is applied to any video, it can analyze the video frame by frame. Then, within multiple frames of the video, the AI ​​model can output an image whose similarity to the scene with the most cumulative views in that local area, learned during training, is greater than or equal to a threshold.

[0076] Returning to Figure 2, an artificial intelligence model can be used to calculate the quality score of the facial regions contained in the image (S40). Specific operations will be described below along with the embodiment in Figure 5.

[0077] Then, it can be determined whether the quality score is greater than or equal to a threshold (S50). If the quality score is less than the threshold, thumbnail generation can be terminated. On the other hand, if the quality score is greater than or equal to the threshold, the extracted image can be converted into a poster format (S60).

[0078] In one embodiment, if the resolution of the extracted image is inconsistent with that of the poster format, upscaling or a deep learning-based algorithm can be applied to improve the image resolution.

[0079] In one embodiment, if the aspect ratio of the extracted image is inconsistent with that of the poster format, a generative artificial intelligence model can be used to make the aspect ratio of the extracted image consistent with that of the poster format.

[0080] Specifically, if the extracted image does not match the proportions of the poster format, and the extracted image is not cropped to meet the resolution of the poster format, the generative AI model can generate a background image for the blank area to match the proportions of the poster format.

[0081] For example, as shown in Figure 17, image 20 can be extracted. The preset poster can be a vertical poster, and the preset poster format can be a 34-border design. The extracted image 20 can be placed within the 34-border design. At this time, within the 34-border design, whitespace 35a and 35b may appear above and below the extracted image 20. Whitespace 35a and 35b can represent the areas within the 34-border design that do not contain the extracted image. Then, for the upper and lower whitespace 35a and 35b, the generative AI model can generate background images. At this time, the positions of the whitespace 35a and 35b, along with the extracted image 20, can be input into the generative AI model. In response to this input, background images 36a and 36b related to the extracted image can be generated. The generated background images 36a and 36b and the extracted image 20 can be synthesized to generate a new poster-formatted image 30.

[0082] In one embodiment, the poster format can be determined by considering the location of the facial region and the location of the blank area in the extracted image. The blank area can refer to the region in the image that does not contain any characters.

[0083] Specifically, the location of facial areas and white space can be considered to determine the poster format, such as portrait, landscape, or square. When modifying the extracted image according to the poster format, it can be confirmed whether the modified image contains facial areas exceeding a threshold, or whether the modified image contains any main characters that were previously removed. Based on the confirmation results, the poster format can be determined to include facial areas exceeding the threshold without removing main characters.

[0084] For example, as shown in Figure 18, image 20 can be extracted from one of multiple scene intervals. Image 20 can be an image containing two football players. The preset poster format can be either a horizontal or vertical poster format. When the extracted image 20 is modified according to the border 34a of the horizontal poster format, the modified image can contain the two main football players, and the facial area of ​​each player is above a threshold. Conversely, when the extracted image 20 is modified according to the border 34b of the vertical poster format, the modified image may only include a portion of the two main football players, and the facial area of ​​each player is also smaller than the threshold. Therefore, the preset poster format for the extracted image 20 can be determined as a horizontal poster format. Then, the extracted image 20 as a horizontal poster can be modified to generate image 30 for thumbnails.

[0085] It should be noted that the poster format is not limited to the examples above. In some embodiments, the poster format (e.g., resolution, aspect ratio, width, and height) can be determined by the user.

[0086] Returning to Figure 2, the converted image can be sent to the user terminal (S70). The sent image can be displayed on the user terminal's screen. For example, as shown in Figure 9, multiple images 30 for video thumbnails can be displayed on a webpage 100 that manages video content.

[0087] Then, when one of the images displayed on the user's terminal is selected, the selected image, or an image modified based on the selected image, can be used as a thumbnail of the corresponding video content. For example, as shown in Figure 10, a video thumbnail 40 can be displayed on a webpage 100 that manages video content.

[0088] In one embodiment, images used for video thumbnails can be displayed in order of their quality scores. For example, in image 30 shown in FIG9, images with higher quality scores for video thumbnails can be displayed closer to the left and top. A high-quality-score image can refer to an image whose brightness and sharpness meet standards, and whose facial region position, size, eye size, and facial expression score meet preset conditions.

[0089] The thumbnail generation method according to an embodiment of the present disclosure has been illustrated above with reference to FIG2. The operations described above will now be explained in detail.

[0090] Figure 3 is an example of the operations that can be performed after extracting at least one or more images containing the faces of characters from each of multiple scene intervals.

[0091] First, as shown in Figure 3, after extracting at least one or more images containing the faces of characters from each of multiple scene intervals, an embedding vector corresponding to the facial region contained in each extracted image can be generated (S31). Specifically, the process of generating the embedding vector of the facial region in the image can begin with the step of recognizing faces in the image through a face detection algorithm. After standardization and preprocessing, the detected facial regions can be input into a deep learning-based face recognition model. The face recognition model extracts the unique features of the face and converts them into a high-dimensional embedding vector, which can refer to a unique numerical value representing the facial features.

[0092] In one embodiment, the process of generating embedding vectors for facial regions can begin with detecting faces in an image, for which a deep learning-based detection algorithm can be used. Next, facial landmarks representing key features such as the eyes, nose, and mouth can be extracted from the detected facial regions to perform face alignment and normalization. The aligned facial images are then fed into a deep learning-based face recognition model, which can extract unique facial features and convert them into high-dimensional embedding vectors.

[0093] Returning to Figure 3, clustering (S32) can be performed using the generated embedding vectors. Clustering groups images containing the same person based on the similarity of the embedding vectors. The image grouping process, measured using facial embedding vector similarity, can utilize mathematical methods such as cosine similarity or Euclidean distance to measure the similarity between the generated embedding vectors. Cosine similarity calculates similarity based on the angle between vectors; the closer the value is to 1, the greater the probability that they are the same person. A specific threshold can be set based on these similarity scores to group images containing the same person.

[0094] Returning to Figure 3, image clusters for each character can be generated through grouping (S33). For example, the embedding vector can be grouped with multiple images similar to the image containing the first character to generate an image cluster for the first character. Similarly, the embedding vector can be grouped with multiple images similar to the image containing the second character to generate an image cluster for the second character.

[0095] As a specific example, as shown in Figure 13, multiple extracted images 20 can be classified according to the characters appearing in the images. Among the multiple extracted images 20, images containing the first character can form a first character image cluster 21. Furthermore, among the multiple extracted images 20, images containing the second character can form a second character image cluster 22. Furthermore, among the multiple extracted images 30, images containing the third character can form a third character image cluster 23. Furthermore, among the multiple extracted images 30, one image containing the fourth character can form a fourth character image cluster 24. Furthermore, among the multiple extracted images 30, one image containing the fifth character can form a fifth character image cluster 25.

[0096] Returning to Figure 3, if the number of images in a cluster of images of characters appearing in a scene is greater than or equal to a threshold, then that character can be identified as a main character (S34). For example, the threshold can be assumed to be 10 images. If the number of images in the first cluster of images of characters appearing in a scene is 20, then the first character can be identified as a main character. Conversely, if the number of images in the second cluster of images of characters appearing in a scene is 5, then the second character may not be identified as a main character. Thus, the main characters that match the plot of the video content can be automatically identified. Images of characters that include the main character or are close to the main character in the video content can be extracted without additional manual settings.

[0097] Returning to Figure 3, images that do not contain the main characters can be deleted (S35). Therefore, only images containing the main characters can be selected for the thumbnails.

[0098] As a specific example, as shown in Figure 13, the fourth and fifth character image clusters may each contain only one image. Here, we assume the threshold for determining the main character is three images. In this case, the fourth and fifth characters are not identified as main characters. Therefore, the images included in the fourth and fifth character image clusters 24 and 25 can be deleted. Ultimately, images containing the fourth and fifth characters will not be generated as images for the thumbnail.

[0099] That is, this embodiment can group images extracted from multiple scene ranges according to the characters appearing in the images. Furthermore, this embodiment can determine the main characters based on the grouped images. Additionally, this embodiment can generate only images containing the main characters as thumbnails. Ultimately, this embodiment can generate images containing the main characters as thumbnails.

[0100] In one embodiment, images for thumbnails can be displayed on a user terminal based on the aforementioned cluster of images of the characters appearing in the images. For example, in the webpage 100 shown in FIG9, the first row of the area containing the images 30 for thumbnails may contain only images of the first character appearing in the images. Meanwhile, the second row of the area containing the images 30 for thumbnails may contain only images of the second character appearing in the images.

[0101] Therefore, users on the user terminal can identify the main characters without viewing the video content, and can easily generate thumbnails for each character.

[0102] Figure 4 is another example of the operations that can be performed after extracting at least one or more images containing the faces of characters from each of multiple scene intervals.

[0103] First, as shown in Figure 4, after extracting at least one or more images containing the faces of characters from each of multiple scene intervals, an artificial intelligence model can be used to detect images that do not meet the review criteria (S36). Specifically, the extracted images can be input into the artificial intelligence model to output images that do not meet the review criteria. More specifically, an embedding vector corresponding to the input image can be generated. Then, the similarity between the embedding vector corresponding to the image that meets the review criteria and the generated embedding vector can be measured. If the similarity is greater than or equal to a threshold, the input image can be determined to meet the review criteria. Additionally, the similarity between the embedding vector corresponding to the image that does not meet the review criteria and the generated embedding vector can be measured. If the similarity is greater than or equal to a threshold, the input image can be determined to not meet the review criteria.

[0104] Among them, the artificial intelligence model can be a model trained to judge whether an image meets the review criteria using images that do not meet the review criteria and images that meet the review criteria.

[0105] The review standards can include statutory review standards and the review standards of the streaming platform itself. Statutory review standards, based on the applicable laws and regulations of various countries, can be review standards aimed at maintaining public order and social values ​​by regulating elements such as obscenity, violence, hate speech, copyright infringement, and defamation. The review standards of the streaming platform itself are based on platform policies and community guidelines, and can consider more detailed regulations related to user experience and brand image. Platform review standards can also regulate elements such as violent scenes, political sensitivity, and false information.

[0106] Therefore, an AI model can detect images that may contain violence or obscenity, hate speech, or spoilers from extracted images based on pre-set censorship criteria. The AI ​​model can be trained using images that do not meet the censorship criteria.

[0107] Then, the detected images can be deleted (S37). Thus, only images that meet the review criteria can be identified as images for use as thumbnails.

[0108] Figure 5 is an example illustrating the operation of using an artificial intelligence model to calculate the quality score of facial regions contained in an image.

[0109] First, as shown in Figure 5, the quality score can be calculated using the location of the facial region (S41). Specifically, the more similar the location of the extracted facial region in the image is to the location of a preset facial region, the higher the calculated quality score can be; conversely, the more different the location of the extracted facial region from the preset facial region, the lower the calculated quality score can be. The preset facial region location can be a location where the entire facial region is contained within the image. Furthermore, the preset facial region location can be the exact center of the image, or the intersection of a vertical line dividing the image into thirds and a horizontal line dividing the image into thirds.

[0110] For example, as shown in Figure 16, assume that the preset facial region position 201 in the image is the exact center of the extracted image. In this case, the facial region 202 of the first image 200 in Figure 16 is further from the center than the facial region 203 of the second image 210. Therefore, the quality score of the first image 200 can be calculated to be lower than that of the second image 210.

[0111] Returning to Figure 5, the quality score can be calculated using the size of the facial region (S42). Specifically, the more similar the size of the extracted facial region is to the preset facial region size, the higher the calculated quality score can be; conversely, the more different the size of the extracted facial region is from the preset facial region size, the lower the calculated quality score can be. The preset facial region size can be the size of one of nine regions created by dividing the image into thirds using vertical and horizontal lines.

[0112] For example, as shown in Figure 16, assume that the preset size of the facial region 204 is the same as that shown in the attached figure. However, the difference between the size of the facial region 205 in the third image 220 of Figure 16 and the preset size of the facial region 204 may be greater than the difference between the size of the facial region 204 in the fourth image 230 and the preset size of the facial region 204. Therefore, the quality score of the third image 220 can be calculated to be lower than the quality score of the fourth image 230.

[0113] Returning to Figure 5, the quality score can be calculated using the character's eye size (S43). Specifically, if the character's eyes are closed and therefore small in size, the quality score can be calculated as low; if the character's eyes are open and therefore large in size, the quality score can be calculated as high.

[0114] For example, as shown in Figure 16, assume the preset eye size is the size when the eyes are open. The fifth image 240 in Figure 16 can be a state 207a with one eye closed. Conversely, the sixth image 250 in Figure 16 can be a state 207b with both eyes open. Therefore, the quality score of the fifth image 240 can be calculated to be lower than the quality score of the sixth image 250.

[0115] In the above description, the extracted image can be input into an artificial intelligence model and output to obtain the size of the facial region, the position of the facial region, and the region corresponding to the eye. Additionally, the quality score can be calculated arithmetically using the difference between the size of the facial region and a preset size, or by inputting the size of the facial region and the preset size into the artificial intelligence model. Furthermore, the quality score can be calculated arithmetically using the difference between the position of the facial region and a preset position in the image, or by inputting the position of the facial region and the preset position in the image into the artificial intelligence model. Finally, the quality score can be calculated arithmetically using the difference between the features of the region corresponding to the eye (e.g., size, position, degree of eye opening and closing, number of times the eyes open and close, etc.) and preset features, or by inputting the features of the region corresponding to the eye and preset features into the artificial intelligence model.

[0116] Returning to Figure 5, the quality score can be calculated using expression scores (S44). First, an image containing a face can be input to feed into a trained deep learning model. Then, the deep learning model can output a classification of the facial expressions in the image (e.g., smiling, surprised, angry). Furthermore, the deep learning model can output the degree to which the facial expressions convey emotion through expression scores. The greater the degree to which an expression conveys emotion, the higher the calculated expression score; conversely, the less pronounced the expression, the lower the calculated expression score. Then, the quality score can be calculated based on the expression scores. It is important to note that regardless of the type of emotion expressed, the higher the degree to which an expression conveys emotion, the higher the calculated expression score. For example, images containing crying expressions and images containing hearty laughter can both yield high expression scores. In other words, expressions that clearly express emotion can be calculated with higher expression scores, while expressions with unclear emotion, such as no expression, can be calculated with lower expression scores.

[0117] As a specific example, the seventh image 260 in Figure 16 contains a smiling expression 208, and the eighth image 270 in Figure 16 contains a laughing expression 208. As a result of sentiment analysis, the smiling expression 208 expresses a lower degree of emotion than the laughing expression 209; therefore, the expression score of the seventh image 260 can be calculated to be lower than that of the eighth image 270. Consequently, the quality score of the seventh image 260 can be calculated to be lower than that of the eighth image 270.

[0118] In one embodiment, images with an expression score greater than or equal to a threshold may not be selected for use as video thumbnails. The expression score threshold may be a different value than the quality score threshold. Images with an expression score greater than or equal to the threshold may be images that evoke excessive emotions (e.g., images of extreme anger or crying). Excluding these images when generating images for video thumbnails prevents the generation of thumbnails that may cause discomfort to the user.

[0119] The calculated quality score can be used to determine the final quality score, and the determined quality score can be used to determine whether the corresponding image should be converted to a poster format.

[0120] Figure 6 shows an example of the operations that can be performed after the thumbnail generation method described with reference to Figure 2.

[0121] First, as shown in Figure 6, the user terminal can receive a selection of images from the sent images (S80). Receiving this information may include the selected image itself and information about which image was selected. The user may be a user of a webpage or application that creates video thumbnails.

[0122] Secondly, the selected images can be used to train an artificial intelligence model (S90). The artificial intelligence model can be trained to predict the likelihood that a user will select an image as a thumbnail from among the images contained in the video. The video can include videos different from the target video used for training the artificial intelligence model.

[0123] Then, the trained artificial intelligence model can be used to extract images for thumbnails from the video (S100). Specifically, the artificial intelligence model can be trained by having a user select a subset of images from the generated thumbnails after multiple images for the first video have been generated. Furthermore, if a second video is input into the artificial intelligence model, the model can generate multiple images for thumbnails of the second video. Through training, the artificial intelligence model can generate images that are highly likely to be selected by the user for video thumbnails.

[0124] For example, according to some embodiments of the thumbnail generation method disclosed herein, a first image and a second image for a thumbnail of a first video can be generated. The first image can be selected to create the thumbnail of the first video, and the thumbnail can be created based on the first image. The first image can be an image containing only the face of the main character, and the second image can be an image containing the faces of the main character and other characters. An artificial intelligence model can learn, by utilizing the difference between the generated image and the selected image, that a user is more likely to choose an image containing only the face of the main character. Then, the trained artificial intelligence model can be used to generate a third image and a fourth image for a thumbnail of the second video. Both the third and fourth images can be images containing only the face of the main character.

[0125] Figure 7 is another example of the operations that can be performed after the thumbnail generation method described with reference to Figure 2.

[0126] First, as shown in FIG7, a modified image can be received from the user terminal (S81). The modified image may be an image generated by a thumbnail generation method according to some embodiments of this disclosure for use as a video thumbnail, which has been modified.

[0127] Secondly, the modified images can be used to train an artificial intelligence model (S91). The artificial intelligence model can be trained to extract images from the video that have a similarity to the modified images greater than or equal to a preset threshold. The video can include videos different from the training target video of the artificial intelligence model.

[0128] Then, the trained artificial intelligence model can be used to extract images for thumbnails from the video (S101). Specifically, the artificial intelligence model can learn the differences between images generated by the thumbnail generation method according to some embodiments of this disclosure and images modified based on the generated images. Furthermore, if a second video is input into the artificial intelligence model, the model can generate multiple images for thumbnails of the second video. Through training, the artificial intelligence model can generate images similar to the modified images.

[0129] For example, according to some embodiments of the thumbnail generation method disclosed herein, a first image for a thumbnail of a first video can be generated. The thumbnail of the first video can be created based on the first image. The first image may be an image containing the faces of characters and a large amount of white space, while the thumbnail of the first video may be an image with the large white space removed, leaving the faces of characters occupying most of the image.

[0130] At this point, the AI ​​model can be trained to extract an image containing only the faces of the characters without any white space, based on the differences between the generated and modified images. Then, the trained AI model can be used to generate a third image for the thumbnail of the second video. This third image can be an image containing only the faces of the characters without significant white space.

[0131] In the above explanation, similarity refers to the similarity of embedding vectors. An embedding vector corresponding to each piece of data can be generated, and the similarity of the generated embedding vectors can be calculated. If the calculated similarity is greater than or equal to a threshold, the two pieces of data can be considered similar. Mathematical methods such as cosine similarity or Euclidean distance can be used to calculate the similarity. Cosine similarity is based on the angle between vectors; the closer the value is to 1, the more similar the two pieces of data are considered.

[0132] The effects of the technical ideas disclosed herein are not limited to the aforementioned technical problems; those skilled in the art will clearly understand from the description in this disclosure other effects not mentioned.

[0133] So far, thumbnail generation methods according to some embodiments of the present disclosure have been described with reference to FIGS. 2 to 18. Next, the hardware structure of an exemplary computing device according to some embodiments of the present disclosure will be described with reference to FIG. 19. This computing device may be a computing device that drives the thumbnail generation device of the present disclosure.

[0134] Figure 19 is an exemplary hardware configuration diagram of a computing device that can be implemented in various embodiments of this disclosure. A computing device 1000 according to this embodiment may include one or more processors 1100, a system bus 1600, a communication interface 1200, memory 1400 loading a computer program 1500 executed by the processor 1100, and storage 1300 storing the computer program 1500. Figure 19 only shows technical features relevant to embodiments of this disclosure. Therefore, those skilled in the art to which this disclosure pertains will understand that other general technical features may be included in addition to those shown in Figure 19.

[0135] Processor 1100 controls the overall operation of the various structures of computing system 1000. Processor 1100 may include at least one of a central processing unit (CPU), a microprocessor unit (MPU), a microcontroller unit (MCU), a graphics processing unit (GPU), or any type of processor known in the industry. Furthermore, processor 1100 can perform calculations for at least one application or program to perform methods / operations according to various embodiments of this disclosure. Computing device 1000 may have two or more processors.

[0136] Memory 1400 stores various contents, commands, and / or information. Memory 1400 can load one or more programs 1500 from storage 1300 to perform the methods and / or actions of various embodiments of this disclosure. Examples of memory 1400 may be RAM, but are not limited thereto. System bus 1600 provides communication functionality between the technical features of computing device 1000.

[0137] The system bus 1600 can be implemented through various types of buses, such as address bus, data bus, and control bus. The communication interface 1200 supports wired and wireless communication of the computing device 1000. The communication interface 1200 can also support various communication methods other than network communication. Therefore, the communication interface 1200 can include communication modules known in the art to which this disclosure pertains. The storage device 1300 can non-temporarily store one or more computer programs 1500. The storage device 1300 can be non-volatile memory such as flash memory, a hard drive, a removable hard drive, or any form of computer-readable recording medium known in the art to which this disclosure pertains.

[0138] Computer program 1500 may include one or more instructions for implementing the methods / actions of the various embodiments of the present disclosure. When computer program 1500 is loaded into memory 1400, processor 1100 may execute the methods / actions of the various embodiments of the present disclosure by executing the one or more instructions.

[0139] For example, computer program 1500 may include instructions to perform the following operations: receiving from a user terminal the selection of a first video as the target for thumbnail generation; detecting scene switching in the first video and segmenting the first video into multiple scene intervals based on the scene switching; extracting at least one or more first images containing the faces of characters appearing in each of the multiple scene intervals; calculating a quality score of the facial regions contained in the first image using a first artificial intelligence model; if the quality score is greater than or equal to a threshold, converting the first image into a preset format to generate a second image; and sending the second image to the user terminal.

[0140] Although the various operations are shown in a specific order in the accompanying drawings, it should not be construed as meaning that the operations must be performed in the specific order shown or sequentially, or that all the shown operations must be performed to obtain the desired result. In some cases, multiplexing and parallel processing may also be advantageous. Moreover, it should be understood that the separation of the various structures in the embodiments described above should not be construed as requiring such separation; the described program elements and systems can generally be integrated together as a single software product, or packaged into multiple software products.

[0141] The embodiments of this specification have been described above with reference to the accompanying drawings. However, those skilled in the art will understand that the embodiments of this specification can be implemented in other specific forms without altering the technical concept or essential features disclosed herein. Therefore, it should be understood that the embodiments described above are exemplary and not limiting in all respects. The scope of protection of this disclosure should be interpreted by the appended claims, and all technical ideas within their equivalent scope should be understood to be included within the scope of the technical ideas defined through this disclosure.

[0142] 1: User terminal 2: Thumbnail generation device 3: Artificial Intelligence Model 10: Video Images 20, 26a, 26c, 27a, 27c: 21, 22, 23, 24, 25: Clusters of images of characters appearing in the scene. 26b: Mailbox 27b: Mailbox 30, 40: Video thumbnails 100: Webpage 101: Button to generate an image for thumbnails 200, 210, 220, 230, 240, 250, 260, 270: Images 201: The location of the preset facial region in the image 202, 203, 205, 206: Facial areas 204: Preset size of the facial area 207a: Image with one eye closed 207b: Image with eyes open 208, 209: Expressions 34: Preset poster format borders 34a: Border for horizontal poster format 34b: Border for vertical poster format 35a, 35b: White space 36a, 36b: Background images 1000: Computing device 1100: Processor 1200: Communication Interface 1300: Storage 1400: Memory 1500: Computer Program f1, f2, f3, f4, f5, f6, f7, f8, f9, f10, f11, f12, f13, f14, f15, f16, f17, f18, f19, f20, f21, f22, f23, f24, f25: frames S1, S2, S3, S4, S5: Intervals S10, S20, S30, S31, S32, S33, S34, S35, S36, S37, S40, S41, S42, S43, S44, S50, S60, S70, S80, S81, S90, S91, S100, S101: Steps t1, t2: Video duration

Claims

1. A method for generating image thumbnails for videos, executed via a computing device, comprising the following steps: receiving a first video selected as the target for thumbnail generation from a user terminal; detecting scene switching in the first video and segmenting the first video into multiple scene intervals based on the scene switching; extracting at least one or more first images containing the faces of characters appearing in each of the multiple scene intervals; inputting the first images into a first artificial intelligence model and outputting a quality score of the facial regions contained in the first image; if the quality score is greater than or equal to a preset first threshold, converting the first image into a preset format to generate a second image; and sending the second image to the user terminal, wherein... The step of extracting at least one or more first images containing the faces of characters from each of the multiple scene intervals includes the following steps: generating an embedding vector corresponding to the facial regions contained in each of the extracted first images; performing clustering based on the similarity of the embedding vectors to generate a first character image cluster and a second character image cluster; if the number of images included in the first character image cluster is greater than or equal to a second threshold, then the first character is identified as the main character; and deleting first images from the at least one or more first images that do not contain the main character.

2. The image generation method for video thumbnails as described in claim 1, wherein, It also includes the following steps: arranging the second image based on the cluster of images of the characters and displaying it on the user terminal.

3. The image generation method for video thumbnails as described in claim 1, wherein, The step of extracting at least one or more first images containing the faces of characters appearing in each of the multiple scene intervals includes the following steps: deleting blank areas from each of the first images to maintain the aspect ratio.

4. The image generation method for video thumbnails as described in claim 1, wherein, The step of extracting at least one or more first images containing the faces of characters appearing in each of the multiple scene intervals includes the following steps: inputting the extracted first image into a second artificial intelligence model to detect the first image that does not meet the review criteria; and deleting the first image detected in the at least one or more first images.

5. The image generation method for video thumbnails as described in claim 1, wherein, The step of extracting at least one or more first images containing the faces of characters appearing in each of the multiple scene intervals includes the following steps: inputting multiple frames constituting the multiple scene intervals into a second artificial intelligence model to output at least one or more first images containing the faces of characters appearing in each of the multiple scene intervals, wherein the second artificial intelligence model is an artificial intelligence model trained to extract images from the first video that have a similarity greater than or equal to a third threshold to scenes in the second video that have been viewed more than or equal to a preset number of times using viewing records of the second video.

6. The image generation method for video thumbnails as described in claim 1, wherein, The step of inputting the first image into the first artificial intelligence model and outputting the quality score of the facial region contained in the first image includes the following steps: inputting the difference between the position of the facial region and the preset position in the first image into the first artificial intelligence model and outputting the quality score of the first image.

7. The image generation method for video thumbnails as described in claim 1, wherein, The steps of inputting the first image into the first artificial intelligence model and outputting the quality score of the facial region contained in the first image include the following steps: inputting the difference between the size of the facial region and a preset size into the first artificial intelligence model and outputting the quality score of the first image.

8. The image generation method for video thumbnails as described in claim 1, wherein, The steps of inputting the first image into the first artificial intelligence model and outputting the quality score of the facial region contained in the first image include the following steps: detecting the region corresponding to the eye in the facial region; inputting the difference between the size of the region corresponding to the eye and a preset size into the first artificial intelligence model and outputting the quality score of the first image.

9. The image generation method for video thumbnails as described in claim 1, wherein, The step of inputting the first image into the first artificial intelligence model and outputting the quality score of the facial region contained in the first image includes the following steps: inputting the first image into the first artificial intelligence model, calculating the expression score of the facial region; and using the expression score to calculate the quality score.

10. The image generation method for video thumbnails as described in claim 9, wherein, The steps to generate the second image include the following: If the quality score is greater than or equal to the first threshold and the expression score is less than the second threshold, then the first image is converted into a preset format to generate the second image.

11. The image generation method for video thumbnails as described in claim 1, wherein, The first video is a video whose resolution has been reduced through image downscaling. The steps to generate the second image also include the following steps: increasing the resolution of the second image.

12. The image generation method for video thumbnails as described in claim 1, wherein, The steps for generating the second image include the following steps: If the size of the first image is smaller than the size of a preset format, the first image is placed within the border of the preset format; and the first image is input into a generative artificial intelligence model to generate a background image for the border that does not contain the first image; and the first image and the background image are synthesized to generate the second image.

13. The image generation method for video thumbnails as described in claim 1, wherein, The steps for generating the second image include the following: considering the position of the facial region and the white space of the first image, selecting a first format from multiple preset formats; and converting the first image to the first format to generate the second image.

14. The image generation method for video thumbnails as described in claim 1, wherein, It also includes the following steps: arranging the second image based on the quality score and displaying it on the user terminal.

15. The image generation method for video thumbnails as described in claim 1, wherein, It also includes the following steps: the user receiving the user terminal selects a portion of the image in the second image; and using the selected second image, trains a third artificial intelligence model to predict the likelihood of the user's selection.

16. The image generation method for video thumbnails as described in claim 15, wherein, It also includes the following steps: inputting the second video into the third artificial intelligence model that has completed the training, and extracting images for the thumbnails of the second video, which is a different video from the first video.

17. The image generation method for video thumbnails as described in claim 1, wherein, It also includes the following steps: receiving a third image modified based on the second image from the user terminal; using the difference between the second image and the third image, training a fourth artificial intelligence model for extracting images from the video whose similarity to the third image is greater than or equal to a preset second threshold.

18. The image generation method for video thumbnails as described in claim 17, wherein, It also includes the following steps: inputting the second video into the fourth artificial intelligence model that has completed the training, and extracting images for the thumbnails of the second video, which is a different video from the first video.

19. An image generation apparatus for video thumbnails, comprising: Communication interface; Memory, loading computer programs; The system includes one or more processors executing a computer program comprising instructions to perform the following operations: receiving a selection of a first video as a target for thumbnail generation from a user terminal; detecting scene transitions in the first video and segmenting the first video into multiple scene intervals based on the scene transitions; extracting at least one or more first images containing the faces of characters appearing in each of the multiple scene intervals; inputting the first images into a first artificial intelligence model and outputting a quality score for the facial regions contained in the first image; if the quality score is greater than or equal to a preset first threshold, converting the first image into a preset format to generate a second image; and sending the second image to the user terminal, wherein the operation of extracting at least one or more first images containing the faces of characters appearing in each of the multiple scene intervals includes the following operations: generating an embedding vector corresponding to the facial regions contained in each of the extracted first images; performing clustering based on the similarity of the embedding vectors to generate a first character image cluster and a second character image cluster; if the number of images included in the first character image cluster is greater than or equal to a second threshold, identifying the first character as the main character. Delete the first image that does not contain the main character from at least one or more first images.

Images