Video thumbnail processing method and apparatus, electronic device, and computer-readable storage medium

Abstract

Embodiments of the present application disclose a video thumbnail processing method and device, electronic equipment and a computer readable storage medium. The method comprises: obtaining a thumbnail splicing rule from a server; loading a thumbnail splicing picture of a video from the server; the thumbnail splicing picture is spliced by a plurality of thumbnails according to the thumbnail splicing rule; in response to a thumbnail display request, obtaining a playing time of the video; according to the playing time and the thumbnail splicing rule, cutting a thumbnail corresponding to the playing time from a target thumbnail splicing picture corresponding to the playing time; and displaying the thumbnail corresponding to the playing time. Thus, the present application can smoothly display the thumbnail of the video.

Description

Technical Field

[0001] This application relates to the field of video processing technology, specifically to a video thumbnail processing method, apparatus, electronic device, and computer-readable storage medium. Background Technology

[0002] Video thumbnails are still images extracted from a video, typically keyframes. Thumbnails help users quickly understand the video content. In related technologies, when pausing or fast-forwarding a video, the terminal sends a thumbnail request to the server based on the video's playback time. The server then returns the thumbnail corresponding to that playback time, which the terminal loads. However, this method struggles to load thumbnails promptly, resulting in choppy thumbnail display. Summary of the Invention

[0003] This application provides a video thumbnail processing method, apparatus, electronic device, and computer-readable storage medium that can smoothly display video thumbnails.

[0004] In a first aspect, embodiments of this application provide a video thumbnail processing method, applied to a terminal, comprising:

[0005] Retrieve thumbnail stitching rules from the server;

[0006] The server loads a thumbnail stitched image of the video; the thumbnail stitched image is composed of multiple thumbnails stitched together according to the thumbnail stitching rules;

[0007] In response to a thumbnail display request, obtain the playback time of the video;

[0008] Based on the playback time and the thumbnail stitching rules, extract the thumbnail corresponding to the playback time from the target thumbnail stitching image corresponding to the playback time; and

[0009] Display a thumbnail corresponding to the playback time.

[0010] In one embodiment, the step of extracting the thumbnail corresponding to the playback time from the target thumbnail stitching image corresponding to the playback time, based on the playback time and the thumbnail stitching rules, includes:

[0011] Based on the thumbnail stitching rules, the playback time range corresponding to each of the thumbnail stitched images is determined;

[0012] The thumbnail stitched image corresponding to the playback time range to which the playback time belongs is determined as the target thumbnail stitched image;

[0013] Obtain the number of stitches represented by the thumbnail stitching rule;

[0014] Based on the playback time, the number of stitches, and the target playback time range corresponding to the target thumbnail stitching image, determine the index of the thumbnail corresponding to the playback time in the target thumbnail stitching image;

[0015] Based on the index and the thumbnail stitching rules, a thumbnail corresponding to the playback time is extracted from the target thumbnail stitching image.

[0016] In one embodiment, determining the index of the thumbnail corresponding to the playback time in the target thumbnail splicing image based on the playback time, the number of stitches, and the target playback time range corresponding to the target thumbnail splicing image includes:

[0017] Based on the target playback time range, determine the playback duration corresponding to the target thumbnail splicing image;

[0018] Determine the starting playback time within the target playback time range, and obtain the difference between the playback time and the starting playback time;

[0019] Obtain the quotient of the difference and the playback duration, and determine the index of the thumbnail corresponding to the playback time in the target thumbnail splicing image based on the quotient and the number of splices.

[0020] In one embodiment, the step of extracting the thumbnail corresponding to the playback time from the target thumbnail stitching image according to the index and the thumbnail stitching rules includes:

[0021] Obtain the resolution and arrangement rules represented by the thumbnail stitching rules;

[0022] Based on the index, the resolution, and the arrangement rules, a thumbnail corresponding to the playback time is extracted from the target thumbnail stitching image.

[0023] Secondly, embodiments of this application provide a video thumbnail processing method, applied to a server, comprising:

[0024] Generate multiple thumbnails of the video based on the target playback duration and resolution;

[0025] According to the thumbnail stitching rules, multiple thumbnails are stitched together into a thumbnail stitched image;

[0026] The thumbnail stitched image and the thumbnail stitching rules are sent to the terminal; the thumbnail stitched image is used to: extract the thumbnail corresponding to the playback time from the target thumbnail stitched image corresponding to the playback time, based on the thumbnail stitched image and the playback time of the video in the terminal.

[0027] In one embodiment, the step of stitching multiple thumbnails into a thumbnail stitched image according to thumbnail stitching rules includes:

[0028] Obtain the number of stitches and the arrangement rules represented by the thumbnail stitching rules;

[0029] The multiple thumbnails of the video are divided into multiple groups of thumbnails according to the time sequence and the number of stitches;

[0030] According to the time sequence and the arrangement rules, each group of thumbnails is stitched together to form the thumbnail stitched image.

[0031] Thirdly, embodiments of this application provide a video thumbnail processing apparatus, applied to a terminal, comprising:

[0032] The rule acquisition module is used to obtain thumbnail stitching rules from the server;

[0033] A loading module is used to load a thumbnail stitched image of a video from the server; the thumbnail stitched image is composed of multiple thumbnails stitched together according to the thumbnail stitching rules;

[0034] The time acquisition module is used to acquire the playback time of the video in response to the thumbnail display request;

[0035] The cropping module is used to crop a thumbnail corresponding to the playback time from a target thumbnail stitched image corresponding to the playback time, based on the playback time and the thumbnail stitching rules; and

[0036] The display module is used to display thumbnails corresponding to the playback time.

[0037] In one embodiment, the interception module includes:

[0038] The range unit is used to determine the playback time range corresponding to each of the thumbnail splicing images according to the thumbnail splicing rules.

[0039] The target determination unit is used to determine the thumbnail stitched image corresponding to the playback time range to which the playback time belongs as the target thumbnail stitched image;

[0040] The quantity acquisition unit is used to acquire the number of stitches represented by the thumbnail stitching rules;

[0041] An index determination unit is used to determine the index of the thumbnail corresponding to the playback time in the target thumbnail splicing image based on the playback time, the number of splices, and the target playback time range corresponding to the target thumbnail splicing image;

[0042] The cropping unit is used to crop the thumbnail corresponding to the playback time from the target thumbnail stitching image according to the index and the thumbnail stitching rules.

[0043] In one embodiment, the index determination unit includes:

[0044] The duration determination subunit is used to determine the playback duration corresponding to the target thumbnail splicing image based on the target playback time range;

[0045] The difference acquisition subunit is used to determine the starting playback time within the target playback time range and acquire the difference between the playback time and the starting playback time.

[0046] The index determination subunit is used to obtain the quotient of the difference and the playback duration, and determine the index of the thumbnail corresponding to the playback time in the target thumbnail splicing image based on the quotient and the number of splices.

[0047] In one embodiment, the interception unit includes:

[0048] A sub-unit is used to obtain the resolution and arrangement rules represented by the thumbnail stitching rules;

[0049] The cropping subunit is used to crop the thumbnail corresponding to the playback time from the target thumbnail stitching image according to the index, the resolution and the arrangement rules.

[0050] Fourthly, embodiments of this application provide a video thumbnail processing apparatus, applied to a server, comprising:

[0051] The generation module is used to generate multiple thumbnails of the video based on the target playback duration and resolution;

[0052] The stitching module is used to stitch multiple thumbnails into a thumbnail stitched image according to thumbnail stitching rules;

[0053] The sending module is used to send the thumbnail stitched image and the thumbnail stitching rules to the terminal; the thumbnail stitched image is used to: extract the thumbnail corresponding to the playback time from the target thumbnail stitched image corresponding to the playback time, based on the thumbnail stitched image and the playback time of the video in the terminal.

[0054] In one embodiment, the splicing module includes:

[0055] The acquisition unit is used to acquire the number of stitches and the arrangement rules represented by the thumbnail stitching rules;

[0056] A segmentation unit is used to divide the multiple thumbnails of the video into multiple groups of thumbnails according to the time sequence and the number of stitches.

[0057] The stitching unit is used to stitch together each group of thumbnails into the thumbnail stitched image according to the time sequence and the arrangement rules.

[0058] Fifthly, embodiments of this application also provide an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the steps in the video thumbnail processing method described above.

[0059] Sixthly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in the video thumbnail processing method described above.

[0060] In a seventh aspect, embodiments of this application also provide a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in the various optional implementations described in embodiments of this application.

[0061] In summary, in this embodiment of the application, since the thumbnail stitching image includes multiple thumbnails, loading a single thumbnail stitching image is more efficient than loading multiple thumbnails. When it is necessary to display a video thumbnail, the corresponding thumbnail can be directly extracted from the target thumbnail stitching image loaded in the terminal according to the video playback time and thumbnail stitching rules, without having to send a request to the server based on the video playback time and obtain the thumbnail returned by the server. This saves transmission time and enables fast and smooth thumbnail display. Attached Figure Description

[0062] To more clearly illustrate the technical solutions in this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0063] Figure 1 This is a schematic diagram of the steps of a video thumbnail processing method provided in an embodiment of this application;

[0064] Figure 2 This is a schematic diagram of the steps of a video thumbnail processing method provided in an embodiment of this application;

[0065] Figure 3This is a schematic diagram of the interaction between a terminal and a server provided in an embodiment of this application;

[0066] Figure 4 This is a schematic flowchart of a video thumbnail processing method provided in an embodiment of this application;

[0067] Figure 5 This is a schematic diagram of the structure of a video thumbnail processing device provided in an embodiment of this application;

[0068] Figure 6 This is a schematic diagram of the structure of a video thumbnail processing device provided in an embodiment of this application.

[0069] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0070] The technical solutions of this application will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0071] In one embodiment, such as Figure 1 As shown, a video thumbnail processing method is provided. Although the logical order is shown in the schematic diagram, in some cases, the steps shown or described may be performed in a different order than that shown in the figures. Specifically, this video thumbnail processing method can be applied to a terminal, which may include, but is not limited to, one or more of smartphones, tablets, laptops, desktop computers, and in-vehicle computers.

[0072] The following sections provide detailed descriptions of each example. It should be noted that the order in which the embodiments are described is not intended to limit the priority of the embodiments.

[0073] according to Figure 1 The video thumbnail processing method shown includes at least steps S110 to S150, which are described in detail below:

[0074] In step S110, thumbnail stitching rules are obtained from the server.

[0075] Thumbnail stitching rules are the rules for combining multiple thumbnails into a single thumbnail stitched image. These rules may include, but are not limited to: the number of thumbnails included in a single stitched image, the position of each thumbnail within the stitched image, the resolution of each thumbnail, and / or the playback time corresponding to each thumbnail. Thumbnail resolution includes both width and height information. For example, a thumbnail resolution of 192x108 pixels indicates a width of 192 pixels and a height of 108 pixels.

[0076] In one embodiment, when a terminal opens a video, the terminal sends a thumbnail stitching rule retrieval request to the server, and the server responds to the thumbnail stitching rule retrieval request by sending the thumbnail stitching rules to the terminal.

[0077] In step S120, a thumbnail stitched image of the video is loaded from the server; the thumbnail stitched image is composed of multiple thumbnails stitched together according to the thumbnail stitching rules.

[0078] The server can have storage capabilities, either using its own storage or by utilizing Object Storage Service (OSS) to store pre-generated thumbnail stitched images of videos. A video can include multiple thumbnail stitched images, and each thumbnail stitched image can include multiple thumbnails.

[0079] The server extracts multiple thumbnails from the video and stitches them together into a single thumbnail image according to thumbnail stitching rules. This extraction can be achieved by dividing the video into multiple segments and generating a thumbnail for each segment based on a key video frame or the nth frame, where n is a positive integer. Key video frames are those highly relevant to the content of the video segment; the method for determining key video frames can refer to relevant technologies.

[0080] In one embodiment, when a terminal opens or plays a video, the terminal sends a thumbnail stitching image retrieval request to the server. The server responds to the request by sending multiple thumbnail stitching images of the video to the terminal, which then loads these images. Alternatively, the server may proactively send thumbnail stitching images to the terminal without requiring a request from the terminal.

[0081] In step S130, in response to the thumbnail display request, the playback time of the video is obtained.

[0082] A thumbnail display request is used to request the display of a thumbnail corresponding to the playback time of the video. This request may be triggered in response to a target action on the video, such as pausing, fast-forwarding, rewinding, and / or dragging the video playback progress bar. It's understood that the video's playback time is the time elapsed after the target action on the video was performed. For example, if a user fast-forwards the video by 10 seconds at 1 minute and 40 seconds, the video's playback time is 1 minute and 50 seconds. Similarly, if a user places the cursor at a specific position on the video progress bar, or drags (and holds) the video progress bar to a specific position, the time corresponding to that position is the video's playback time.

[0083] In step S140, based on the playback time and the thumbnail stitching rules, a thumbnail corresponding to the playback time is extracted from the target thumbnail stitching image corresponding to the playback time.

[0084] Based on the video's playback time, the target thumbnail stitched together from multiple thumbnail images of the loaded video can be determined. According to the video's playback time and thumbnail stitching rules, the image area occupied by the thumbnail corresponding to that playback time within the target thumbnail stitched together can be determined. By cropping this image area, the thumbnail corresponding to that playback time can be obtained.

[0085] In one embodiment, thumbnails corresponding to each playback time can be pre-extracted from the various thumbnail stitching images based on the video's playback times and thumbnail stitching rules. In response to a thumbnail display request, the pre-extracted thumbnail corresponding to the playback time specified in the thumbnail display request is directly retrieved.

[0086] In step S150, a thumbnail corresponding to the playback time is displayed.

[0087] Display a thumbnail corresponding to the extracted playback time. Optionally, the extracted thumbnail can be displayed as a small image above the video playback progress bar, or the extracted thumbnail can occupy the entire video playback page.

[0088] The technical solution adopted in this application embodiment, because the thumbnail stitching image includes multiple thumbnails, can save resources and improve efficiency by loading a single thumbnail stitching image compared to loading multiple thumbnails. When it is necessary to display a video thumbnail, it is only necessary to directly extract the corresponding thumbnail from the target thumbnail stitching image loaded in the terminal according to the video playback time and thumbnail stitching rules, and then display it. There is no need to send a request to the server according to the video playback time and obtain the thumbnail returned by the server, which can save transmission time and achieve fast and smooth thumbnail display.

[0089] Based on the above technical solution, as an embodiment, the step of extracting the thumbnail corresponding to the playback time from the target thumbnail stitching image corresponding to the playback time according to the playback time and the thumbnail stitching rules may include: determining the playback time range corresponding to each thumbnail stitching image according to the thumbnail stitching rules; determining the thumbnail stitching image corresponding to the playback time range to which the playback time belongs as the target thumbnail stitching image; obtaining the stitching quantity represented by the thumbnail stitching rules; determining the index of the thumbnail corresponding to the playback time in the target thumbnail stitching image according to the playback time, the stitching quantity, and the target playback time range corresponding to the target thumbnail stitching image; and extracting the thumbnail corresponding to the playback time from the target thumbnail stitching image according to the index and the thumbnail stitching rules.

[0090] Given the playback time range of the video segment corresponding to each thumbnail, the number of thumbnails included in the stitched thumbnail image can be determined according to the thumbnail stitching rules. Since the thumbnails in the stitched thumbnail image are stitched in order according to their corresponding playback times, the playback time range corresponding to each thumbnail can be determined based on the playback time range of the video segment and the number of stitches. If a playback time falls within the playback time range of a thumbnail stitched image, then the thumbnail corresponding to that playback time is within that thumbnail stitched image. Therefore, the thumbnail stitched image corresponding to the playback time range to which the playback time belongs is determined as the target thumbnail stitched image.

[0091] In one embodiment, a thumbnail is generated based on every 10-second video segment. These 20 thumbnails are then stitched together into a single thumbnail stitched image. The playback time range corresponding to the first thumbnail stitched image is (0, 200], the playback time range corresponding to the second thumbnail stitched image is (200, 400], and so on. When the playback time is 260 seconds, since 260 seconds falls within the playback time range corresponding to the second thumbnail stitched image, the second thumbnail stitched image can be determined as the target thumbnail stitched image corresponding to the 260-second playback time.

[0092] All thumbnails in the thumbnail mosaic have the same resolution. Therefore, by determining the index of the thumbnail corresponding to the playback time in the thumbnail mosaic, the image area occupied by the thumbnail corresponding to the playback time in the thumbnail mosaic can be determined.

[0093] The index of a thumbnail in a thumbnail mosaic can be determined based on the playback time corresponding to the thumbnail, the number of stitches, and the target playback time range corresponding to the target thumbnail mosaic. Optionally, the order of the playback time within the target playback time range is determined based on the playback time corresponding to the thumbnail and the target playback time range. The order of the playback time within the target playback time range is consistent with the order of the thumbnails corresponding to the playback time in the thumbnail mosaic; therefore, the index of the thumbnail in the target thumbnail mosaic can be determined based on the order of the playback time within the target playback time range and the number of stitches corresponding to the thumbnail mosaic.

[0094] In one embodiment, determining the index of the thumbnail corresponding to the playback time in the target thumbnail splicing image based on the playback time, the number of stitches, and the target playback time range corresponding to the target thumbnail splicing image may include: determining the playback duration corresponding to the target thumbnail splicing image based on the target playback time range; determining the starting playback time within the target playback time range and obtaining the difference between the playback time and the starting playback time; obtaining the quotient of the difference and the playback duration, and determining the index of the thumbnail corresponding to the playback time in the target thumbnail splicing image based on the quotient and the number of stitches.

[0095] Calculate the difference between the playback time and the starting playback time, and then calculate the quotient of this difference with the playback duration. This quotient essentially represents the order of the playback time within the target playback time range. Therefore, based on the quotient and the number of stitches, the index of the thumbnail corresponding to the playback time in the target thumbnail stitched image can be determined.

[0096] Continuing with the previous example, the playback time is 260 seconds, the playback time range corresponding to the target thumbnail stitching image is (200, 400], and the number of stitches is 20. Therefore, the difference between the playback time and the starting playback time is 260 - 200 = 60 seconds. The quotient of this difference and the playback duration is 60 ÷ (400 - 200) = 6 / 20. This quotient indicates that the playback time is the 60 / 200th second from the starting playback time. The index of the thumbnail corresponding to the 260th second playback time in the target thumbnail stitching image is 60 / 200 × 20 = 6, meaning this thumbnail is the 6th thumbnail in the target thumbnail stitching image.

[0097] When the calculated index is not an integer, it can be rounded up, rounded down, or rounded to obtain the final index. In one embodiment, the index of the thumbnail corresponding to the 261st second in the target thumbnail stitching image is 61 / 200×20=6.1. Rounding 6.1 up yields a final index of 7, so the thumbnail corresponding to the 261st second is the 7th thumbnail in the target thumbnail stitching image.

[0098] In this way, the index of the thumbnail corresponding to the playback time in the thumbnail splicing image can be determined based on the playback time, the number of splices, and the target playback time range. Subsequently, the thumbnail corresponding to the playback time can be extracted from the target thumbnail splicing image based on the index.

[0099] In one embodiment, extracting the thumbnail corresponding to the playback time from the target thumbnail stitching image according to the index and the thumbnail stitching rules may include: obtaining the resolution and arrangement rules represented by the thumbnail stitching rules; and extracting the thumbnail corresponding to the playback time from the target thumbnail stitching image according to the index, the resolution, and the arrangement rules.

[0100] The thumbnail stitching rules specify the resolution of the thumbnails and the arrangement rules for multiple thumbnails. Based on the thumbnail resolution, the image area occupied by each thumbnail can be determined; based on the arrangement rules for multiple thumbnails, the number of rows and columns corresponding to the thumbnails of each index can be determined. Based on the number of rows and columns corresponding to the thumbnails, and the thumbnail resolution, the image area of ​​each index's thumbnail in the stitched thumbnail image can be determined.

[0101] In one embodiment, the resolution of the thumbnail is 192x108 pixels; one thumbnail consists of 20 thumbnails, and the arrangement rule of the thumbnails indicates that the 20 thumbnails are arranged in an orderly manner according to the format of 5 rows × 4 columns. With the top left corner of the thumbnail splicing image as the origin, the vertical downward direction is the positive direction of the vertical axis, and the horizontal right direction is the positive direction of the horizontal axis, the coordinates of the top left pixel of the image area corresponding to the thumbnail with index 6 are (192, 108), the coordinates of the bottom left pixel are (192, 216), the coordinates of the top right pixel are (384, 108), and the coordinates of the bottom right pixel are (384, 216).

[0102] In this way, by simply using the thumbnail index and thumbnail stitching rules, the image area of ​​each index's thumbnail in the thumbnail stitching image can be quickly determined.

[0103] The technical solution adopted in this application embodiment does not require a separate request from the server for the thumbnail corresponding to the playback time. Since the target thumbnail stitching image is already loaded, when it is necessary to display the thumbnail corresponding to the playback time, only an image cropping operation is needed to display the thumbnail, which has the advantages of being fast and efficient.

[0104] In one embodiment, such as Figure 2As shown, a video thumbnail processing method is provided. Although the logical order is illustrated in the step diagram, in some cases, the steps shown or described can be performed in a different order than that shown in the diagram. Specifically, this video thumbnail processing method can be applied to a server, which can be a physical server or a cloud server providing various cloud services. It is worth noting that this application does not limit the number of terminals or servers. Depending on the implementation needs, there can be any number of terminals or servers. For example, the server can be a single server or a server cluster composed of multiple servers, etc.

[0105] The following sections provide detailed descriptions of each example. It should be noted that the order in which the embodiments are described is not intended to limit the priority of the embodiments.

[0106] according to Figure 2 The video thumbnail processing method shown includes at least steps S210 to S230, which are described in detail below:

[0107] In step S210, multiple thumbnails of the video are generated based on the target playback duration and resolution.

[0108] The target playback duration is a pre-configured playback duration corresponding to each thumbnail. The server divides the video into multiple video segments based on the target playback duration, with each segment's playback duration being the target duration. A thumbnail is then generated for each video segment.

[0109] Optionally, a thumbnail corresponding to each video segment can be generated based on the key video frame or the nth frame in each video segment, where n is a positive integer. The key video frame can be a video frame whose content relevance to the video segment is higher than a threshold. The method for determining the key video frame can refer to relevant technologies.

[0110] After determining the key video frame or the nth frame in the video clip, the key video frame or the nth frame in the video clip can be adjusted according to the resolution defined by the thumbnail stitching rules so that the key video frame or the nth frame in the video clip has that resolution, thereby obtaining the thumbnail corresponding to the video clip.

[0111] In step S220, multiple thumbnails are stitched together into a thumbnail stitched image according to the thumbnail stitching rules.

[0112] The thumbnail stitching rules define the number of thumbnails included in a thumbnail stitched image, as well as the arrangement rules for multiple thumbnails. Following the thumbnail stitching rules, the specified number of thumbnails are stitched together sequentially to form a thumbnail stitched image.

[0113] In one embodiment, stitching multiple thumbnails into a thumbnail stitched image according to thumbnail stitching rules may include: obtaining the stitching quantity and arrangement rules represented by the thumbnail stitching rules; dividing the multiple thumbnails of the video into multiple groups of thumbnails according to the time order and the stitching quantity; and stitching each group of thumbnails into the thumbnail stitched image according to the time order and the arrangement rules.

[0114] The arrangement rules represent the number of rows and columns of thumbnails in the thumbnail splicing image. Multiple thumbnails are divided into multiple groups based on their playback time sequence and the number of thumbnails included in the splicing image. Each group of thumbnails is then spliced ​​together to form the final thumbnail splicing image, according to the stated time sequence and the number of rows and columns in the thumbnail splicing image.

[0115] In one embodiment, the number of thumbnails is 20. The arrangement rule of the thumbnails indicates that the 20 thumbnails are arranged in an orderly manner according to the format of 5 rows × 4 columns. Then, according to the time order of each thumbnail, the number of rows and columns of each thumbnail in the thumbnail splicing image can be determined in turn, so that each thumbnail can be spliced ​​to obtain the thumbnail splicing image.

[0116] In step S230, the thumbnail stitching image and the thumbnail stitching rules are sent to the terminal; the thumbnail stitching image is used to: extract the thumbnail corresponding to the playback time from the target thumbnail stitching image corresponding to the playback time according to the thumbnail stitching image and the playback time of the video in the terminal.

[0117] In one embodiment, when a terminal opens a video, the terminal sends a thumbnail stitching rule retrieval request and a thumbnail stitching image retrieval request to the server. The server responds to the thumbnail stitching rule retrieval request and the thumbnail stitching image retrieval request by sending the thumbnail stitching rule and multiple thumbnail stitching images to the terminal.

[0118] In response to a thumbnail display request triggered by a target operation on the video, the terminal obtains the video playback time and extracts the thumbnail corresponding to the playback time from the target thumbnail stitching image.

[0119] The technical solution adopted in this application embodiment includes multiple thumbnails. Therefore, compared with sending multiple thumbnails to the terminal, the server sending the thumbnail stitched image can reduce the number of images sent, save transmission resources and improve efficiency. When the terminal needs to display a thumbnail of a video, it only needs to directly extract the corresponding thumbnail from the target thumbnail stitched image according to the video playback time and thumbnail stitching rules and display it. There is no need to send a request to the server according to the video playback time and obtain the thumbnail returned by the server. This can save transmission time and achieve fast and smooth thumbnail display.

[0120] In one embodiment, Figure 3 This is a schematic diagram of communication between a terminal and a server provided in an embodiment of this application, as shown below. Figure 3 As shown, the server generates multiple thumbnails based on the video, stitches them together according to thumbnail stitching rules, and stores these thumbnails on a storage server. This storage server can be independent of the server that generated the thumbnail stitching, or it can be the same server as the one that generated the thumbnail stitching. When the terminal opens the video, it sends a request to the server to retrieve the thumbnail stitching rules. The server responds to this request by sending the thumbnail stitching rules to the terminal.

[0121] The terminal can send a request to the storage server to retrieve the thumbnail mosaic image while playing or opening a video. The storage server will then return the thumbnail mosaic image to the terminal, which will then preload the image.

[0122] In response to a thumbnail display request triggered by a target operation on the video, the terminal obtains the video's playback time, extracts the thumbnail corresponding to the playback time from the target thumbnail stitching image, and displays it.

[0123] In one embodiment, such as Figure 4 As shown, the server defines thumbnail stitching rules and distributes these rules to the terminals. The server uses FFmpeg to generate thumbnails and then uses ImageMagick to stitch them together according to the rules into a composite thumbnail image, which is then stored on the storage server. FFmpeg is an open-source computer program used to record and convert digital audio and video, and to stream them; ImageMagick is an application for creating, editing, and compositing images.

[0124] Considering that third parties providing videos typically offer video links, and that video files are usually in m3u8 format, it's necessary to support recognizing video playback links and retrieving thumbnails. Video editing software can only edit local videos, while FFmpeg is suitable for scenarios requiring efficient batch processing and precise control, and OpenCV (a cross-platform computer vision and machine learning software library) is suitable for scenarios requiring flexible integration and automated processing. Given that FFmpeg is an open-source multimedia framework that provides a wealth of tools and libraries for processing video files, the server can extract video thumbnails by calling FFmpeg.

[0125] The target playback duration of the video clip for generating thumbnails can be determined based on actual needs. A target playback duration that is too long will result in fewer thumbnails and a poor user experience; a target playback duration that is too short will incur significant costs. Considering that most videos are played on streaming aggregation platforms (such as Roku), and that a single fast-forward on such platforms corresponds to 10 seconds, the target playback duration of the video clip is set to 10 seconds to ensure that users see a thumbnail every time they fast-forward. If the video playback time falls within 10 seconds, the thumbnail closest to the playback time will be used for display.

[0126] Considering that the current TV resolution is 16:9, the image resolution is also set to 16:9. Combining the resolution of commonly used video playback software, small thumbnails have a resolution of 120x90 pixels, medium-sized thumbnails have a resolution of 320x180 pixels, and large-sized thumbnails have a resolution of 640x360 pixels. Finally, 192x108 pixels was chosen as the thumbnail resolution.

[0127] When the terminal powers on, it sends a request to the server to obtain the thumbnail stitching rules. The server responds to the request by sending the thumbnail stitching rules back to the terminal. When the terminal opens a video, it preloads the various thumbnail stitched images. When the user fast-forwards, it extracts a thumbnail from the stitched images and displays it.

[0128] In related technologies, each time a user fast-forwards, the terminal requests a thumbnail from the server and displays it. When the user fast-forwards continuously, the terminal needs to continuously request thumbnails from the server. Since requesting thumbnails takes time, the user will see that the thumbnails are not loaded in time. This problem can be solved by preloading multiple thumbnails. However, since no thumbnails are displayed when the user does not fast-forward, many thumbnails that will not be displayed may be loaded. Moreover, the problem of thumbnails not loading in time will still occur when fast-forwarding in areas where thumbnails are not preloaded.

[0129] To mitigate the issue of thumbnails not displaying during fast-forwarding, this embodiment employs a thumbnail stitching scheme. Multiple thumbnails are stitched together into a single large thumbnail image for distribution. A new thumbnail is generated every 10 seconds, and 10 thumbnails are stitched together into a single image before being sent to the terminal. When the terminal plays a video, the thumbnail image can be preloaded. When displaying a thumbnail, the resolution of the thumbnail within the thumbnail image is calculated based on the playback time, and then cropped and displayed on the terminal. This ensures that images can be loaded quickly during fast-forwarding within 100 seconds, reducing the probability of users having to immediately download the next image while fast-forwarding. Within the time period covered by the thumbnail image, each thumbnail can be displayed quickly, improving the user experience.

[0130] In one embodiment, the server names the thumbnails according to the format of "{domain name issued by the cloud} + {fixed prefix} + {unique identifier of media asset content provider} + {unique identifier of media asset} + number of rows x number of columns + {resolution} + index".

[0131] The technical solution adopted in this application embodiment, because the thumbnail stitching image includes multiple thumbnails, can save resources and improve efficiency by loading a single thumbnail stitching image compared to loading multiple thumbnails. When it is necessary to display a video thumbnail, it is only necessary to directly extract the corresponding thumbnail from the target thumbnail stitching image loaded in the terminal according to the video playback time and thumbnail stitching rules, and then display it. There is no need to send a request to the server according to the video playback time and obtain the thumbnail returned by the server, which can save transmission time and achieve fast and smooth thumbnail display.

[0132] To facilitate better implementation of the video thumbnail processing method of this application, this application also provides a video thumbnail processing apparatus based on the above-described video thumbnail processing method. The meanings of the terms used are the same as in the above-described video thumbnail processing method, and specific implementation details can be found in the descriptions of the method embodiments.

[0133] Please see Figure 5 , Figure 5 This is a schematic diagram of the structure of a video thumbnail processing device provided in an embodiment of this application, wherein the video thumbnail processing device is applied to a terminal, and the video thumbnail processing device includes:

[0134] Rule acquisition module 501 is used to obtain thumbnail stitching rules from the server;

[0135] The loading module 502 is used to load a thumbnail stitched image of a video from the server; the thumbnail stitched image is composed of multiple thumbnails stitched together according to the thumbnail stitching rules;

[0136] The time acquisition module 503 is used to acquire the playback time of the video in response to the thumbnail display request;

[0137] The cropping module 504 is used to crop the thumbnail corresponding to the playback time from the target thumbnail splicing image corresponding to the playback time according to the playback time and the thumbnail splicing rules;

[0138] The display module 505 is used to display a thumbnail corresponding to the playback time.

[0139] In one embodiment, the interception module 504 includes:

[0140] The range unit is used to determine the playback time range corresponding to each of the thumbnail splicing images according to the thumbnail splicing rules.

[0141] The target determination unit is used to determine the thumbnail stitched image corresponding to the playback time range to which the playback time belongs as the target thumbnail stitched image;

[0142] The quantity acquisition unit is used to acquire the number of stitches represented by the thumbnail stitching rules;

[0143] An index determination unit is used to determine the index of the thumbnail corresponding to the playback time in the target thumbnail splicing image based on the playback time, the number of splices, and the target playback time range corresponding to the target thumbnail splicing image;

[0144] The cropping unit is used to crop the thumbnail corresponding to the playback time from the target thumbnail stitching image according to the index and the thumbnail stitching rules.

[0145] In one embodiment, the index determination unit includes:

[0146] The duration determination subunit is used to determine the playback duration corresponding to the target thumbnail splicing image based on the target playback time range;

[0147] The difference acquisition subunit is used to determine the starting playback time within the target playback time range and acquire the difference between the playback time and the starting playback time.

[0148] The index determination subunit is used to obtain the quotient of the difference and the playback duration, and determine the index of the thumbnail corresponding to the playback time in the target thumbnail splicing image based on the quotient and the number of splices.

[0149] In one embodiment, the interception unit includes:

[0150] A sub-unit is used to obtain the resolution and arrangement rules represented by the thumbnail stitching rules;

[0151] The cropping subunit is used to crop the thumbnail corresponding to the playback time from the target thumbnail stitching image according to the index, the resolution and the arrangement rules.

[0152] The technical solution adopted in this application embodiment, because the thumbnail stitching image includes multiple thumbnails, can save resources and improve efficiency by loading a single thumbnail stitching image compared to loading multiple thumbnails. When it is necessary to display a video thumbnail, it is only necessary to directly extract the corresponding thumbnail from the target thumbnail stitching image loaded in the terminal according to the video playback time and thumbnail stitching rules, and then display it. There is no need to send a request to the server according to the video playback time and obtain the thumbnail returned by the server, which can save transmission time and achieve fast and smooth thumbnail display.

[0153] Please see Figure 6 , Figure 6 This is a schematic diagram of the structure of a video thumbnail processing device provided in an embodiment of this application, wherein the video thumbnail processing device is applied to a server, and the video thumbnail processing device includes:

[0154] The generation module 601 is used to generate multiple thumbnails of the video based on the target playback duration and resolution;

[0155] The stitching module 602 is used to stitch multiple thumbnails into a thumbnail stitched image according to thumbnail stitching rules;

[0156] The sending module 603 is used to send the thumbnail stitched image and the thumbnail stitching rules to the terminal; the thumbnail stitched image is used to: extract the thumbnail corresponding to the playback time from the target thumbnail stitched image corresponding to the playback time according to the thumbnail stitched image and the playback time of the video in the terminal.

[0157] In one embodiment, the splicing module 602 includes:

[0158] The acquisition unit is used to acquire the number of stitches and the arrangement rules represented by the thumbnail stitching rules;

[0159] A segmentation unit is used to divide the multiple thumbnails of the video into multiple groups of thumbnails according to the time sequence and the number of stitches.

[0160] The stitching unit is used to stitch together each group of thumbnails into the thumbnail stitched image according to the time sequence and the arrangement rules.

[0161] The technical solution adopted in this application embodiment includes multiple thumbnails. Therefore, compared with sending multiple thumbnails to the terminal, the server sending the thumbnail stitched image can reduce the number of images sent, save transmission resources and improve efficiency. When the terminal needs to display a thumbnail of a video, it only needs to directly extract the corresponding thumbnail from the target thumbnail stitched image according to the video playback time and thumbnail stitching rules and display it. There is no need to send a request to the server according to the video playback time and obtain the thumbnail returned by the server. This can save transmission time and achieve fast and smooth thumbnail display.

[0162] Specific limitations regarding the video thumbnail processing device can be found in the limitations of the video thumbnail processing method described above, and will not be repeated here. Each module in the aforementioned video thumbnail processing device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0163] In addition, this application also provides an electronic device, such as Figure 7 As shown, it illustrates the structural diagram of the electronic device involved in this application, specifically:

[0164] The electronic device may include components such as a processor 701 with one or more processing cores and a memory 702 with one or more computer-readable storage media. Those skilled in the art will understand that... Figure 7 The electronic device structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0165] The processor 701 is the control center of the electronic device. It connects various parts of the electronic device via various interfaces and lines. By running or executing software programs and / or modules stored in the memory 702, and by calling data stored in the memory 702, it performs various functions and processes data, thereby providing overall monitoring of the electronic device. Optionally, the processor 701 may include one or more processing cores; preferably, the processor 701 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 701.

[0166] The memory 702 can be used to store software programs and modules. The processor 701 executes various functional applications and data processing by running the software programs and modules stored in the memory 702. The memory 702 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 702 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 702 may also include a memory controller to provide the processor 701 with access to the memory 702.

[0167] In one embodiment, the electronic device further includes a power supply 703 that supplies power to the various components. Preferably, the power supply 703 can be logically connected to the processor 701 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 703 may also include one or more DC or AC power supplies, recharging systems, power equipment debugging circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0168] In one embodiment, the electronic device may further include an input unit 704, which can be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0169] Although not shown, the electronic device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 701 in the electronic device loads the executable files corresponding to the processes of one or more applications into the memory 702 according to the following instructions, and the processor 701 runs the applications stored in the memory 702, thereby implementing the steps in any of the video thumbnail processing methods provided in the embodiments of this application.

[0170] Those skilled in the art will understand that Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the electronic device to which the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0171] In one embodiment, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the methods described in any embodiment of this application.

[0172] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the method described in any embodiment of this application.

[0173] In some embodiments, a computer program product is also provided, including a computer program or instructions that, when executed by a processor, implement the methods described in any embodiment of this application.

[0174] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0175] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0176] Therefore, this application provides a computer-readable storage medium storing a computer program that can be loaded by a processor to perform the steps of any of the video thumbnail processing methods provided in this application.

[0177] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0178] The computer-readable storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0179] Since the instructions stored in the computer-readable storage medium can execute the steps of any of the video thumbnail processing methods provided in this application, the beneficial effects that any of the video thumbnail processing methods provided in this application can achieve can be realized, as detailed in the preceding embodiments, and will not be repeated here.

[0180] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0181] The foregoing has provided a detailed description of a video thumbnail processing method, apparatus, electronic device, and computer-readable storage medium provided in this application. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A video thumbnail processing method, characterized in that, Applied to terminals, including: Obtain thumbnail stitching rules from the server; the thumbnail stitching rules specify the resolution of the thumbnails and the arrangement rules of multiple thumbnails; The server loads a thumbnail stitched image of the video; the thumbnail stitched image is composed of multiple thumbnails stitched together according to the thumbnail stitching rules; In response to a thumbnail display request, obtain the playback time of the video; According to the playback time and the thumbnail stitching rules, the thumbnail corresponding to the playback time is extracted from the target thumbnail stitching image corresponding to the playback time, including: calculating the difference between the playback time and the starting playback time of the target thumbnail stitching image; calculating the quotient of the difference and the playback duration corresponding to the target thumbnail stitching image, where the quotient represents the order of the playback time within the playback time range corresponding to the target thumbnail stitching image; determining the index of the thumbnail corresponding to the playback time in the target thumbnail stitching image based on the product of the quotient and the number of stitches corresponding to the target thumbnail stitching image; determining the image area occupied by each thumbnail based on the resolution of the thumbnail; determining the number of rows and columns corresponding to each index of the thumbnail based on the arrangement rules of multiple thumbnails; determining the image area of ​​each index of the thumbnail in the thumbnail stitching image based on the number of rows and columns corresponding to the thumbnail and the resolution of the thumbnail; and Display a thumbnail corresponding to the playback time.

2. The method according to claim 1, characterized in that, The step of extracting the thumbnail corresponding to the playback time from the target thumbnail stitching image corresponding to the playback time according to the playback time and the thumbnail stitching rules includes: Based on the thumbnail stitching rules, the playback time range corresponding to each of the thumbnail stitched images is determined; The thumbnail stitched image corresponding to the playback time range to which the playback time belongs is determined as the target thumbnail stitched image; Obtain the number of stitches represented by the thumbnail stitching rule; Based on the index and the thumbnail stitching rules, a thumbnail corresponding to the playback time is extracted from the target thumbnail stitching image.

3. The method according to claim 2, characterized in that, The method further includes: Based on the target playback time range, determine the playback duration corresponding to the target thumbnail splicing image.

4. The method according to claim 2, characterized in that, The step of extracting the thumbnail corresponding to the playback time from the target thumbnail stitching image according to the index and the thumbnail stitching rules includes: Obtain the resolution and arrangement rules represented by the thumbnail stitching rules; Based on the index, the resolution, and the arrangement rules, a thumbnail corresponding to the playback time is extracted from the target thumbnail stitching image.

5. A video thumbnail processing method, characterized in that, Applied to servers, including: Generate multiple thumbnails of the video based on the target playback duration and resolution; According to the thumbnail stitching rules, multiple thumbnails are stitched together to form a thumbnail stitched image; the thumbnail stitching rules specify the resolution of the thumbnails and the arrangement rules of multiple thumbnails; The thumbnail stitched image and the thumbnail stitching rules are sent to the terminal; the thumbnail stitched image is used to: extract the thumbnail corresponding to the playback time from the target thumbnail stitched image corresponding to the playback time, based on the thumbnail stitched image and the playback time of the video in the terminal; The step of extracting the thumbnail corresponding to the playback time from the target thumbnail stitched image includes: calculating the difference between the playback time and the starting playback time of the target thumbnail stitched image; calculating the quotient of the difference and the playback duration corresponding to the target thumbnail stitched image, where the quotient represents the order of the playback time within the playback time range corresponding to the target thumbnail stitched image; determining the index of the thumbnail corresponding to the playback time in the target thumbnail stitched image based on the product of the quotient and the number of stitches corresponding to the target thumbnail stitched image; determining the image area occupied by each thumbnail based on the resolution of the thumbnail; determining the number of rows and columns corresponding to each index of the thumbnail based on the arrangement rules of multiple thumbnails; and determining the image area of ​​each index of the thumbnail in the thumbnail stitched image based on the number of rows and columns corresponding to the thumbnail and the resolution of the thumbnail.

6. The method according to claim 5, characterized in that, The step of stitching multiple thumbnails into a thumbnail stitched image according to thumbnail stitching rules includes: Obtain the number of stitches and the arrangement rules represented by the thumbnail stitching rules; The multiple thumbnails of the video are divided into multiple groups of thumbnails according to the time sequence and the number of stitches; According to the time sequence and the arrangement rules, each group of thumbnails is stitched together to form the thumbnail stitched image.

7. A video thumbnail processing device, characterized in that, Applied to terminals, including: The rule acquisition module is used to obtain thumbnail stitching rules from the server; the thumbnail stitching rules specify the resolution of the thumbnails and the arrangement rules of multiple thumbnails; A loading module is used to load a thumbnail stitched image of a video from the server; the thumbnail stitched image is composed of multiple thumbnails stitched together according to the thumbnail stitching rules; The time acquisition module is used to acquire the playback time of the video in response to the thumbnail display request; The extraction module is used to extract a thumbnail corresponding to the playback time from a target thumbnail stitching image corresponding to the playback time, based on the playback time and the thumbnail stitching rules. This includes: calculating the difference between the playback time and the starting playback time of the target thumbnail stitching image; calculating the quotient of the difference and the playback duration corresponding to the target thumbnail stitching image, where the quotient represents the order of the playback time within the playback time range corresponding to the target thumbnail stitching image; determining the index of the thumbnail corresponding to the playback time in the target thumbnail stitching image based on the quotient and the number of stitches corresponding to the target thumbnail stitching image; determining the image area occupied by each thumbnail based on the resolution of the thumbnail; determining the number of rows and columns corresponding to each index's thumbnail based on the arrangement rules of multiple thumbnails; and determining the image area of ​​each index's thumbnail in the thumbnail stitching image based on the number of rows and columns corresponding to the thumbnail and the resolution of the thumbnail. The display module is used to show thumbnails corresponding to the playback time.

8. A video thumbnail processing device, characterized in that, Applied to servers, including: The generation module is used to generate multiple thumbnails of the video based on the target playback duration and resolution; The stitching module is used to stitch multiple thumbnails into a thumbnail stitched image according to thumbnail stitching rules; the thumbnail stitching rules specify the resolution of the thumbnails and the arrangement rules of the multiple thumbnails; The sending module is used to send the thumbnail stitched image and the thumbnail stitching rules to the terminal; the thumbnail stitched image is used to: extract the thumbnail corresponding to the playback time from the target thumbnail stitched image corresponding to the playback time according to the thumbnail stitched image and the playback time of the video in the terminal; The step of extracting the thumbnail corresponding to the playback time from the target thumbnail stitched image includes: calculating the difference between the playback time and the starting playback time of the target thumbnail stitched image; calculating the quotient of the difference and the playback duration corresponding to the target thumbnail stitched image, where the quotient represents the order of the playback time within the playback time range corresponding to the target thumbnail stitched image; determining the index of the thumbnail corresponding to the playback time in the target thumbnail stitched image based on the quotient and the number of stitches corresponding to the target thumbnail stitched image; determining the image area occupied by each thumbnail based on the resolution of the thumbnail; determining the number of rows and columns corresponding to each index's thumbnail based on the arrangement rules of multiple thumbnails; and determining the image area of ​​each index's thumbnail in the thumbnail stitched image based on the number of rows and columns corresponding to the thumbnail and the resolution of the thumbnail.

9. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the video thumbnail processing method as described in any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the video thumbnail processing method as described in any one of claims 1 to 6.

Images