Video playing method, processing method, device, equipment and storage medium

Abstract

Embodiments of the present application disclose a video playing method, a processing method, a device, equipment and a storage medium, and belong to the technical field of multimedia playing. The method comprises: playing a target video in a video playing window, the size of the video playing window being smaller than the size of a screen of a video playing device; in response to the target video playing to a first time period, displaying associated content related to the video content of the first time period; wherein the display level of the associated content is below the video playing window, and the display area of the associated content comprises a screen area outside the video playing window. The present application does not need to make the associated content into the original video when playing the video, and is decoupled from the original video, thereby improving the production efficiency and playing flexibility of the associated content, and enabling the video content and the associated content to produce a linkage effect, effectively attracting users to watch, and improving the video playing effect.

Description

Technical Field

[0001] This application relates to the field of multimedia playback technology, and in particular to a video playback method, processing method, apparatus, device, and storage medium. Background Technology

[0002] Users can watch the videos by playing them on devices such as mobile phones.

[0003] Taking mobile phones as an example, mobile phones typically display a video playback window on the screen and play videos within that window. In portrait mode, the video playback window is usually displayed in the center of the screen, and its size is smaller than the screen size, approximately one-third of the screen size. To avoid large blank areas on the screen, related technologies manually re-edit the original video, adding text content such as video titles, tags, and comments above or below the original video, generating a new video that the client then plays. This allows viewers to learn additional information while watching the video content.

[0004] However, the above method requires manual re-editing of the original video to generate a new video, which is inefficient and lacks flexibility. Summary of the Invention

[0005] This application provides a video playback method, processing method, apparatus, device, and storage medium. The technical solution is as follows:

[0006] According to one aspect of the embodiments of this application, a video playback method is provided, the method comprising:

[0007] Play the target video in a video playback window, the size of which is smaller than the screen size of the video playback device;

[0008] In response to the target video playing to a first time period, related content associated with the video content of the first time period is displayed;

[0009] The display hierarchy of the associated content is located below the video playback window, and the display area of ​​the associated content includes the screen area outside the video playback window.

[0010] According to one aspect of the embodiments of this application, a video processing method is provided, the method comprising:

[0011] Acquire the target video;

[0012] Identify the video content of the first time segment in the target video;

[0013] Generate associated content related to the video content of the first time period, and the associated content is used to be displayed when the target video is played to the first time period;

[0014] Wherein, the size of the video playback window of the target video is smaller than the screen size of the video playback device, the display layer of the associated content is located below the video playback window, and the display area of ​​the associated content includes the screen area outside the video playback window.

[0015] According to one aspect of the embodiments of this application, a video playback apparatus is provided, the apparatus comprising:

[0016] The video playback module is used to play a target video in a video playback window, the size of which is smaller than the screen size of the video playback device;

[0017] The content display module is used to display related content to the video content of the first time period in response to the target video playing to the first time period;

[0018] The display hierarchy of the associated content is located below the video playback window, and the display area of ​​the associated content includes the screen area outside the video playback window.

[0019] According to one aspect of the embodiments of this application, a video processing apparatus is provided, the apparatus comprising:

[0020] The video acquisition module is used to acquire the target video;

[0021] The video recognition module is used to identify the video content of the first time segment in the target video;

[0022] A content generation module is used to generate associated content related to the video content of the first time period, and the associated content is used to be displayed when the target video is played to the first time period.

[0023] Wherein, the size of the video playback window of the target video is smaller than the screen size of the video playback device, the display layer of the associated content is located below the video playback window, and the display area of ​​the associated content includes the screen area outside the video playback window.

[0024] According to one aspect of the embodiments of this application, a computer device is provided, the computer device including a processor and a memory, the memory storing at least one instruction, at least one program, code set or instruction set, the at least one instruction, the at least one program, the code set or instruction set being loaded and executed by the processor to implement the above-described video playback method, or to implement the above-described video processing method.

[0025] According to one aspect of the embodiments of this application, a computer-readable storage medium is provided, the storage medium storing at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the above-described video playback method, or to implement the above-described video processing method.

[0026] According to one aspect of the embodiments of this application, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the aforementioned video playback method or the aforementioned video processing method.

[0027] The technical solution provided in this application can bring the following beneficial effects:

[0028] By displaying related content when the target video reaches a specific time segment, this approach achieves two goals: First, the related content does not need to be created into the original video, thus decoupling it from the original video and overcoming the inefficiency and inflexibility of manual re-creation. This helps improve the efficiency of related content creation and playback flexibility. Second, by displaying related content associated with the characters, scenes, and plot of the video during a specific time segment, the video content and related content create a synergistic effect, effectively attracting users to watch and enhancing the video playback experience.

[0029] In addition, by setting the display level of related content below the video playback window, and including the screen area outside the video playback window in the display area of ​​related content, not only can the blank area in the screen be effectively utilized to increase the amount of information displayed on the screen, but the display of related content will not have any adverse effects such as obstructing the video in the video playback window. Attached Figure Description

[0030] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0031] Figure 1 This is a schematic diagram of the implementation environment of a solution provided in one embodiment of this application;

[0032] Figure 2 This is a flowchart of a video playback method provided in one embodiment of this application;

[0033] Figure 3 This is a schematic diagram of a video playback window provided in one embodiment of this application;

[0034] Figure 4 This is a schematic diagram of the interface for displaying associated content provided in one embodiment of this application;

[0035] Figure 5 This is a schematic diagram of a partial frame in an associated video provided in one embodiment of this application;

[0036] Figure 6 This is a schematic diagram of partial frame movement in an associated video provided in one embodiment of this application;

[0037] Figure 7 This is a schematic diagram of associated animation provided in one embodiment of this application;

[0038] Figure 8 This is a flowchart of a video playback method provided in another embodiment of this application;

[0039] Figure 9 This is a schematic diagram showing the introduction card provided in one embodiment of this application;

[0040] Figure 10 This is a schematic diagram shown on the introduction card provided in another embodiment of this application;

[0041] Figure 11 This is a flowchart of a video processing method provided in one embodiment of this application;

[0042] Figure 12 This is a flowchart illustrating the generation of background video according to one embodiment of this application;

[0043] Figure 13 This is a block diagram of a video playback device provided in one embodiment of this application;

[0044] Figure 14 This is a block diagram of a video processing apparatus provided in one embodiment of this application;

[0045] Figure 15 This is a schematic diagram of the structure of a computer device provided in one embodiment of this application. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0047] Please refer to Figure 1This diagram illustrates an implementation environment provided by one embodiment of the present application. This implementation environment can be implemented as a video playback system. The implementation environment may include: a video playback device 10 and a server 20.

[0048] The video playback device 10 can be a terminal device such as a mobile phone, tablet computer, wearable device, smart TV, or multimedia playback device, and this application does not limit its scope. The video playback device 10 is used to play videos. A client application with video playback functionality can be installed and run on the video playback device 10. For example, the target application can be a browser, video playback application, instant messaging application, social networking application, news application, or lifestyle service application, and this application does not limit its scope.

[0049] Server 20 can be a single server, a server cluster consisting of multiple servers, or a cloud computing service center. Server 20 can be the backend server for the target application described above, used to provide backend services to the client of the target application. In some embodiments, server 20 stores a media library for providing video resources.

[0050] The video playback device 10 and the server 20 can communicate via a network.

[0051] The technical solution provided in this application allows the video playback device 10 (such as a client of the target application) to display related content during video playback. The video and its corresponding related content can be provided to the video playback device 10 by the server 20.

[0052] The technical solution of this application will be described below through several embodiments.

[0053] Please refer to Figure 2 The diagram illustrates a flowchart of a video playback method provided in one embodiment of this application. The execution entity for each step of this method may be... Figure 1 In the implementation environment of the illustrated scheme, the video playback device 10 can perform each step by a client of the target application installed and running on the video playback device 10. The method may include the following steps (210-220):

[0054] Step 210: Play the target video in the video playback window. The size of the video playback window is smaller than the screen size of the video playback device.

[0055] A video playback window is a window used to play videos, or in other words, a window used to display video content. The video playback window is displayed on the screen of the video playback device. The size of the video playback window is smaller than the screen size of the video playback device; that is, the video playback window only occupies a portion of the screen area for display, rather than being displayed full-screen. The video playback window can be located in the center, top, or bottom of the screen; this application does not limit the display position of the video playback window.

[0056] Because the video playback window is smaller than the screen size of the video playback device, meaning it only occupies a portion of the screen area, this results in blank areas on the screen. These blank areas typically do not display any content and are usually filled with the same color (such as black or white).

[0057] For example, consider a video playback device screen that is rectangular (or rectangle-like, such as a rounded rectangle), and a video playback window that is also rectangular (or rectangle-like, such as a rounded rectangle). Figure 3 As shown, the video playback device displays in portrait mode. Assume the screen 30 has a length of L1 and a width of W1, where L1 is greater than W1. Figure 3 In the image, screen 30 has four vertices A1, B1, C1, and D1. Screen 30 is a rectangular area enclosed by a first wide side A1B1, a first long side B1C1, a second wide side C1D1, and a second long side D1A1. The lengths of the first wide side A1B1 and the second wide side C1D1 are W1, and the lengths of the first long side B1C1 and the second long side D1A1 are L1. In the screen 30 of the video playback device, a video playback window 31 is displayed in landscape mode. The length of the video playback window 31 is L2, and the width is W2, where L2 is greater than W2; for example, the aspect ratio could be 16:9. Figure 3 In the video playback window 31, the four vertices are A2, B2, C2, and D2. The video playback window 31 is a rectangular area enclosed by the first long side A2B2, the first wide side B2C2, the second long side C2D2, and the second wide side D2A2. The length of the first long side A2B2 and the second long side C2D2 is L2, and the length of the first wide side B2C2 and the second wide side D2A2 is W2. In some embodiments, such as Figure 3As shown, the long side of the video playback window 31 (including the first long side A2B2 and the second long side C2D2) is parallel to the wide side of the screen 30 (including the first wide side A1B1 and the second wide side C1D1), and the length L2 of the video playback window 31 is equal to the width W1 of the screen 30; furthermore, the wide side of the video playback window 31 coincides with the long side of the screen 30, such as the first wide side B2C2 of the video playback window 31 coinciding with the first long side B1C1 of the screen 30, and the second wide side D2A2 of the video playback window 31 coinciding with the second long side D1A1 of the screen 30. The width W2 of the video playback window 31 is less than the length L1 of the screen 30, for example, W2 = L1 / 3 or W2 = L1 / 4, etc. Additionally, in Figure 3 In this design, the video playback window 31 is positioned in the center of the screen 30, meaning that S1 and S2 are equal or nearly equal. S1 is the distance between the first wide side A1B1 of the screen 30 and the first long side A2B2 of the video playback window 31, and S2 is the distance between the second wide side C1D1 of the screen 30 and the second long side C2D2 of the video playback window 31. The statement that S1 and S2 are nearly equal means that the absolute value of the difference between them is less than or equal to a threshold value. This threshold value could be 1 cm, 2 cm, 100 pixels, 200 pixels, etc., and can be reasonably set based on the size and proportion of the screen 30 and the video playback window 31. This application does not impose any limitations on this.

[0058] In some embodiments, the client displays a video playback interface, which includes a video playback window. Optionally, the size of the video playback interface is the same as the screen size of the video playback device, i.e., the video playback interface is displayed in full screen. The size of the video playback window is smaller than the size of the video playback interface, and the video playback window occupies only a portion of the area of ​​the video playback interface for display.

[0059] Additionally, the client plays the target video in the video playback window, which can be a video selected by the user. In this embodiment, the type of the target video is not limited; for example, the target video can be a movie, TV series, short film, animation, or other similar video.

[0060] Step 220: In response to the target video playing to the first time segment, display related content associated with the video content of the first time segment. The related content is displayed below the video playback window, and its display area includes the screen area outside the video playback window.

[0061] During the playback of a target video, the client displays related content relevant to the video's content. Specifically, the displayed video content varies at different times; for example, characters, scenes, and plot points differ at different times. In this embodiment, time is used as a reference. When the target video reaches the first time segment, related content relevant to the video content of that first time segment is displayed. Assuming the first time segment starts at time t1 and ends at time t2, the client begins displaying related content when the target video reaches time t1, and stops displaying related content when the target video reaches time t2. This related content is content related to the video content of the first time segment, such as content related to characters, scenes, and plot points within the first time segment. The related content can be in the form of images, videos, or animations, etc., as described in the embodiments below.

[0062] The client's UI (User Interface) has a display hierarchy, similar to layers. Content in the upper display hierarchy can cover content in the lower display hierarchy. In this embodiment, the display hierarchy of the associated content is below the video playback window, and the display area of ​​the associated content includes the screen area outside the video playback window. For example, the size of the display area of ​​the associated content is larger than the size of the video playback window; therefore, the video playback window does not completely cover the associated content. The associated content displayed in the screen area outside the video playback window is still visible to the user. The size of the display area of ​​the associated content can be equal to the screen size of the video playback device, meaning the associated content is displayed full-screen on the video playback device; or, the size of the display area of ​​the associated content can be smaller than the screen size of the video playback device but larger than the size of the video playback window.

[0063] In some embodiments, such as Figure 4 As shown, Figure 4 Part (a) shows a schematic diagram of the video playback window 31. Figure 4 Part (b) shows a schematic diagram of related content 32. Figure 4Part (c) of the diagram illustrates the display of related content during the playback of the target video by the client. As can be seen from the diagram, the display hierarchy of the video playback window 31 is above the related content 32. The display of the related content 32 does not affect the playback of the target video in the video playback window 31. That is, the target video continues to play without pausing while the related content 32 is displayed. Furthermore, the related content 32 does not obscure or cover the target video playing in the video playback window 31, allowing users to view related content while watching the target video.

[0064] It should be understood that the first time segment can be any time segment within the target video. For the target video, relevant content can be set for a single time segment, or multiple time segments can be set with their own relevant content. The relevant content for each time segment is related to the video content within that time segment. For example, if the target video includes characters A and B, when playing a video segment related to character A, relevant content related to character A will be displayed; when playing a video segment related to character B, relevant content related to character B will be displayed.

[0065] The technical solution provided in this application displays related content to the video content during a specific time period when the target video is played. On the one hand, the related content does not need to be created in the original video, thus decoupling it from the original video and overcoming the problems of low efficiency and poor flexibility caused by manual re-creation. This helps to improve the production efficiency and playback flexibility of related content. On the other hand, when playing to a specific time period, displaying related content associated with the video content such as characters, scenes, and plots during that time period creates a linkage effect between the video content and the related content, which can effectively attract users to watch and improve the video playback effect.

[0066] In addition, by setting the display level of related content below the video playback window, and including the screen area outside the video playback window in the display area of ​​related content, not only can the blank area in the screen be effectively utilized to increase the amount of information displayed on the screen, but the display of related content will not have any adverse effects such as obstructing the video in the video playback window.

[0067] In some embodiments, the associated content is a related video displayed in video format. For example, when the target video is playing up to a first segment, the client plays an associated video related to the content of that first segment. The associated video is displayed below the video playback window, and its display area includes the screen area outside the video playback window. Furthermore, the playback of the associated video does not affect the normal playback of the target video. The user can view both the target video and its associated videos simultaneously.

[0068] Optionally, the client uses two players to play the target video and its associated videos respectively. For example, it includes a first player and a second player. The client uses the first player to play the target video and the second player to play the associated videos, so that the playback of the target video and the associated videos is independent and does not affect each other.

[0069] For example, the video playback window used to play the target video, as described above, is called the target video playback window, and the video playback window used to play the associated video is called the associated video playback window. This is illustrated by the example where the screen of the video playback device, the target video playback window, and the associated video playback window are all rectangles (or rectangle-like structures, such as rounded rectangles). Figure 3 As shown, the four vertices of screen 30 are A1, B1, C1, and D1; the four vertices of the target video playback window 31 are A2, B2, C2, and D2; and the four vertices of the associated video playback window are A3, B3, C3, and D3. The display hierarchy of the associated video playback window is below that of the target video playback window 31, and the size of the associated video playback window is larger than that of the target video playback window 31. Optionally, the size of the associated video playback window can be equal to or smaller than the size of screen 30. Figure 3 Taking the size of the associated video playback window as an example, which is equal to 30 pixels of the screen, the associated video can be played in full screen for a better immersive experience.

[0070] In some embodiments, the client displays a first associated video related to the video content of the first time period, the first associated video being a video created based on image materials related to the video content of the first time period.

[0071] Image materials can be pictures related to the people, scenes, plots, etc., appearing in the first time segment. These image materials can be obtained from sources other than the target video, or they can be extracted from the target video. The methods for obtaining image materials and generating the first associated video will be described in the examples below.

[0072] In some embodiments, the client displays a second associated video related to the video content of the first time period, the second associated video being the same as and played synchronously with the video content of the first time period.

[0073] The second associated video is a video segment from the target video. For example, a video segment from the first time period of the target video is extracted to obtain the second associated video. The method for generating the second associated video will be described in the following examples.

[0074] Optionally, the resolution of the second associated video is greater than the resolution (or size) of the associated video playback window, so that only a portion of the image frames of the second associated video is displayed in the associated video playback window, that is, only a partial view of the second associated video is displayed. During the playback of the second associated video, the position of the displayed partial view can remain constant or change dynamically. For example, the position of this partial view 51 can be a fixed position within the image frame 52 of the second associated video, such as the center position of the image, or... Figure 5 The left-hand illustration shows this. For example, the position of this partial frame 51 could also be a dynamically changing position within the image frame 52 of the second associated video, such as... Figure 5 As shown, the position of the partial image 51 can dynamically change from left to right. Of course, in some other embodiments, the position of the partial image can also dynamically change from right to left, or dynamically change with the position of the key element in the image frame.

[0075] In some embodiments, during the display of the second associated video, the client identifies key elements in the target video and displays a partial view of the second associated video centered on these key elements. The key elements in the target video can be a single scene content, such as a person, a face, or an object, or a pixel region or pixel position. Since the second associated video is a video with the same content as and played synchronously with the first segment of the target video, the aforementioned key elements also exist in the second associated video, such as the same face, the same object, or the same pixel region or pixel position. The client then displays the partial view of the second associated video centered on these key elements. Because the position of the key element in the video frame dynamically changes as the video plays, the position of the partial view also changes dynamically.

[0076] In one possible implementation, the client identifies image frames of the target video to determine key elements within the video. For example, the client can use facial recognition technology to identify faces within the image frames of the target video as key elements. It should be noted that the image frames used to identify key elements refer to those in the first segment of the target video. Furthermore, if the key elements are elements other than faces, other recognition methods can be used accordingly, and this application does not limit this approach. This method determines key elements based on the video content, enabling the identification of key content within the video frame as key elements.

[0077] In another possible implementation, the client uses eye-tracking technology to obtain the user's eye focus position in the target video and then identifies the corresponding image element as the key element. Eye-tracking technology tracks the user's gaze on the screen, allowing the identification of the user's eye focus position within the target video. The corresponding image element (such as a pixel region or pixel position) is then identified as the key element. This approach uses the user's eye focus as the basis for determining key elements, enabling the selection of content of user interest as the key element.

[0078] In some embodiments, the client can also dynamically adjust the position of the partial frame in response to user operations. During the display of the second associated video, in response to a rotation operation on the video playback device, the client displays a partial frame from the second associated video corresponding to the orientation of the video playback device.

[0079] The orientation of the video playback device can be determined based on sensor data collected by a motion sensor in the video playback device. This motion sensor may include an accelerometer, an angular velocity sensor, etc. In some embodiments, such as... Figure 6 As shown, taking a mobile phone as an example, the client displays a video playback interface 35, which contains a video playback window 31. The client plays the target video 33 within the video playback window 31. Additionally, the background video is a second associated video 34, and only a portion of this video is displayed on the screen. As the user rotates the phone to change its orientation, the portion of the second associated video 34 dynamically changes; for example, if the phone moves from left to right, the portion of the second associated video 34 also moves from left to right.

[0080] In this embodiment of the application, by displaying associated videos in video form, when the associated video is a video made from image materials related to the video content, users can view related image materials from the associated video and obtain some other information related to the target video; when the associated video is a video with the same content as the video and played synchronously, the picture effect has a sense of layering and immersion.

[0081] In addition, this application also supports the client to dynamically adjust the partial view of the associated video based on the content of the screen or the content that the user is interested in, or to dynamically change the partial view of the associated video based on the user's adjustment of the device orientation, so that the display of associated videos is more flexible and enhances the visual appeal and impact.

[0082] In some embodiments, the associated content is displayed as an associated animation.

[0083] Optionally, in response to the target video reaching the first time segment, the client displays related animations associated with the video content of that first time segment. These related animations highlight the currently playing content of the target video through animations, special effects, sounds, and text.

[0084] For example, such as Figure 7 As shown, the target video 33 is played in the video playback window 31 of the video playback interface 35 in the left figure. For example, the target video 33 can be a clip of Thor's appearance in the movie "Thor". By displaying related motion effects 32, such as lightning effects and lightning sound effects, the shock and expressiveness of Thor's appearance are enhanced in conjunction with Thor's appearance.

[0085] For example, such as Figure 7 As shown, the target video 33 is played in the video playback window 31 of the video playback interface 35 on the right. For example, the target video 33 can be a video clip of lovers kissing. By displaying related animation effects 32, such as the animation and sound effects of fireworks blooming, the kissing scene is made more romantic and expressive.

[0086] In this embodiment of the application, by adding motion effects related to the target video during the playback of the target video, the playback of the target video becomes more expressive.

[0087] Please refer to Figure 8 The diagram illustrates a flowchart of a video playback method provided in another embodiment of this application. The entity executing each step of this method may be... Figure 1 In the implementation environment of the illustrated scheme, the video playback device 10 can perform each step by a client of the target application installed and running on the video playback device 10. The method may include the following steps (810-850):

[0088] Step 810: Play the target video in the video playback window. The size of the video playback window is smaller than the screen size of the video playback device.

[0089] Step 820: In response to the target video playing to the first time segment, display related content associated with the video content of the first time segment. The display hierarchy of the related content is located below the video playback window, and the display area of ​​the related content includes the screen area outside the video playback window.

[0090] For a description of steps 810 to 820, please refer to the above embodiments; they will not be repeated here.

[0091] Step 830: In response to the target video playing to the second time period, a first introductory card related to the video content of the second time period is displayed. The first introductory card is used to display at least one of the following introductory information: first character information, first character actor information, first plot information, and first comment information.

[0092] Optionally, the introduction card can be located at the same level as the target video's playback window, above the target video's playback window, or below the target video's playback window and above the background video. The aforementioned "first character information" introduces the first character in the target video, "first character actor information" introduces the actor playing the first character, "first plot information" introduces the first plot in the target video (e.g., the first plot could be related to the content of the second segment of the video), and "first comment information" refers to one or more comments related to the target video.

[0093] The mapping relationship between the second time period and the first introduction card can be communicated to the client by the server.

[0094] Optionally, the introduction cards can be switched based on the appearance of people or events in the target video, or based on time points in the target video. This application does not limit the method of switching between introduction cards.

[0095] In some embodiments, such as Figure 9 As shown, Figure 9 The demonstration shows the process of displaying the introduction card that introduces the character information. The target video 33 is played in the video playback window 31 of the video playback interface 35. Before the first character appears in the target video 33, the introduction card 36 only introduces the plot information and the plot poster. After the first character appears, the first character information and the first character actor information are obtained through AI face recognition technology, and the introduction card 36 that introduces the first character information and / or the first character actor information is displayed.

[0096] In some embodiments, Figure 9 After the first character in the target video 33 leaves the scene, the introduction card 36 introducing the first character's information and / or the actor's information will not disappear with the first character's departure, or the introduction card 36 introducing the first character's information and the actor's information will disappear with the first character's departure.

[0097] In some embodiments, a second character appears at the same time as the first character exits. AI facial recognition technology is used to obtain the second character's information and the second character's actor information, and an introduction card for the second character's information and / or the second character's actor information is displayed.

[0098] In some embodiments, a second character enters before the first character leaves the stage. AI facial recognition technology is used to obtain the second character's information and the actor's information, and simultaneously, introduction cards for the first character and / or the actor's information, as well as introduction cards for the second character and / or the actor's information, are displayed. Optionally, when a second character enters before the first character leaves the stage, AI facial recognition technology is used to obtain the second character's information and the actor's information, and the introduction cards for the first character and / or the actor's information are converted into introduction cards for the second character and / or the actor's information. Multiple character cards can be switched using video software functions.

[0099] The character information and actor information can be displayed simultaneously, or only one can be displayed by setting it to be displayed, or the two can be switched by using the video software function.

[0100] Step 840: In response to the first operation on the first introduction card, the first introduction card is switched from a simplified display style to a detailed display style, wherein the detailed display style contains more introduction information than the simplified display style.

[0101] In some embodiments, such as Figure 10 As shown, the introduction card in the left image is a simplified display style 37, which only introduces the actor's English and Chinese names. The introduction card in the right image is a detailed display style 38, which introduces the actor's English and Chinese names, as well as actor information. The first operation expands the simplified display style 37 to obtain the detailed display style 38, as shown in the process from the left image to the right image.

[0102] In addition, the first operation can be to tap the simplified display style of the character on the touch screen to get the detailed display style, or to switch the simplified display style of the character to the detailed display style by shaking the phone. This application does not limit the type of the first operation.

[0103] Step 850, in response to a second operation on the first introduction card, the first introduction card is switched to a second introduction card, the second introduction card being used to display at least one of the following introductory information: second character information, second character actor information, second plot information, and second commentary information.

[0104] The aforementioned second character information is used to introduce the second character in the target video drama, and this second character is different from the first character. The second character actor information is used to introduce the actor who plays the second character. The second plot information is used to introduce the second plot in the target video (for example, this second plot can be another plot different from the first plot). The second comment information refers to one or more comments related to the target video, and this second comment information is different from the first comment information.

[0105] In some embodiments, a first person and a second person appear simultaneously in the target video. Information about either the first person or the second person is displayed. If the first person's information and / or the first person's actor information are displayed, the display of the second person's information and / or the second person's actor information is switched via a second operation on the first introduction card. Similarly, if the second person's information and / or the second person's actor information are displayed, the display of the first person's information and / or the first person's actor information is switched via a second operation on the second introduction card.

[0106] In some embodiments, when multiple people (three or more) appear in the target video, the second operation is performed multiple times to switch the person information multiple times.

[0107] In some embodiments, when multiple people appear in the target video, information on two or three people can be displayed simultaneously. The second operation can be used to switch between information on two or three people at the same time, or the second operation can be used to switch between information on one person.

[0108] In addition, the second operation can be switching between the first and second introduction cards by swiping on the touch screen, or by shaking the phone. This application does not limit the type of the second operation.

[0109] In this embodiment, the introduction cards allow viewers to better understand the character and actor information, enriching the video content. Furthermore, the switching and expanding function of the introduction cards increases human-computer interaction, further introducing character and actor information beyond the original cards, thus further enriching the video content.

[0110] Please refer to Figure 11 The diagram illustrates a flowchart of a video processing method provided in one embodiment of this application. The entity executing each step of the method may be... Figure 1 The server 20 in the implementation environment of the scheme shown may include the following steps (1110-1130):

[0111] Step 1110: Obtain the target video.

[0112] The target video is the video to be processed. The video content in the target video is searched and a material library for the target video is generated. The material library for the target video contains any one or more of the following: character information of the target video, plot information of the target video, plot poster of the target video, character and actor information of the target video, etc.

[0113] Step 1120: Identify the video content of the first time segment in the target video.

[0114] The server can identify video content at different times within a target video. Taking the first time segment as an example, the server can identify the people, scenes, plot, and other video content within that segment.

[0115] Step 1130: Generate associated content related to the video content of the first time segment. The associated content is used to display when the target video plays to the first time segment.

[0116] The target video's playback window is smaller than the screen size of the video playback device. The related content is displayed below the video playback window, and the display area of ​​the related content includes the area outside the video playback window.

[0117] As mentioned above, associated content can be either associated videos or associated animations. Accordingly, there are two ways to generate associated content:

[0118] Optionally, the server generates associated videos related to the video content of the first time segment, which are used to display when the target video plays to the first time segment.

[0119] In some embodiments, image materials related to the video content of the first time period are obtained, and a first associated video related to the video content of the first time period is generated based on the image materials. The first associated video is used by the client to play in the background video.

[0120] The aforementioned image materials can be obtained from sources other than the target video, or they can be extracted from the target video. For example, the server obtains image materials of the target video from the network. The server can search for relevant image materials from the network based on information such as the name of the target video and the names of the characters, such as poster images of the target video, poster images of the characters, and still images of the characters. This application does not limit this. Alternatively, the server can also use AI (Artificial Intelligence) facial recognition technology to identify characters in each image frame of the target video, thereby identifying image frames containing the desired target character, and then using these image frames as image materials, or extracting a portion of the image frame as image materials. Furthermore, after obtaining the image materials, they can be placed into a template to generate the first associated video.

[0121] In some embodiments, a video segment of a first time period is extracted from the target video, and the resolution of the video segment of the first time period is increased to obtain a second associated video related to the video content of the first time period. The second associated video is used by the client to play in the background video.

[0122] The server analyzes the video content and segments the video based on the analysis results to obtain video clips. This analysis can be performed by observing the appearance and exit of characters, scene transitions, or changes in viewing distance; this application does not limit the method of video content analysis. The video clips include character video clips, plot video clips, and event video clips, etc.; this application does not limit the type of video clip.

[0123] Optionally, the video clips can be video clips of people. AI facial recognition technology is used to mark the time a person appears in the video as the start time and the time a person leaves the video as the end time. Video clips of the marked people are extracted, and the second associated video is the video clip of that person. The time a person appears in the video refers to the time when the marked person appears, and the time a person leaves the video refers to the time when the marked person leaves the video. The marked person is a person appearing in the video footage library. The video clips of the people refer to video clips of the main people in the video, including the main person's entrance, exit, and emotional expressions, etc., which are not limited in this application.

[0124] Optionally, the second associated video can also be a plot video clip, with the start time of the plot as the start time and the end time of the plot as the end time. The start time refers to the point in the video where the plot begins, and the end time is the point in the video where the plot ends. The second associated video is a segment of the plot video clip. This plot video clip can be a climax or a plot twist, etc.; this application does not limit the type of plot video clip. Specifically, a climax is a segment in the video where the plot reaches its climax, and a plot twist is a segment in the video where the plot takes a turn.

[0125] Optionally, the server generates associated animations related to the video content of the first time segment, which are displayed when the target video plays to the first time segment.

[0126] In some embodiments, such as Figure 12 As shown, three methods for generating background videos are introduced:

[0127] 1. Generate background videos using the video names and plot information related to the target videos in the target video material library.

[0128] 2. Using AI facial recognition technology, identify the person who appears most frequently in the target video. Based on the time points when the person appears and disappears in the target video, slice the video to obtain video segments of that person and generate the background video.

[0129] 3. Generate a background video by creating corresponding dynamic effects based on the video content in the target video.

[0130] Of course, in some other embodiments, background videos can also be generated in other ways, and this application does not limit this.

[0131] The technical solution provided in this application displays related content to the video content during a specific time period when the target video is played. On the one hand, the related content does not need to be created in the original video, thus decoupling it from the original video and overcoming the problems of low efficiency and poor flexibility caused by manual re-creation. This helps to improve the production efficiency and playback flexibility of related content. On the other hand, when playing to a specific time period, displaying related content associated with the video content such as characters, scenes, and plots during that time period creates a linkage effect between the video content and the related content, which can effectively attract users to watch and improve the video playback effect.

[0132] The following are embodiments of the apparatus described in this application, which can be used to execute the embodiments of the method described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method described in this application.

[0133] Please refer to Figure 13 This diagram illustrates a block diagram of a video playback device according to an embodiment of this application. The device has the function of implementing the video playback method described above; this function can be implemented in hardware or by hardware executing corresponding software. The device can be the user terminal described above, or it can be installed within a user terminal. The device 1300 may include: a video playback module 1310 and a content display module 1320.

[0134] The video playback module 1310 is used to play a target video in a video playback window, the size of which is smaller than the screen size of the video playback device.

[0135] The content display module 1320 is used to display related content to the video content of the first time segment in response to the target video playing to the first time segment.

[0136] The display hierarchy of the associated content is located below the video playback window, and the display area of ​​the associated content includes the screen area outside the video playback window.

[0137] In some embodiments, the content display module 1320 is configured to display associated videos related to the video content of the first time period in response to the target video being played to the first time period.

[0138] In some embodiments, the content display module 1320 is used to display a first associated video related to the video content of the first time period, the first associated video being a video created based on image materials related to the video content of the first time period.

[0139] In some embodiments, the content display module 1320 is used to display a second associated video related to the video content of the first time period, wherein the second associated video is a video that is the same as the video content of the first time period and is played synchronously.

[0140] In some embodiments, the content display module 1320 is configured to, in response to a rotation operation on the video playback device, display a partial image of the second associated video corresponding to the orientation of the video playback device during the process of displaying the second associated video.

[0141] In some embodiments, the content display module 1320 is configured to display associated dynamics related to the video content of the first time period in response to the target video being played to the first time period.

[0142] In some embodiments, the device 1300 further includes a card display module 1330.

[0143] The card display module 1330 is used to display a first introduction card related to the video content of the second time period in response to the playback of the target video to the second time period. The first introduction card is used to display at least one of the following introductory information: first character information, first plot information, first plot poster, and first character actor information.

[0144] In some embodiments, the card display module 1330 is further configured to switch the first introduction card from a simplified display style to a detailed display style in response to a first operation on the first introduction card, wherein the detailed display style contains more introduction information than the simplified display style.

[0145] In some embodiments, the card display module 1330 is further configured to switch the first introduction card to a second introduction card in response to a second operation on the first introduction card. The second introduction card is configured to display at least one of the following introductory information: second character information, second plot information, second plot poster, and second character actor information.

[0146] Please refer to Figure 14 This diagram illustrates a block diagram of a video processing apparatus according to an embodiment of this application. The apparatus has the function of implementing the video processing method described above; this function can be implemented in hardware or by hardware executing corresponding software. The apparatus can be the server described above, or it can be located within a server. The apparatus 1400 may include: a video acquisition module 1410, a video recognition module 1420, and a content generation module 1430.

[0147] The video acquisition module 1410 is used to acquire the target video.

[0148] The video recognition module 1420 is used to identify the video content of the first time segment in the target video.

[0149] The content generation module 1430 is used to generate associated content related to the video content of the first time segment. The associated content is used to be displayed when the target video is played to the first time segment.

[0150] The target video's playback window is smaller than the screen size of the video playback device. The display hierarchy of the associated content is located below the video playback window, and the display area of ​​the associated content includes the screen area outside the video playback window.

[0151] In some embodiments, the content generation module 1430 is used to generate an associated video related to the video content of the first time period, and the associated video is used to be displayed when the target video is played to the first time period.

[0152] In some embodiments, the content generation module 1430 is used to acquire image materials related to the video content of the first time period; and based on the image materials, generate a first associated video related to the video content of the first time period.

[0153] In some embodiments, the content generation module 1430 is used to extract a video segment of a first time period from the target video; and to perform resolution upscaling processing on the video segment of the first time period to obtain a second associated video related to the video content of the first time period.

[0154] In some embodiments, the content generation module 1430 is used to generate associated animation effects related to the video content of the first time period, and the associated animation effects are used to be displayed when the target video is played to the first time period.

[0155] It should be noted that the apparatus provided in the above embodiments is only illustrated by the division of the above functional modules when implementing its functions. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the content structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0156] Please refer to Figure 15 This illustration shows a schematic diagram of a computer device provided in one embodiment of this application. The computer device can be any electronic device with data computing, processing, and storage functions, such as the video playback device or server described above. The computer device can be implemented as a video playback device to implement the video playback method provided in the above embodiments; or, the computer device can be implemented as a server to implement the video processing method provided in the above embodiments. Specifically:

[0157] The computer device 1500 includes a processing unit (such as a CPU (Central Processing Unit), GPU (Graphics Processing Unit), and FPGA (Field Programmable Gate Array)) 1501, a system memory 1504 including RAM (Random-Access Memory) 1502 and ROM (Read-Only Memory) 1503, and a system bus 1505 connecting the system memory 1504 and the central processing unit 1501. The computer device 1500 also includes a basic input / output system 1506 to facilitate information transfer between various devices within the server, and a mass storage device 1507 for storing the operating system 1513, application programs 1514, and other program modules 1515.

[0158] The basic input / output system 1506 includes a display 1508 for displaying information and an input device 1509 for user input, such as a mouse or keyboard. Both the display 1508 and the input device 1509 are connected to the central processing unit 1501 via an input / output controller 1510 connected to the system bus 1505. The basic input / output system 1506 may also include the input / output controller 1510 for receiving and processing input from multiple other devices such as a keyboard, mouse, or electronic stylus. Similarly, the input / output controller 1510 also provides output to a display screen, printer, or other types of output devices.

[0159] The mass storage device 1507 is connected to the central processing unit 1501 via a mass storage controller (not shown) connected to the system bus 1505. The mass storage device 1507 and its associated computer-readable media provide non-volatile storage for the computer device 1500. That is, the mass storage device 1507 may include computer-readable media (not shown) such as a hard disk or a CD-ROM (Compact Disc Read-Only Memory) drive.

[0160] Without loss of generality, the computer-readable medium may include computer storage media and communication media. Computer storage media include volatile and non-volatile, removable and non-removable media implemented using any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media include RAM, ROM, EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory or other solid-state storage technologies, CD-ROM, DVD (Digital Video Disc) or other optical storage, magnetic tape cassettes, magnetic tape, disk storage, or other magnetic storage devices. Of course, those skilled in the art will recognize that the computer storage medium is not limited to the above-mentioned types. The system memory 1504 and mass storage device 1507 described above can be collectively referred to as memory.

[0161] According to an embodiment of this application, the computer device 1500 can also be connected to a remote computer on a network, such as the Internet. That is, the computer device 1500 can be connected to the network 1512 through the network interface unit 1511 connected to the system bus 1505, or the network interface unit 1511 can be used to connect to other types of networks or remote computer systems (not shown).

[0162] The memory also includes at least one instruction, at least one program, code set, or instruction set, which is stored in the memory and configured to be executed by one or more processors to implement the above-described video playback method or video processing method.

[0163] In an exemplary embodiment, a computer-readable storage medium is also provided, which stores at least one instruction, at least one program, code set, or instruction set. When executed by a processor of a computer device, the at least one instruction, at least one program, code set, or instruction set implements the video playback method or video processing method provided in the above embodiments.

[0164] Optionally, the computer-readable storage medium may include: ROM (Read-Only Memory), RAM (Random-Access Memory), SSD (Solid State Drives), or optical disc, etc. The random access memory may include ReRAM (Resistance Random Access Memory) and DRAM (Dynamic Random Access Memory).

[0165] In an exemplary embodiment, a computer program product or computer program is also provided, comprising computer instructions stored in a computer-readable storage medium. The processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the aforementioned video playback method or video processing method.

[0166] It should be understood that "multiple" as used herein refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. Furthermore, the step numbers described herein are merely illustrative of one possible execution order. In some other embodiments, the steps may not be executed in numerical order, such as two steps with different numbers being executed simultaneously, or two steps with different numbers being executed in the reverse order of the illustration. This application does not limit this.

[0167] The above are merely exemplary embodiments of this application and are not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application shall be included within the protection scope of this application.

Claims

1. A video playback method, characterized in that, The method includes: Play the target video in a video playback window, the size of which is smaller than the screen size of the video playback device; In response to the target video playing to a first time segment, a second associated video related to the video content of the first time segment is displayed. The second associated video is a video segment of the first time segment extracted from the target video. The second associated video is played synchronously with the video content of the first time segment. The display level of the second associated video is located below the video playback window, and the display area of ​​the second associated video includes the screen area outside the video playback window. During the display of the second associated video, in response to a rotation operation on the video playback device, a partial view of the second associated video corresponding to the orientation of the video playback device is displayed based on the orientation of the video playback device; In response to the target video playing to a second time segment, a first introduction card related to the video content of the second time segment is displayed; before the first character appears in the video segment of the second time segment, first plot information and first comment information are displayed in the first introduction card; after the first character appears in the video segment of the second time segment, first character information and first character actor information are added to the first introduction card; after the first character leaves the video segment of the second time segment, the first character information and first character actor information continue to be displayed in the first introduction card; the first character information is used to introduce the first character, and the first character actor information is used to introduce the actor playing the first character; In response to a second operation on the first introductory card, the first introductory card is switched to a second introductory card, the second introductory card being used to display at least one of the following introductory information: second character information, second character actor information, second plot information, and second commentary information.

2. The method according to claim 1, characterized in that, The method further includes: During the display of the second associated video, key elements in the target video are identified; Display a partial view of the second associated video centered on the key element.

3. The method according to claim 2, characterized in that, The process of determining the key elements in the target video includes: Identify the image frames of the target video to determine the key elements in the target video; or, Eye-tracking technology is used to obtain the user's eye focus position in the target video, and the image element corresponding to the focus position is identified as the key element.

4. The method according to claim 1, characterized in that, Following the display of the first introductory card related to the video content of the second time period, the method further includes: In response to a first operation on the first introduction card, the first introduction card is switched from a simplified display style to a detailed display style, wherein the detailed display style contains more introduction information than the simplified display style.

5. A video processing method, characterized in that, The method includes: Acquire the target video; Identify the video content of the first time segment in the target video; Generate a second associated video related to the video content of the first time period. The second associated video is used to be displayed when the target video is played to the first time period. The second associated video is a video segment of the first time period extracted from the target video. The second associated video is used to be played synchronously with the video content of the first time period. Wherein, the size of the video playback window of the target video is smaller than the screen size of the video playback device, the display layer of the second associated video is located below the video playback window, and the display area of ​​the second associated video includes the screen area outside the video playback window; the video playback device is used to, during the display of the second associated video, in response to a rotation operation on the video playback device, display a partial image of the second associated video corresponding to the orientation of the video playback device; the video playback device is also used to, when the target video plays to the second time segment, display a first introductory card related to the video content of the second time segment; the video playback device is also used to, before the first character appears in the video segment of the second time segment, display in the first introductory card... The first plot information and the first comment information, and for adding and displaying the first character information and the first character actor information in the first introduction card after the first character appears in the video segment of the second time period, and for continuing to display the first character information and the first character actor information in the first introduction card after the first character leaves the video segment of the second time period; the first character information is used to introduce the first character, and the first character actor information is used to introduce the actor who plays the first character; in response to a second operation on the first introduction card, the first introduction card is switched to a second introduction card, the second introduction card being used to display at least one of the following introductory information: second character information, second character actor information, second plot information, and second comment information.

6. The method according to claim 5, characterized in that, The generation of a second associated video related to the video content of the first time period includes: Extract a video segment from the first time period from the target video; The video segment of the first time period is subjected to resolution upscaling to obtain the second associated video related to the video content of the first time period.

7. A video playback device, characterized in that, The device includes: The video playback module is used to play a target video in a video playback window, the size of which is smaller than the screen size of the video playback device; The content display module is configured to, in response to the target video playing to a first time segment, display a second associated video related to the video content of the first time segment. The second associated video is a video segment extracted from the target video of the first time segment. The second associated video is played synchronously with the video content of the first time segment. The display layer of the second associated video is located below the video playback window, and the display area of ​​the second associated video includes the screen area outside the video playback window. During the display of the second associated video, in response to a rotation operation on the video playback device, a partial image of the second associated video corresponding to the orientation of the video playback device is displayed. A card display module is configured to, in response to the target video playing to a second time segment, display a first introduction card related to the video content of the second time segment; before the first character appears in a video segment of the second time segment, display first plot information and first comment information in the first introduction card; after the first character appears in a video segment of the second time segment, add and display first character information and first character actor information in the first introduction card; after the first character exits a video segment of the second time segment, continue to display first character information and first character actor information in the first introduction card; the first character information is used to introduce the first character, and the first character actor information is used to introduce the actor playing the first character; in response to a second operation on the first introduction card, switch the first introduction card to a second introduction card, the second introduction card being used to display at least one of the following introductory information: second character information, second character actor information, second plot information, and second comment information.

8. A video processing apparatus, characterized in that, The device includes: The video acquisition module is used to acquire the target video; The video recognition module is used to identify the video content of the first time segment in the target video; The content generation module is used to generate a second associated video related to the video content of the first time period. The second associated video is used to be displayed when the target video is played to the first time period. The second associated video is a video segment of the first time period extracted from the target video. The second associated video is used to be played synchronously with the video content of the first time period. Wherein, the size of the video playback window of the target video is smaller than the screen size of the video playback device, the display layer of the second associated video is located below the video playback window, and the display area of ​​the second associated video includes the screen area outside the video playback window; the video playback device is used to, during the display of the second associated video, in response to a rotation operation on the video playback device, display a partial image of the second associated video corresponding to the orientation of the video playback device; the video playback device is also used to, when the target video plays to the second time segment, display a first introductory card related to the video content of the second time segment; the video playback device is also used to, before the first character appears in the video segment of the second time segment, display in the first introductory card... The first plot information and the first comment information, and for adding and displaying the first character information and the first character actor information in the first introduction card after the first character appears in the video segment of the second time period, and for continuing to display the first character information and the first character actor information in the first introduction card after the first character leaves the video segment of the second time period; the first character information is used to introduce the first character, and the first character actor information is used to introduce the actor who plays the first character; in response to a second operation on the first introduction card, the first introduction card is switched to a second introduction card, the second introduction card being used to display at least one of the following introductory information: second character information, second character actor information, second plot information, and second comment information.

9. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing at least one program, the at least one program being loaded and executed by the processor to implement the video playback method as described in any one of claims 1 to 4, or to implement the video processing method as described in any one of claims 5 to 6.

10. A computer-readable storage medium, characterized in that, The storage medium stores at least one program, which is loaded and executed by a processor to implement the video playback method as described in any one of claims 1 to 4, or the video processing method as described in any one of claims 5 to 6.

11. A computer program product, characterized in that, The computer program product includes computer instructions stored in a computer-readable storage medium, and a processor reads and executes the computer instructions from the computer-readable storage medium to implement the video playback method as described in any one of claims 1 to 4, or the video processing method as described in any one of claims 5 to 6.

Images