Display method, electronic device, storage medium and computer program product

Abstract

The invention provides a display method, electronic equipment, a storage medium and a computer program product. The method comprises the following steps: acquiring playing time of each video frame in a first video stream, and acquiring playing content displayed by each video frame; determining first countdown information corresponding to each video frame according to the playing time and the type information corresponding to the playing content, and sending each video frame and the first countdown information to the terminal, so that the terminal adds the first countdown information to a first display area in each video frame, and displaying each added video frame on the first display interface.

Description

Technical Field

[0001] This application relates to artificial intelligence technology, and more particularly to a display method, electronic device, storage medium, and computer program product. Background Technology

[0002] In related technologies, users typically obtain countdown information related to the playback event based on the time displayed on the playback screen, which results in the inability to obtain countdown information in real time. Summary of the Invention

[0003] This application provides a display method, electronic device, storage medium, and computer program product that can ensure the real-time nature of the acquired countdown information.

[0004] The technical solution of this application embodiment is implemented as follows: This application provides a display method applied to a server, the method comprising: Get the playback time of each video frame in the first video stream, and get the playback content displayed in each video frame; The first countdown information corresponding to each video frame is determined based on the playback time and the type information corresponding to the playback content. Each video frame and the first countdown information are sent to the terminal, so that the terminal adds the first countdown information to the first display area of ​​each video frame and displays each video frame after adding the information on the first display interface.

[0005] This application provides an electronic device, the electronic device comprising: Memory is used to store executable instructions or computer programs. The processor, when executing computer-executable instructions or computer programs stored in the memory, implements the method provided in the embodiments of this application.

[0006] This application provides a storage medium storing a computer program or computer-executable instructions, which, when executed by a processor, implements the display method provided in this application.

[0007] This application provides a computer program product, including a computer program or computer executable instructions, which, when executed by a processor, implements the display method provided in this application.

[0008] The embodiments of this application have the following beneficial effects: The first countdown information for each video frame is determined by considering the playback time and the type of content displayed in each video frame. This comprehensive approach improves the accuracy of the countdown information. Furthermore, the determination of the first countdown information incorporates the playback time of each video frame, ensuring that the obtained first countdown information matches the playback time of each video frame, thereby guaranteeing the real-time nature of the obtained first countdown information. Attached Figure Description

[0009] Figure 1 A schematic flowchart illustrating a display method provided in an embodiment of this application; Figure 2 A flowchart illustrating an exemplary method for presenting a countdown to a live sporting event, provided as an embodiment of this application; Figure 3 An exemplary display result diagram provided for an embodiment of this application; Figure 4 This is a schematic diagram of the structure of a display device provided in an embodiment of this application; Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0010] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0011] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0012] In the following description, references to "some embodiments" refer to a subset of all possible embodiments. It is understood that "some embodiments" may be the same or different subsets of all possible embodiments and may be combined with each other without conflict. It should also be noted that the terms "first," "second," etc., used in the embodiments of this application are merely for distinguishing similar objects and do not represent a specific ordering of objects. It is understood that "first," "second," etc., may be interchanged in a specific order or sequence where permissible, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0013] At the game, a countdown timer is displayed on signs, and its start and stop are controlled by on-site staff based on the referee's actions. During live broadcasts, viewers rely on the countdown timer captured in the feed to determine possession. However, the live broadcast frequently switches and resets the countdown, such as in basketball games, where information like remaining time for the offensive team or time across half-court is frequently reset during changes of possession or fouls. Limited by the feed, viewers can only obtain this information when the countdown is displayed. When the countdown is hidden, viewers struggle to accurately grasp the game's progress.

[0014] This application provides a display method. Figure 1 This is a flowchart illustrating a display method provided in an embodiment of this application; as shown below. Figure 1 As shown, applied to a server, the method includes; S101. Obtain the playback time of each video frame in the first video stream, and obtain the playback content displayed in each video frame.

[0015] It should be noted that the display method can be understood as the display method for the countdown of live sports events.

[0016] It should be noted that the first video stream can be understood as a live video stream. The number of video frames in the first video stream can be determined according to the actual situation and is not limited here. Playback time can be understood as the presentation time of each video frame, or the frame presentation time stamp (PTS). Playback content can be understood as the image displayed in each video frame.

[0017] S102. Determine the first countdown information corresponding to each video frame based on the playback time and the type information corresponding to the playback content.

[0018] It should be noted that the type information includes a first type and a second type; the first type indicates that no abnormal events occurred during playback; the second type indicates that abnormal events were sent during playback. In the context of live basketball games, the first type can be understood as the normal situation, i.e., the basketball game is proceeding normally; abnormal events include at least one of the following events: pause, reset, or events corresponding to countdown information; in practical applications, abnormal events can be illustrated as ball out of bounds, inbounds pass, ball hitting the basket, change of possession, foul, violation, pause, pause end, inbounds pass, crossing the center line, replay start, and replay end.

[0019] S103. Send each video frame and the first countdown information to the terminal so that the terminal adds the first countdown information to the first display area of ​​each video frame and displays each video frame on the first display interface after adding the information.

[0020] It should be noted that the first display area can be understood as any area of ​​each video frame, and can be determined according to the actual situation; no limitation is made here. The first display interface is the terminal's display interface. Each video frame and the first countdown information are sent to the terminal so that the terminal adds the first countdown information to the first display area of ​​each video frame and displays each video frame on the first display interface after adding the first countdown information. This can be understood as the server sending each video frame and the first countdown information to the terminal, the terminal adding the first countdown information to the first display area of ​​each video frame, and displaying each video frame on the first display interface after adding the first countdown information; so that users viewing the first display interface can accurately obtain the countdown for each video frame.

[0021] The solution in this application determines the first countdown information of each video frame by taking into account the playback time of each video frame and the type information corresponding to the playback content displayed in each video frame. By comprehensively considering the playback time and the type information of the playback content, the accuracy of the obtained countdown can be improved. Furthermore, the playback time of each video frame is combined in the process of determining the first countdown information to ensure that the obtained first countdown information conforms to the playback time of each video frame, thereby ensuring the real-time nature of the obtained first countdown information.

[0022] In this embodiment of the application, the process of determining the first countdown information corresponding to each video frame based on the type information corresponding to the playback time and playback content specifically includes: determining the second countdown information corresponding to each video frame based on the type information corresponding to the playback time and playback content; and performing time calibration on the second countdown information based on each video frame to obtain the first countdown information.

[0023] It should be noted that the second countdown information corresponding to each video frame is determined based on the playback time and the type information corresponding to the playback content. The second countdown information is then time-calibrated based on each video frame to obtain the first countdown information. This can be understood as follows: the event type determination model is used to determine the type information corresponding to the playback content, and the second countdown information corresponding to each video frame is determined based on the playback time and the type information corresponding to the playback content. Each video frame is then input into the time calibration model to time-calibrate the second countdown information to obtain the first countdown information.

[0024] In this embodiment of the application, the process of determining the second countdown information corresponding to each video frame based on the type information corresponding to the playback time and playback content specifically includes: when the type information corresponding to the playback content of the first video frame in each video frame represents a first type, determining the second countdown information based on the playback time of the first video frame and the playback time of the second video frame; the second video frame is the video frame preceding the first video frame; the first type is that no abnormal event has occurred in the playback content; the abnormal event includes at least one of the following events: pause, reset, or the event corresponding to the countdown information; when the type information corresponding to the playback content of the first video frame represents a second type, determining the second countdown information based on the playback time and type information; the second type is that an abnormal event has occurred in the playback content.

[0025] It should be noted that in the scenario of live basketball games, the first type is when no abnormal events occur in the playback content, i.e., the basketball game is proceeding normally; the second type is when abnormal events occur in the playback content, including at least one of the following events: pause, reset, or events corresponding to countdown information. In practical applications, abnormal events can be illustrated as ball out of bounds, inbounds pass, ball hitting the basket, change of possession, foul, violation, pause, pause end, inbounds pass, crossing the center line, replay start, and replay end. When the type information corresponding to the playback content of the first video frame in each video frame represents the first type, the second countdown information is determined based on the playback time of the first video frame and the playback time of the second video frame. This can be understood as follows: if the playback content of the first video frame is a basketball game proceeding normally, the difference between the playback time of the first video frame and the playback time of the second video frame is obtained. The countdown information corresponding to the second video frame is then obtained by subtracting this difference from the countdown information corresponding to the second video frame to obtain the first countdown information corresponding to the first video frame. Alternatively, it can be illustrated as follows: the countdown timer corresponding to the previous video frame is subtracted from the PTS difference between the two frames, and this becomes the countdown timer for the current video frame.

[0026] It should be noted that when the type information corresponding to the playback content of the first video frame represents the second type, determining the second countdown information based on the playback time and type information can be understood as follows: if the playback content of the first video frame is ball out of bounds, inbounds pass, ball hits the basket, possession changes, foul, violation, pause, pause ends, inbounds pass, crosses the center line, replay starts, or replay ends, the previous video frame is obtained based on the playback time, and the countdown corresponding to the previous video frame is obtained. Then, based on the second type, the countdown corresponding to the previous video frame is intervened accordingly, serving as the first countdown information for this video frame. For ease of understanding, an example can be given: if the type information corresponding to the playback content of the first video frame represents ball hits the basket, the 24 seconds required to reset after the ball hits the basket are directly used as the first countdown information for this video frame; if the type information corresponding to the playback content of the first video frame represents a stop (i.e., pause), the countdown corresponding to the previous video frame is used as the first countdown information for this video frame.

[0027] The solution in this application determines the first countdown information corresponding to each video frame by obtaining the type and playback time of the playback content displayed in each video frame. This can take into account various situations in the video frame and obtain the countdown corresponding to each video frame more accurately.

[0028] In this embodiment of the application, the process of time-calibrating the second countdown information based on the first video frame to obtain the first countdown information specifically includes: if the third countdown information is obtained from the playback content displayed in each video frame, the third countdown information is used as the first countdown information; if the third countdown information is not obtained from the playback content displayed in each video frame, the second countdown information is used as the first countdown information.

[0029] It should be noted that if the third countdown information is obtained from the playback content displayed in each video frame, the third countdown information is used as the first countdown information; if the third countdown information is not obtained from the playback content displayed in each video frame, the second countdown information is used as the first countdown information. This can be understood as follows: the first N video frames and the last M video frames of each video frame are selected from the first video stream according to the playback time to form the video frame set corresponding to each video frame; the video frame set is input into the time calibration model to obtain the output content (the screen corresponding to the playback content). If the third countdown information is obtained from the output content, the third countdown information is used as the first countdown information; if the third countdown information is not obtained from the output content, the second countdown information is used as the first countdown information.

[0030] It should be noted that the third countdown information can be understood as the countdown information directly obtained from the output image corresponding to each video frame. Using the third countdown information as the first countdown information, or using the second countdown information as the first countdown information, both refer to the countdown of the same frame.

[0031] In this embodiment of the application, the process of using the third countdown information as the first countdown information specifically includes: determining the time level information corresponding to the third countdown information; when the time level information is at the millisecond level, using the third countdown information as the first countdown information; when the time level information is at the second level and the integer second value corresponding to the third countdown information is different from the integer second value corresponding to the second countdown information, using the integer second value corresponding to the third countdown information as the first countdown information.

[0032] It should be noted that when the time level information is at the millisecond level, the third countdown information is used as the first countdown information; when the time level information is at the second level, and the integer second value corresponding to the third countdown information is different from the integer second value corresponding to the second countdown information, the integer second value corresponding to the third countdown information is used as the first countdown information. This can be understood as follows: when the third countdown information (the countdown information directly obtained from the screen) is at the millisecond level, it is directly used as the first countdown information; when the third countdown information is at the second level, and the integer second value corresponding to the third countdown information is different from the integer second value corresponding to the second countdown information (the value at the second level has changed), the integer second value corresponding to the third countdown information is used as the first countdown information.

[0033] The solution in this application embodiment obtains the third countdown information directly from the playback content displayed in each video frame to calibrate the first countdown information, thereby obtaining the countdown information corresponding to each video frame more accurately.

[0034] In this embodiment of the application, the method further includes: selecting the first N video frames and the last M video frames of each video frame from the first video stream to form a video frame set corresponding to each video frame; performing type determination on the video frame set corresponding to each video frame to obtain type information corresponding to the playback content displayed in each video frame.

[0035] It should be noted that the specific values ​​of N and M in the first N video frames and the last M video frames can be determined according to the actual situation and are not limited here. Selecting the first N video frames and the last M video frames from each video frame in the first video stream can be understood as selecting the first N video frames and the last M video frames with the playback time (PTS) of each video frame as a reference point. Type determination is performed on the set of video frames corresponding to each video frame to obtain the type information corresponding to the playback content displayed in each video frame. This can be understood as inputting the set of video frames into the event determination model to obtain the output queue corresponding to the set of video frames, and then determining the type information by judging the output queue. The type information includes a first type and a second type, which have been described above and will not be repeated here.

[0036] The solution in this application embodiment can accurately determine whether additional operations are needed for the countdown of each video frame by determining the type identifier corresponding to each video frame.

[0037] In this embodiment of the application, the process of sending each video frame and the first countdown information to the terminal specifically includes: binding each video frame and the first countdown information to obtain the binding relationship corresponding to each video frame; encoding the first video stream and inserting the binding relationship during the encoding process to obtain the second video stream; and sending the second video stream to the terminal.

[0038] It should be noted that binding each video frame with the first countdown information can be understood as defining the playback time (PTS) of each video frame with the first countdown information. Encoding the first video stream and inserting the binding relationship during the encoding process yields the second video stream. Sending the second video stream to the terminal can be understood as encoding the first video stream, writing Supplemental Enhancement Information (SEI) (i.e., inserting the binding relationship) during the encoding process, obtaining the second video stream, and then sending the second video stream to the terminal. Upon receiving the second video stream, the terminal decodes it to obtain the binding relationship corresponding to each video frame.

[0039] The solution in this application embodiment binds each video frame with first countdown information and sends it to the terminal, so that the terminal adds the first countdown information to the first display area of ​​each video frame according to the binding relationship, so that the user viewing the first display interface can obtain the countdown information corresponding to each video frame in real time, thereby improving the user experience.

[0040] To facilitate understanding, an example of the above method is provided here. Specifically, by combining information appearing in the live broadcast source, the timing of the stop and start of the clock is determined based on events in the match footage. The countdown to possession of the ball is calculated, and the countdown hidden in the frame is continued and displayed on the live stream. The calculated countdown is calibrated based on occasional countdowns from the filming location, achieving an accurate, real-time countdown that is relevant to the scene. Specific steps include: training an artificial intelligence (AI) model to increase the accuracy of countdown acquisition and reset timing judgment; analyzing the live broadcast source footage to ensure that the countdown calculated in the background matches the scene presentation time and guarantees real-time performance; obtaining the countdown from the preceding footage, analyzing it, and then using it for the following footage, increasing the operability and accuracy of calibration.

[0041] This application embodiment uses two trained models to analyze the live stream footage: (1) Make judgments on events that occur during the event and pinpoint the time of occurrence.

[0042] (2) Quantify the countdown that appears in the picture and use it as a reference for calibration through continuous analysis.

[0043] Through logical judgment, a countdown timer based on the frame's moment is output and maintained automatically in the backend. Each frame is bound to a custom SEI (Search Engine Identity) through video encoding. Finally, a live event stream containing the optional countdown timer is output for the receiving end to view.

[0044] Figure 2 A flowchart illustrating an exemplary method for presenting a countdown timer for a live sporting event, as provided in this application embodiment; Figure 2 As shown, the specific steps are as follows.

[0045] 1. External input.

[0046] It should be noted that this is an externally input live stream.

[0047] 2. Content creation.

[0048] 3. Event Judgment Model.

[0049] It should be noted that the determination of table running / stopping / resetting is performed through an event-based determination model.

[0050] 4. Time calibration model.

[0051] It should be noted that the time calibration model performs time calibration.

[0052] 5. Encoding components.

[0053] It should be noted that the live stream and production information are sent to the encoding component.

[0054] 6. Distribution network.

[0055] It should be noted that the live stream + SEI is sent to the distribution network.

[0056] 7. Terminal.

[0057] It should be noted that the live stream + SEI is sent to the terminal.

[0058] The above steps are explained in detail here.

[0059] Step 1: The system first creates a self-maintained background countdown timer for subsequent use. Upon receiving the input live stream, it decodes it, constructing a set containing several frames, with the number of frames adjustable through configuration. A binding relationship is then established between each frame and the constructed set. Detailed steps are as follows.

[0060] (1) When a live stream is input from an external source, the system generates a self-maintained background countdown timer and sets its status to "stop". The initial default value can be set to 0 or the maximum countdown value. Since the timer is stopped, it does not affect the subsequent logic.

[0061] (2) The decoding component breaks down the live stream into consecutive frames.

[0062] (3) Based on the timeline, the PTS of each frame is used as the reference point X.

[0063] (4) Based on the configuration parameter Y, select several frames before and after the reference point X as a dataset Z, where Y is used to describe the number of frames before and after the selected reference frame and adjust the judgment accuracy.

[0064] (5) Binding the continuous reference point X to the dataset Z<X,Z> Make two copies, named T. x and S x .

[0065] Step Two: Analyze the binding relationship dataset using the trained event judgment model to generate an event identifier for the video frame. This event identifier indicates whether any additional operations requiring a continuation countdown, such as resetting, stopping, or resetting, occurred within the video frame. Normally, the continuation countdown can be calculated by subtracting from the time difference between video frames. Detailed steps are as follows.

[0066] (1) Continue to use T x As input, the trained event determination model is passed in to obtain an output list P for X.x This includes determining whether various on-court events have occurred, including but not limited to the following events: ball out of bounds, inbounds pass, ball hitting the basket, change of possession, foul, violation, timeout, timeout end, inbounds pass, crossing the center line, replay start and replay end.

[0067] (2) For P x The analysis is performed, and the countdown event determination results are given based on the screen time mapped from the reference point X. These events include, but are not limited to, the following: the countdown needs to be reset for 24 seconds, the countdown needs to be reset for 14 seconds, the countdown stops, and the countdown starts.

[0068] (3) Based on the above judgment results, determine whether events such as pause, reset, or table reading occur. If they occur, intervene accordingly based on the background continuation countdown of the previous frame and use it as the continuation countdown of this frame. If no event occurs, subtract the PTS difference between the two frames from the continuation countdown of the previous frame and use it as the continuation countdown of this frame.

[0069] Step 3: Analyze the binding relationship dataset using the trained time calibration model to provide a time calibration decision for the video frame. If a countdown appears in the frame, determine whether the countdown moment in the preceding and following frames can be used as a calibration reference. If so, continue the countdown using that moment for calibration; otherwise, no additional steps are required. Detailed steps are as follows.

[0070] (1) Bind another relationship T x The time-calibrated model is continuously trained using X as input, resulting in an output column Q with respect to X. x This includes, but is not limited to, the following: the appearance of a left-side timer, the appearance of a right-side timer, the appearance of an official timer, the appearance of a mid-screen timer, the presence of a screen with a minimum second level of display, a screen with a minimum millisecond level of display, a screen without a countdown, the timer display time, and whether the current frame time has changed.

[0071] (2) Based on the output of the time calibration model, determine whether the background countdown of the system maintenance needs to be calibrated: if no screen is obtained, no calibration is required; if countdown information is obtained from the screen, determine the time level. If it is at the millisecond level, the specific time in the screen can be used as the calibration value; if it is at the second level, when the value at that level changes, the whole second value is used as the calibration value.

[0072] Step 4: Bind the calculated countdown timer to the PTS (Presentation Time Sequence) of the video frame. Then, write the description of the binding relationship as a custom parameter into the SEI (Search Engine Information) during the encoding process. Repeat the above steps to output a live stream, obtaining a countdown timer value for each frame. Detailed steps are as follows.

[0073] (1) Based on the above-processed background countdown, bind it to the PTS corresponding to the reference point X. This binding relationship can be given for each X; or it can be given when events such as pause, reset, or reading the table or calibration occur. The binding relationship of each frame in the subsequent time interval can be obtained through the offset of PTS between frames through the previous binding relationship.

[0074] However, with the latter method, when the receiving end accesses the intermediate point, it is impossible to obtain the frame binding relationship normally before the time point occurs.

[0075] (2) When encoding live content, insert the binding relationship in SEI, that is, each frame can directly or indirectly obtain a set of mapping binding relationships.

[0076] Step 5: Transmit the data over the network and finally display the effect on the terminal. Detailed steps are as follows.

[0077] (1) The processed live stream is transmitted over the network and can eventually be obtained by the receiving end.

[0078] (2) At the receiving end, the binding relationship is obtained by decoding according to the predetermined data structure, and can be customized to be presented in a certain area of ​​the live broadcast screen. Figure 3 An exemplary display result diagram provided for an embodiment of this application; such as Figure 3 As shown, Figure 3 (a) The countdown is displayed when no time information is shown on the screen; Figure 3 (b) is the countdown displayed when time information appears on the screen.

[0079] The event determination model described above is trained in the following way: Based on historical event data, events where the countdown status changes are categorized into several scenarios: no change required, need to reset for 24 seconds, need to reset for 14 seconds, the countdown stops, and the countdown starts. The category is used as V, and the frames before and after the event are used as the frame set W.

[0080] (2) with<V,W> The set of binding relationships is taken as input, and the countdown change state is taken as output.

[0081] The neural network model was trained in the above manner, with each neural network having 5 layers, and the final output success rate was greater than 99%.

[0082] Based on the above neural network model, an event determination model is constructed. The output states of the neural network model are merged and output in a voting manner, and the output result with more than half of the results is taken as the standard.

[0083] The time calibration model described above is trained in the following manner: (1) Using historical event data, capture the countdown that appears on the screen, bind it with the actual countdown value of the screen, label it G, and construct a set of <screen, G> binding relationships.

[0084] (2) The set of <screen, G> binding relationships is used as input, and the specific countdown value is used as output.

[0085] (3) Based on historical event data, select and classify the situations in which countdown appears on the screen, and use the screen with the lowest second level, the lowest millisecond level, and the screen without countdown as different sample sets.

[0086] (4) The sample set classification identifier F and the set are used as inputs, and the classification result is used as output.

[0087] (5) Using historical event data, capture the nodes where the countdown changes on the screen to obtain a series of consecutive frames where the value changes. Use a set of frames as elements to construct a set of samples; and construct another set of samples where the countdown does not change, and use whether the countdown changes as the state value S.

[0088] (6) Take the set of <screen group, S> binding relationships as input and the state value of whether it changes as output.

[0089] The neural network model was trained in the above manner, with each neural network having 5 layers, and the final output success rate was greater than 99%.

[0090] A time calibration model is constructed based on the above neural network model. The output states of the neural network model are merged and output in a voting manner, and the output result with more than half of the results is taken as the standard.

[0091] This application provides a display device. Figure 4 This is a schematic diagram of the structure of a display device provided in an embodiment of this application; as shown below. Figure 4 As shown, the display device 400 includes: The acquisition unit 401 is used to acquire the playback time of each video frame in the first video stream and acquire the playback content displayed in each video frame. Determining unit 402 is used to determine the first countdown information corresponding to each video frame based on the playback time and the type information corresponding to the playback content; The sending unit 403 is used to send each video frame and the first countdown information to the terminal, so that the terminal adds the first countdown information to the first display area of ​​each video frame and displays each video frame after adding the information on the first display interface.

[0092] In some embodiments, the determining unit 402 is further configured to determine the second countdown information corresponding to each video frame based on the playback time and the type information corresponding to the playback content; and to perform time calibration on the second countdown information based on each video frame to obtain the first countdown information.

[0093] In some embodiments, the determining unit 402 is further configured to: determine the second countdown information based on the playback time of the first video frame and the playback time of the second video frame when the type information corresponding to the playback content of the first video frame in each video frame represents a first type; the second video frame is the video frame preceding the first video frame; the first type is that no abnormal event has occurred in the playback content; the abnormal event includes at least one of the following events: pause, reset, or an event corresponding to the countdown information; and determine the second countdown information based on the playback time and the type information when the type information corresponding to the playback content of the first video frame represents a second type; the second type is that an abnormal event has occurred in the playback content.

[0094] In some embodiments, the determining unit 402 is further configured to, when obtaining third countdown information from the playback content displayed in each video frame, use the third countdown information as the first countdown information; and when not obtaining the third countdown information from the playback content displayed in each video frame, use the second countdown information as the first countdown information.

[0095] In some embodiments, the determining unit 402 is further configured to determine the time level information corresponding to the third countdown information; when the time level information is at the millisecond level, the third countdown information is used as the first countdown information; when the time level information is at the second level and the whole second value corresponding to the third countdown information is different from the whole second value corresponding to the second countdown information, the whole second value corresponding to the third countdown information is used as the first countdown information.

[0096] In some embodiments, the display device 400 further includes a determination unit, configured to select the first N video frames and the last M video frames of each video frame from the first video stream to form a video frame set corresponding to each video frame; and to perform type determination on the video frame set corresponding to each video frame to obtain type information corresponding to the playback content displayed in each video frame.

[0097] In some embodiments, the sending unit 403 is further configured to bind each video frame and the first countdown information to obtain a binding relationship corresponding to each video frame; encode the first video stream and insert the binding relationship during the encoding process to obtain a second video stream; and send the second video stream to the terminal.

[0098] This application also provides an electronic device. Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application; as shown below. Figure 5 As shown, the electronic device 500 includes a processor 501 and a memory 503. Optionally, the electronic device 500 may also include a communication bus 502.

[0099] In specific embodiments, the processor 501 described above can be at least one of the following: Application Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Digital Signal Processing Device (DSPD), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), CPU, controller, microcontroller, and microprocessor. It is understood that for different devices, the electronic device used to implement the above processor function can also be other types, and this embodiment does not specifically limit it.

[0100] In this embodiment, the communication bus 502 is used to establish communication between the processor 501 and the memory 503; when the processor 501 executes the running program stored in the memory 503, it implements the following display method: The playback time of each video frame in the first video stream is obtained, and the playback content displayed in each video frame is obtained; the first countdown information corresponding to each video frame is determined according to the type information corresponding to the playback time and the playback content; each video frame and the first countdown information are sent to the terminal, so that the terminal adds the first countdown information to the first display area of ​​each video frame, and displays each video frame after adding the countdown information on the first display interface.

[0101] Furthermore, the processor 501 is also configured to determine the second countdown information corresponding to each video frame based on the playback time and the type information corresponding to the playback content; and to perform time calibration on the second countdown information based on each video frame to obtain the first countdown information.

[0102] Furthermore, the processor 501 is also configured to, when the type information corresponding to the playback content of the first video frame in each video frame represents a first type, determine the second countdown information based on the playback time of the first video frame and the playback time of the second video frame; the second video frame is the video frame preceding the first video frame; the first type is that no abnormal event has occurred in the playback content; the abnormal event includes at least one of the following events: pause, reset, or an event corresponding to the countdown information; when the type information corresponding to the playback content of the first video frame represents a second type, determine the second countdown information based on the playback time and the type information; the second type is that an abnormal event has occurred in the playback content.

[0103] Furthermore, the processor 501 is also configured to use the third countdown information as the first countdown information when the third countdown information is obtained from the playback content displayed in each video frame; and to use the second countdown information as the first countdown information when the third countdown information is not obtained from the playback content displayed in each video frame.

[0104] Furthermore, the processor 501 is also configured to determine the time level information corresponding to the third countdown information; when the time level information is at the millisecond level, the third countdown information is used as the first countdown information; when the time level information is at the second level and the whole second value corresponding to the third countdown information is different from the whole second value corresponding to the second countdown information, the whole second value corresponding to the third countdown information is used as the first countdown information.

[0105] Furthermore, the processor 501 is also configured to select the first N video frames and the last M video frames of each video frame from the first video stream to form a video frame set corresponding to each video frame; and to perform type determination on the video frame set corresponding to each video frame to obtain type information corresponding to the playback content displayed in each video frame.

[0106] Furthermore, the processor 501 is also configured to bind each video frame to the first countdown information to obtain a binding relationship corresponding to each video frame; encode the first video stream and insert the binding relationship during the encoding process to obtain a second video stream; and send the second video stream to the terminal.

[0107] This application provides a storage medium storing a computer program thereon. The computer-readable storage medium stores one or more programs, which can be executed by one or more processors. The computer program implements the display method described above.

[0108] Based on the above embodiments, this application provides a computer program product, including a computer program that can be executed by one or more processors, and the computer program implements the display method as described above.

[0109] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0110] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the related technology, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause an image display device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this disclosure.

[0111] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application.

Claims

1. A display method, characterized in that, Applied to a server, the method includes: Get the playback time of each video frame in the first video stream, and get the playback content displayed in each video frame; The first countdown information corresponding to each video frame is determined based on the playback time and the type information corresponding to the playback content. Each video frame and the first countdown information are sent to the terminal, so that the terminal adds the first countdown information to the first display area of ​​each video frame and displays each video frame after adding the information on the first display interface.

2. The method according to claim 1, characterized in that, The step of determining the first countdown information corresponding to each video frame based on the playback time and the type information corresponding to the playback content includes: The second countdown information corresponding to each video frame is determined based on the playback time and the type information corresponding to the playback content. The second countdown information is time-calibrated based on each video frame to obtain the first countdown information.

3. The method according to claim 2, characterized in that, The step of determining the second countdown information corresponding to each video frame based on the playback time and the type information corresponding to the playback content includes: When the type information corresponding to the playback content of the first video frame in each video frame represents a first type, the second countdown information is determined based on the playback time of the first video frame and the playback time of the second video frame; the second video frame is the video frame preceding the first video frame; the first type is that no abnormal event occurs in the playback content; the abnormal event includes at least one of the following events: pause, reset, or the event corresponding to the countdown information; When the type information corresponding to the playback content of the first video frame represents the second type, the second countdown information is determined based on the playback time and the type information; the second type is an abnormal event that occurred in the playback content.

4. The method according to claim 2, characterized in that, The step of time-calibrating the second countdown information based on each video frame to obtain the first countdown information includes: If the third countdown information is obtained from the playback content displayed in each video frame, the third countdown information shall be used as the first countdown information; If the third countdown information is not obtained from the playback content displayed in each video frame, the second countdown information is used as the first countdown information.

5. The method according to claim 4, characterized in that, The step of using the third countdown information as the first countdown information includes: Determine the time level information corresponding to the third countdown information; When the time level information is at the millisecond level, the third countdown information is used as the first countdown information; If the time level information is at the second level and the integer second value corresponding to the third countdown information is different from the integer second value corresponding to the second countdown information, then the integer second value corresponding to the third countdown information shall be used as the first countdown information.

6. The method according to claim 1, characterized in that, The method further includes: Select the first N video frames and the last M video frames of each video frame from the first video stream to form the video frame set corresponding to each video frame; The type of the video frame set corresponding to each video frame is determined to obtain the type information corresponding to the playback content displayed in each video frame.

7. The method according to claim 1, characterized in that, Sending each video frame and the first countdown information to the terminal includes: Each video frame is bound to the first countdown information to obtain the binding relationship corresponding to each video frame; The first video stream is encoded, and the binding relationship is inserted during the encoding process to obtain a second video stream; the second video stream is then sent to the terminal.

8. An electronic device, characterized in that, The electronic device includes: Memory is used to store executable instructions or computer programs. A processor, when executing computer-executable instructions or computer programs stored in the memory, implements the method according to any one of claims 1 to 7.

9. A storage medium having a computer program or computer-executable instructions stored thereon, characterized in that, When the computer program or the computer-executable instructions are executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer program product comprising a computer program or computer-executable instructions, characterized in that, When the computer program or the computer-executable instructions are executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images