Media stream abnormal supplementary transmission method and device based on key frame and medium

Abstract

The invention discloses a key frame-based media stream abnormal supplementary transmission method, equipment and a medium, belongs to the technical field of media stream transmission, and is used for solving the problems that the existing media stream transmission technology has certain limitation in the aspect of guaranteeing the integrity and security of media stream contents, the user experience is poor, and the user experience is poor. And detection of media stream abnormity and effective supplementary transmission missing are difficult to realize. The method comprises the following steps: transmitting a target media stream and corresponding head end unique identification information to terminal equipment; performing unique identification calculation of related terminal key frames on the received media stream which is in the playing state after receiving to obtain terminal unique identification information; performing media stream abnormity judgment on the unique identification information of the head end and the unique identification information of the terminal, and determining abnormal supplementary transmission request information; performing positioning processing of the supplementary transmission content on the target media stream to obtain supplementary transmission content information; and performing data integration processing on the supplementary transmission content information and the abnormal receiving media stream to obtain a supplementary transmission completed media stream.

Description

Technical Field

[0001] This application relates to the field of media streaming transmission, and in particular to a method, device and medium for abnormal media streaming transmission based on keyframes. Background Technology

[0002] In today's era of rapid digital information dissemination, media streaming applications are becoming increasingly widespread, encompassing online video playback, live streaming, video conferencing, and many other fields. However, media streaming faces numerous challenges during transmission, with the integrity and security of its content being particularly prominent. During the transmission of media streams from the headend server to the terminal device, the complex network environment becomes a "destabilizing factor" for the data. Network congestion, signal interference, and malicious attacks are common occurrences. For example, during peak network periods, large-scale data transmission may lead to the loss of some media stream data; malicious attackers may tamper with critical information in the media stream for various malicious purposes. Furthermore, hardware failures and software errors in the terminal device itself can also damage the media stream content. Currently, some technologies addressing media streaming transmission issues focus primarily on encryption protection of data transmission, using security protocols such as SSL / TLS to encrypt data during transmission and prevent theft or tampering. However, this approach has limitations; it cannot detect in real time whether the media stream content is truly complete and unaltered during terminal playback. While some detection methods can identify anomalies in the media stream, they lack effective retransmission mechanisms and cannot promptly restore the integrity of the media stream. For example, when a keyframe in a media stream is detected to be missing or tampered with, existing technologies often can only interrupt playback and notify the user of an error, rather than automatically requesting a replacement of the missing or incorrect content, which greatly affects the user experience.

[0003] This problem is particularly severe in live streaming scenarios. Viewers expect to watch live content smoothly and completely. If there are media stream anomalies that cannot be promptly corrected, it can lead to video stuttering, audio interruptions, or displaying incorrect images, severely damaging the viewing experience and potentially causing viewers to leave. In video conferencing, incomplete media streams can result in inaccurate information delivery, disrupting the normal progress of the meeting.

[0004] In summary, existing media streaming technologies are clearly insufficient in ensuring the integrity and security of media stream content. There is an urgent need for a technical solution that can detect media stream anomalies in real time and effectively retransmit missing or erroneous content to improve user experience and meet the growing demand for multimedia communication. Summary of the Invention

[0005] This application provides a method, device, and medium for retransmitting abnormal media streams based on keyframes, which solves the following technical problems: Existing media stream transmission technologies have certain limitations in ensuring the integrity and security of media stream content, resulting in poor user experience and difficulty in detecting media stream anomalies and effectively retransmitting missing data.

[0006] The embodiments of this application adopt the following technical solutions: On one hand, embodiments of this application provide a method for abnormal media stream retransmission based on keyframes, comprising: calculating a unique identifier for a keyframe in a target media stream to be transmitted using a headend server to obtain a unique headend identifier for the keyframe; transmitting the target media stream and the corresponding unique headend identifier to a terminal device; calculating a unique identifier for a terminal keyframe in a received media stream that is in playback mode using the terminal device to obtain a unique terminal identifier; performing a media stream anomaly judgment on the unique headend identifier and the unique terminal identifier to determine an abnormal retransmission request sent to the headend server; locating the retransmission content in the target media stream according to the abnormal retransmission request to obtain retransmission content information; and integrating the retransmission content information with the abnormal received media stream using the terminal device to obtain a retransmission completed media stream.

[0007] This application's embodiments, by uniquely identifying keyframes, ensure accurate location of content requiring retransmission in the event of anomalies, thereby improving the overall reliability of the media stream transmission. Since only the faulty keyframes are retransmitted, rather than the entire media stream, network bandwidth consumption and data transmission volume are significantly reduced. Furthermore, by quickly retransmitting lost or corrupted keyframes, playback interruptions and stuttering are reduced, improving the user viewing experience. Simultaneously, the intelligent retransmission mechanism avoids unnecessary data transmission, optimizing network resource utilization. The processing of retransmission requests and retransmission content can be performed quickly, reducing delays caused by data loss. It can also handle anomalies in media stream transmission, enhancing the overall robustness of the system. It also simplifies the anomaly handling process, reduces operational costs, and enables real-time monitoring of media stream quality.

[0008] In one feasible implementation, a unique identifier is calculated for each headend keyframe in the target media stream to be transmitted via a headend server. This process specifically includes: performing keyframe identification processing on the target media stream using video and audio analysis algorithms to obtain the headend keyframe located on the headend server; wherein the video keyframes in the headend keyframes include scene transition frames and the start of major actions; the audio keyframes in the headend keyframes include sound start frames, sound end frames, and sound volume change frames; combining and encrypting the timestamp, content hash value, and sequence number of the keyframe in the media stream using an encryption algorithm to obtain the unique identifier information of the headend keyframe; and associating the unique identifier information with the metadata of the headend keyframe to obtain the headend unique identifier information; wherein the metadata includes at least: keyframe location information and keyframe timestamp.

[0009] In one feasible implementation, transmitting the target media stream and the corresponding unique headend identifier to the terminal device specifically includes: receiving the target media stream and the unique headend identifier from the headend server through the terminal device; storing the target media stream and the unique headend identifier in the cache area of ​​the terminal device; wherein, the cache area is related to the media stream playback settings.

[0010] In one feasible implementation, the terminal device performs a unique identifier calculation on the received media stream that is in playback mode after being received, to obtain unique terminal identifier information. Specifically, this includes: controlling the playback of the received media stream in the buffer area of ​​the terminal device; extracting terminal keyframes of the playback segment from the received media stream in real time using the terminal device and based on video and audio analysis algorithms; combining and encrypting the timestamp, content hash value, and sequence number of the keyframe in the media stream using the same encryption algorithm as in the headend server to obtain unique identifier information for the terminal keyframe; and associating the unique identifier information with the corresponding metadata in the playback segment to obtain the unique terminal identifier information.

[0011] In one feasible implementation, the media stream anomaly determination is performed by comparing the unique identifier information of the headend with the unique identifier information of the terminal to determine the abnormal retransmission request information to be sent to the headend server. Specifically, this includes: comparing and identifying the same playback frame period according to a preset playback frame interval period in the received media stream in playback mode to obtain a comparison frame period; based on the comparison frame period, obtaining the corresponding unique identifier information of the terminal device and the corresponding unique identifier information of the headend in the headend server; if the unique identifier information of the headend and the unique identifier information of the terminal are inconsistent, the received media stream is in a playback abnormal state, and abnormal playback information is obtained; based on the abnormal playback information and through the corresponding abnormal unique identifier information of the terminal device, generating the abnormal retransmission request information to be sent to the headend server; wherein, the abnormal retransmission request information includes: metadata, abnormal unique identifier information, and corresponding abnormal keyframe position information.

[0012] In one feasible implementation, based on the abnormal retransmission request information, the target media stream is processed to locate the retransmission content to obtain the retransmission content information. Specifically, this includes: receiving the abnormal retransmission request information through the headend server; extracting the abnormal unique identifier information and the corresponding abnormal keyframe position information from the abnormal retransmission request information; and locating and extracting the corresponding keyframe content and related media stream segments in the target media using the abnormal unique identifier information and the corresponding abnormal keyframe position information to obtain the retransmission content information.

[0013] In one feasible implementation, the terminal device integrates the retransmission content information with the abnormal received media stream to obtain a retransmission completed media stream. Specifically, this includes: parsing the retransmission frame position information and corresponding timestamps in the retransmission content information; performing abnormal frame replacement and integration processing on key frames and related frames in the retransmission content information based on the retransmission frame position information and corresponding timestamps to determine the retransmission completed related frames; and obtaining the retransmission completed media stream based on the retransmission completed related frames in the media stream on the terminal device.

[0014] In one feasible implementation, if the retransmission completed media stream is identified, the replay start point of the received media stream in the playback state is set to the initial frame of the retransmission completed relevant frame; and playback start control is performed on the received media stream through the initial frame of the retransmission completed relevant frame.

[0015] Secondly, embodiments of this application also provide a media stream anomaly retransmission device based on keyframes, the device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to execute a media stream anomaly retransmission method based on keyframes as described in any of the above embodiments.

[0016] Thirdly, embodiments of this application also provide a non-volatile computer storage medium, wherein the storage medium is a non-volatile computer-readable storage medium, and the non-volatile computer-readable storage medium stores at least one program, each program including instructions, wherein when the instructions are executed by a terminal, the terminal performs a media stream abnormal retransmission method based on key frames as described in any of the above embodiments.

[0017] This application provides a method, device, and medium for abnormal media stream retransmission based on keyframes. Compared with the prior art, the embodiments of this application have the following beneficial technical effects: 1. From a content security perspective, this significantly enhances the security and reliability of media streaming playback. By generating unique identifiers for keyframe content and comparing them on the terminal, it can accurately detect whether the media stream has been tampered with. Once tampering is detected, playback is immediately stopped, effectively preventing viewers from receiving tampered, false, or harmful information and ensuring the authenticity and integrity of the information. Regarding user experience, although a detection mechanism has been added, because it operates based on keyframes, the computational complexity is relatively low, and the impact on terminal device performance is minimal. This ensures that the media stream can play smoothly under normal circumstances, without playback stuttering or delays caused by complex detection algorithms. Simultaneously, timely stopping of tampered content prevents users from being forced to watch problematic content, improving the comfort and satisfaction of the user viewing experience.

[0018] 2. For content providers and copyright holders, this method helps protect their intellectual property and brand image. Preventing media stream tampering ensures that the original content is presented to users in the correct format, avoiding negative impacts such as reputational damage and user churn caused by malicious content alteration. Furthermore, the abnormal information recorded by the terminal provides content providers with strong evidence to trace the source of tampering and strengthen security measures.

[0019] 3. Regarding the network environment, preventing the spread of tampered media streams helps purify cyberspace and reduce the spread of harmful information. This is of positive significance for maintaining a healthy and orderly network environment, especially in today's era of rapid and widespread information dissemination. It can effectively curb the spread of maliciously tampered content and reduce its negative impact on the entire network ecosystem. In summary, this invention plays an important role in ensuring the security of media stream content, improving user experience, protecting copyright, and maintaining the network environment, and has significant beneficial effects. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings: Figure 1 A flowchart of a media stream anomaly retransmission method based on keyframes provided in this application embodiment; Figure 2 This is a schematic diagram of a media stream anomaly retransmission device based on keyframes, provided in an embodiment of this application. Detailed Implementation

[0021] To enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this application.

[0022] It should be noted that the core focus of this application lies in the generation and application of unique keyframe identifiers. The unique keyframe identifiers generated through a unique algorithm are not only the basis for detecting media stream anomalies but also crucial for achieving accurate retransmission. The accuracy of the identifier generation by the headend server, the real-time comparison of the identifier by the terminal device, and the efficiency of retransmission requests and responses based on the identifier constitute the key aspects of this method. Furthermore, the consistency between the headend and terminal in extracting keyframes, the identifier generation algorithms, and the accurate integration of retransmission content on the terminal are also important factors ensuring the effective implementation of this method.

[0023] This application's method for media stream anomaly retransmission based on keyframes also includes algorithms and processes for keyframe extraction and unique identifier generation by the headend server, including generating stable and unique identifiers by combining timestamps, hash values, and other information. It also covers the complete process of keyframe monitoring, comparison, anomaly detection, and retransmission requests by the terminal device, setting a reasonable comparison frequency to balance performance and detection accuracy. Furthermore, it includes a mechanism for the headend server to locate and transmit retransmission content after receiving a retransmission request, as well as algorithms for the terminal device to receive and integrate the retransmission content. This aims to prevent unauthorized use of similar technical solutions to achieve media stream anomaly detection and retransmission.

[0024] This application provides a method for abnormal media stream retransmission based on keyframes, such as... Figure 1 As shown, the media stream anomaly retransmission method based on keyframes specifically includes steps S101-S106: S101. Through the headend server, the headend keyframes in the target media stream to be transmitted are uniquely identified to obtain the headend unique identifier information of the headend keyframes.

[0025] Specifically, it is necessary to first use video analysis algorithms and audio analysis algorithms to perform keyframe recognition processing on the target media stream, obtaining the headend keyframes in the headend server. Among these, the video keyframes include scene transition frames and the start of major actions. The audio keyframes include sound start frames, sound end frames, and sound volume change frames.

[0026] Furthermore, by using an encryption algorithm, the timestamp, content hash value, and sequence number of the keyframe in the media stream are combined and encrypted to obtain the unique identification information of the keyframe.

[0027] Furthermore, the unique identifier information is associated with the metadata of the header keyframe to obtain the header unique identifier information. The metadata includes at least: keyframe location information and keyframe timestamp.

[0028] In one embodiment, the headend server first analyzes the media stream to be transmitted, using video and audio analysis algorithms to identify keyframes. For video, keyframes typically contain important information such as scene transitions and the start of major actions; for audio, keyframes correspond to the start, end, and volume changes of sounds. After extracting keyframes, the headend server uses a specific algorithm to generate a unique identifier for each keyframe (headend unique identifier information). For example, the keyframe's timestamp, content hash value (e.g., calculated using the SHA-256 algorithm), and sequence number in the media stream can be combined and then processed using an encryption algorithm to generate a unique identifier. For example, the identifier of a video keyframe might be in the form of "SHA256 (timestamp + frame content + sequence number) + encryption key". The headend server associates and stores the generated unique identifier of the keyframe with the keyframe's position information in the media stream, timestamp, and other relevant metadata for subsequent querying and transmission.

[0029] S102. Transmit the target media stream and its corresponding unique header identifier to the terminal device.

[0030] Specifically, terminal devices are needed to receive the target media stream and the unique identifier information of the headend from the headend server.

[0031] Furthermore, the target media stream and its unique header identifier are stored in the cache area of ​​the terminal device. This cache area is related to the media stream playback settings.

[0032] In one embodiment, the terminal device receives the media stream, keyframe identifiers, and metadata from the headend server and stores them in a local cache area. The cache area should be configured to meet the real-time playback requirements of the media stream, possessing efficient read and write performance to ensure that relevant information can be quickly retrieved during playback.

[0033] S103. Through the terminal device, the unique identifier of the received media stream that has been received and is in playback state is calculated for the terminal key frame to obtain the terminal unique identifier information.

[0034] Specifically, the media stream received in the buffer area of ​​the terminal device is first controlled for playback.

[0035] Furthermore, the terminal keyframes of the playback segments in the received media stream are extracted in real time through the terminal device and based on video analysis algorithms and audio analysis algorithms.

[0036] Furthermore, using the same encryption algorithm as the headend server, the timestamp, content hash value, and sequence number of the keyframe in the media stream are combined and encrypted to obtain the unique identification information of the terminal keyframe.

[0037] Furthermore, the unique identifier information is then associated with the corresponding metadata in the playback segment to obtain the terminal's unique identifier information.

[0038] In one embodiment, during media stream playback, the terminal device extracts keyframes of the currently playing segment in real time. It uses the same keyframe extraction and identifier generation algorithms as the headend server to generate unique identifiers for the extracted keyframes, i.e., unique terminal identifier information.

[0039] S104. Perform media stream anomaly judgment by comparing the unique identifier information of the headend with the unique identifier information of the terminal, and determine the abnormal retransmission request information sent to the headend server.

[0040] Specifically, the same playback frame period must be compared and identified according to the preset playback frame interval period in the received media stream that is in playback state to obtain the comparison frame period.

[0041] Furthermore, based on the comparison frame period, the corresponding terminal unique identifier information in the terminal device and the corresponding headend unique identifier information in the headend server are obtained.

[0042] Furthermore, if there is an inconsistency between the unique identifier information of the headend and the unique identifier information of the terminal, the received media stream is in an abnormal playback state, and abnormal playback information is obtained.

[0043] Furthermore, based on the abnormal playback information and using the corresponding unique identifier of the abnormal terminal in the terminal device, an abnormal retransmission request is generated to be sent to the headend server. The abnormal retransmission request information includes: metadata, unique identifier of the abnormality, and the corresponding abnormal keyframe position information.

[0044] In one embodiment, the generated terminal unique identifier information is compared with the corresponding headend unique identifier information sent by the pre-stored headend server. The comparison process can be performed frame-by-frame, or, depending on the actual situation, every certain number of frames to balance detection accuracy and device performance consumption. For example, for video playback, keyframe identifier comparison can be performed every 5 frames. Anomaly detection and retransmission request: Once the terminal device detects that the generated keyframe identifier is inconsistent with the identifier sent by the headend server, it immediately determines that the media stream is abnormal. At this time, the terminal device, based on the inconsistent keyframe identifier, i.e., the abnormal terminal unique identifier information, finally sends a retransmission request (abnormal retransmission request information) to the headend server. The abnormal retransmission request information includes the unique identifier of the abnormal keyframe and related metadata such as the position of the keyframe in the media stream, so that the headend server can accurately and quickly locate the content that needs to be retransmitted.

[0045] S105. Based on the abnormal retransmission request information, perform retransmission content location processing on the target media stream to obtain the retransmission content information.

[0046] Specifically, the abnormal retransmission request information is received through the headend server.

[0047] Furthermore, it is also necessary to extract the unique identifier of the exception and the corresponding location information of the exception keyframe from the exception retransmission request information.

[0048] Furthermore, by using the unique identifier information of the anomaly and the corresponding keyframe location information, the corresponding keyframe content and related media stream segments in the target media are located and extracted to obtain the retransmitted content information.

[0049] In one embodiment, after receiving a retransmission request (abnormal retransmission request information) from a terminal device, the headend server quickly locates and extracts the corresponding keyframe content and related media stream segments in the target media based on the keyframe identifier and metadata in the request, and then sends them to the terminal device to obtain the retransmission content information for subsequent integration and retransmission.

[0050] S106. Using the terminal device, the retransmitted content information is integrated with the abnormal received media stream to obtain the retransmitted media stream.

[0051] Specifically, the position information of the retransmission frame and the corresponding timestamp are first parsed from the retransmission content information.

[0052] Furthermore, based on the location information of the retransmitted frames and the corresponding timestamps, the key frames and related frames in the retransmitted content information are replaced and integrated with abnormal frames whose location information is related to abnormal key frames, and the relevant frames for retransmission completion are determined.

[0053] Furthermore, based on the retransmission completion frames in the media stream in the terminal device, the retransmission completed media stream is obtained.

[0054] In one embodiment, after receiving the retransmitted content (retransmitted content information), the terminal device accurately integrates the retransmitted content into the corresponding position in the local media stream based on the location information and timestamps carried in the retransmitted content. For example, if the retransmitted content consists of video keyframes and related frames, the insertion position in the video stream is determined based on the timestamp information to ensure the continuity of video playback. For the audio portion, it is also accurately integrated based on time information to ensure synchronization between audio and video.

[0055] As a feasible implementation, if a media stream that has been successfully retransmitted is detected, the replay starting point for the received media stream in playback mode is set to the initial frame of the retransmission completion frame. Playback control of the received media stream is then initiated using this initial frame. After integration, the terminal device resumes playback of the media stream from the point of anomaly, achieving complete and smooth playback. Through anomaly identification and retransmission processing between the headend server and the terminal device, media stream anomaly retransmission based on the unique identifier of the keyframe is achieved, effectively ensuring the content integrity and security during media stream playback.

[0056] As a feasible implementation method, existing media streaming technologies have shortcomings in ensuring content integrity and security. Most focus only on transmission encryption and lack effective detection and retransmission mechanisms for content anomalies during terminal playback. This application addresses these deficiencies and limitations by making the following improvements: (1) In the media stream processing flow, the headend server uniquely identifies and names the keyframes of the media stream content. Keyframes hold a core position in the media stream, carrying the main visual and audio information and determining the key characteristics of the media content. The headend server generates a unique identifier for each keyframe using a specific algorithm, such as combining the keyframe's timestamp and content hash value. This identifier not only accurately represents the content of the keyframe but also has high stability; even if other parts of the media stream change, the keyframe identifier will not be affected.

[0057] (2) Regarding the terminal device, it bears the important responsibility of monitoring the content of the media stream. During the playback of the media stream, the terminal extracts the keyframes of the currently playing segment in real time and generates a unique identifier for the keyframe according to the same algorithm as the headend server. Subsequently, the generated identifier is compared with the identifier predefined and transmitted by the headend server. Once a discrepancy is found, it is determined that the current media stream segment is abnormal, and there may be a situation where the keyframe is lost or tampered with. When an anomaly is detected, the terminal device uses the unique identifier of the keyframe to send a request to the headend server to retransmit the relevant content. Because the unique identifier is unique, the headend server can quickly locate the accurate content required by the terminal device and send it to the terminal.

[0058] (3) After receiving the retransmitted content from the headend server, the terminal device integrates the local media stream content. This integration process requires precisely inserting the retransmitted content into the correct position in the media stream to ensure the continuity of the media stream in terms of time and content. For example, the specific position of the retransmitted content in the media stream is determined by the timestamp information of key frames, thereby achieving seamless connection. After integration, the media stream is restored to integrity, and the terminal device can continue to play it completely and smoothly.

[0059] In addition, embodiments of this application also provide a media stream anomaly retransmission device based on keyframes, such as... Figure 2 As shown, the media stream anomaly retransmission device 200 based on keyframes specifically includes: At least one processor 201; and a memory 202 communicatively connected to the at least one processor 201; wherein the memory 202 stores instructions executable by the at least one processor 201 to enable the at least one processor 201 to execute: The unique identifier of the headend keyframe in the target media stream to be transmitted is calculated by the headend server to obtain the unique identifier information of the headend keyframe. Transmit the target media stream and its corresponding unique header identifier to the terminal device; The terminal device calculates the unique identifier of the terminal key frame by receiving the media stream that is in playback state after receiving it, and obtains the unique identifier information of the terminal. The unique identifier information of the headend and the unique identifier information of the terminal are used to determine the abnormal media stream anomaly and identify the abnormal retransmission request information sent to the headend server. Based on the abnormal retransmission request information, the target media stream is located to obtain the retransmission content information; The terminal device integrates the retransmitted content information with the abnormal received media stream to obtain the retransmitted media stream.

[0060] This application's embodiments, by uniquely identifying keyframes, ensure accurate location of content requiring retransmission in the event of anomalies, thereby improving the overall reliability of the media stream transmission. Since only the faulty keyframes are retransmitted, rather than the entire media stream, network bandwidth consumption and data transmission volume are significantly reduced. Furthermore, by quickly retransmitting lost or corrupted keyframes, playback interruptions and stuttering are reduced, improving the user viewing experience. Simultaneously, the intelligent retransmission mechanism avoids unnecessary data transmission, optimizing network resource utilization. The processing of retransmission requests and retransmission content can be performed quickly, reducing delays caused by data loss. It can also handle anomalies in media stream transmission, enhancing the overall robustness of the system. It also simplifies the anomaly handling process, reduces operational costs, and enables real-time monitoring of media stream quality.

[0061] The various embodiments in this application are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device and medium embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the description of the method embodiments.

[0062] The devices and media provided in this application are one-to-one with the methods. Therefore, the devices and media also have similar beneficial technical effects as their corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the devices and media will not be repeated here.

[0063] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0064] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0065] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0066] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0067] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0068] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0069] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0070] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0071] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of this specification.

Claims

1. A method for abnormal retransmission of media streams based on keyframes, characterized in that, The method includes: The unique identifier of the headend keyframe in the target media stream to be transmitted is calculated by the headend server to obtain the headend unique identifier information of the headend keyframe. The target media stream and the corresponding unique header identifier information are transmitted to the terminal device; The terminal device calculates the unique identifier of the terminal key frame of the received media stream that is in playback state after being received, and obtains the unique identifier information of the terminal. The unique identifier information of the headend and the unique identifier information of the terminal are used to determine the media stream anomaly and identify the abnormal retransmission request information sent to the headend server. Based on the abnormal retransmission request information, the target media stream is processed to locate the retransmission content, and the retransmission content information is obtained. The terminal device integrates the retransmitted content information with the abnormal received media stream to obtain the retransmitted media stream.

2. The method for abnormal media stream retransmission based on keyframes according to claim 1, characterized in that, The header server performs a unique identifier calculation on the header keyframes in the target media stream to be transmitted, obtaining the header unique identifier information of the header keyframes, specifically including: Using video analysis algorithms and audio analysis algorithms, the target media stream is processed to identify key frames of the video and audio streams, thereby obtaining the headend key frames in the headend server; wherein, the video key frames in the headend key frames include scene transition frames and the start of the main action; the audio key frames in the headend key frames include sound start frames, sound end frames, and sound volume change frames. By using an encryption algorithm, the timestamp, content hash value, and sequence number of the keyframe in the media stream are combined and encrypted to obtain the unique identification information of the keyframe. The unique identifier information is associated with the metadata of the head-end keyframe to obtain the head-end unique identifier information; wherein, the metadata includes at least: keyframe location information and keyframe timestamp.

3. The method for abnormal media stream retransmission based on keyframes according to claim 1, characterized in that, Transmitting the target media stream and the corresponding unique header identifier information to the terminal device specifically includes: The terminal device receives the target media stream and the unique identifier information of the headend from the headend server. The target media stream and the unique identifier of the headend are stored in the cache area of ​​the terminal device; wherein, the cache area is related to the media stream playback settings.

4. The method for abnormal media stream retransmission based on keyframes according to claim 1, characterized in that, The terminal device calculates the unique identifier of the terminal keyframes of the received media stream that is in playback state after being received, and obtains the unique identifier information of the terminal, specifically including: The terminal device receives media streams in the buffer area and controls their playback. Using the terminal device and based on video and audio analysis algorithms, the terminal keyframes of the playback segments in the received media stream are extracted in real time. Using the same encryption algorithm as the headend server, the timestamp, content hash value and sequence number of the keyframe in the media stream of the terminal keyframe are combined and encrypted to obtain the unique identification information of the terminal keyframe. The unique identifier information is associated with the corresponding metadata in the playback segment to obtain the terminal's unique identifier information.

5. The method for abnormal media stream retransmission based on keyframes according to claim 1, characterized in that, The unique identifier information of the headend and the unique identifier information of the terminal are used to determine media stream anomalies, and the abnormal retransmission request information sent to the headend server is determined, specifically including: Based on the preset playback frame interval period in the received media stream that is in playback state, the same playback frame period is compared and identified to obtain the comparison frame period. Based on the comparison frame period, obtain the terminal unique identifier information corresponding to the terminal device and the headend unique identifier information corresponding to the headend server. If the unique identifier information of the head end and the unique identifier information of the terminal are inconsistent, the received media stream is in an abnormal playback state, and abnormal playback information is obtained. Based on the abnormal playback information and using the corresponding abnormal terminal unique identifier information in the terminal device, an abnormal retransmission request information is generated for sending to the headend server; wherein, the abnormal retransmission request information includes: metadata, abnormal unique identifier information and corresponding abnormal keyframe position information.

6. The method for abnormal media stream retransmission based on keyframes according to claim 1, characterized in that, Based on the abnormal retransmission request information, the target media stream is processed to locate the retransmission content, and the retransmission content information is obtained, specifically including: The abnormal retransmission request information is received through the headend server; Extract the unique identifier of the abnormality and the corresponding location information of the abnormal keyframe from the abnormal retransmission request information; Using the unique identifier information of the anomaly and the corresponding keyframe location information, the corresponding keyframe content and related media stream segments in the target media are located and extracted to obtain the retransmitted content information.

7. The method for abnormal media stream retransmission based on keyframes according to claim 1, characterized in that, The terminal device integrates the retransmitted content information with the abnormal received media stream to obtain the retransmitted media stream, specifically including: Parse the retransmission frame position information and the corresponding timestamp from the retransmission content information; Based on the retransmission frame position information and the corresponding timestamp, the key frames and related frames in the retransmission content information are replaced and integrated with abnormal frames related to the abnormal key frame position information to determine the retransmission completion related frames. The retransmission completed media stream is obtained based on the retransmission completed relevant frames in the media stream in the terminal device.

8. The method for abnormal media stream retransmission based on keyframes according to claim 1, characterized in that, If the retransmission completed media stream is identified, the replay start point of the received media stream in the playback state is set to the initial frame of the retransmission completed relevant frame. The initial frame of the relevant frames is completed through the retransmission, and the playback control of the received media stream is started.

9. A media stream anomaly retransmission device based on keyframes, characterized in that, The device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor to enable the at least one processor to perform a keyframe-based media stream anomaly retransmission method according to any one of claims 1-8.

10. A non-volatile computer storage medium, characterized in that, The storage medium is a non-volatile computer-readable storage medium that stores at least one program, each program including instructions that, when executed by a terminal, cause the terminal to perform a media stream anomaly retransmission method based on keyframes according to any one of claims 1-8.

Images