A key frame-based media stream abnormality retransmission method, device and medium

Abstract

The application discloses a kind of based on key frame's media stream exception supplementary transmission method, equipment and medium, belong to media stream transmission technical field, for solving the existing media stream transmission technology, there is certain limitation in the integrity and security of guaranteeing media stream content, user experience is poor, it is difficult to realize the technical problem of detecting media stream exception and effectively supplementing missing.The method comprises the following steps: transmitting the target media stream and the corresponding head-end unique identification information into the terminal device;The unique identification of the relevant terminal key frame is calculated for the received media stream in the playing state after receiving, and the terminal unique identification information is obtained;The head-end unique identification information is compared with the terminal unique identification information to determine the abnormal transmission request information;The positioning processing of the target media stream is carried out to obtain the supplementary transmission content information;Supplementary transmission content information and the received media stream with abnormality are integrated and processed to obtain the completed media stream.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of media streaming, and particularly relates to a key frame-based media stream abnormality retransmission method, device and medium. BACKGROUND

[0002] In today's era of rapid digital information dissemination, media stream application scenarios are increasingly widespread, covering online video playback, live streaming, video conferencing and other fields. However, media streams face many challenges during transmission, among which the completeness and security of media stream content are particularly prominent. During the transmission of media streams from head-end servers to terminal devices, complex network environments become "unstable factors" for media stream data. Network congestion, signal interference, malicious attacks and other situations occur from time to time. For example, during network peak hours, a large amount of data transmission may cause some media stream data to be lost; and malicious attackers may tamper with key information in the media stream for various malicious purposes. In addition, hardware failures, software errors and other problems of terminal devices may also damage the media stream content. Currently, some technologies for media stream transmission mainly focus on the encryption protection of data transmission, and use SSL / TLS and other security protocols to encrypt the data during transmission to prevent data from being stolen or tampered with. However, this approach has limitations, as it cannot detect in real time whether the media stream content is complete and has not been tampered with during terminal playback. Some detection methods can detect abnormalities in media streams, but lack effective retransmission mechanisms and cannot restore the completeness of media streams in a timely manner. For example, when a key frame in a media stream is detected to be missing or tampered with, existing technologies often only interrupt the playback and prompt the user of an error, but cannot automatically request retransmission of the missing or erroneous content, which greatly affects the user experience.

[0003] Moreover, in live streaming scenarios, this problem is particularly serious. Viewers want to watch live streaming content smoothly and completely, and once media stream abnormalities cannot be timely completed, it may cause live streaming pictures to freeze, sound to interrupt or error pictures to appear, seriously damaging the viewing experience of viewers and even causing viewers to leave. In video conferencing, incomplete media streams may cause inaccurate information transmission and affect the normal progress of the conference.

[0004] In summary, existing media stream transmission technologies have obvious deficiencies in ensuring the completeness and security of media stream content, and there is an urgent need for a technical solution that can detect media stream abnormalities in real time and effectively retransmit missing or erroneous content to improve user experience and meet the growing demand for multimedia communication. SUMMARY

[0005] The embodiment of the application provides a media stream abnormality retransmission method based on key frames, equipment and medium, and is used for solving the following technical problems: the existing media stream transmission technology has certain limitations in guaranteeing the integrity and security of media stream content, the user experience is poor, and it is difficult to realize detection of media stream abnormality and effective retransmission of loss.

[0006] The embodiment of the application adopts the following technical scheme:

[0007] In one aspect, the embodiment of the application provides a media stream abnormality retransmission method based on key frames, including: through a head-end server, performing unique identification calculation on a head-end key frame in a target media stream to be transmitted to obtain head-end unique identification information of the head-end key frame; transmitting the target media stream and the corresponding head-end unique identification information into a terminal device; through the terminal device, performing unique identification calculation on a received media stream in a playing state to obtain terminal unique identification information; performing media stream abnormality judgment on the head-end unique identification information and the terminal unique identification information to determine abnormality retransmission request information sent to the head-end server; performing positioning processing on retransmission content of the target media stream according to the abnormality retransmission request information to obtain retransmission content information; and through the terminal device, performing data integration processing on the retransmission content information and the received media stream with the abnormality to obtain a retransmission completed media stream.

[0008] The embodiment of the application can ensure that the content needing retransmission can be accurately positioned when an abnormality occurs by uniquely identifying key frames, thereby improving the transmission reliability of the entire media stream. Since only the abnormal key frames are retransmitted, but not the entire media stream, this can significantly reduce the consumption of network bandwidth and the amount of data transmission. The loss or damage of key frames can be quickly retransmitted, which can reduce the interruption and lag in playing, thereby improving the user viewing experience. At the same time, through the intelligent retransmission mechanism, unnecessary data transmission can be avoided, thereby optimizing the use of network resources. The processing of retransmission request and retransmission content can be quickly performed, reducing the delay caused by data loss. And it can handle abnormal situations in media stream transmission, enhance the overall robustness of the system. It also simplifies the abnormality processing process, reduces the operation and maintenance cost, and realizes real-time monitoring of the quality of the media stream.

[0009] In an embodiment, the head-end key frame in the target media stream to be transmitted is uniquely identified by the head-end server to obtain head-end unique identification information of the head-end key frame, which includes: the head-end key frame in the head-end server is obtained by video analysis algorithm and audio analysis algorithm for key frame identification processing of the target media stream about video stream and audio stream; the video key frame in the head-end key frame includes scene switching frame and main action start; the audio key frame in the head-end key frame includes sound start frame, sound end frame and sound volume mutation frame; the timestamp, content hash value and sequence number of the key frame in the head-end key frame are combined and data encrypted by encryption algorithm to obtain the unique identification information of the head-end key frame; the unique identification information is associated with the metadata of the head-end key frame to obtain the head-end unique identification information; the metadata at least includes key frame position information and key frame timestamp.

[0010] In an embodiment, the target media stream and the corresponding head-end unique identification information are transmitted to the terminal device, which includes: the target media stream and the head-end unique identification information from the head-end server are received by the terminal device; the target media stream and the head-end unique identification information are stored in the cache area of the terminal device; the cache area is related to media stream playing setting.

[0011] In an embodiment, the received media stream in the playing state after receiving is related to terminal key frame unique identification calculation by the terminal device to obtain terminal unique identification information, which includes: the received media stream in the cache area of the terminal device is played; the terminal key frame of the playing segment of the received media stream is extracted in real time by the terminal device based on video analysis algorithm and audio analysis algorithm; the timestamp, content hash value and sequence number of the key frame in the terminal key frame are combined and data encrypted according to the same encryption algorithm as the head-end server to obtain the unique identification information of the terminal key frame; the unique identification information is data associated with the corresponding metadata of the playing segment to obtain the terminal unique identification information.

[0012] In an implementable embodiment, the head-end unique identification information and the terminal unique identification information are subjected to media stream anomaly judgment, and an abnormal retransmission request information sent to the head-end server is determined, specifically including: according to a preset playing frame interval period in a received media stream in a playing state, the same playing frame period is subjected to comparison and identification processing, and a comparison frame period is obtained; based on the comparison frame period, corresponding terminal unique identification information in the terminal device and corresponding head-end unique identification information in the head-end server are obtained; if the head-end unique identification information and the terminal unique identification information are identification inconsistent information, the received media stream is in an abnormal playing state, and abnormal playing information is obtained; based on the abnormal playing information, and through corresponding abnormal terminal unique identification information in the terminal device, the abnormal retransmission request information for sending to the head-end server is generated; wherein the abnormal retransmission request information includes: metadata, abnormal unique identification information and corresponding abnormal key frame position information.

[0013] In an implementable embodiment, according to the abnormal retransmission request information, positioning processing of retransmission content of the target media stream is performed, and retransmission content information is obtained, specifically including: the head-end server receives the abnormal retransmission request information; abnormal unique identification information and corresponding abnormal key frame position information in the abnormal retransmission request information are extracted; through the abnormal unique identification information and the corresponding abnormal key frame position information, corresponding key frame content and related media stream segments in the target media are positioned and extracted, and the retransmission content information is obtained.

[0014] In an implementable embodiment, through the terminal device, the retransmission content information and the received media stream with an abnormality are subjected to data integration processing, and a retransmission completed media stream is obtained, specifically including: retransmission frame position information and corresponding time stamps in the retransmission content information are parsed; according to the retransmission frame position information and the corresponding time stamps, abnormal frames related to key frames and related frames in the retransmission content information are subjected to abnormal frame replacement and integration processing with respect to abnormal key frame position information, and retransmission completed related frames are determined; based on the retransmission completed related frames in the media stream in the terminal device, the retransmission completed media stream is obtained.

[0015] In an implementable embodiment, if the retransmission completed media stream is identified, a re-playing starting point of the received media stream in a playing state is set as an initial frame of the retransmission completed related frames; the received media stream is subjected to starting playing control through the initial frame of the retransmission completed related frames.

[0016] In a second aspect, the embodiments of the present application further provide a key frame-based media stream abnormal retransmission device, the device comprising: at least one processor; and a memory in communication connection with the at least one processor; wherein the memory stores instructions capable of being executed by the at least one processor, so as to enable the at least one processor to execute the key frame-based media stream abnormal retransmission method according to any one of the embodiments.

[0017] In a third aspect, the embodiments of the present application further provide a nonvolatile computer storage medium, the storage medium being a nonvolatile computer readable storage medium, the nonvolatile computer readable storage medium storing at least one program, each of the programs comprising instructions, which, when executed by a terminal, cause the terminal to execute the key frame-based media stream abnormal retransmission method according to any one of the embodiments.

[0018] The present application provides a key frame-based media stream abnormal retransmission method, device and medium, compared with the prior art, the embodiments of the present application have the following beneficial technical effects:

[0019] 1. From the perspective of content security, the security and reliability of media stream playback are greatly improved. By generating a unique identifier for the key frame content and comparing it on the terminal, it can accurately detect whether the media stream has been tampered with. Once tampering is detected, playback is immediately stopped, effectively preventing the audience from receiving false or harmful information that has been tampered with, and ensuring the authenticity and integrity of the information. In terms of user experience, although a detection mechanism is added, the computational complexity is relatively low based on key frames, and the performance impact on the terminal device is small. This ensures that the media stream can be played smoothly under normal circumstances, and does not cause problems such as playback lag, delay, etc. due to complex detection algorithms. At the same time, stopping playback of tampered content in a timely manner avoids forcing users to watch problematic content, improving the comfort and satisfaction of user viewing experience.

[0020] 2. For content providers and copyright holders, this method helps to protect their intellectual property rights and brand image. Preventing media stream tampering can ensure that the original content provided by them is presented to users in the correct form, avoiding negative impacts such as damage to reputation and user loss due to malicious tampering of content. In addition, the abnormal information recorded by the terminal can also provide strong evidence for content providers to trace the source of tampering and strengthen security measures.

[0021] 3. In terms of network environment, preventing the spread of tampered media stream helps to purify cyberspace and reduce the spread of undesirable information. This has a positive significance for maintaining a healthy and orderly network environment, especially in today's era of rapid and extensive information dissemination, effectively containing the spread of malicious tampered content and reducing its negative impact on the entire network ecosystem. In summary, the present application plays an important role in ensuring media stream content security, improving user experience, protecting copyrights, and maintaining network environment, and has significant beneficial effects. BRIEF DESCRIPTION OF DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor. In the drawings:

[0023] Figure 1 A key frame-based media stream abnormality retransmission method flowchart is provided for the embodiments of the present application;

[0024] Figure 2 A structure schematic diagram of a key frame-based media stream abnormality retransmission device is provided for the embodiments of the present application. DETAILED DESCRIPTION

[0025] In order to make the person skilled in the art better understand the technical solutions in the present application, the following will combine the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative labor should be within the scope of protection of the present application.

[0026] It should be noted that the core key point of the present application is the generation and use of key frame unique identification. The key frame unique identification generated by the unique algorithm is not only the basis for detecting media stream abnormalities, but also the key to realizing accurate retransmission. The accuracy of the identification generated by the head-end server, the real-time comparison of the identification by the terminal device, and the efficiency of the retransmission request and response based on the identification are the key links of the present method. In addition, the consistency of the key frame extraction and identification generation algorithm between the head-end and the terminal, as well as the accurate integration of the retransmission content in the terminal, are also important factors to ensure the effective implementation of the method.

[0027] Wherein, the key frame-based media stream abnormality retransmission method provided by the application further comprises the algorithm and process of key frame extraction and unique identification generation of the head-end server, including generating stable and unique identification combining timestamp, hash value and other information. The complete process of terminal device key frame monitoring, comparison, abnormality detection and retransmission request, setting reasonable comparison frequency to balance performance and detection accuracy. It also includes the mechanism of head-end server positioning and transmission of retransmission content after receiving the retransmission request, and the receiving and integration algorithm of terminal device for retransmission content. Prevent others from using similar technical solutions without permission to realize the detection and retransmission of media stream abnormality.

[0028] The embodiment of the application provides a key frame-based media stream abnormality retransmission method, as shown in the figure, the key frame-based media stream abnormality retransmission method specifically comprises steps S101-S106: Figure 1

[0029] S101, through the head-end server, the head-end key frame in the target media stream to be transmitted is uniquely identified to obtain the head-end unique identification information of the head-end key frame.

[0030] Specifically, the video analysis algorithm and the audio analysis algorithm need to be used first to identify and process the key frames of the video stream and the audio stream of the target media stream, and obtain the head-end key frame in the head-end server. Among them, the video key frame in the head-end key frame includes scene switching frame and main action start. The audio key frame in the head-end key frame includes sound start frame, sound end frame and sound volume mutation frame.

[0031] Further, the timestamp, content hash value and sequence number of the key frame in the head-end key frame are combined and data encrypted by an encryption algorithm to obtain the unique identification information of the head-end key frame.

[0032] Further, the unique identification information and the metadata of the head-end key frame are associated to obtain the head-end unique identification information. Among them, the metadata at least includes: key frame position information and key frame timestamp.

[0033] ​In one embodiment, the end server first analyzes the media stream to be transmitted, and identifies key frames therein using video and audio analysis algorithms. For video, the key frames are usually frames containing important information such as scene changes, start of major actions, etc.; for audio, the key frames correspond to positions such as the start, end, volume mutation, etc. of the sound. After extracting the key frames, the head-end server generates a unique identifier (head-end unique identifier information) for each key frame using a specific algorithm. For example, the timestamp of the key frame, the hash value of the content (e.g. calculated using the SHA-256 algorithm), and the sequence number of the key frame in the media stream can be combined and then processed by an encryption algorithm to generate a unique identifier. For example, the identifier of a video key frame can be in the form of "SHA256 (timestamp + frame content + sequence number) + encryption key". The head-end server stores the generated key frame unique identifier in association with relevant metadata such as the position information and timestamp of the key frame in the media stream, for subsequent querying and transmission.

[0034] S102, transmitting the target media stream and the corresponding head-end unique identifier information to the terminal device.

[0035] Specifically, the terminal device is also used to receive the target media stream and the head-end unique identifier information from the head-end server.

[0036] Further, the target media stream and the head-end unique identifier information are stored in a cache area of the terminal device. The cache area is related to the media stream playback settings.

[0037] In one embodiment, the terminal device receives the media stream, key frame identifier and metadata from the head-end server and stores them in a local cache area. The cache area is set to take into account the real-time playback requirements of the media stream and has high read-write performance to ensure that relevant information can be quickly obtained during playback.

[0038] S103, using the terminal device to calculate the terminal unique identifier information based on the received media stream in the playback state and the unique identifier of the terminal key frame.

[0039] Specifically, the received media stream in the cache area of the terminal device is first controlled for playback.

[0040] Further, the terminal device is used to extract the terminal key frame of the playback segment of the received media stream in real time based on the video analysis algorithm and the audio analysis algorithm.

[0041] Further, according to the same encryption algorithm as the head-end server, the timestamp, content hash value and sequence number of the key frame in the media stream are combined and data encrypted to obtain the unique identifier information of the terminal key frame.

[0042] Further, the unique identification information is data-associated with the corresponding metadata in the playing segment to obtain terminal unique identification information.

[0043] In one embodiment, during the media stream playing process, the terminal device extracts the key frame of the current playing segment in real time. The same key frame extraction algorithm and identification generation algorithm as the head-end server are used to generate a unique identification for the extracted key frame, i.e., terminal unique identification information.

[0044] S104, the head-end unique identification information and the terminal unique identification information are used for media stream anomaly judgment to determine the abnormal retransmission request information sent to the head-end server.

[0045] Specifically, the same playing frame period is also compared and identified according to the preset playing frame interval period in the received media stream in the playing state to obtain a comparison frame period.

[0046] Further, the corresponding terminal unique identification information in the terminal device and the corresponding head-end unique identification information in the head-end server are obtained based on the comparison frame period.

[0047] Further, if the head-end unique identification information and the terminal unique identification information are identification inconsistent information, the received media stream is in an abnormal playing state, and abnormal playing information is obtained.

[0048] Further, based on the abnormal playing information, the abnormal retransmission request information for sending to the head-end server is generated through the corresponding abnormal terminal unique identification information in the terminal device. The abnormal retransmission request information includes metadata, abnormal unique identification information, and corresponding abnormal key frame position information.

[0049] In one embodiment, the generated terminal unique identification information is compared with the corresponding head-end unique identification information sent by the head-end server in advance. The comparison process can use a frame-by-frame comparison method, or according to the actual situation, comparison can be performed every certain number of frames to balance the detection accuracy and device performance consumption. For example, for video playing, key frame identification comparison can be performed every 5 frames. Once the terminal device finds that the generated key frame identification is inconsistent with the identification sent by the head-end server, it is immediately determined that the media stream is abnormal. At this time, the terminal device sends a retransmission request (abnormal retransmission request information) to the head-end server according to the inconsistent key frame identification, i.e., abnormal terminal unique identification information. The abnormal retransmission request information contains the unique identification of the abnormal key frame and the position of the key frame in the media stream and other related metadata, so that the head-end server can accurately and quickly locate the content to be retransmitted.

[0050] S105, positioning processing of the target media stream according to the abnormality retransmission request information, to obtain retransmission content information.

[0051] Specifically, the abnormality retransmission request information is received by the head-end server.

[0052] Further, the abnormality unique identification information and the corresponding abnormality key frame position information in the abnormality retransmission request information need to be extracted.

[0053] Further, the corresponding key frame content and related media stream segments in the target media are located and extracted according to the abnormality unique identification information and the corresponding abnormality key frame position information, to obtain the retransmission content information.

[0054] In one embodiment, after the head-end server receives the retransmission request (abnormality retransmission request information) of the terminal device, the corresponding key frame content and related media stream segments in the target media are quickly located and extracted according to the key frame identification and metadata in the request, and then sent to the terminal device to obtain the retransmission content information, for subsequent integration and retransmission.

[0055] S106, data integration processing of the retransmission content information and the received media stream with abnormality by the terminal device, to obtain a retransmission completed media stream.

[0056] Specifically, the retransmission frame position information and the corresponding time stamp in the retransmission content information are first parsed.

[0057] Further, the key frame and related frames in the retransmission content information are replaced and integrated according to the abnormality frame position information, to determine the retransmission completed related frames.

[0058] Further, the retransmission completed media stream is obtained based on the retransmission completed related frames in the terminal device.

[0059] In one embodiment, after the terminal device receives the retransmission content (retransmission content information), the retransmission content is accurately integrated into the corresponding position of the local media stream according to the position information and time stamp carried by the retransmission content and other metadata. For example, if the video key frame and its related frames are retransmitted, the insertion position in the video stream is determined according to the time stamp information, to ensure the continuity of video playback. For the audio part, accurate integration is also performed according to the time information, to ensure the synchronization of audio and video.

[0060] As a feasible implementation, if the supplement completed media stream is identified, the replay starting point of the received media stream in the playing state is set to the initial frame of the supplement completed related frame. The start playing control is performed on the received media stream through the initial frame of the supplement completed related frame. After the integration is completed, the terminal device continues to play the media stream from the abnormal position, and complete and smooth playing is realized. Through the abnormal identification and supplement processing between the head-end server and the terminal device, the media stream abnormal supplement based on the unique identification of the key frame is realized, and the content integrity and security in the media stream playing process are effectively guaranteed.

[0061] As a feasible implementation, the existing media stream transmission technology has defects in guaranteeing content integrity and security, and mostly only focuses on transmission encryption, lacking effective detection and supplement mechanism for content abnormalities during terminal playing. In view of the above defects and limitations, the following improvements are made:

[0062] (1) In the processing flow of the media stream, the head-end server uniquely identifies and names the key frames of the media stream content. The key frame has a core position in the media stream, carries the main picture and audio information, and determines the key features of the media content. The head-end server generates a unique identification for each key frame through a specific algorithm, such as combining the timestamp, content hash value and other information of the key frame. This identification can accurately represent the content of the key frame, and has high stability, even if other parts of the media stream change, the key frame identification will not be affected.

[0063] (2) In terms of terminal devices, it undertakes the important responsibility of monitoring media stream content. The terminal extracts the key frame of the current playing segment in real time during playing the media stream, and generates a unique identification of the key frame according to the same algorithm as the head-end server. Then, the generated identification is compared with the identification defined and transmitted by the head-end server in advance. If they are inconsistent, it is determined that the current media stream segment is abnormal, and there may be key frame loss or tampering. When the abnormality is detected, the terminal device sends a request for supplement of related content to the head-end server by means of the unique identification of the key frame. Since the unique identification is unique, the head-end server can quickly locate the accurate content required by the terminal device and send it to the terminal.

[0064] (3) After receiving the supplement content from the head-end server, the terminal device integrates the local media stream content. This integration process needs to accurately insert the supplement content into the correct position in the media stream to ensure the continuity of the media stream in time and content. For example, the specific position of the supplement content in the media stream is determined through the timestamp information of the key frame, so as to realize seamless connection. After integration, the media stream recovers integrity, and the terminal device can continue to play completely and smoothly.

[0065] In addition, the embodiment of the present application also provides a key frame-based media stream abnormality retransmission device, as shown in the drawings, the key frame-based media stream abnormality retransmission device 200 specifically comprises: Figure 2

[0066] at least one processor 201; and a memory 202 connected with the at least one processor 201; wherein the memory 202 stores instructions executable by the at least one processor 201, so that the at least one processor 201 can execute:

[0067] through the head-end server, the head-end key frame in the target media stream to be transmitted is uniquely identified to obtain head-end unique identification information of the head-end key frame;

[0068] the target media stream and the corresponding head-end unique identification information are transmitted to the terminal device;

[0069] through the terminal device, the received media stream in the playing state is uniquely identified to obtain terminal unique identification information;

[0070] the head-end unique identification information and the terminal unique identification information are used for media stream abnormality judgment to determine the abnormality retransmission request information sent to the head-end server;

[0071] according to the abnormality retransmission request information, the target media stream is subjected to positioning processing of retransmission content to obtain retransmission content information;

[0072] through the terminal device, the retransmission content information and the received media stream with the abnormality are subjected to data integration processing to obtain a retransmission completed media stream.

[0073] The embodiment of the present application can ensure accurate positioning of the content that needs to be retransmitted when an abnormality occurs by uniquely identifying the key frame, thereby improving the transmission reliability of the entire media stream. Since only the abnormal key frame is retransmitted, but not the entire media stream, this can significantly reduce the consumption of network bandwidth and the amount of data transmission. By quickly retransmitting the lost or damaged key frame, the playback interruption and lag can be reduced, thereby improving the user viewing experience. At the same time, through the intelligent retransmission mechanism, unnecessary data transmission can be avoided, thereby optimizing the use of network resources. The processing of retransmission request and retransmission content can be quickly performed, reducing the delay caused by data loss. And it can handle abnormal situations in media stream transmission, enhance the overall robustness of the system. It also simplifies the abnormality processing procedure, reduces the operation and maintenance cost, and realizes real-time monitoring of the quality of the media stream.

[0074] ​The various embodiments in the present application are described in a progressive manner, and the same or similar parts among the various embodiments can be referred to each other. Each embodiment focuses on the difference from other embodiments. In particular, the device and medium embodiments are described simply because they are substantially similar to the method embodiments, and the relevant parts can be referred to the description of the method embodiments.

[0075] The device and medium provided by the embodiments of the present application are one-to-one corresponding to the method, and therefore, the device and medium also have the similar beneficial technical effects as the method. Since the beneficial technical effects of the method have been described in detail above, the beneficial technical effects of the device and medium will not be described here.

[0076] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage media, etc.) containing computer-usable program code.

[0077] The present application is described with reference to flowcharts and / or block diagrams of the method, device (system), and computer program product according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing apparatus produce a device implemented in the flowcharts and / or block diagrams. Figure 1 The function specified in one flow or multiple flows and / or blocks Figure 1 The device for implementing the function specified in one block or multiple blocks.

[0078] These computer program instructions can also be stored in a computer-readable memory that can guide the computer or other programmable data processing apparatus to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including instruction devices that implement the flowcharts and / or block diagrams. Figure 1 The function specified in one flow or multiple flows and / or blocks Figure 1 The device for implementing the function specified in one block or multiple blocks.

[0079] These computer program instructions can also be loaded into a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 Figure 1

[0080] In one typical configuration, the computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0081] The memory can include non-persistent memory and / or volatile memory, such as random access memory (RAM) and / or cache memory, non-volatile memory, such as read-only memory (ROM), EPROM, and / or flash memory, etc. The memory is an example of computer readable media.

[0082] Computer readable media includes permanent and non-permanent, movable and non-movable media that can be implemented by any method or technology for storing information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible by a computing device. According to the definition herein, computer readable media does not include transitory media, such as modulated data signals and carriers.

[0083] It should also be noted that the terms "comprising", "containing", or any other variant thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article or apparatus that comprises a list of elements does not only include those elements, but can also include other elements not expressly listed or inherent to such process, method, article or apparatus. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article or apparatus that includes the element.

[0084] ​​The above merely provides an example of the present application, but is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the present application should be included in the scope of the present application.

Claims

1. A method for key frame based media stream abnormal retransmission, characterized in that, The method comprises: Through the head-end server, unique identification calculation is performed on the head-end key frame in the target media stream to be transmitted to obtain head-end unique identification information of the head-end key frame; The target media stream and the corresponding head-end unique identification information are transmitted to a terminal device; Through the terminal device, unique identification calculation is performed on the received media stream in a playing state to obtain terminal unique identification information; The head-end unique identification information and the terminal unique identification information are subjected to media stream abnormality judgment to determine an abnormal retransmission request information sent to the head-end server, specifically comprising: According to a preset playing frame interval period in the received media stream in a playing state, the same playing frame period is subjected to comparison and identification processing to obtain a comparison frame period; Based on the comparison frame period, the corresponding terminal unique identification information in the terminal device and the corresponding head-end unique identification information in the head-end server are obtained; If the head-end unique identification information and the terminal unique identification information are identification inconsistent information, the received media stream is in an abnormal playing state, and abnormal playing information is obtained; Based on the abnormal playing information, the abnormal retransmission request information sent to the head-end server is generated through the corresponding abnormal terminal unique identification information in the terminal device; wherein the abnormal retransmission request information comprises metadata, abnormal unique identification information and corresponding abnormal key frame position information; According to the abnormal retransmission request information, positioning processing of retransmission content is performed on the target media stream to obtain retransmission content information; Through the terminal device, data integration processing is performed on the retransmission content information and the received media stream with an abnormality to obtain a retransmission completed media stream.

2. The method of claim 1, wherein, Through the head-end server, unique identification calculation is performed on the head-end key frame in the target media stream to be transmitted to obtain head-end unique identification information of the head-end key frame, specifically comprising: Through a video analysis algorithm and an audio analysis algorithm, key frame identification processing is performed on the target media stream related to the video stream and the audio stream to obtain the head-end key frame in the head-end server; wherein the video key frame in the head-end key frame comprises a scene switching frame and a main action start; the audio key frame in the head-end key frame comprises a sound start frame, a sound end frame and a sound volume mutation frame; Through an encryption algorithm, the timestamp, the content hash value and the sequence number of the key frame in the media stream in the head-end key frame are combined and subjected to data encryption processing to obtain the unique identification information of the head-end key frame; The unique identification information and the metadata of the head-end key frame are subjected to association processing to obtain the head-end unique identification information; wherein the metadata at least comprises key frame position information and key frame timestamp.

3. The method of claim 1, wherein, The target media stream and the corresponding head-end unique identification information are transmitted to a terminal device, specifically comprising: Through the terminal device, the target media stream and the head-end unique identification information from the head-end server are received; Store the target media stream and the head-end unique identification information to a cache area of the terminal device; wherein the cache area is related to media stream playing settings.

4. The method of claim 1, wherein, Through the terminal device, calculate the terminal unique identification information of the received media stream in a playing state in relation to terminal key frames, specifically including: Control playing of the received media stream in the cache area of the terminal device; Through the terminal device, extract terminal key frames of a playing segment of the received media stream in real time based on a video analysis algorithm and an audio analysis algorithm; According to the same encryption algorithm as in the head-end server, combine and encrypt the timestamp, content hash value, and sequence number of the key frame in the media stream in the terminal key frame to obtain the terminal unique identification information of the key frame; Data associate the unique identification information with corresponding metadata in the playing segment to obtain the terminal unique identification information.

5. The method of claim 1, wherein, According to the abnormal retransmission request information, perform positioning processing on the retransmission content of the target media stream to obtain retransmission content information, specifically including: Through the head-end server, receive the abnormal retransmission request information; Extract the abnormal unique identification information and corresponding abnormal key frame position information in the abnormal retransmission request information; Through the abnormal unique identification information and corresponding abnormal key frame position information, position and extract corresponding key frame content and related media stream segments in the target media to obtain the retransmission content information.

6. The method of claim 1, wherein, Through the terminal device, perform data integration processing on the retransmission content information and the received media stream with the abnormality to obtain a retransmission completed media stream, specifically including: Parse the retransmission frame position information and corresponding timestamp in the retransmission content information; According to the retransmission frame position information and corresponding timestamp, replace and integrate the key frame and related frames in the retransmission content information in relation to the abnormal key frame position information to determine the retransmission completed related frames; Based on the retransmission completed related frames of the media stream in the terminal device, obtain the retransmission completed media stream.

7. The method of claim 1, wherein, If the retransmission completed media stream is identified, set the re-play start point of the received media stream in a playing state to the initial frame of the retransmission completed related frames; Through the initial frame of the retransmission completed related frames, control starting playing of the received media stream.

8. A keyframe-based media stream abnormal retransmission device, characterized in that, The device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to execute a key frame-based media stream abnormal retransmission method according to any one of claims 1-7.

9. A non-transitory computer storage medium, comprising: The storage medium is a non-volatile computer readable storage medium, and the non-volatile computer readable storage medium stores at least one program, and each program includes instructions that, when executed by a terminal, cause the terminal to execute a key frame-based media stream abnormal retransmission method according to any one of claims 1-7.

Images