Delay Judgment Method, Device, and Storage Medium in the Process of Streaming Media Protocol Transformation

Abstract

The present application provides a method, device, and storage medium for delay determination during a streaming media protocol transformation process. The method includes: obtaining SRT streaming media data; converting the SRT streaming media data into NDI streaming media data; and determining the delay during the streaming media protocol transformation process based on I-frames in the SRT streaming media data and the NDI streaming media data. The present application first converts the SRT streaming media data into NDI streaming media data, and then determines the delay during the streaming media protocol transformation process based on I-frames in the SRT streaming media data and the NDI streaming media data, achieving a fast determination of the number of delayed frames without decoding during the SRT-NDI streaming media protocol transformation process.

Description

Technical Field

[0001] This application relates to the field of audio and video production technologies, and in particular, to a method, device, and storage medium for determining latency during the transformation of streaming media protocols. Background Art

[0002] In current integrated media live broadcasts of radio and television stations, it is often necessary to reasonably handle Internet signal transmission and in-station private network production and transmission. Therefore, it is often necessary to combine two different video compression encodings. For example, the Internet uses SRT (Secure Reliable Transport), and the internal network uses NDI (Network Device Interface) for unified scheduling and processing.

[0003] Usually, when performing streaming media processing, it involves the conversion from SRT to NDI (which may include operations such as transcoding, adding logos, adjusting exposure, adjusting hue, etc., and the superposition of these operations). Inevitably, latency will occur during this process. In order to maintain synchronization between different signals, it is necessary to know the number of delayed frames generated during the conversion process. Summary of the Invention

[0004] To solve one of the above technical defects, this application provides a method, device, and storage medium for determining latency during the transformation of streaming media protocols.

[0005] In the first aspect of this application, a method for determining latency during the transformation of streaming media protocols is provided. The method includes:

[0006] Obtain Secure Reliable Transport (SRT) streaming media data;

[0007] Convert the SRT streaming media data into Network Device Interface (NDI) streaming media data;

[0008] Determine the latency during the transformation of the streaming media protocol based on the I-frames in the SRT streaming media data and the NDI streaming media data.

[0009] Optionally, converting the SRT streaming media data into Network Device Interface (NDI) streaming media data includes:

[0010] Decode the SRT streaming media data to obtain raw video and audio data. The raw video and audio data includes video data in YUV format and audio data in Pulse Code Modulation (PCM);

[0011] Re-encode the raw video and audio data using the NDI-HX protocol to obtain NDI streaming media data.

[0012] Optionally, judging the delay in the streaming protocol transformation according to the I-frames in the SRT streaming data and the NDI streaming data includes:

[0013] Calibrate the I-frames of the SRT streaming data to obtain a first sequence;

[0014] Calibrate the I-frames of the NDI streaming data to obtain a second sequence;

[0015] Judge the delay in the streaming protocol transformation according to the first sequence and the second sequence.

[0016] Optionally, calibrating the I-frames of the SRT streaming data includes:

[0017] Take the I-frame of the first complete GOP group in the SRT streaming data as the calibration starting frame I0;

[0018] Determine the amount of frame data between all subsequent I-frames and I0;

[0019] Form a first sequence [a1, a2, …, a n according to the positional relationship between the I-frames, where n is the subsequent I-frame identifier, and a n is the amount of frame data between the nth subsequent I-frame and I0.

[0020] Optionally, calibrating the I-frames of the NDI streaming data includes:

[0021] Take the I-frame of the first complete GOP group in the NDI streaming data as the calibration starting frame I′0;

[0022] Determine the amount of frame data between all I-frames within a subsequent preset duration and I′0;

[0023] Form a second sequence [A1, A2, …, A m according to the positional relationship between the I-frames, where m is the I-frame identifier within the subsequent preset duration, and A m is the amount of frame data between the mth I-frame within the subsequent preset duration and I′0.

[0024] Optionally, judging the delay in the streaming protocol transformation according to the first sequence and the second sequence includes:

[0025] Obtain a preset comparison quantity Q;

[0026] Set the comparison initial value q = 1;

[0027] Determine the qth position deviation according to the first sequence, the second sequence, and q;

[0028] If q < Q, then after q = q + 1, re-execute the step of determining the qth position deviation according to the first sequence, the second sequence, and q;

[0029] If q = Q, the delay in the streaming media protocol transformation process is judged according to each position deviation.

[0030] Optionally, determining the q-th position deviation according to the first sequence, the second sequence, and q includes:

[0031] The q-th position deviation

[0032] Optionally, judging the delay in the streaming media protocol transformation process according to each position deviation includes:

[0033] Determining the number of 0s among each position deviation;

[0034] Determining the q corresponding to the position deviation with the most 0s as the delay in the streaming media protocol transformation process.

[0035] In a second aspect of the present application, an electronic device is provided, including:

[0036] A memory;

[0037] A processor; and

[0038] A computer program;

[0039] Wherein, the computer program is stored in the memory and is configured to be executed by the processor to implement the method described in the first aspect above.

[0040] In a third aspect of the present application, a computer-readable storage medium is provided, on which a computer program is stored; the computer program is executed by a processor to implement the method described in the first aspect above.

[0041] The present application provides a method, device, and storage medium for judging the delay in the streaming media protocol transformation process. The method includes: acquiring SRT streaming media data; converting the SRT streaming media data into NDI streaming media data; and judging the delay in the streaming media protocol transformation process according to the I-frames in the SRT streaming media data and the NDI streaming media data. The present application first converts the SRT streaming media data into NDI streaming media data, and then judges the delay in the streaming media protocol transformation process according to the I-frames in the SRT streaming media data and the NDI streaming media data, realizing a fast judgment of directly judging the number of delayed frames without decoding in the SRT-NDI streaming media protocol transformation process. Description of the Drawings

[0042] The drawings described herein are used to provide a further understanding of the present application, and constitute a part of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation of the present application. In the drawings:

[0043] Figure 1 Schematic flowchart of a method for determining delay in the process of streaming media protocol transformation provided by an embodiment of the present application;

[0044] Figure 2 Schematic diagram of the I-frame distribution of SRT streaming media data provided by an embodiment of the present application;

[0045] Figure 3 Schematic diagram of the I-frame distribution and GOP change in SRT stream and NDI stream provided by an embodiment of the present application;

[0046] Figure 4 Schematic diagram of the frame delay after converting an SRT stream to an NDI stream provided by an embodiment of the present application;

[0047] Figure 5 Schematic diagram of the starting I-frame calibration in an SRT stream provided by an embodiment of the present application;

[0048] Figure 6 Schematic diagram of the starting I-frame calibration in an NDI stream provided by an embodiment of the present application;

[0049] Figure 7 Schematic diagram of the I-frame distribution of SRT streaming media data provided by an embodiment of the present application. Detailed implementation manners

[0050] In order to make the technical solutions and advantages in the embodiments of the present application clearer and more understandable, the following further details the exemplary embodiments of the present application with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than an exhaustive list of all embodiments. It should be noted that, without conflict, the embodiments in the present application and the features in the embodiments can be combined with each other.

[0051] In the process of implementing the present application, the inventor found that when performing streaming media processing, it usually involves the conversion from SRT to NDI. Inevitably, there will be a delay during this process. In order to maintain the synchronization between different signals, it is necessary to know the number of delayed frames generated during the conversion process.

[0052] In view of the above problems, an embodiment of the present application provides a method, device, and storage medium for determining delay during the transformation of streaming media protocols. The method includes: obtaining SRT streaming media data; converting the SRT streaming media data into NDI streaming media data; and determining the delay during the transformation of streaming media protocols based on I-frames in the SRT streaming media data and the NDI streaming media data. In the present application, the SRT streaming media data is first converted into NDI streaming media data, and then the delay during the transformation of the SRT-NDI streaming media protocol is determined based on the I-frames in the SRT streaming media data and the NDI streaming media data, realizing a fast determination of the number of delayed frames without decoding during the SRT-NDI streaming media protocol transformation process.

[0053] See Figure 1 , the implementation process of the method for determining delay during the transformation of streaming media protocols provided in this embodiment is as follows:

[0054] 101. Obtain SRT streaming media data.

[0055] SRT (Secure Reliable Transport) streaming media data is SRT media stream data encoded using H.264. Its generation process encapsulates media data based on the SRT protocol to obtain compressed H.264-encoded media data, which is the SRT streaming media data. This step obtains the SRT streaming media data.

[0056] 102. Convert the SRT streaming media data into NDI streaming media data.

[0057] In implementation, first decode the SRT streaming media data to obtain raw video and audio data. The raw video and audio data includes video data in the YUV format and audio data in Pulse Code Modulation (PCM). Then, re-encode the raw video and audio data using the NDI-HX protocol to obtain NDI (Network Device Interface) streaming media data.

[0058] Among them, the YUV format is a color encoding format. Y represents luminance, that is, the grayscale value, and UV represent chrominance and chroma respectively. Its function is to describe the color and saturation of the image and is used to specify the color of pixels.

[0059] The NDI-HX protocol is a low-bandwidth version of NDI and is a compressed long-GOP H.264 variant. Its advantage is low bandwidth requirements, allowing multiple video streams to be carried on a simple gigabit network without occupying a large amount of network traffic.

[0060] In step 102, the SRT streaming media data is decoded to obtain raw video and audio data (the compressed and coded data of the video stream is restored to YUV image data, and the compressed and coded data of the audio stream is PCM audio data). The compressed and coded data of the video stream is restored to YUV image data, and the compressed and coded data of the audio stream is restored to PCM audio data for re-encoding by NDI-HX, where NDI-HX also uses H.264, and then the NDI SDK is used to perform protocol encapsulation on the coded data to complete the streaming media protocol conversion from SRT to NDI.

[0061] 103, judging the delay in the process of streaming media protocol conversion according to the I frame in the SRT streaming media data and the NDI streaming media data.

[0062] Executing step 102 completes the conversion of the streaming media protocol from SRT to NDI. In the streaming media data transmitted by SRT, according to the encoding rules of H.264, it includes I frames, P frames, and B frames, among which I frames are key frames. In H.264, I frames will be compressed intra-frame, and P frames and B frames are inter-frame prediction data. A video stream includes several transition segments, among which due to the characteristics of the encoder, the transition frame (the first frame of each transition segment) is often encoded as an I frame due to the drastic change of the picture. At the same time, each transition includes several GOPs (Group Of Pictures), and the length of each GOP is uncertain. The range of GOP is generally between 1s and tens of seconds. The first frame of each GOP is also an I frame. It should be noted that an I frame is not necessarily a transition frame. Transition frames are often I frames. The I frame distribution diagram of SRT streaming media data is shown as follows. Figure 2 shown.

[0063] After NDI protocol conversion, the same number of transition segments as in the original SRT media stream data are still included, and the frame length of the transition segments and the number of transition frames are also the same as in the original SRT media stream data. Since the media stream data has been decoded and re-encoded, the length of GOP has been redefined, resulting in a different number of I frames in GOP. However, since the transition segments are still the same after protocol conversion, the number of transition I frames remains the same, such as Figure 3 As shown, the transition frame in the SRT stream is defined as The transition frame in the NDI stream is

[0064] In practical application, such as Figure 4 As shown, a certain delay will be generated after the SRT to NDI protocol conversion, and the delay after the SRT to NDI conversion is calculated by calibrating the I frame.

[0065] Therefore, step 103 determines the delay in the streaming protocol conversion process based on the I frame in the SRT streaming data and the NDI streaming data.

[0066] Specifically,

[0067] 1. Calibrate the I-frames of the SRT streaming media data to obtain the first sequence.

[0068] During specific implementation,

[0069] 1) Use the I-frame of the first complete GOP group in the SRT streaming media data as the calibration starting frame I0.

[0070] 2) Determine the amount of frame data between all subsequent I-frames and I0.

[0071] For example, the amount of frame data between the nth I-frame after I0 and I0 is a n .

[0072] 3) Form the first sequence [a1, a2, …, a n according to the positional relationship between the I-frames.

[0073] Among them, n is the identifier of the subsequent I-frame, and a n is the amount of frame data between the nth subsequent I-frame and I0.

[0074] During the process of obtaining the first sequence, the I-frames of the SRT encoded data will be calibrated. Use the I-frame of the first complete GOP group in the SRT stream as the calibration starting frame I0, as Figure 5 shown. All frames in the SRT stream are arranged in an integer sequence. Define a as the number of frames between the I-frame and I0, and thus obtain the description of the positional relationship between the I-frames in the SRT encoded data, that is, the first sequence [a1, a2, …, a n .

[0075] 2. Calibrate the I-frames of the NDI streaming media data to obtain the second sequence.

[0076] During specific implementation,

[0077] 1) Use the I-frame of the first complete GOP group in the NDI streaming media data as the calibration starting frame I′0.

[0078] 2) Determine the amount of frame data between all I-frames within the subsequent preset duration (such as 1 minute) and I′0.

[0079] For example, the amount of frame data between the mth I-frame within one minute after I′0 and I′0 is A m .

[0080] 3) Form the second sequence [A1, A2, …, A m according to the positional relationship between the I-frames, where m is the identifier of the I-frame within the subsequent preset duration, and A m is the amount of frame data between the mth I-frame within the subsequent preset duration and I′0.

[0081] During the process of obtaining the second sequence, perform I-frame calibration on the encoded data of NDI. Use the I-frame of the first complete GOP group in the converted NDI stream as the calibration starting frame I′0, as Figure 6 shown. All frames are arranged in an integer sequence within the subsequent specified time (such as 1 minute). Define A as the number of frames between the I-frame and I′0, and thus obtain the description of the positional relationship between I-frames in the encoded data of NDI, that is, the second sequence [A1, A2, …, A m .

[0082] 3. Determine the delay during the transformation process of the streaming media protocol according to the first sequence and the second sequence.

[0083] Specifically, when implementing,[[]]

[0084] 1) Obtain the preset comparison quantity Q.

[0085] For example, Q = 100.

[0086] 2) Set the comparison initial value q = 1.

[0087] Among them, q is used for counting, ensuring that the subsequent comparison is performed Q times.

[0088] 3) Determine the q-th position deviation according to the first sequence, the second sequence, and q.

[0089] Among them, the q-th position deviation

[0090] Taking Q = 100 as an example, when q = 1,[[]]

[0091] when q = 2,[[]]

[0092] …

[0093] when q = 100,[[]]

[0094] 4) If q < Q, after q = q + 1, re-execute the step of determining the q-th position deviation according to the first sequence, the second sequence, and q.

[0095] If q < Q, it means that the number of comparison times is less than Q times, and continuous comparison is required. Therefore, the count value is incremented by one, and then 3) is re-executed. Loop to execute 3) until the number of comparison times reaches Q times, that is, until q = Q.

[0096] 5) If q = Q, judge the delay during the transformation process of the streaming media protocol according to each position deviation.

[0097] If q = Q, it means that the number of comparison times reaches Q times, and delay judgment can be performed.

[0098] During the judgment, the number of 0s in each position deviation is determined. The q corresponding to the position deviation with the largest number of 0s is determined as the delay in the streaming media protocol transformation process.

[0099] Taking Q = 100 as an example, during the judgment, r is used to record the number of 0s in each matrix in z1 to z 100 to obtain [r1, r2, …, r 100 , and the maximum value among r1, r2, …, r 100 is taken. The subscript coefficient of r corresponding to the maximum value is the most likely number of delayed frames, that is, the delay in the streaming media protocol transformation process (since the number of transition I-frames is the same, this is the value when the maximum number of I-frames coincide).

[0100] In addition, during the specific implementation, the above first sequence and second sequence queues can be continuously updated as the video progresses, and the delay is obtained through step 103. If the delay is stable for a certain period of time, the entire system locks this delay.

[0101] The method provided in this embodiment, its implementation process is as Figure 7 shown. For the streaming media data of the SRT protocol encoded with H.264 (GOP>1), the SRT streaming media data is first converted into NDI streaming media data, and then the delay in the streaming media protocol transformation process is judged according to the I-frames in the SRT streaming media data and the NDI streaming media data, realizing a fast judgment of directly judging the number of delayed frames without decoding in the SRT-NDI streaming media protocol transformation process.

[0102] This embodiment provides a method for judging the delay in the streaming media protocol transformation process, which includes: obtaining SRT streaming media data; converting the SRT streaming media data into NDI streaming media data; judging the delay in the streaming media protocol transformation process according to the I-frames in the SRT streaming media data and the NDI streaming media data, realizing a fast judgment of directly judging the number of delayed frames without decoding in the SRT-NDI streaming media protocol transformation process.

[0103] Based on the same inventive concept of the method for judging the delay in the streaming media protocol transformation process, this embodiment provides an electronic device, which includes: a memory, a processor, and a computer program.

[0104] Wherein, the computer program is stored in the memory and is configured to be executed by the processor to implement the above method for judging the delay in the streaming media protocol transformation process.

[0105] Specifically,

[0106] obtain the secure and reliable transport protocol SRT streaming media data.

[0107] Convert SRT streaming media data into NDI streaming media data of the local area network device interface protocol.

[0108] Judge the delay in the streaming media protocol transformation process according to the I-frames in the SRT streaming media data and the NDI streaming media data.

[0109] Optionally, converting SRT streaming media data into NDI streaming media data of the local area network device interface protocol includes:

[0110] Decode the SRT streaming media data to obtain raw video and audio data, where the raw video and audio data includes video data in YUV format and audio data in Pulse Code Modulation (PCM).

[0111] Re-encode the raw video and audio data using the NDI-HX protocol to obtain NDI streaming media data.

[0112] Optionally, judging the delay in the streaming media protocol transformation process according to the I-frames in the SRT streaming media data and the NDI streaming media data includes:

[0113] Perform I-frame calibration on the SRT streaming media data to obtain the first sequence.

[0114] Perform I-frame calibration on the NDI streaming media data to obtain the second sequence.

[0115] Judge the delay in the streaming media protocol transformation process according to the first sequence and the second sequence.

[0116] Optionally, performing I-frame calibration on the SRT streaming media data includes:

[0117] Use the I-frame of the first complete GOP group in the SRT streaming media data as the calibration starting frame I0.

[0118] Determine the amount of frame data between all subsequent I-frames and I0.

[0119] Form the first sequence [a1, a2, …, a n , where n is the subsequent I-frame identifier, and a n is the amount of frame data between the nth subsequent I-frame and I0.

[0120] Optionally, performing I-frame calibration on the NDI streaming media data includes:

[0121] Use the I-frame of the first complete GOP group in the NDI streaming media data as the calibration starting frame I′0.

[0122] Determine the amount of frame data between all I-frames within the subsequent preset duration and I′0.

[0123] Form a second sequence [A1, A2, …, A m according to the positional relationship between I-frames, where m is the I-frame identifier within a subsequent preset duration, and A m is the amount of frame data between the m-th I-frame and I′0 within the subsequent preset duration.

[0124] Optionally, determine the delay during the streaming media protocol transformation according to the first sequence and the second sequence, including:

[0125] Obtain a preset comparison quantity Q.

[0126] Set the comparison initial value q = 1.

[0127] Determine the q-th position deviation according to the first sequence, the second sequence, and q.

[0128] If q < Q, after q = q + 1, re-execute the step of determining the q-th position deviation according to the first sequence, the second sequence, and q.

[0129] If q = Q, determine the delay during the streaming media protocol transformation according to each position deviation.

[0130] Optionally, determine the q-th position deviation according to the first sequence, the second sequence, and q, including:

[0131] The q-th position deviation

[0132] Optionally, determine the delay during the streaming media protocol transformation according to each position deviation, including:

[0133] Determine the number of 0s among each position deviation.

[0134] Determine the q corresponding to the position deviation with the largest number of 0s as the delay during the streaming media protocol transformation.

[0135] The electronic device provided in this embodiment, when the computer program thereon is executed by the processor, first converts SRT streaming media data into NDI streaming media data, and then determines the delay during the streaming media protocol transformation according to the I-frames in the SRT streaming media data and the NDI streaming media data, realizing a fast judgment of directly determining the number of delayed frames without decoding during the SRT-NDI streaming media protocol transformation.

[0136] Based on the same inventive concept of the method for determining the delay during the streaming media protocol transformation, this embodiment provides a computer, and a computer program is stored thereon. The computer program is executed by the processor to implement the above method for determining the delay during the streaming media protocol transformation.

[0137] Specifically,

[0138] Obtain SRT streaming media data with secure and reliable transmission protocol.

[0139] Convert the SRT streaming media data into NDI streaming media data of the local area network device interface protocol.

[0140] Judge the delay in the streaming media protocol transformation process based on the I-frames in the SRT streaming media data and the NDI streaming media data.

[0141] Optionally, converting the SRT streaming media data into NDI streaming media data of the local area network device interface protocol includes:

[0142] Decode the SRT streaming media data to obtain raw video and audio data, where the raw video and audio data includes video data in YUV format and audio data in Pulse Code Modulation (PCM).

[0143] Re-encode the raw video and audio data using the NDI-HX protocol to obtain NDI streaming media data.

[0144] Optionally, judging the delay in the streaming media protocol transformation process based on the I-frames in the SRT streaming media data and the NDI streaming media data includes:

[0145] Calibrate the I-frames of the SRT streaming media data to obtain the first sequence.

[0146] Calibrate the I-frames of the NDI streaming media data to obtain the second sequence.

[0147] Judge the delay in the streaming media protocol transformation process based on the first sequence and the second sequence.

[0148] Optionally, calibrating the I-frames of the SRT streaming media data includes:

[0149] Use the I-frame of the first complete GOP group in the SRT streaming media data as the calibration starting frame I0.

[0150] Determine the amount of frame data between all subsequent I-frames and I0.

[0151] Form the first sequence [q1, q2, …, an] according to the positional relationship between the I-frames, where n is the subsequent I-frame identifier, and an is the amount of frame data between the nth subsequent I-frame and I0. n , where n is the subsequent I-frame identifier, and an n is the amount of frame data between the nth subsequent I-frame and I0.

[0152] Optionally, calibrating the I-frames of the NDI streaming media data includes:

[0153] Use the I-frame of the first complete GOP group in the NDI streaming media data as the calibration starting frame I′0.

[0154] Determine the amount of frame data between all I-frames within the subsequent preset duration and I′0.

[0155] Form a second sequence [A1, A2, …, A m according to the positional relationship between I-frames, where m is the I-frame identifier within a subsequent preset duration, and A m is the amount of frame data from the m-th I-frame to I′0 within a subsequent preset duration.

[0156] Optionally, determining the delay during the streaming media protocol transformation according to the first sequence and the second sequence includes:

[0157] Obtain a preset comparison quantity Q.

[0158] Set the comparison initial value q = 1.

[0159] Determine the q-th position deviation according to the first sequence, the second sequence, and q.

[0160] If q < Q, after q = q + 1, re-execute the step of determining the q-th position deviation according to the first sequence, the second sequence, and q.

[0161] If q = Q, determine the delay during the streaming media protocol transformation according to each position deviation.

[0162] Optionally, determining the q-th position deviation according to the first sequence, the second sequence, and q includes:

[0163] The q-th position deviation

[0164] Optionally, determining the delay during the streaming media protocol transformation according to each position deviation includes:

[0165] Determine the number of 0s among each position deviation.

[0166] Determine the q corresponding to the position deviation with the largest number of 0s as the delay during the streaming media protocol transformation.

[0167] The computer-readable storage medium provided in this embodiment, when the computer program thereon is executed by a processor, first converts SRT streaming media data into NDI streaming media data, and then determines the delay during the streaming media protocol transformation according to the I-frames in the SRT streaming media data and the NDI streaming media data, realizing a fast judgment of directly judging the number of delayed frames without decoding during the SRT-NDI streaming media protocol transformation.

[0168] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memory, CD-ROM, optical memory, etc.) that contain computer-usable program code. The solutions in the embodiments of the present application can be implemented using various computer languages. For example, object-oriented programming languages such as Java and interpreted scripting languages such as JavaScript, etc.

[0169] The present application is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram, as well as the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0170] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including instruction means that implement the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0171] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0172] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the description of this application, "a plurality of" means at least two, such as two, three, etc., unless otherwise specifically defined.

[0173] Although the preferred embodiments of this application have been described, additional changes and modifications can be made to these embodiments by those skilled in the art once they learn of the basic creative concept. Therefore, the appended claims are intended to be construed to include the preferred embodiments as well as all changes and modifications that fall within the scope of this application.

[0174] Obviously, those skilled in the art can make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalent technologies, this application is also intended to include these changes and modifications.

Claims

1. A method for determining delay in the process of streaming media protocol transformation, characterized in that, The method includes: Obtaining Secure Reliable Transport (SRT) streaming media data; Converting the SRT streaming media data into Local Area Network Device Interface (NDI) protocol streaming media data; Judging the delay in the streaming media protocol transformation process according to the I-frames in the SRT streaming media data and the NDI streaming media data; Performing I-frame calibration on the SRT streaming media data to obtain a first sequence; Performing I-frame calibration on the NDI streaming media data to obtain a second sequence; Judging the delay in the streaming media protocol transformation process according to the first sequence and the second sequence; The performing I-frame calibration on the SRT streaming media data includes: Taking the I-frame of the first complete Group of Pictures (GOP) in the SRT streaming media data as the calibration starting frame I0; Determining the amount of frame data between all subsequent I-frames and I0; Form a first sequence [a1, a2, …, a according to the positional relationship between I-frames n , where n is the identifier of the subsequent I-frame, and a n is the amount of frame data between the nth subsequent I-frame and I0; The performing I-frame calibration on the NDI streaming media data includes: Taking the I-frame of the first complete GOP in the NDI streaming media data as the calibration starting frame I′0; Determining the amount of frame data between all I-frames within a subsequent preset duration and I′0; Form a second sequence [A1, A2, …, A m according to the positional relationship between I frames, where m is the I frame identifier within a subsequent preset duration, and A m is the amount of frame data between the m-th I frame and I′0 within the subsequent preset duration; The judging the delay in the streaming media protocol transformation process according to the first sequence and the second sequence includes: Obtaining a preset comparison quantity Q; Setting a comparison initial value q = 1; Determining the q-th position deviation according to the first sequence, the second sequence, and q; If q < Q, then after q = q + 1, re-perform the step of determining the q-th position deviation according to the first sequence, the second sequence, and q; If q = Q, then judging the delay in the streaming media protocol transformation process according to each position deviation; The determining the q-th position deviation according to the first sequence, the second sequence, and q includes: The q-th position deviation The judging the delay in the streaming media protocol transformation process according to each position deviation includes: Determining the number of 0s among each position deviation; Determining the q corresponding to the position deviation with the largest number of 0s as the delay in the streaming media protocol transformation process.

2. The method according to claim 1, characterized in that, The converting the SRT streaming media data into NDI protocol streaming media data includes: Decoding the SRT streaming media data to obtain raw video and audio data, where the raw video and audio data includes video data in YUV format and audio data in Pulse Code Modulation (PCM); Re-encoding the raw video and audio data using the NDI-HX protocol to obtain NDI streaming media data.

3. An electronic device, characterized in that, It includes: A memory; A processor; And A computer program; Wherein, the computer program is stored in the memory and is configured to be executed by the processor to implement the method according to any one of claims 1-2.

4. A computer-readable storage medium, characterized in that, A computer program is stored thereon; the computer program is executed by a processor to implement the method according to any one of claims 1-2.

Images