Transfer packaging verification method and device

Abstract

The embodiment of the invention provides a repackaging verification method, and the method comprises the steps: obtaining a first audio and video and a second audio and video, the second audio and video is obtained by repackaging the first audio and video, the first audio and video comprises a plurality of first frames, and the second audio and video comprises a plurality of second frames; acquiring frame data and metadata of the plurality of first frames and the plurality of second frames according to the first audio and video and the second audio and video; determining a plurality of frame groups according to the frame data of the plurality of first frames and the plurality of second frames, each frame group comprising a first frame and a second frame, the frame data of the first frames being matched with the frame data of the second frames; and determining a verification result according to the frame data and the metadata of the plurality of frame groups, the verification result comprising an error of the second audio and video relative to the first audio and video. According to the technical scheme provided by the embodiment of the invention, the corresponding frames before and after repackaging can be automatically positioned and verified, so that the troubleshooting efficiency of repackaging problems is effectively improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of multimedia technology, and in particular to a transpackaging verification method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Art

[0002] With the popularization of Internet technology, live streaming is being accepted and loved by more and more people. In live streaming, the host terminal can upload the collected audio and video to the server, and the server can transcode the audio and video according to the playback requirements, and then provide the transcoded audio and video to the audience terminal. In actual applications, the audience terminal has many and strict requirements on the audio and video format, and it is difficult for developers to accurately obtain the specific requirements of the audience terminal, which easily leads to the transcoding result not meeting the requirements, thus causing playback problems such as screen distortion and audio and video asynchrony.

[0003] However, the current troubleshooting of transpackaging problems relies on manual observation and developer assistance, which is time-consuming, complex, and costly, resulting in low troubleshooting efficiency and poor user experience.

[0004] It should be noted that the above content is not necessarily prior art, nor is it intended to limit the scope of patent protection of this application. Summary of the invention

[0005] The embodiments of the present application provide a transpackaging verification method, apparatus, computer equipment, computer-readable storage medium, and computer program product to solve or alleviate one or more of the technical problems raised above.

[0006] One aspect of an embodiment of the present application provides a transpackaging verification method, the method comprising: Acquire a first audio and video and a second audio and video, where the second audio and video is obtained by transpackaging the first audio and video, the first audio and video includes a plurality of first frames, and the second audio and video includes a plurality of second frames; Acquire frame data and metadata of the plurality of first frames and the plurality of second frames according to the first audio and video and the second audio and video; Determine a plurality of frame groups according to the frame data of the plurality of first frames and the plurality of second frames, each frame group including a first frame and a second frame, and the frame data of the first frame and the second frame match; A verification result is determined based on the frame data and metadata of the multiple frame groups, where the verification result includes an error of the second audio and video relative to the first audio and video.

[0007] Optionally, the frame data includes content information; correspondingly, determining a plurality of frame groups according to the frame data of the plurality of first frames and the plurality of second frames includes: Based on the content information of the plurality of first frames and the plurality of second frames, acquiring hash values ​​of the content information of the plurality of first frames and the plurality of second frames; The first frame and the second frame having the same content information hash value are determined as a frame group to obtain the plurality of frame groups.

[0008] Optionally, determining verification results according to the frame data and metadata of the plurality of frame groups includes: Based on the metadata of the plurality of first frames and the plurality of second frames, obtaining metadata hash values ​​of the plurality of first frames and the plurality of second frames; For each frame group, compare the metadata hash values ​​of the first frame and the second frame; When the two are inconsistent, the current frame group is determined as an abnormal frame group; Based on the abnormal frame group, the verification result is generated.

[0009] Optionally, the frame data includes a timestamp; correspondingly, determining a verification result according to the frame data and metadata of the plurality of frame groups includes: Acquire multiple timestamp differences according to the timestamps of the first frame and the second frame of the multiple frame groups, where one timestamp difference corresponds to one frame group; Determine a reference timestamp difference and a fluctuation timestamp difference according to the multiple timestamp differences, wherein the reference timestamp difference is a timestamp difference whose occurrence number exceeds a preset threshold among the multiple timestamp differences, and the fluctuation timestamp difference is a timestamp difference whose difference from the reference timestamp difference exceeds a preset range; Acquire a frame group corresponding to the fluctuation timestamp difference, and determine the corresponding frame group as an abnormal frame group; Based on the abnormal frame group, the verification result is generated.

[0010] Optionally, the transpackaging verification method further includes: Based on the first audio and video, determine the sequence of the multiple first frames; Based on the second audio and video, determine the sequence of the plurality of second frames; For each frame group, compare the order of the first frame and the second frame; When the two are inconsistent, the current frame group is determined as an abnormal frame group; Based on the abnormal frame group, the verification result is generated.

[0011] Optionally, the transpackaging verification method further includes: Determine the sequence of the plurality of second frames according to the second audio and video; Based on the order of the plurality of second frames, determining whether the timestamp of a latter second frame among two adjacent second frames is smaller than the timestamp of a former second frame; When the timestamp of the subsequent second frame is less than the timestamp of the previous second frame, determining the subsequent second frame as an abnormal frame; Based on the abnormal frame, the verification result is generated.

[0012] Optionally, the transpackaging verification method further includes: Obtaining durations of the plurality of second frames; Determine whether the timestamp of a latter second frame of two adjacent second frames is equal to the sum of the timestamp of the former second frame and the duration; In the case of inequality, the second frame is determined as an abnormal frame; Based on the abnormal frame, the verification result is generated.

[0013] Optionally, the transpackaging verification method further includes: Determine a difference frame according to the plurality of first frames, the plurality of second frames and the plurality of frame groups, the difference frame comprising a first frame and / or a second frame that does not belong to the plurality of frame groups; Based on the difference frame, the verification result is generated.

[0014] Another aspect of an embodiment of the present application provides a transpackaging verification device, the device comprising: A first acquisition module is used to acquire a first audio and video and a second audio and video, where the second audio and video is obtained by transpackaging the first audio and video, the first audio and video includes a plurality of first frames, and the second audio and video includes a plurality of second frames; A second acquisition module, used to acquire frame data and metadata of the plurality of first frames and the plurality of second frames according to the first audio and video and the second audio and video; A first determining module, configured to determine a plurality of frame groups according to the frame data of the plurality of first frames and the plurality of second frames, each frame group including a first frame and a second frame, and the frame data of the first frame and the second frame match; The second determination module is used to determine a verification result according to the frame data and metadata of the multiple frame groups, and the verification result includes an error of the second audio and video relative to the first audio and video.

[0015] Another aspect of an embodiment of the present application provides a computer device, including: at least one processor; and a memory communicatively coupled to the at least one processor; Wherein: the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method as described above.

[0016] Another aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the method described above is implemented.

[0017] Another aspect of an embodiment of the present application provides a computer program product, including a computer program, which implements the method described above when executed by a processor.

[0018] The above technical solution adopted in the embodiment of the present application may have the following advantages: A first audio and video and a second audio and video obtained by transpackaging the first audio and video are obtained, wherein the first audio and video include multiple first frames, and the second audio and video include multiple second frames. Based on the first audio and video and the second audio and video, frame data and metadata of the multiple first frames and multiple second frames are obtained. Based on the frame data, the corresponding first frame and second frame are determined to construct multiple frame groups. Based on the frame data and metadata of the multiple frame groups, the verification result of the transpackaging is determined, wherein the verification result includes the error of the second audio and video relative to the first audio and video. It can be seen that the embodiment of the present application can determine whether the audio and video are modified during the transpackaging process that may cause playback problems by respectively extracting the frame data and metadata of the audio and video before and after the transpackaging, automatically locating the corresponding frames before and after the transpackaging, and performing verification, thereby effectively improving the efficiency of troubleshooting transpackaging problems and optimizing user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings exemplarily illustrate the embodiments and constitute a part of the specification, and together with the text description of the specification, are used to explain the exemplary implementation of the embodiments. The embodiments shown are for illustrative purposes only and do not limit the scope of the claims. In all drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0020] Figure 1 The following schematically shows an operating environment diagram of the transpackaging verification method according to the first embodiment of the present application; Figure 2 The flowchart of the transpackaging verification method according to the first embodiment of the present application is schematically shown; Figure 3 Schematically shows Figure 2 Flow chart of sub-steps of step S204; Figure 4 Schematically shows Figure 2 Flow chart of sub-steps of step S206; Figure 5 Schematically shows Figure 2 Flow chart of sub-steps of step S206; Figure 6 The newly added flow chart of the transpackaging verification method according to the first embodiment of the present application is schematically shown; Figure 7 The newly added flow chart of the transpackaging verification method according to the first embodiment of the present application is schematically shown; Figure 8 The newly added flow chart of the transpackaging verification method according to the first embodiment of the present application is schematically shown; Fig. 9 The newly added flow chart of the transpackaging verification method according to the first embodiment of the present application is schematically shown; Fig.10 This is an application example diagram of the transpackaging verification method according to the first embodiment of the present application; Fig.11 A block diagram of a transpackaging verification device according to Embodiment 2 of the present application is schematically shown; and Fig.12 The hardware architecture diagram of the computer device according to the third embodiment of the present application is schematically shown. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0022] It should be noted that the descriptions involving "first", "second", etc. in the embodiments of the present application are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0023] In the description of the present application, it should be understood that the numerical labels before the steps do not indicate the order in which the steps are executed, but are only used to facilitate the description of the present application and to distinguish each step, and therefore should not be understood as a limitation on the present application.

[0024] First, the following terms are explained: Server: Also known as a server, a computer that provides data processing and storage functions on the Internet. In the live broadcast system, the server receives the original audio and video data from the anchor terminal, processes it uniformly and converts it into the same packaging format, and then transcodes and distributes it.

[0025] Client: refers to the software used by users, such as anchor terminals and audience terminals, which can be used to send audio and video data to the server, or receive audio and video data from the server, process and display it to users. Different clients may support different media formats and playback technologies, so the server needs to provide audio and video formats that are compatible with its playback capabilities.

[0026] Transpackaging: Converting audio and video content from one packaging format to another. The purpose is to unify the original audio and video data format and perform compatibility processing to ensure that it meets the network transmission and playback requirements of different clients.

[0027] Metadata: Data that describes various attributes of audio and video content, such as video encoding type, decoding parameters, audio information, etc. Decoding parameters include resolution, frame rate, bit rate, etc. There may be dozens or even hundreds of specific parameters.

[0028] Frame data: refers to the specific data of each individual image frame that constitutes the video, or the specific data of each individual audio frame that constitutes the audio. Each frame has a corresponding timestamp and content information.

[0029] Timestamp: Used to mark the exact time position of a video frame or audio frame in the entire stream, indicating the length of time from the start of the video or audio to the current frame.

[0030] Secondly, in order to facilitate those skilled in the art to understand the technical solutions provided in the embodiments of the present application, the relevant technologies are described below: With the popularization of Internet technology, live streaming is being accepted and loved by more and more people. In live streaming, the host terminal can upload the collected audio and video to the server, and the server can transcode the audio and video according to the playback requirements, and then provide the transcoded audio and video to the audience terminal. In actual applications, the audience terminal has many and strict requirements on the format of audio and video, and it is difficult for the audience terminal to accurately obtain the specific requirements of the audience terminal, which easily leads to the transcoding result not meeting the requirements, thus causing playback problems such as screen distortion and audio and video asynchrony.

[0031] However, the applicant has learned that the related technology discovers problems by manually observing the transpackaging results, and then the client and server developers work together to troubleshoot and fix the specific transpackaging errors. This involves many people, a complex process, a long time, and high costs, which seriously affects the efficiency of troubleshooting transpackaging problems and provides a poor user experience.

[0032] To this end, the embodiment of the present application provides a transpackaging verification technical solution. In this technical solution: (1) A verification method is provided for end-to-end automatic verification of whether the transpackaging result of the server meets the client requirements from the perspective of the media format, and a verification process is specifically customized: by comparing the media files sent by the anchor terminal and the media files received by the audience terminal, it automatically senses and locates whether the audio and video have been modified that may cause playback problems, thereby assisting in problem troubleshooting and timely alarming. (2) Based on the characteristics that the specific content information in the metadata and frame data before and after transpackaging is not modified, the frame data and metadata of the audio and video before and after transpackaging are extracted respectively, and the specific frames corresponding to the transpackaging before and after are located based on the content information in the frame data. It can be verified whether the metadata of each frame has been modified, and whether the timestamp of the audio and video stream is correct, so as to realize automatic verification of the live transpackaging results, thereby realizing automatic alarming and assisting manual troubleshooting of the specific causes of playback errors. See below for details.

[0033] Finally, for ease of understanding, an exemplary operating environment is provided below.

[0034] like Figure 1 As shown, the operating environment diagram includes: service platform 2, anchor terminals (4A, 4B, ..., 4M), and audience terminals (6A, 6B, ..., 6N). In the live broadcast scenario, the anchor terminal (4A, 4B, ..., 4M) logs in to the service platform 2, and the service platform 2 pushes the live broadcast data to the audience terminal (6A, 6B, ..., 6N) in real time.

[0035] Service platform 2 can provide live broadcast service, which can be a single server, a server cluster or a cloud computing service center. The server can include a business server, a video cloud server, etc.

[0036] The anchor terminal (4A, 4B, ..., 4M) is used to generate live broadcast data in real time and perform streaming operations on the live broadcast data. The live broadcast data may include audio data or video data. The anchor terminal may be an electronic device such as a smart phone or a tablet computer. Of course, the anchor terminal may be a virtual computing instance in the service platform 2.

[0037] The audience terminal (6A, 6B, ..., 6N) may be configured to receive the live broadcast data of the anchor terminal in real time. The audience terminal (6A, 6B, ..., 6N) may be any type of computing device, such as a smart phone, a tablet device, a laptop computer, a smart TV, a vehicle-mounted terminal, etc. The audience terminal (6A, 6B, ..., 6N) may have a built-in browser or a special program, and receive the live broadcast data through the browser or the special program to output content to the user. The content may include video, audio, comments, text data and / or the like.

[0038] The audience terminal (6A, 6B, ..., 6N) may include a player. The player outputs (e.g., displays, presents) content to the user. The content may include video, audio, comments, text data and / or the like. The audience terminal (6A, 6B, ..., 6N) may include an interface, which may include an input element (touch screen). For example, the input element may be configured to receive user instructions, and the user instructions may enable the audience terminal (6A, 6B, ..., 6N) to perform various operations, such as sending a bullet screen, entering a comment, giving a gift, etc.

[0039] The anchor terminals (4A, 4B, ..., 4M), the audience terminals (6A, 6B, ..., 6N) and the service platform 2 may be connected via a network. The network may include various network devices, such as routers, switches, multiplexers, hubs, modems, bridges, repeaters, firewalls and / or proxy devices, etc. The network may include physical links, such as coaxial cable links, twisted pair cable links, optical fiber links, combinations thereof and / or the like. The network may include wireless links, such as cellular links, satellite links, Wi-Fi links and / or the like.

[0040] It should be noted that Figure 1 The number of the host terminal and the audience terminal is only illustrative and is not used to limit the scope of patent protection of this application. According to the actual situation, there can be any number of host terminals and audience terminals, and the host terminal and the audience terminal can be the same terminal. The transpackaging verification method provided in the embodiment of the present application can be executed on the host terminal, the audience terminal or the service platform.

[0041] The technical solutions of the present application are described below through multiple embodiments. It should be noted that these embodiments can be implemented in a variety of different forms and should not be construed as being limited to the embodiments described here.

[0042] Embodiment 1 Figure 2 The flowchart of the transpackaging verification method according to the first embodiment of the present application is schematically shown.

[0043] like Figure 2 As shown, the transpackaging verification method may include steps S200 to S206, wherein: Step S200, obtaining a first audio and video and a second audio and video, where the second audio and video is obtained by transpackaging the first audio and video, the first audio and video includes a plurality of first frames, and the second audio and video includes a plurality of second frames.

[0044] Step S202: Acquire frame data and metadata of the multiple first frames and the multiple second frames according to the first audio and video and the second audio and video.

[0045] Step S204: determining a plurality of frame groups according to the frame data of the plurality of first frames and the plurality of second frames, each frame group including a first frame and a second frame, and the frame data of the first frame and the frame data of the second frame match.

[0046] Step S206: Determine a verification result based on the frame data and metadata of the multiple frame groups, where the verification result includes an error of the second audio and video relative to the first audio and video.

[0047] The transpackaging verification method provided in the present embodiment obtains a first audio and video and a second audio and video obtained by transpackaging the first audio and video, wherein the first audio and video includes multiple first frames, and the second audio and video includes multiple second frames. Based on the first audio and video and the second audio and video, frame data and metadata of multiple first frames and multiple second frames are obtained. Based on the frame data, the corresponding first frame and second frame are determined to construct multiple frame groups. Based on the frame data and metadata of the multiple frame groups, the verification result of the transpackaging is determined, wherein the verification result includes the error of the second audio and video relative to the first audio and video. It can be seen that the embodiment of the present application can determine whether the audio and video are modified during the transpackaging process that may cause playback problems by extracting the frame data and metadata of the audio and video before and after the transpackaging, automatically locating the corresponding frames before and after the transpackaging, and performing verification, thereby effectively improving the efficiency of troubleshooting transpackaging problems and optimizing user experience.

[0048] The following combination Figure 2 , each step in steps S200~S206 and other optional steps are described in detail.

[0049] Step S200 , obtain a first audio and video and a second audio and video, the second audio and video is obtained by transpackaging the first audio and video, the first audio and video includes multiple first frames, and the second audio and video includes multiple second frames.

[0050] The first audio and video may be a media file or a real-time live stream uploaded by the anchor terminal (such as 4A) to the service platform 2. After receiving the first audio and video, the service platform 2 may transpack the first audio and video according to actual business needs to obtain the second audio and video. Among them, the packaging format of the first audio and video and the packaging format of the second audio and video may be different. For example: convert the first audio and video in FLV or BMT format into the second audio and video in HLS or MP4 format. The service platform 2 can also push the second audio and video to the audience terminal (such as 6A) to display and play the second audio and video through the audience terminal. The first audio and video may include multiple first frames, and the first frame may include an audio frame and a video frame. The second audio and video may include multiple second frames, and the second frame may include an audio frame and a video frame.

[0051] Step S202, according to the first audio and video and the second audio and video, obtain the frame data and metadata of the multiple first frames and the multiple second frames.

[0052] Exemplarily, the encapsulation format of the first audio and video can be obtained, and the index information therein can be parsed so as to extract specific binary data blocks from the first audio and video, such as the frame data of each first frame, the corresponding metadata, etc. Among them, the frame data may include content information and timestamp. If the first frame is a video frame, the content information is the image data of the video frame, and the timestamp is the timestamp corresponding to the video frame. If the first frame is an audio frame, the content information is the audio data of the audio frame, and the timestamp is the timestamp corresponding to the audio frame. Similarly, the frame data and metadata of multiple second frames can be obtained based on the second audio and video. Since the server-side transcapsulation may involve the modification of the media data code stream, for example, for the H.264 encoding format, it may be necessary to delete or add SEI (supplemental enhancement information). Therefore, it is necessary to parse the frame data and metadata in the original code stream. Among them, the original code stream may include an audio stream and a video stream, corresponding to the first audio and video. For example, for videos encoded in H.264 format, the binary data blocks of SPS (sequence parameter set) and PPS (picture parameter set) can be extracted from the original bitstream according to the H.264 encoding specification to obtain metadata; the original bitstream can also be standardized, such as removing unnecessary padding data, irrelevant SEI, etc., to effectively extract frame data.

[0053] Step S204 , determining a plurality of frame groups according to the frame data of the plurality of first frames and the plurality of second frames, each frame group including a first frame and a second frame, and the frame data of the first frame and the second frame match.

[0054] The purpose of transpackaging is to convert audio and video from one packaging format to another packaging format without changing its essential content, that is, the content information of the frame data before and after transpackaging is not modified. Based on this feature, multiple frame groups can be established based on the frame data of multiple first frames and multiple second frames, and each frame group can be composed of a first frame and a second frame whose frame data matches. The establishment method may include complete matching of frame data, partial matching of frame data, etc. An exemplary scheme is provided below.

[0055] In an optional embodiment, if Figure 3 As shown, step S204 may include: Step S300: based on the content information of the plurality of first frames and the plurality of second frames, obtain content information hash values ​​of the plurality of first frames and the plurality of second frames.

[0056] Step S302: Determine a first frame and a second frame having the same content information hash value as a frame group to obtain the plurality of frame groups.

[0057] Exemplarily, the binary data block corresponding to the content information of each first frame can be read, and the hash value (MD5 value) of the binary data block can be calculated to obtain the content information hash value of each first frame. Similarly, the content information hash value of each second frame can be calculated based on the content information of multiple second frames. Based on the characteristic that the content information of the frame data is not modified before and after transcapsulation, the first frame and the second frame with the same content information hash value can be grouped into one frame group, thereby constructing multiple frame groups. The first frame and the second frame of each frame group are actually the performance of the same frame in different encapsulation formats.

[0058] In this embodiment, based on the fact that the content information of the frame data remains unchanged before and after transcapsulation, the correspondence between the frames before and after transcapsulation is established through the MD5 value of the content information of the frame data, so as to subsequently verify whether each frame is mapped incorrectly or modified incorrectly before and after transcapsulation.

[0059] Step S206 , determining a verification result based on the frame data and metadata of the multiple frame groups, the verification result including an error of the second audio and video relative to the first audio and video.

[0060] During the transpackaging process, the essential content of the audio and video is not modified. For example, the content information of the frame data, metadata, the mapping relationship between metadata and frame data, and the organization of the frames in the audio and video after transpackaging (i.e., the second audio and video) should be consistent with the audio and video before transpackaging (i.e., the first audio and video). Based on these characteristics, the error of the second audio and video relative to the first audio and video can be efficiently located, that is, the error caused by transpackaging can be determined. Exemplarily, by comparing the metadata and timestamps of the frame data of the first frame and the second frame of each frame group, it can be determined whether the metadata of the same frame before and after transpackaging is mapped incorrectly and whether the timestamp is modified incorrectly, etc., to obtain a comprehensive and accurate verification result.

[0061] In this embodiment, by comparing the frame data and metadata of corresponding frames before and after transpackaging, it is possible to determine whether the audio and video have been modified during the transpackaging process that may cause playback problems, automate the transpackaging verification process, and improve the efficiency of troubleshooting.

[0062] The following will introduce the automatic verification process of the transpackaging result in combination with multiple embodiments.

[0063] In an optional embodiment, if Figure 4 As shown, step S206 may include: Step S400: acquiring metadata hash values ​​of the plurality of first frames and the plurality of second frames based on metadata of the plurality of first frames and the plurality of second frames.

[0064] Step S402: for each frame group, compare the metadata hash values ​​of the first frame and the second frame.

[0065] Step S404: when the two are inconsistent, the current frame group is determined as an abnormal frame group.

[0066] Step S406: Generate the verification result based on the abnormal frame group.

[0067] Exemplarily, the binary data block corresponding to the metadata of each first frame can be read, and the hash value can be calculated to obtain the metadata hash value of each first frame. Similarly, the metadata hash value of each second frame can be calculated based on the metadata of multiple second frames. For each frame group, the metadata hash values ​​of the first frame and the second frame are compared. If the two are consistent, it means that the metadata of the frame has not been modified before and after transpackaging. If the two are inconsistent, it means that the metadata of the frame has been incorrectly modified during the transpackaging process. The frame group with inconsistent metadata hash values ​​can be determined as an abnormal frame group, and further, the second frame in the abnormal frame group can also be determined as an abnormal frame. Based on the abnormal frame group or abnormal frame, a verification result can be generated. The verification result may include the timestamp of the abnormal frame group or the abnormal frame, etc., so as to quickly locate the problem.

[0068] In this embodiment, by calculating and comparing the metadata hash value of each frame group, erroneous modifications to the metadata during the transpackaging process can be effectively identified, thereby generating a verification result. Based on the verification result, the problem can be efficiently located, further improving the efficiency of problem troubleshooting.

[0069] Since server-side transcapsulation involves the conversion of multiple encapsulation formats, the timestamp of the frame data may also be modified based on actual business needs. Therefore, it is not possible to directly perform transcapsulation verification by comparing the timestamps of the frame groups. Incorrect timestamp modification will cause playback problems such as freezes and asynchrony between audio and video. To this end, the following will provide several exemplary solutions to verify whether the timestamp has been modified incorrectly during the transcapsulation process.

[0070] In an optional embodiment, if Figure 5 As shown, step S206 may include: Step S500: acquiring a plurality of timestamp differences according to the timestamps of the first frame and the second frame of the plurality of frame groups, wherein one timestamp difference corresponds to one frame group.

[0071] Step S502: Determine a reference timestamp difference and a fluctuation timestamp difference based on the multiple timestamp differences. The reference timestamp difference is a timestamp difference whose occurrence frequency exceeds a preset threshold among the multiple timestamp differences, and the fluctuation timestamp difference is a timestamp difference whose difference from the reference timestamp difference exceeds a preset range.

[0072] Step S504, obtaining a frame group corresponding to the fluctuation timestamp difference, and determining the corresponding frame group as an abnormal frame group.

[0073] Step S506: Generate the verification result based on the abnormal frame group.

[0074] Exemplarily, when the sampling rate is the same, the timestamp difference corresponding to each frame group can be calculated according to the timestamp of the first frame and the timestamp of the second frame of each frame group to obtain multiple timestamp differences. By statistically analyzing multiple timestamp differences, a reference timestamp difference and a fluctuating timestamp difference can be obtained. Among them, the reference timestamp difference can be a timestamp difference whose number of occurrences exceeds a preset threshold in multiple timestamp differences, for example: the majority of multiple timestamp differences. The fluctuating timestamp difference can be a timestamp difference whose difference with the reference timestamp difference exceeds a preset range, that is, a timestamp difference that deviates greatly from the reference timestamp difference. The fluctuating timestamp difference can indicate that the difference between the timestamp of each frame after transpackaging and the timestamp before transpackaging at the same sampling rate fails to remain constant, which can be regarded as a timestamp modification error. The frame group corresponding to the fluctuating timestamp difference can be obtained, and the corresponding frame group can be determined as an abnormal frame group. Further, the second frame in the abnormal frame group can also be determined as an abnormal frame. Based on the abnormal frame group or abnormal frame, a verification result can be generated. The verification result can include the timestamp of the abnormal frame group or the abnormal frame, etc., so as to quickly locate the problem.

[0075] In this embodiment, through statistical analysis of timestamp differences, timestamp modification errors sent during the transpackaging process can be effectively identified, thereby generating verification results. Based on the verification results, the problem can be efficiently located, the playback problem caused by timestamp modification errors can be solved, and the user experience can be optimized.

[0076] In an optional embodiment, if Figure 6 As shown, the transpackaging verification method may also include: Step S600: determining the sequence of the plurality of second frames according to the second audio and video.

[0077] Step S602: Based on the order of the plurality of second frames, determine whether the timestamp of the latter second frame of two adjacent second frames is smaller than the timestamp of the former second frame.

[0078] Step S604: when the timestamp of the next second frame is smaller than the timestamp of the previous second frame, the next second frame is determined as an abnormal frame.

[0079] Step S606: Generate the verification result based on the abnormal frame.

[0080] Taking the second frame including the video frame as an example, the corresponding video frame sequence can be determined according to the second audio and video, and the video frame sequence includes multiple second frames arranged in sequence. Based on the video frame sequence, the sequence of each second frame can be determined, that is, the position in the video frame sequence. According to the sequence of multiple second frames, it can be determined in sequence whether the timestamp of the latter second frame in two adjacent second frames is less than the timestamp of the previous second frame. The decoding timestamp of each frame in the second audio and video obtained by transpackaging the first audio and video is monotonically non-decreasing, that is, the decoding timestamp of the next frame in sequence is required to be greater than or equal to the previous frame, otherwise it indicates that a timestamp calculation error occurs during the transpackaging process. Therefore, in the case where the timestamp of the latter second frame is less than the timestamp of the previous second frame, the latter second frame can be determined as an abnormal frame. A verification result can be generated based on the abnormal frame. The verification result can include the timestamp of the abnormal frame group or the abnormal frame, etc., so as to quickly locate the problem. Similarly, if the second frame includes an audio frame, the audio frame sequence can also be obtained based on the second audio and video and the timestamp in the sequence can be checked to be monotonically non-decreasing, so as to determine whether a timestamp modification error occurs during the transpackaging process.

[0081] In this embodiment, by verifying the decoding timestamp of each second frame after transpackaging, timestamp modification errors can be effectively identified, thereby optimizing the verification results and further improving the efficiency of transpackaging problem troubleshooting.

[0082] In an optional embodiment, if Figure 7 As shown, the transpackaging verification method may also include: Step S700, obtaining the duration of the plurality of second frames.

[0083] Step S702: Determine whether the time stamp of the latter second frame of two adjacent second frames is equal to the sum of the time stamp of the former second frame and the duration.

[0084] Step S704: If they are not equal, the next second frame is determined as an abnormal frame.

[0085] Step S706: Generate the verification result based on the abnormal frame.

[0086] Exemplarily, the encapsulation format of the second audio and video can be obtained, and the index information therein can be parsed to extract the binary data block representing the duration from the second audio and video. The duration of the second frame can also be calculated according to the sampling length and the sampling rate. For example, the sampling length can be 1024, the sampling rate can be 48000HZ, and the duration of the second frame can be 1024 / 48000*1000=21.3ms. The above example takes the fixed sampling rate setting as an example. In the variable sampling rate setting, the duration of each second frame can be read or calculated separately. Determine whether the timestamp of the second frame after the two adjacent second frames is equal to the sum of the timestamp and duration of the previous second frame to determine whether the multiple second frames are continuous. For example: the decoding timestamp of the previous second frame can be 0ms, and the duration can be 21.3ms. Considering the error caused by rounding off the decimal place, the timestamp of the second frame after can be equal to 21ms or 22ms. If they are not equal, it means that the two adjacent second frames are not continuous in time, indicating that the timestamp was incorrectly modified during the transpackaging process. Therefore, the second frame after the abnormal frame can be determined as an abnormal frame. A verification result can be generated based on the abnormal frame. The verification result can include the timestamp of the abnormal frame group or the abnormal frame, etc., so as to quickly locate the problem. It should be noted that the continuity check can be performed only on the audio frame in the second frame, which reduces the calculation burden while meeting the requirements of the player. Of course, the continuity check can also be performed on the video frame in the second frame, which is not limited here.

[0087] In this embodiment, by detecting whether the timestamps of multiple second frames are continuous, it is possible to effectively identify incorrect timestamp modifications, accurately locate transcapsulation problems, improve the reliability of the second audio and video, and optimize user experience.

[0088] The above embodiments describe how to verify whether a timestamp modification error occurs during the transpackaging process and how to locate the problem frame. The following describes how to verify whether an incorrect mapping of the frame organization mode occurs during the transpackaging process.

[0089] In an optional embodiment, if Figure 8 As shown, the transpackaging verification method may also include: Step S800: determining the sequence of the plurality of first frames based on the first audio and video.

[0090] Step S802: determine the sequence of the plurality of second frames based on the second audio and video.

[0091] Step S804: for each frame group, compare the order of the first frame and the second frame.

[0092] Step S806: When the two are inconsistent, the current frame group is determined as an abnormal frame group.

[0093] Step S808: Generate the verification result based on the abnormal frame group.

[0094] Exemplarily, the sequence position of each first frame and the sequence position of each second frame can be determined respectively according to the first audio and video and the second audio and video. For each frame group, the sequence positions of the first frame and the second frame therein are compared. In the case of inconsistency between the two, it means that the arrangement order of the same frame before and after transpackaging is different, and the organization mode of the frame does not correspond before and after transpackaging. Therefore, the frame group with inconsistent sequence position can be determined as an abnormal frame group, and further, the second frame in the abnormal frame group can also be determined as an abnormal frame. Based on the abnormal frame group or the abnormal frame, a verification result can be generated. The verification result may include the timestamp of the abnormal frame group or the abnormal frame, etc., so as to quickly locate the problem.

[0095] In this embodiment, by comparing the sequence position of each frame in the first audio and video and the second audio and video, it is possible to effectively identify whether the organization of the frames before and after transpackaging is consistent, thereby quickly locating the transpackaging problem and improving the efficiency of troubleshooting.

[0096] In an optional embodiment, if Fig. 9 As shown, the transpackaging verification method may also include: Step S900: determining a difference frame according to the plurality of first frames, the plurality of second frames and the plurality of frame groups, wherein the difference frame includes the first frame and / or the second frame that does not belong to the plurality of frame groups.

[0097] Step S902: Generate the verification result based on the difference frame.

[0098] Exemplarily, based on multiple first frames, multiple second frames and multiple frame groups, a difference frame can be quickly acquired. The difference frame may include a first frame and / or a second frame that do not belong to multiple frame groups. It can be seen that the difference frame has no corresponding frame before or after transpackaging, indicating that frame loss and multiple frame problems occur in the transpackaging process. Based on the difference frame, a verification result can be generated. The verification result may include a timestamp of the difference frame, etc., so as to quickly locate the problem.

[0099] In this embodiment, by comparing multiple first frames, multiple second frames and multiple frame groups, the difference frames can be quickly identified, the problem of frame loss or multiple frames in the transpackaging process can be effectively detected, and the verification result can be further optimized.

[0100] In order to make this application easier to understand, the following Fig.10 An exemplary application is provided.

[0101] S11: Obtain a first audio and video and a second audio and video, wherein the second audio and video is obtained by transpackaging the first audio and video, the first audio and video includes a plurality of first frames, and the second audio and video includes a plurality of second frames.

[0102] The first frame may include an audio frame and a video frame, and the second frame may include an audio frame and a video frame.

[0103] S12: extracting content data, metadata, and timestamps of multiple first frames according to the index information of the encapsulation format of the first audio and video, and calculating a hash value of the content information and a hash value of the metadata. Extracting content data, metadata, and timestamps of multiple second frames according to the index information of the encapsulation format of the second audio and video, and calculating a hash value of the content information and a hash value of the metadata.

[0104] S13: Determine multiple frame groups according to the content information hash values ​​of the multiple first frames and the content information hash values ​​of the multiple second frames. Specifically, the first frame and the second frame with the same content information hash value are determined as a frame group, that is, a corresponding relationship is established. The first frame and the second frame of each frame group are actually the performance of the same frame in different encapsulation formats.

[0105] S14: For each frame group, compare the metadata hash values ​​of the first frame and the second frame to see if they are consistent. If they are inconsistent, it means that the metadata is incorrectly mapped, and the second frame is determined as an abnormal frame.

[0106] S15: Determine whether a timestamp rollback problem occurs based on the timestamps of the multiple second frames. That is, whether the timestamp of the latter second frame of two adjacent second frames is smaller than the timestamp of the former second frame. If smaller, it means that the timestamp is incorrectly modified, and the latter second frame is determined as an abnormal frame.

[0107] S16: Determine the sequence positions of the first frames and the second frames, and determine whether the sequence positions of the first frame and the second frame of the same frame group are consistent. If they are inconsistent, it means that the frame arrangement is wrong, and the second frame is determined as an abnormal frame.

[0108] S17: Obtain the timestamp difference of each frame group, determine the fluctuating timestamp difference therein through statistical analysis, determine the frame group corresponding to the fluctuating timestamp difference as an abnormal frame group, and determine the second frame of the abnormal frame group as an abnormal frame.

[0109] S18: Obtain the duration of the second frame and perform continuity detection on multiple second frames. That is, whether the timestamp of the second frame after two adjacent second frames is equal to the sum of the timestamp and the duration stamp of the previous second frame. If not, it means that the timestamp has been modified incorrectly, and the next second frame is determined as an abnormal frame.

[0110] S19: Generate a verification result according to the abnormal frame and abnormal frame group determined in the previous step, where the verification result includes an error of the second audio and video relative to the first audio and video.

[0111] In this exemplary application: (1) A verification method is provided for end-to-end automatic verification of whether the server-side transpackaging results meet the client requirements from the perspective of media format, and a specific verification process is customized: by comparing the media files sent by the anchor terminal and the media files received by the audience terminal, it automatically senses and locates whether the audio and video have been modified that may cause playback problems, thereby assisting in problem troubleshooting and timely alarming. (2) Based on the characteristics that the specific content information in the metadata and frame data before and after transpackaging is not modified, the frame data and metadata of the audio and video before and after transpackaging are extracted respectively, and the specific frames corresponding to the transpackaging before and after are located based on the content information in the frame data. It can be verified whether the metadata of each frame has been modified, and whether the timestamp of the audio and video stream is correct, so as to realize automatic verification of the live transpackaging results, thereby realizing automatic alarming and assisting manual troubleshooting of the specific causes of playback errors.

[0112] Embodiment 2 Fig.11 The block diagram of the transpackaging verification device according to the second embodiment of the present application is schematically shown. The device can be divided into one or more program modules, one or more program modules are stored in a storage medium, and are executed by one or more processors to complete the embodiment of the present application. The program module referred to in the embodiment of the present application refers to a series of computer program instruction segments that can complete specific functions. The following description will specifically introduce the functions of each program module in this embodiment. Fig.11 As shown, the apparatus 1000 may include: a first acquisition module 1100, a second acquisition module 1200, a first determination module 1300 and a second determination module 1400, wherein: A first acquisition module 1100 is used to acquire a first audio and video and a second audio and video, where the second audio and video is obtained by transpackaging the first audio and video, the first audio and video includes a plurality of first frames, and the second audio and video includes a plurality of second frames; A second acquisition module 1200 is used to acquire frame data and metadata of the plurality of first frames and the plurality of second frames according to the first audio and video and the second audio and video; A first determining module 1300 is used to determine a plurality of frame groups according to the frame data of the plurality of first frames and the plurality of second frames, each frame group including a first frame and a second frame, and the frame data of the first frame and the second frame match; The second determination module 1400 is used to determine a verification result according to the frame data and metadata of the multiple frame groups, where the verification result includes an error of the second audio and video relative to the first audio and video.

[0113] As an optional embodiment, the frame data includes content information; correspondingly, determining a plurality of frame groups according to the frame data of the plurality of first frames and the plurality of second frames includes: Based on the content information of the plurality of first frames and the plurality of second frames, acquiring hash values ​​of the content information of the plurality of first frames and the plurality of second frames; The first frame and the second frame having the same content information hash value are determined as a frame group to obtain the plurality of frame groups.

[0114] As an optional embodiment, determining the verification result according to the frame data and metadata of the multiple frame groups includes: Based on the metadata of the plurality of first frames and the plurality of second frames, obtaining metadata hash values ​​of the plurality of first frames and the plurality of second frames; For each frame group, compare the metadata hash values ​​of the first frame and the second frame; When the two are inconsistent, the current frame group is determined as an abnormal frame group; Based on the abnormal frame group, the verification result is generated.

[0115] As an optional embodiment, the frame data includes a timestamp; correspondingly, determining the verification result according to the frame data and metadata of the multiple frame groups includes: Acquire multiple timestamp differences according to the timestamps of the first frame and the second frame of the multiple frame groups, where one timestamp difference corresponds to one frame group; Determine a reference timestamp difference and a fluctuation timestamp difference according to the multiple timestamp differences, wherein the reference timestamp difference is a timestamp difference whose occurrence number exceeds a preset threshold among the multiple timestamp differences, and the fluctuation timestamp difference is a timestamp difference whose difference from the reference timestamp difference exceeds a preset range; Acquire a frame group corresponding to the fluctuation timestamp difference, and determine the corresponding frame group as an abnormal frame group; Based on the abnormal frame group, the verification result is generated.

[0116] As an optional embodiment, the device 1000 is further used for: Based on the first audio and video, determine the sequence of the multiple first frames; Based on the second audio and video, determine the sequence of the plurality of second frames; For each frame group, compare the order of the first frame and the second frame; When the two are inconsistent, the current frame group is determined as an abnormal frame group; Based on the abnormal frame group, the verification result is generated.

[0117] As an optional embodiment, the apparatus 1000 is further used for: Determine the sequence of the plurality of second frames according to the second audio and video; Based on the order of the plurality of second frames, determining whether the timestamp of a latter second frame among two adjacent second frames is smaller than the timestamp of a former second frame; When the timestamp of the subsequent second frame is less than the timestamp of the previous second frame, determining the subsequent second frame as an abnormal frame; Based on the abnormal frame, the verification result is generated.

[0118] As an optional embodiment, the device 1000 is further used for: Obtaining durations of the plurality of second frames; Determine whether the timestamp of a latter second frame of two adjacent second frames is equal to the sum of the timestamp of the former second frame and the duration; In the case of inequality, the second frame is determined as an abnormal frame; Based on the abnormal frame, the verification result is generated.

[0119] As an optional embodiment, the device 1000 is further used for: Determine a difference frame according to the plurality of first frames, the plurality of second frames and the plurality of frame groups, the difference frame comprising a first frame and / or a second frame that does not belong to the plurality of frame groups; Based on the difference frame, the verification result is generated.

[0120] Embodiment 3 Fig.12 The schematic diagram of the hardware architecture of a computer device 10000 suitable for implementing the transpackaging verification method according to the third embodiment of the present application is shown schematically. In some embodiments, the computer device 10000 may be a terminal device such as a smart phone, a wearable device, a tablet computer, a personal computer, a vehicle terminal, a game console, a virtual device, a workbench, a digital assistant, a set-top box, a robot, etc. In other embodiments, the computer device 10000 may be a rack server, a blade server, a tower server, or a cabinet server (including an independent server, or a server cluster composed of multiple servers), etc. Fig.12 As shown, the computer device 10000 includes but is not limited to: a memory 10010, a processor 10020, and a network interface 10030 that can communicate with each other through a system bus. Among them: The memory 10010 includes at least one type of computer-readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (such as SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 10010 can be an internal storage module of the computer device 10000, such as a hard disk or memory of the computer device 10000. In other embodiments, the memory 10010 can also be an external storage device of the computer device 10000, such as a plug-in hard disk equipped on the computer device 10000, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Of course, the memory 10010 can also include both the internal storage module of the computer device 10000 and its external storage device. In this embodiment, the memory 10010 is generally used to store the operating system and various application software installed in the computer device 10000, such as the program code of the transpackaging verification method, etc. In addition, the memory 10010 can also be used to temporarily store various data that have been output or will be output.

[0121] In some embodiments, the processor 10020 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other chips. The processor 10020 is generally used to control the overall operation of the computer device 10000, such as performing control and processing related to data interaction or communication with the computer device 10000. In this embodiment, the processor 10020 is used to run the program code stored in the memory 10010 or process data.

[0122] The network interface 10030 may include a wireless network interface or a wired network interface, and the network interface 10030 is generally used to establish a communication link between the computer device 10000 and other computer devices. For example, the network interface 10030 is used to connect the computer device 10000 to an external terminal through a network, and to establish a data transmission channel and a communication link between the computer device 10000 and the external terminal. The network may be a wireless or wired network such as an intranet, the Internet, the Global System of Mobile communication (GSM), Wideband Code Division Multiple Access (WCDMA), 4G network, 5G network, Bluetooth, Wi-Fi, etc.

[0123] It should be pointed out that Fig.12 Only a computer device having components 10010 - 10030 is shown, but it should be understood that implementation of all of the components shown is not a requirement, and more or fewer components may alternatively be implemented.

[0124] In this embodiment, the transpackaging verification method stored in the memory 10010 may also be divided into one or more program modules and executed by one or more processors (such as the processor 10020) to complete the embodiment of the present application.

[0125] Embodiment 4 An embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored, wherein when the computer program is executed by a processor, the steps of the transpackaging verification method in the embodiment are implemented.

[0126] In this embodiment, the computer-readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the computer-readable storage medium may be an internal storage unit of a computer device, such as a hard disk or memory of the computer device. In other embodiments, the computer-readable storage medium may also be an external storage device of a computer device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the computer device. Of course, the computer-readable storage medium may also include both an internal storage unit of a computer device and an external storage device thereof. In this embodiment, the computer-readable storage medium is generally used to store an operating system and various application software installed on a computer device, such as the program code of the transpackaging verification method in the embodiment, etc. In addition, the computer-readable storage medium may also be used to temporarily store various types of data that have been output or are to be output.

[0127] Embodiment 5 An embodiment of the present application also provides a computer program product, including a computer program, which implements the method in the above embodiment when executed by a processor.

[0128] Obviously, those skilled in the art should understand that the modules or steps of the above-mentioned embodiments of the present application can be implemented by general-purpose computer devices, they can be concentrated on a single computer device, or distributed on a network composed of multiple computer devices, optionally, they can be implemented by executable program codes of computer devices, so that they can be stored in a storage device and executed by the computer device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. In this way, the embodiments of the present application are not limited to any specific combination of hardware and software.

[0129] It should be noted that the above are only preferred embodiments of the present application, and the patent protection scope of the present application is not limited thereto. Any equivalent structure or equivalent process transformation made using the contents of the specification and drawings of the present application, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A transpackaging verification method, characterized in that: The method comprises: Acquire a first audio and video and a second audio and video, where the second audio and video is obtained by transpackaging the first audio and video, the first audio and video includes a plurality of first frames, and the second audio and video includes a plurality of second frames; Acquire frame data and metadata of the plurality of first frames and the plurality of second frames according to the first audio and video and the second audio and video; Determine a plurality of frame groups according to the frame data of the plurality of first frames and the plurality of second frames, each frame group including a first frame and a second frame, and the frame data of the first frame and the second frame match; A verification result is determined based on the frame data and metadata of the multiple frame groups, where the verification result includes an error of the second audio and video relative to the first audio and video.

2. The method according to claim 1, characterized in that The frame data includes content information; correspondingly, determining a plurality of frame groups according to the frame data of the plurality of first frames and the plurality of second frames includes: Based on the content information of the plurality of first frames and the plurality of second frames, acquiring hash values ​​of the content information of the plurality of first frames and the plurality of second frames; The first frame and the second frame having the same content information hash value are determined as a frame group to obtain the plurality of frame groups.

3. The method according to claim 1, characterized in that Determining verification results according to the frame data and metadata of the plurality of frame groups includes: Based on the metadata of the plurality of first frames and the plurality of second frames, obtaining metadata hash values ​​of the plurality of first frames and the plurality of second frames; For each frame group, compare the metadata hash values ​​of the first frame and the second frame; When the two are inconsistent, the current frame group is determined as an abnormal frame group; Based on the abnormal frame group, the verification result is generated.

4. The method according to claim 1, characterized in that: The frame data includes a timestamp; correspondingly, determining a verification result according to the frame data and metadata of the plurality of frame groups includes: Acquire multiple timestamp differences according to the timestamps of the first frame and the second frame of the multiple frame groups, where one timestamp difference corresponds to one frame group; Determine a reference timestamp difference and a fluctuation timestamp difference according to the multiple timestamp differences, wherein the reference timestamp difference is a timestamp difference whose occurrence number exceeds a preset threshold among the multiple timestamp differences, and the fluctuation timestamp difference is a timestamp difference whose difference from the reference timestamp difference exceeds a preset range; Acquire a frame group corresponding to the fluctuation timestamp difference, and determine the corresponding frame group as an abnormal frame group; Based on the abnormal frame group, the verification result is generated.

5. The method according to claim 1, characterized in that Also includes: Based on the first audio and video, determine the sequence of the multiple first frames; Based on the second audio and video, determine the sequence of the plurality of second frames; For each frame group, compare the order of the first frame and the second frame; When the two are inconsistent, the current frame group is determined as an abnormal frame group; Based on the abnormal frame group, the verification result is generated.

6. The method according to claim 1, characterized in that Also includes: Determine the sequence of the plurality of second frames according to the second audio and video; Based on the order of the plurality of second frames, determining whether the timestamp of a latter second frame among two adjacent second frames is smaller than the timestamp of a former second frame; When the timestamp of the subsequent second frame is less than the timestamp of the previous second frame, determining the subsequent second frame as an abnormal frame; Based on the abnormal frame, the verification result is generated.

7. The method according to claim 6, characterized in that Also includes: Obtaining durations of the plurality of second frames; Determine whether the timestamp of a latter second frame of two adjacent second frames is equal to the sum of the timestamp of the former second frame and the duration; In the case of inequality, the second frame is determined as an abnormal frame; Based on the abnormal frame, the verification result is generated.

8. The method according to any one of claims 1 to 7, characterized in that: Also includes: Determine a difference frame according to the plurality of first frames, the plurality of second frames and the plurality of frame groups, the difference frame comprising a first frame and / or a second frame that does not belong to the plurality of frame groups; Based on the difference frame, the verification result is generated.

9. A transpackaging verification device, characterized in that: The device comprises: A first acquisition module is used to acquire a first audio and video and a second audio and video, where the second audio and video is obtained by transpackaging the first audio and video, the first audio and video includes a plurality of first frames, and the second audio and video includes a plurality of second frames; A second acquisition module, used to acquire frame data and metadata of the plurality of first frames and the plurality of second frames according to the first audio and video and the second audio and video; A first determining module, configured to determine a plurality of frame groups according to the frame data of the plurality of first frames and the plurality of second frames, each frame group including a first frame and a second frame, and the frame data of the first frame and the second frame match; The second determination module is used to determine a verification result according to the frame data and metadata of the multiple frame groups, and the verification result includes an error of the second audio and video relative to the first audio and video.

10. A computer device, characterized in that: include: at least one processor; and a memory communicatively coupled to the at least one processor; wherein: The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the method according to any one of claims 1 to 8 is implemented.

12. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to claims 1 to 8 are implemented.

Images