Sound and picture asynchronization detection method and device

Abstract

The embodiment of the invention provides a sound and picture asynchronization detection method, and the method comprises the steps: obtaining an original video and a target video, and enabling the target video to be obtained through the transcoding of the original video; determining a first scene switching point in the original video and a second scene switching point in the target video; determining a video transcoding error according to the first video timestamp of the first scene switching point and the second video timestamp of the second scene switching point; determining an audio transcoding error according to a first audio corresponding to the first video timestamp and a second audio corresponding to the second video timestamp; and according to the video transcoding error and the audio transcoding error, determining whether sound and picture asynchronization occurs in the target video. Therefore, whether audio and video are synchronous or not can be automatically detected by utilizing the characteristic that encoding and decoding do not change scene related information, the detection efficiency and the universality of the detection method are improved, and the quality of a transcoded video is improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer technology, and in particular, to a method, apparatus, computer device, computer-readable storage medium, and computer program product for detecting audio and video asynchrony. Background Art

[0002] In distributed transcoding, the audio and video may be out of sync due to a variety of reasons. When transcoding distributed audio and video, frame rate conversion (such as 60 frames to 30 frames) may cause cumulative errors in the number of frames, resulting in audio and video out of sync.

[0003] The existing method for detecting audio and video asynchrony can only detect specific video sources and has a low degree of automation.

[0004] It should be noted that the above content is not necessarily prior art, nor is it intended to limit the scope of patent protection of this application. Summary of the invention

[0005] The embodiments of the present application provide a method, apparatus, computer device, computer-readable storage medium, and computer program product for detecting audio and video asynchrony, so as to solve or alleviate one or more of the technical problems mentioned above.

[0006] One aspect of an embodiment of the present application provides a method for detecting audio and video asynchrony, the method comprising: Obtaining an original video and a target video, wherein the target video is obtained by transcoding the original video; Determining a first scene switching point in the original video and a second scene switching point in the target video; Determine a video transcoding error according to a first video timestamp of the first scene switching point and a second video timestamp of the second scene switching point; Determining an audio transcoding error according to a first audio corresponding to the first video timestamp and a second audio corresponding to the second video timestamp; It is determined whether the target video has audio and video asynchronism according to the video transcoding error and the audio transcoding error.

[0007] Optionally, determining a first scene switching point in the original video and a second scene switching point in the target video includes: Acquire multiple video frames of a video to be detected, where the video to be detected is the original video or the target video; Determining a scene index for each of the video frames; When the scene index is greater than a preset threshold, determining the video frame as a target scene switching point of the video to be detected; When the video to be detected is the original video, the target scene switching point is the first scene switching point; when the video to be detected is the target video, the target scene switching point is the second scene switching point.

[0008] Optionally, a plurality of the first scene switching points correspond one-to-one to a plurality of the second scene switching points; and the method further comprises: determining a video transcoding error sequence according to a first video timestamp of each of the first scene switching points and a second video timestamp of each of the second scene switching points; According to the video transcoding error sequence, it is determined whether cumulative errors occur in the target video during the transcoding process.

[0009] Optionally, the number of the first scene switching points is different from the number of the second scene switching points; and the method further includes: sequentially matching the first scene switching point with each of the second scene switching points; Obtain multiple sets of matching first scene switching points and second scene switching points; The video transcoding error sequence is obtained according to the video timestamp difference between each group of matched first scene switching points and second scene switching points.

[0010] Optionally, determining the audio transcoding error according to the first audio corresponding to the first video timestamp and the second audio corresponding to the second video timestamp includes: Determine, according to the first audio, a first feature sequence corresponding to the first audio, wherein the first feature sequence corresponds to a first audio timestamp sequence; Determine, according to the second audio, a second feature sequence corresponding to the second audio, wherein the second feature sequence corresponds to a second audio timestamp sequence; The audio transcoding error is determined according to the first feature sequence and the second feature sequence.

[0011] Optionally, determining whether the target video has audio and video asynchronism according to the video transcoding error and the audio transcoding error includes: Determining whether the video transcoding error is the same as the audio transcoding error; In a case where the video transcoding error is the same as the audio transcoding error, determining whether the video transcoding error is less than a preset standard; When the video transcoding error is less than the preset standard, it is determined that the target video does not have audio and video asynchronism.

[0012] Another aspect of an embodiment of the present application provides a device for detecting audio and video asynchrony, the device comprising: An acquisition module, used to acquire an original video and a target video, wherein the target video is obtained by transcoding the original video; A first determining module, used to determine a first scene switching point in the original video and a second scene switching point in the target video; A second determining module, configured to determine a video transcoding error according to a first video timestamp of the first scene switching point and a second video timestamp of the second scene switching point; A third determination module, configured to determine an audio transcoding error according to a first audio corresponding to the first video timestamp and a second audio corresponding to the second video timestamp; The fourth determination module is used to determine whether the target video has audio and video asynchronism according to the video transcoding error and the audio transcoding error.

[0013] Another aspect of an embodiment of the present application provides a computer device, including: at least one processor; and a memory communicatively coupled to the at least one processor; Wherein: the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method as described above.

[0014] Another aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the method described above is implemented.

[0015] Another aspect of an embodiment of the present application provides a computer program product, including a computer program, which implements the method described above when executed by a processor.

[0016] The above technical solution adopted in the embodiment of the present application may have the following advantages: Using the scene switching point as an anchor point, the scene switching points of the video before and after transcoding are compared to determine whether the transcoded video has audio and video asynchrony. Therefore, the characteristic of encoding and decoding that does not change scene-related information can be used to automatically detect whether the audio and video are synchronized, improve detection efficiency and the versatility of the detection method, and improve the quality of the transcoded video. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings exemplarily illustrate the embodiments and constitute a part of the specification, and together with the text description of the specification, are used to explain the exemplary implementation of the embodiments. The embodiments shown are for illustrative purposes only and do not limit the scope of the claims. In all drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0018] Figure 1 The operating environment diagram of the audio-video asynchrony detection method according to the first embodiment of the present application is schematically shown; Figure 2 The flowchart of the method for detecting audio and video asynchrony according to the first embodiment of the present application is schematically shown; Figure 3 Schematically shows Figure 2 Flow chart of sub-steps of step S202; Figure 4 The newly added flow chart of the method for detecting audio and video asynchrony according to the first embodiment of the present application is schematically shown; Figure 5 An example flow chart of determining a video transcoding error sequence in an embodiment of the present application is schematically shown; Figure 6 Another newly added flow chart of the method for detecting audio and video asynchrony according to the first embodiment of the present application is schematically shown; Figure 7 Another example flow chart of determining a video transcoding error sequence in an embodiment of the present application is schematically shown; Figure 8 Schematically shows Figure 2 Flow chart of sub-steps of step S206; Fig. 9 An example flow chart of determining an audio transcoding error in an embodiment of the present application is schematically shown; Fig.10 Schematically shows Figure 2 Flow chart of sub-steps of step S208; Fig.11 is an application example diagram of the audio-video asynchrony detection method according to an embodiment of the present application; Fig.12 A block diagram schematically shows a device for detecting audio and video asynchrony according to Embodiment 2 of the present application; Fig.13 The hardware architecture diagram of the computer device according to the third embodiment of the present application is schematically shown. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0020] It should be noted that the descriptions involving "first", "second", etc. in the embodiments of the present application are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0021] In the description of the present application, it should be understood that the numerical labels before the steps do not indicate the order in which the steps are executed, but are only used to facilitate the description of the present application and to distinguish each step, and therefore should not be understood as a limitation on the present application.

[0022] First, the following terms are explained: Audio and video are out of sync: The sound and picture of the video are inconsistent, for example, the sound lags behind the picture, or the picture lags behind the sound.

[0023] Transcoding: is the process of converting digital media files (such as video) from one encoding format to another.

[0024] Distributed transcoding: It is a technology for processing large-scale video file transcoding by dividing the video file into multiple small segments, then processing these segments in parallel on multiple computing nodes, and finally merging the processed segments into a complete video file.

[0025] Encapsulation: It is a key step in the transcoding process, which involves combining video and audio data according to specific format specifications to form a complete multimedia file.

[0026] Editlist: It is a parameter in the video transcoding process, which is used to deal with the timestamp offset problem of video and audio.

[0027] Frame rate: Frame Rate refers to the number of still images (frames) displayed per second, usually expressed in FPS (Frames Per Second). Frame rate is an important parameter for measuring the smoothness of video playback, which determines the smoothness of the picture during video playback.

[0028] PSNR: It is an objective evaluation indicator for measuring image quality. It provides a basis for the performance evaluation of image processing algorithms by quantifying the error between the original image and the distorted image.

[0029] Global perspective decision frame adjustment strategy: It is a strategy used in a distributed transcoding system to ensure that video frames remain synchronized and consistent during the transcoding process. This strategy involves adjusting the video frames from the perspective of the overall system to ensure the quality of the transcoded video.

[0030] Fast direct conversion point: refers to the process of synchronously recording the live stream as on-demand video, and supports a series of operations including media asset management, media processing (such as transcoding, content review, smart cover and other AI processing), content production (such as cloud editing), CDN distribution acceleration, etc.

[0031] Scenecut: is a feature in the x264 encoder that identifies transition points between different scenes in a video.

[0032] Secondly, in order to facilitate those skilled in the art to understand the technical solutions provided in the embodiments of the present application, the relevant technologies are described below: In the video transcoding system, the problem of audio and video asynchrony may be caused by a variety of factors. First, improper processing of encapsulation parameters, such as improper use of editlist, may cause the phenomenon of audio and video asynchrony from the beginning of the video. Secondly, when performing frame rate conversion, such as converting a 60-frame video to a 30-frame video, the accumulated frame number error may also cause audio and video asynchrony. Finally, problems with the original data source, such as the loss of certain abnormal fragments during the transcoding process, may also cause audio and video asynchrony. For the problem of audio and video asynchrony in distributed transcoding, the existing solution is to solve it by deciding the frame adjustment strategy through a global perspective. However, the inventors found that in the "fast straight switch point" scenario, due to the lack of a global perspective, a frame compensation strategy cannot be implemented, and audio and video asynchrony is more likely to occur.

[0033] To this end, the embodiment of the present application provides a technical solution for detecting audio-video asynchrony. In this technical solution, a parameter-free audio-video asynchrony detection method is provided to automatically detect audio-video asynchrony problems during video transcoding, including but not limited to the cumulative error before and after transcoding or the initial error at the beginning of the video, as well as the synchronization between audio and video, thereby improving the detection efficiency of audio-video asynchrony and the quality of video playback. See below for details.

[0034] Finally, for ease of understanding, an exemplary operating environment is provided below.

[0035] like Figure 1 As shown, the operating environment diagram includes: service platform 2, network 4, client 6, where: The service platform 2 may be composed of a single or multiple computing devices. The multiple computing devices may include virtualized computing instances. Virtualized computing instances may include virtual machines, such as simulations of computer systems, operating systems, servers, etc. The computing device may load a virtual machine based on a virtual image and / or other data defining specific software (e.g., operating system, dedicated application, server) for simulation. As the demand for different types of processing services changes, different virtual machines may be loaded and / or terminated on one or more computing devices. A hypervisor may be implemented to manage the use of different virtual machines on the same computing device.

[0036] The service platform 2 may be configured to communicate with the client 6 or the like via a network 4. The network 4 includes various network devices, such as routers, switches, multiplexers, hubs, modems, bridges, repeaters, firewalls, proxy devices, and / or the like. The network 4 may include physical links, such as coaxial cable links, twisted pair cable links, optical fiber links, combinations thereof, and the like, or wireless links, such as cellular links, satellite links, Wi-Fi links, and the like.

[0037] The service platform 2 can provide storage, reading, writing, querying, deleting and other services, such as providing video data transcoding and audio and video asynchrony detection services for the client.

[0038] The client 6 may be an electronic device running an operating system such as Windows, Android™ or iOS, such as a smart phone, a tablet device, a laptop computer, a virtual reality device, a gaming device, a set-top box, a vehicle terminal, or a smart TV. Based on the above operating system, various applications may be run, such as an application for video data transcoding and detecting audio and video asynchrony.

[0039] The client 6 can provide / configure a user access page for controlling the service platform 2 or uploading objects (such as videos), etc.

[0040] It should be noted that the above devices are exemplary, and the number and type of devices are adjustable in different scenarios or according to different needs.

[0041] The following describes the technical solution of the present application through multiple embodiments, taking the service platform 2 or the client 6 as the execution subject. It should be noted that these embodiments can be implemented in a variety of different forms and should not be interpreted as being limited to the embodiments described here.

[0042] Embodiment 1 Figure 2 The flowchart of the method for detecting audio and video asynchrony according to the first embodiment of the present application is schematically shown.

[0043] like Figure 2As shown, the audio-video asynchrony detection method may include steps S200 to S208, wherein: Step S200, obtaining an original video and a target video, wherein the target video is obtained by transcoding the original video.

[0044] Step S202: determining a first scene switching point in the original video and a second scene switching point in the target video.

[0045] Step S204: determining a video transcoding error according to a first video timestamp of the first scene switching point and a second video timestamp of the second scene switching point.

[0046] Step S206: determining an audio transcoding error according to the first audio corresponding to the first video timestamp and the second audio corresponding to the second video timestamp.

[0047] Step S208: Determine whether the target video has audio and video asynchronism according to the video transcoding error and the audio transcoding error.

[0048] Using the scene switching point as an anchor point, the scene switching points of the video before and after transcoding are compared to determine whether the transcoded video has audio and video asynchrony. Therefore, the characteristic of encoding and decoding that does not change scene-related information can be used to automatically detect whether the audio and video are synchronized, improve detection efficiency and the versatility of the detection method, and improve the quality of the transcoded video.

[0049] The following combination Figure 2 , each step in steps S200~S208 and other optional steps are explained in detail.

[0050] Step S200 , obtaining an original video and a target video, wherein the target video is obtained by transcoding the original video.

[0051] In specific implementation, the original video and the target video can be movies, documentaries, animation videos, teaching videos, etc. on a network platform or a video sharing website. According to actual conditions, it can also be other available videos.

[0052] In some embodiments, the transcoding process may be performed using tools such as multimedia processing tools, distributed computing components, or media transcoding service components. In other embodiments, other tools or methods may be used for transcoding.

[0053] Step S202 , determining a first scene switching point in the original video and a second scene switching point in the target video.

[0054] The inventors have found that existing solutions usually rely on manual recognition or complex algorithms to locate key frames when detecting whether the audio and video after transcoding are out of sync. This method is not only inefficient, but also easily interfered by human factors, resulting in inconsistency and inaccuracy in the detection results.

[0055] In view of this, in this embodiment, the scene switching point is used as an anchor point, and the scene switching points of the video before and after transcoding are compared to determine whether the transcoded video has audio and video asynchrony. In this embodiment, the characteristic that video transcoding does not change the scene-related information is used to achieve accurate scene and picture matching, so that the matched picture is used to detect the video synchronization before and after transcoding, without the need to input additional parameters, so that it can adapt to various types of videos. In this way, the automation and versatility of the audio and video asynchrony detection method in the embodiment can be improved, while ensuring the accuracy of the detection result.

[0056] In an optional embodiment, if Figure 3 As shown, step S202 includes: S300, obtaining multiple video frames of a video to be detected, where the video to be detected is the original video or the target video.

[0057] S302: Determine a scene index of each of the video frames.

[0058] S304: When the scene index is greater than a preset threshold, determine the video frame as a target scene switching point of the video to be detected.

[0059] When the video to be detected is the original video, the target scene switching point is the first scene switching point; when the video to be detected is the target video, the target scene switching point is the second scene switching point.

[0060] In specific implementation, Scenecut can be used for video detection to obtain the scene value of the video frame as the scene index, and the preset threshold can be set to 0.9. According to actual needs, other methods can also be used to determine the scene switching point, and correspondingly other data can also be used as the scene index, and the preset threshold can be set to other values.

[0061] In this embodiment, by quantifying the scene index of each video frame, the scene switching point in the video can be accurately determined, thereby providing a reliable anchor point for audio and video asynchrony detection and ensuring the accuracy and automation of the detection process.

[0062] Step S204 , determining a video transcoding error according to a first video timestamp of the first scene switching point and a second video timestamp of the second scene switching point.

[0063] In this embodiment, the error generated during the transcoding process is determined based on the timestamps of the same scene switching point in the video before and after transcoding. Thus, the accuracy of video transcoding can be quantitatively evaluated, and the audio and video asynchrony problem introduced by transcoding can be discovered and corrected in a timely manner.

[0064] In an optional embodiment, a plurality of the first scene switching points correspond one-to-one to a plurality of the second scene switching points, such as Figure 4 As shown, the method also includes: S400: Determine a video transcoding error sequence according to a first video timestamp of each of the first scene switching points and a second video timestamp of each of the second scene switching points.

[0065] S402: Determine whether cumulative errors occur in the target video during the transcoding process according to the video transcoding error sequence.

[0066] In some embodiments, if the video transcoding error sequence is an increasing sequence, it can be determined that the target video has accumulated errors during the transcoding process. In other embodiments, if the video transcoding errors are all fixed values, it can be determined that the target video has audio and video asynchronism from the beginning, that is, there is a starting error. According to actual needs, the transcoding quality of the target video can also be adaptively determined based on the actual situation of the video transcoding error sequence. The example flow chart for determining the video transcoding error sequence in this embodiment is shown in FIG. Figure 5 shown.

[0067] In this embodiment, multiple scene switching points in the video are used to determine whether there is a cumulative error in the target video, and the synchronization error in the transcoding process can be quantified to highlight the obvious audio and video asynchrony phenomenon caused by the error accumulation, thereby more accurately and comprehensively evaluating the quality of video transcoding.

[0068] In an optional embodiment, the number of the plurality of first scene switching points is different from the number of the plurality of second scene switching points. Figure 6 As shown, the method also includes: S600, sequentially matching the first scene switching point with each of the second scene switching points; S602, obtaining multiple sets of matching first scene switching points and second scene switching points; S604: Acquire the video transcoding error sequence according to the video timestamp difference between each group of matched first scene switching points and second scene switching points.

[0069] In specific implementation, the PSNR method can be used to match the scene switching points. According to actual needs, other methods can also be used for matching. Figure 7 shown.

[0070] In this embodiment, by matching different numbers of first scene switching points and second scene switching points, scene switching points that only exist in the video before or after transcoding can be excluded. Thus, a video transcoding error sequence can be accurately generated, and the process of detecting the asynchrony between audio and video can be further optimized, thereby improving the accuracy and efficiency of detection.

[0071] Step S206 , determining an audio transcoding error according to a first audio corresponding to the first video timestamp and a second audio corresponding to the second video timestamp.

[0072] In a specific implementation, the first audio may be the audio of the original video near the first video timestamp. For example, if the first video timestamp is 1 minute 15 seconds, the first audio may be the audio from 1 minute 10 seconds to 1 minute 20 seconds of the original video. Correspondingly, the second audio may be the audio of the target video near the second video timestamp. Other methods may also be used to determine the first audio and the second audio according to actual needs.

[0073] In this embodiment, the audio near the scene switching point is compared to determine whether an audio error occurs during the transcoding process, thereby ensuring the accuracy and quality of audio transcoding and providing a reliable audio quality assessment for the video transcoding effect.

[0074] In an optional embodiment, if Figure 8 As shown, step S206 includes: S800: Determine, according to the first audio, a first feature sequence corresponding to the first audio, where the first feature sequence corresponds to a first audio timestamp sequence.

[0075] S802: Determine, according to the second audio, a second feature sequence corresponding to the second audio, where the second feature sequence corresponds to a second audio timestamp sequence.

[0076] S804: Determine the audio transcoding error according to the first feature sequence and the second feature sequence.

[0077] In specific implementation, a vector set or sequence set of audio can be extracted by a vectorization or serialization method to form a feature sequence. According to actual needs, the feature sequence of audio can also be determined by other methods. Fig. 9 shown.

[0078] In this embodiment, the audio is analyzed for errors during the transcoding process based on the feature sequences of the audio before and after transcoding, thereby evaluating the transcoding quality, timely discovering and correcting potential audio problems, and ensuring the accuracy of audio transcoding.

[0079] Step S208 , according to the video transcoding error and the audio transcoding error, determine whether the target video has audio and video asynchronism.

[0080] In specific implementation, the video transcoding error and the audio transcoding error may be the error of a certain scene switching point in the video, or may be an error sequence of all or part of the scene switching points in the video. Preferably, the errors of the scene switching points in the second half of the video may be selected for comparison, so as to take advantage of the fact that the second half of the video is more likely to have obvious cumulative errors. According to actual needs, scene switching points located at other locations may also be used for comparison.

[0081] In this embodiment, by comprehensively considering the obtained video transcoding error and audio transcoding error, the audio and video synchronization of the target video can be accurately grasped, the detection efficiency of audio and video asynchrony and the video playback quality can be improved, and an important basis for subsequent video optimization and adjustment can be provided. The following is an example to illustrate the method for determining audio and video asynchrony.

[0082] In an optional embodiment, if Fig.10 As shown, step S208 includes: S1000: Determine whether the video transcoding error is the same as the audio transcoding error.

[0083] S1002: When the video transcoding error is the same as the audio transcoding error, determine whether the video transcoding error is less than a preset standard.

[0084] S1004: When the video transcoding error is less than the preset standard, determine that the target video does not have audio and video asynchronism.

[0085] In actual situations, if the difference between the video transcoding error and the audio transcoding error is very small, it can be considered that it does not affect the audience's viewing experience, and the two can be considered the same. According to actual needs, other judgment criteria can also be set for whether the two are the same.

[0086] In this embodiment, the video with the same video transcoding error and audio transcoding error is judged as not having the phenomenon of audio and video asynchrony, which can improve the accuracy of video quality detection and video playback quality. At the same time, setting relevant standards to limit the transcoding error from being too large can ensure the smoothness and stability of video playback, improve the audience's viewing experience, and reduce playback problems caused by transcoding errors.

[0087] In order to make this application easier to understand, the following Fig.11 An exemplary application is provided. S11, obtaining an original video A and a target video B, where the target video B is obtained by transcoding the original video A.

[0088] S12, using the scenecut algorithm to analyze each video frame in the original video A and the target video B, extracting 3 and 4 frames with scene values ​​greater than 0.9 respectively, thereby determining the 3 first scene switching points in the original video A and the 4 second scene switching points in the target video B.

[0089] S13, using a PSNR method to match the three first scene switching points and the four second scene switching points, to obtain three groups of corresponding first scene switching points and second scene switching points.

[0090] S14, according to the timestamps corresponding to the first scene switching point and the second scene switching point, determining that the video transcoding error sequence is 0ms, 0.5ms, and 0.8ms. It can be determined that the target video B has generated cumulative errors during the transcoding process.

[0091] S15: Determine three corresponding groups of first audio and second audio according to the timestamps corresponding to the first scene switching point and the second scene switching point.

[0092] S16, vectorizing the first audio and the second audio respectively to obtain three corresponding groups of feature vectors.

[0093] S17, determining the audio transcoding error sequence as 0ms, 0.5ms, and 0.8ms according to the obtained three sets of feature sequences.

[0094] S18, it can be determined that the video transcoding error and the audio transcoding error of the target video B are the same and smaller than the preset standard of 0.1ms, and it is determined that the target video B does not have audio and video asynchronism during the transcoding process.

[0095] Embodiment 2 Fig.12 The block diagram of the audio-video asynchrony detection device according to the second embodiment of the present application is schematically shown. The device can be divided into one or more program modules, one or more program modules are stored in a storage medium, and are executed by one or more processors to complete the embodiment of the present application. The program module referred to in the embodiment of the present application refers to a series of computer program instruction segments that can complete specific functions. The following description will specifically introduce the functions of each program module in this embodiment. Fig.12 As shown, the apparatus 1000 may include: an acquisition module 1100, a first determination module 1200, a second determination module 1300, a third determination module 1400, and a fourth determination module 1500, wherein: An acquisition module 1100 is used to acquire an original video and a target video, wherein the target video is obtained by transcoding the original video; A first determining module 1200, configured to determine a first scene switching point in the original video and a second scene switching point in the target video; A second determining module 1300, configured to determine a video transcoding error according to a first video timestamp of the first scene switching point and a second video timestamp of the second scene switching point; A third determination module 1400, configured to determine an audio transcoding error according to a first audio corresponding to the first video timestamp and a second audio corresponding to the second video timestamp; The fourth determination module 1500 is configured to determine whether the target video has audio and video asynchronism according to the video transcoding error and the audio transcoding error.

[0096] As an optional embodiment, the first determining module 1200 is further configured to: Acquire multiple video frames of a video to be detected, where the video to be detected is the original video or the target video; Determining a scene index for each of the video frames; When the scene index is greater than a preset threshold, determining the video frame as a target scene switching point of the video to be detected; When the video to be detected is the original video, the target scene switching point is the first scene switching point; when the video to be detected is the target video, the target scene switching point is the second scene switching point.

[0097] As an optional embodiment, a plurality of the first scene switching points correspond one-to-one to a plurality of the second scene switching points, and the apparatus further includes a fifth determining module, which is used to: determining a video transcoding error sequence according to a first video timestamp of each of the first scene switching points and a second video timestamp of each of the second scene switching points; According to the video transcoding error sequence, it is determined whether cumulative errors occur in the target video during the transcoding process.

[0098] As an optional embodiment, the number of the plurality of first scene switching points is different from the number of the plurality of second scene switching points, and the apparatus further includes a sixth determining module, configured to: sequentially matching the first scene switching point with each of the second scene switching points; Obtain multiple sets of matching first scene switching points and second scene switching points; The video transcoding error sequence is obtained according to the video timestamp difference between each group of matched first scene switching points and second scene switching points.

[0099] As an optional embodiment, the third determining module 1400 is further configured to: Determine, according to the first audio, a first feature sequence corresponding to the first audio, wherein the first feature sequence corresponds to a first audio timestamp sequence; Determine, according to the second audio, a second feature sequence corresponding to the second audio, wherein the second feature sequence corresponds to a second audio timestamp sequence; The audio transcoding error is determined according to the first feature sequence and the second feature sequence.

[0100] As an optional embodiment, the fourth determining module 1500 is further configured to: Determining whether the video transcoding error is the same as the audio transcoding error; In a case where the video transcoding error is the same as the audio transcoding error, determining whether the video transcoding error is less than a preset threshold; When the video transcoding error is less than the preset threshold, it is determined that the target video does not have audio and video asynchronism.

[0101] Embodiment 3 Fig.13 The hardware architecture diagram of a computer device 10000 suitable for implementing the audio and video asynchrony detection method according to the third embodiment of the present application is schematically shown. In some embodiments, the computer device 10000 may be a terminal device such as a smart phone, a wearable device, a tablet computer, a personal computer, a vehicle-mounted terminal, a game console, a virtual device, a workbench, a digital assistant, a set-top box, a robot, etc. In other embodiments, the computer device 10000 may be a rack server, a blade server, a tower server, or a cabinet server (including an independent server, or a server cluster composed of multiple servers), etc. Fig.13 As shown, the computer device 10000 includes but is not limited to: a memory 10010, a processor 10020, and a network interface 10030 that can communicate with each other through a system bus. Among them: The memory 10010 includes at least one type of computer-readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (such as SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 10010 can be an internal storage module of the computer device 10000, such as a hard disk or memory of the computer device 10000. In other embodiments, the memory 10010 can also be an external storage device of the computer device 10000, such as a plug-in hard disk equipped on the computer device 10000, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Of course, the memory 10010 can also include both the internal storage module of the computer device 10000 and its external storage device. In this embodiment, the memory 10010 is generally used to store the operating system and various application software installed in the computer device 10000, such as the program code of the audio and video asynchrony detection method, etc. In addition, the memory 10010 can also be used to temporarily store various data that have been output or will be output.

[0102] In some embodiments, the processor 10020 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other chips. The processor 10020 is generally used to control the overall operation of the computer device 10000, such as performing control and processing related to data interaction or communication with the computer device 10000. In this embodiment, the processor 10020 is used to run the program code stored in the memory 10010 or process data.

[0103] The network interface 10030 may include a wireless network interface or a wired network interface, and the network interface 10030 is generally used to establish a communication link between the computer device 10000 and other computer devices. For example, the network interface 10030 is used to connect the computer device 10000 to an external terminal through a network, and to establish a data transmission channel and a communication link between the computer device 10000 and the external terminal. The network may be a wireless or wired network such as an intranet, the Internet, the Global System of Mobile communication (GSM), Wideband Code Division Multiple Access (WCDMA), 4G network, 5G network, Bluetooth, Wi-Fi, etc.

[0104] It should be pointed out that Fig.13 Only a computer device having components 10010 - 10030 is shown, but it should be understood that implementation of all of the components shown is not a requirement, and more or fewer components may alternatively be implemented.

[0105] In this embodiment, the audio and video asynchrony detection method stored in the memory 10010 can also be divided into one or more program modules and executed by one or more processors (such as processor 10020) to complete the embodiment of the present application.

[0106] Embodiment 4 An embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the audio and video asynchrony detection method in the embodiment are implemented.

[0107] In this embodiment, the computer-readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the computer-readable storage medium may be an internal storage unit of a computer device, such as a hard disk or memory of the computer device. In other embodiments, the computer-readable storage medium may also be an external storage device of a computer device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the computer device. Of course, the computer-readable storage medium may also include both an internal storage unit of a computer device and an external storage device thereof. In this embodiment, the computer-readable storage medium is generally used to store an operating system and various application software installed on a computer device, such as the program code of the audio and video asynchrony detection method in the embodiment. In addition, the computer-readable storage medium may also be used to temporarily store various types of data that have been output or are to be output.

[0108] Embodiment 5 An embodiment of the present application also provides a computer program product, including a computer program, which implements the method in the above embodiment when executed by a processor.

[0109] Obviously, those skilled in the art should understand that the modules or steps of the above-mentioned embodiments of the present application can be implemented by general-purpose computer devices, they can be concentrated on a single computer device, or distributed on a network composed of multiple computer devices, optionally, they can be implemented by executable program codes of computer devices, so that they can be stored in a storage device and executed by the computer device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. In this way, the embodiments of the present application are not limited to any specific combination of hardware and software.

[0110] It should be noted that the above are only preferred embodiments of the present application, and the patent protection scope of the present application is not limited thereto. Any equivalent structure or equivalent process transformation made using the contents of the specification and drawings of the present application, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for detecting audio and video asynchrony, characterized in that: The method comprises: Obtaining an original video and a target video, wherein the target video is obtained by transcoding the original video; Determining a first scene switching point in the original video and a second scene switching point in the target video; Determine a video transcoding error according to a first video timestamp of the first scene switching point and a second video timestamp of the second scene switching point; determining an audio transcoding error according to a first audio corresponding to the first video timestamp and a second audio corresponding to the second video timestamp; It is determined whether the target video has audio and video asynchronism according to the video transcoding error and the audio transcoding error.

2. The method according to claim 1, characterized in that: Determining a first scene switching point in the original video and a second scene switching point in the target video includes: Acquire multiple video frames of a video to be detected, where the video to be detected is the original video or the target video; Determining a scene index for each of the video frames; When the scene index is greater than a preset threshold, determining the video frame as a target scene switching point of the video to be detected; When the video to be detected is the original video, the target scene switching point is the first scene switching point; when the video to be detected is the target video, the target scene switching point is the second scene switching point.

3. The method according to claim 1, characterized in that The plurality of first scene switching points correspond one-to-one to the plurality of second scene switching points; the method further comprises: determining a video transcoding error sequence according to a first video timestamp of each of the first scene switching points and a second video timestamp of each of the second scene switching points; According to the video transcoding error sequence, it is determined whether cumulative errors occur in the target video during the transcoding process.

4. The method according to claim 1, characterized in that: The number of the first scene switching points is different from the number of the second scene switching points; and the method further includes: sequentially matching the first scene switching point with each of the second scene switching points; Obtain multiple sets of matching first scene switching points and second scene switching points; The video transcoding error sequence is obtained according to the video timestamp difference between each group of matched first scene switching points and second scene switching points.

5. The method according to any one of claims 1 to 4, characterized in that: Determining an audio transcoding error according to a first audio corresponding to the first video timestamp and a second audio corresponding to the second video timestamp includes: Determine, according to the first audio, a first feature sequence corresponding to the first audio, wherein the first feature sequence corresponds to a first audio timestamp sequence; Determine, according to the second audio, a second feature sequence corresponding to the second audio, wherein the second feature sequence corresponds to a second audio timestamp sequence; The audio transcoding error is determined according to the first feature sequence and the second feature sequence.

6. The method according to any one of claims 1 to 4, characterized in that: Determining whether the target video has audio and video asynchronism according to the video transcoding error and the audio transcoding error includes: Determining whether the video transcoding error is the same as the audio transcoding error; In a case where the video transcoding error is the same as the audio transcoding error, determining whether the video transcoding error is less than a preset standard; When the video transcoding error is less than the preset standard, it is determined that the target video does not have audio and video asynchronism.

7. A device for detecting asynchronism between audio and video, characterized in that: The device comprises: An acquisition module, used to acquire an original video and a target video, wherein the target video is obtained by transcoding the original video; A first determining module, used to determine a first scene switching point in the original video and a second scene switching point in the target video; A second determining module, configured to determine a video transcoding error according to a first video timestamp of the first scene switching point and a second video timestamp of the second scene switching point; A third determination module, configured to determine an audio transcoding error according to a first audio corresponding to the first video timestamp and a second audio corresponding to the second video timestamp; The fourth determination module is used to determine whether the target video has audio and video asynchronism according to the video transcoding error and the audio transcoding error.

8. A computer device, characterized in that: include: at least one processor; and a memory communicatively connected to the at least one processor; wherein: The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the method according to any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to claims 1 to 6 are implemented.

Images