Method, device and equipment for detecting quality of audio and video data

Abstract

The application provides a quality detection method, device and equipment of audio and video data, and relates to big data technology. The method comprises the following steps: obtaining a quality detection request; the quality detection request comprises a data identifier to be detected. Based on the quality detection request, the data to be detected corresponding to the data identifier to be detected is extracted and processed to obtain audio data and video data. The audio data is converted and processed to obtain text information. According to preset key speech information, the text information is matched with the key speech to determine a time set of the key speech. The time set comprises a plurality of time intervals of the key speech. According to the plurality of time intervals in the time set, video frames corresponding to each time interval in the plurality of time intervals are extracted from the video data, each video frame in each time interval is detected, and detection result information is obtained. The method provided by the application reduces the number of key frames of the video data and solves the technical problem that the accuracy of the quality detection of the audio and video data is low.

Description

Technical Field

[0001] This application relates to big data technology, and in particular to a method, apparatus and equipment for quality detection of audio and video data. Background Technology

[0002] Currently, with increasingly stringent regulations on the sale of banking financial products, the China Banking Regulatory Commission (CBRC) requires banking financial institutions to record both video and audio recordings of the sales process of their own wealth management products and products sold on behalf of others. Based on the "dual recording" business scenario, a large number of audio and video files will be generated, and banks must conduct quality inspections on these files to ensure that the "dual recording" business is implemented in a standardized manner.

[0003] In current dual-recording quality inspection solutions, the process is primarily accomplished through a combination of automated system inspection and manual inspection. The system first automates the inspection, and then manual review is conducted on documents that fail or are uncertain. Documents that pass the inspection are also subject to random checks to ensure accuracy.

[0004] However, in the existing technology, the accuracy of quality inspection is relatively low because a large amount of manual labor is still required. Summary of the Invention

[0005] This application provides a method, apparatus, and equipment for quality inspection of audio and video data, in order to solve the technical problem of low accuracy in quality inspection of audio and video data.

[0006] In a first aspect, this application provides a method for quality detection of audio and video data, including:

[0007] Obtain a quality inspection request; wherein, the quality inspection request includes an identifier for the data to be inspected;

[0008] Based on the quality inspection request, the data to be inspected corresponding to the data identifier to be inspected is extracted and processed to obtain audio data and video data.

[0009] The audio data is converted to obtain text information; and based on preset key dialogue information, the text information is matched with key dialogue information to determine the time set in which the matched key dialogue appears; wherein, the time set includes multiple time intervals in which the key dialogue appears;

[0010] Based on the multiple time intervals in the time set, video frames corresponding to each of the multiple time intervals are extracted from the video data, and the video frames of each time interval are detected to obtain detection result information; wherein, the detection result information indicates whether the business corresponding to the data to be inspected is compliant in the processing process.

[0011] Furthermore, the video frames in each time interval are detected to obtain detection result information, including:

[0012] The video frames of each time interval are saved as images, and a set of keyframes for each time interval is generated based on the images.

[0013] Based on the preset mapping relationship between time nodes and image recognition models, the target image recognition model corresponding to each time interval is determined;

[0014] Based on the mapping relationship between the keyframe set of each time interval and the target image recognition model corresponding to each time interval, the keyframe set of the corresponding time interval is detected by the target image recognition model corresponding to each time interval to obtain detection result information; wherein, the preset image recognition model is trained based on key behaviors.

[0015] Furthermore, the audio data is converted to obtain text information, including:

[0016] The audio data is extracted according to the timbre in the audio data to obtain the first audio with different timbres, and the role indicated by the first audio with different timbres is determined.

[0017] The first audio of each character is converted to obtain the text information of each character.

[0018] Further, the step of performing key dialogue matching on the text information based on preset key dialogue information to determine the time set of occurrence of the matched key dialogue includes:

[0019] Based on the preset key dialogue information, search for key dialogues that match the preset key dialogue information in the text information of each character, and determine the time set in which the matched key dialogues appear.

[0020] Furthermore, after converting the audio data to obtain text information, the process also includes:

[0021] Based on preset compliance rules, compliance checks are performed on the text information of each role to obtain compliance check information for the text information of each role; wherein, the compliance check information indicates whether the text is compliant.

[0022] Further, the step of performing key dialogue matching on the text information based on preset key dialogue information to determine the time set of occurrence of the matched key dialogue includes:

[0023] If it is determined that the compliance detection information of the text information of each role represents text compliance, then for the text information of each role, according to the preset key dialogue information, the text information after compliance detection is matched with key dialogue information to determine the time set of the matched key dialogue.

[0024] Secondly, this application provides an audio / video data quality detection device, comprising:

[0025] An acquisition unit is used to acquire a quality inspection request; wherein the quality inspection request includes an identifier of the data to be inspected.

[0026] The first extraction unit is used to extract and process the data to be inspected corresponding to the data identifier to be inspected based on the quality inspection request, so as to obtain audio data and video data.

[0027] The conversion unit is used to convert the audio data to obtain text information;

[0028] The matching unit is used to perform key word matching on the text information according to preset key word information, and determine the time set of the matched key words appearing; wherein, the time set includes multiple time intervals in which the key words appear.

[0029] The second extraction unit is used to extract video frames corresponding to each of the multiple time intervals in the video data based on the multiple time intervals in the time set.

[0030] The detection unit is used to detect video frames in each time interval and obtain detection result information; wherein, the detection result information indicates whether the business corresponding to the data to be inspected is compliant in the process.

[0031] Furthermore, the detection unit includes:

[0032] The generation module is used to save the video frames of each time interval as images, and generate a set of keyframes for each time interval based on the images.

[0033] The determination module is used to determine the target image recognition model corresponding to each time interval based on the preset mapping relationship between time nodes and image recognition models;

[0034] The detection module is used to detect the keyframe set of the corresponding time interval based on the mapping relationship between the keyframe set of each time interval and the target image recognition model corresponding to each time interval, and obtain detection result information; wherein, the preset image recognition model is trained based on key behaviors.

[0035] Further, the conversion unit includes:

[0036] The extraction module is used to extract the audio data according to the timbre in the audio data, obtain the first audio with different timbres, and determine the role indicated by the first audio with different timbres;

[0037] The conversion module is used to convert the first audio of each character to obtain the text information of each character.

[0038] Furthermore, the matching unit is specifically used for:

[0039] Based on the preset key dialogue information, search for key dialogues that match the preset key dialogue information in the text information of each character, and determine the time set in which the matched key dialogues appear.

[0040] Furthermore, it also includes:

[0041] The compliance unit is used to perform compliance checks on the text information of each role according to preset compliance rules after converting and processing the audio data to obtain text information, thereby obtaining compliance check information for the text information of each role; wherein, the compliance check information indicates whether the text is compliant.

[0042] Furthermore, the matching unit is specifically used for:

[0043] If it is determined that the compliance detection information of the text information of each role represents text compliance, then for the text information of each role, according to the preset key dialogue information, the text information after compliance detection is matched with key dialogue information to determine the time set of the matched key dialogue.

[0044] Thirdly, this application provides an electronic device, including a memory and a processor, wherein the memory stores a computer program that can run on the processor, and the processor executes the computer program to implement the method described in the first aspect.

[0045] Fourthly, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the method described in the first aspect.

[0046] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the method described in the first aspect.

[0047] This application provides a method, apparatus, and device for quality inspection of audio and video data, which obtains a quality inspection request, wherein the quality inspection request includes an identifier for data to be inspected. Based on the quality inspection request, the data to be inspected corresponding to the identifier is extracted and processed to obtain audio data and video data. The audio data is converted to obtain text information; and according to preset key dialogue information, the text information is matched with key dialogue to determine the time set in which the matched key dialogue appears; wherein the time set includes multiple time intervals in which the key dialogue appears. Based on the multiple time intervals in the time set, video frames corresponding to each of the multiple time intervals are extracted from the video data, and the video frames of each time interval are inspected to obtain inspection result information; wherein the inspection result information indicates whether the business corresponding to the data to be inspected is compliant during the processing. In this solution, based on the identifier for the data to be inspected in the quality inspection request, the data to be inspected corresponding to the identifier is extracted and processed, and audio files are extracted from the data to be inspected to obtain separate audio data and video data. The audio data is converted to obtain converted text information. Based on preset key dialogue information, the system searches for matching key dialogue in the text information and determines the time intervals in which the matched key dialogue appears, thus defining a time set. Based on multiple time intervals within the time set, video frames for each time interval are extracted from the video data. Then, the video frames in each time interval are inspected to obtain inspection results, thereby determining whether the business corresponding to the data to be inspected is compliant during processing. Therefore, by extracting keyframes from video data based on the context of audio data, the number of keyframes in the video data is significantly reduced, improving the accuracy of quality inspection, increasing the efficiency of video quality inspection, shortening the video quality inspection time, and solving the technical problem of low accuracy in audio and video data quality inspection. Attached Figure Description

[0048] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0049] Figure 1 A flowchart illustrating a method for quality detection of audio and video data provided in an embodiment of this application;

[0050] Figure 2 A flowchart illustrating another method for quality detection of audio and video data provided in an embodiment of this application;

[0051] Figure 3 A schematic diagram of the structure of another audio / video data quality detection device provided in an embodiment of this application;

[0052] Figure 4A schematic diagram of the structure of an audio / video data quality detection device provided in an embodiment of this application;

[0053] Figure 5 A schematic diagram of another audio / video data quality detection device provided in an embodiment of this application;

[0054] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0055] Figure 7 This is a block diagram of an electronic device provided in an embodiment of this application.

[0056] The accompanying drawings have illustrated specific embodiments of this disclosure, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concepts of this disclosure to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0057] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure.

[0058] It should be noted that the user information and data involved in this application (including but not limited to data used for analysis, stored data, and displayed data) are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use, and processing of such data must comply with the relevant laws, regulations, and standards of the relevant countries and regions, and corresponding access points are provided for users to choose to authorize or refuse. In other words, the collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution of this application all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0059] Furthermore, the audio and video data quality detection method, apparatus, and equipment disclosed herein can be used in the big data or financial fields, as well as in any other field besides big data and finance. The application areas of the audio and video data quality detection method, apparatus, and equipment disclosed herein are not limited.

[0060] Currently, with increasingly stringent regulations on the sale of banking financial products, the China Banking Regulatory Commission (CBRC) requires banking financial institutions to record both video and audio recordings of the sales process of their own wealth management products and products sold on behalf of others. Based on the "dual recording" business scenario, a large number of audio and video files will be generated, and banks must conduct quality inspections on these files to ensure that the "dual recording" business is implemented in a standardized manner.

[0061] In one example, current dual-recording quality inspection solutions primarily combine intelligent system inspection with manual inspection. The system first automates the inspection process, then manually reviews files that fail or are uncertain, and randomly checks files that pass inspection to ensure accuracy. Specifically, a typical audio / video dual-recording quality inspection solution consists of two parts: video inspection and audio inspection. First, audio is extracted from the video. Then, speech recognition technology converts the audio into text, which is then inspected for quality indicators such as sensitive words and key phrases. Finally, for video inspection, the video file is broken down frame by frame into image frames. These image frames are then fed into an image recognition model for action recognition. The image recognition model processes these images for quality inspection (e.g., signatures, document presentation). However, in existing technologies, conventional audio and video dual-recording quality inspection solutions often result in many false alarms when detecting the full amount of text generated by the audio quality inspection. These false alarms increase the workload of manual review, requiring a large amount of manpower to be invested in the quality inspection work, thus resulting in low accuracy.

[0062] In one example, for audio and video files, a large number of video frame images are generated during quality inspection. The automated quality inspection of the system requires a lot of computation and comparison. The image recognition model takes a very long time to process the image set, resulting in excessively long batch program execution time and thus low quality inspection efficiency.

[0063] This application provides a method, apparatus, and equipment for quality testing of audio and video data, which aims to solve the above-mentioned technical problems in the prior art.

[0064] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0065] Figure 1 This is a flowchart illustrating a method for quality detection of audio and video data provided in an embodiment of this application, as shown below. Figure 1 As shown, the method includes:

[0066] Step 101: Obtain the quality inspection request; wherein, the quality inspection request includes the identifier of the data to be inspected.

[0067] For example, the executing entity of this embodiment can be an electronic device, a terminal device, an audio / video data quality detection device or device, or other device or device capable of executing this embodiment, and there is no limitation thereto. In this embodiment, the executing entity is described as an electronic device.

[0068] First, a quality inspection request needs to be obtained, which includes an identifier for the data to be inspected. This can be done in response to a user's request on the electronic device, or by receiving quality inspection requests transmitted from other devices. Once the electronic device obtains the quality inspection request, it can locate the data to be inspected corresponding to the identifier in the request. This data can be audio / video data, video data, etc., and the format can be Moving Picture Experts Group 4 (MP4) format, etc.

[0069] Step 102: Based on the quality inspection request, extract and process the data to be inspected corresponding to the data identifier to be inspected, and obtain audio data and video data.

[0070] For example, based on a quality inspection request, the electronic device extracts the data to be inspected corresponding to the data identifier to be inspected, and extracts audio files from the data to be inspected. For example, the audio in the MP4 format data to be inspected is extracted in PCM format, and finally, the separated Pulse Code Modulation (PCM) format audio data and MP4 format video data are obtained.

[0071] Step 103: Convert the audio data to obtain text information; and match the text information with key phrases based on preset key phrase information to determine the time set in which the matched key phrases appear; wherein, the time set includes multiple time intervals in which the key phrases appear.

[0072] For example, the preset key dialogue information is pre-stored information containing multiple key dialogues. The electronic device converts the audio data to obtain converted text information. Based on the preset key dialogue information, it searches for key dialogues that match the preset key dialogue information in the text information, determines the time interval in which the matched key dialogues appear, and then determines the time set P, P = [(T1,T2),(T3,T4)...(T... ... n-1 ,T n The time unit for a time interval can be seconds or minutes, etc.

[0073] Step 104: Based on multiple time intervals in the time set, extract the video frames corresponding to each time interval in the video data, and detect the video frames in each time interval to obtain detection result information; wherein, the detection result information indicates whether the business corresponding to the data to be inspected is compliant in the process.

[0074] For example, the electronic device extracts video frames from MP4 format video data based on multiple time intervals in the time set P and the time units of the time intervals. Then, the video frames in each time interval are inspected to obtain inspection result information. The inspection result information indicates whether the business corresponding to the data to be inspected is compliant during the processing.

[0075] In this embodiment, a quality inspection request is obtained; wherein, the quality inspection request includes an identifier for data to be inspected. Based on the quality inspection request, the data to be inspected corresponding to the identifier is extracted and processed to obtain audio data and video data. The audio data is converted to obtain text information; and according to preset key dialogue information, the text information is matched with key dialogue to determine the time set in which the matched key dialogue appears; wherein, the time set includes multiple time intervals in which the key dialogue appears. Based on the multiple time intervals in the time set, video frames corresponding to each time interval are extracted from the video data, and the video frames of each time interval are detected to obtain detection result information; wherein, the detection result information indicates whether the business corresponding to the data to be inspected is compliant during the processing. In this solution, based on the identifier for data to be inspected in the quality inspection request, the data to be inspected corresponding to the identifier is extracted and processed, and audio files are extracted from the data to be inspected to obtain separate audio data and video data. The audio data is converted to obtain converted text information. Based on preset key dialogue information, the system searches for matching key dialogue in the text information and determines the time intervals in which the matched key dialogue appears, thus defining a time set. Based on multiple time intervals within the time set, video frames for each time interval are extracted from the video data. Then, the video frames in each time interval are inspected to obtain inspection results, thereby determining whether the business corresponding to the data to be inspected is compliant during processing. Therefore, by extracting keyframes from video data based on the context of audio data, the number of keyframes in the video data is significantly reduced, improving the accuracy of quality inspection, increasing the efficiency of video quality inspection, shortening the video quality inspection time, and solving the technical problem of low accuracy in audio and video data quality inspection.

[0076] Figure 2 A flowchart illustrating another audio / video data quality detection method provided in this application embodiment is shown below. Figure 2 As shown, the method includes:

[0077] Step 201: Obtain the quality inspection request; wherein, the quality inspection request includes the identifier of the data to be inspected.

[0078] For example, this step can be referred to Figure 1 Step 101 in the text will not be repeated here.

[0079] Step 202: Based on the quality inspection request, extract and process the data to be inspected corresponding to the data identifier to be inspected, and obtain audio data and video data.

[0080] For example, this step can be referred to Figure 1 Step 102 in the text will not be repeated here.

[0081] Step 203: Extract the audio data according to the timbre in the audio data, obtain the first audio with different timbres, and determine the role indicated by the first audio with different timbres.

[0082] For example, the electronic device can first extract the audio data by timbre, obtain the first audio with different timbres, and determine the role indicated by the first audio with different timbres. That is, the first audio includes the business audio of the business personnel and the customer audio of the customer. Then, the role of the business personnel is determined according to the business audio, and the role of the customer is determined according to the customer audio.

[0083] Step 204: Convert the first audio of each character to obtain the text information of each character.

[0084] For example, an electronic device can call an ASR (speech recognition, speech-to-text) interface to convert the first audio of each character into text, thereby obtaining text information for different characters.

[0085] Step 205: Based on the preset compliance rules, perform compliance checks on the text information of each role to obtain compliance check information for the text information of each role; wherein, the compliance check information indicates whether the text is compliant.

[0086] For example, the preset compliance rule information is pre-stored information, including sensitive words that should not appear. Based on the preset compliance rule information, the electronic device can compare the text information of each role with the preset compliance rule information. If it is determined that the text information does not contain the sensitive words in the compliance rule information, compliance detection information indicating text compliance is obtained; if it is determined that the text information contains the sensitive words in the compliance rule information, compliance detection information indicating text non-compliance is obtained.

[0087] For example, when the generated text information is checked by role, the text information is first segmented into words, and then the string is matched according to the preset compliance rules to determine whether the business is compliant, and finally compliance detection information is obtained.

[0088] Step 206: Based on the preset key dialogue information, perform key dialogue matching on the text information to determine the time set of the matched key dialogues; wherein, the time set includes multiple time intervals in which the key dialogues appear.

[0089] Step 206 includes two implementation methods:

[0090] The first implementation method of step 206: Based on the preset key dialogue information, search for key dialogues that match the preset key dialogue information in the text information of each role, and determine the time set of the matched key dialogues.

[0091] The second implementation of step 206: If it is determined that the compliance detection information of the text information of each role represents text compliance, then for the text information of each role, according to the preset key dialogue information, the text information after compliance detection is matched with key dialogue information to determine the time set of the matched key dialogue.

[0092] For example, before performing key phrase matching, it can be first determined whether the compliance check information of each role's text information is compliant. If the compliance check information of each role's text information is determined to be compliant, key phrase matching can proceed. If the compliance check information of any role's text information is determined to be non-compliant, the non-compliant text information needs to be revised to obtain revised text information. The revised text information then undergoes compliance checks until the compliance check information of the revised text information is determined to be compliant. The revision method can be automatic or manual. Alternatively, if the compliance check information of any role's text information is determined to be non-compliant, the next step of key phrase matching cannot proceed.

[0093] Because there is text information from multiple roles, the electronic device can perform key phrase matching on the text information of different roles separately. Specifically, based on preset key phrase information, the electronic device searches for key phrases that match the preset key phrase information in the text information of each role, determines the time interval in which the matched key phrases appear, and then determines the time set P corresponding to the text information of each role, P = [(T1,T2),(T3,T4)...(T... n-1 ,T n The time unit for a time interval can be seconds or minutes, etc.

[0094] Step 207: Based on the multiple time intervals in the time set, extract the video frames corresponding to each of the multiple time intervals from the video data.

[0095] For example, an electronic device can extract video frames second by second from MP4 format video data based on the time intervals recorded in the time set P.

[0096] Step 208: Save the video frames of each time interval as images, and generate a set of keyframes for each time interval based on the images.

[0097] For example, an electronic device can save video frames for each time interval as images, and generate a set of keyframes for each time interval based on the images for each time interval. The image format can be JPG (Joint Photographic Experts Group) or similar.

[0098] Step 209: Determine the target image recognition model corresponding to each time interval based on the preset mapping relationship between time nodes and image recognition models.

[0099] For example, an electronic device can determine the target image recognition model Q = [V1, V2...V] corresponding to each time interval based on the preset mapping relationship between time nodes and image recognition models. m ].

[0100] Step 210: Based on the mapping relationship between the keyframe set of each time interval and the target image recognition model corresponding to each time interval, the keyframe set of the corresponding time interval is detected by the target image recognition model corresponding to each time interval to obtain the detection result information; wherein, the preset image recognition model is trained based on key behaviors; wherein, the detection result information represents whether the business corresponding to the data to be inspected is compliant in the process.

[0101] For example, the preset image recognition model is trained based on key behaviors. Since each time interval corresponds to a set of keyframes and a target image recognition model, the electronic device can determine the mapping relationship between the set of keyframes and the target image recognition model for each time interval. Based on the mapping relationship between the set of keyframes and the target image recognition model for each time interval, the electronic device inputs the set of keyframes for each time interval into the corresponding target image recognition model Q for detection. If no key behavior is detected in the set of keyframes, it is deemed non-compliant; if a key behavior is detected in the set of keyframes, the loop is terminated and the detection of the next time interval is performed until the end, and the detection result information is obtained.

[0102] In this embodiment, a quality inspection request is obtained; wherein, the quality inspection request includes an identifier for data to be inspected. Based on the quality inspection request, the data to be inspected corresponding to the identifier is extracted and processed to obtain audio data and video data. Audio data is extracted according to the timbre in the audio data to obtain first audio with different timbres, and the role indicated by the first audio with different timbres is determined. The first audio of each role is converted to obtain text information for each role. According to preset compliance rules, compliance detection is performed on the text information of each role to obtain compliance detection information for the text information of each role; wherein, the compliance detection information indicates whether the text is compliant. According to preset key phrase information, key phrase matching is performed on the text information to determine the time set in which the matched key phrases appear; wherein, the time set includes multiple time intervals in which the key phrases appear. Based on the multiple time intervals in the time set, video frames corresponding to each time interval are extracted from the video data. The video frames of each time interval are saved as images, and a set of key frames for each time interval is generated based on the images. According to the preset mapping relationship between time nodes and image recognition models, the target image recognition model corresponding to each time interval is determined. Based on the mapping relationship between the keyframe set of each time interval and the corresponding target image recognition model, the keyframe set of the corresponding time interval is detected using the target image recognition model, and the detection result information is obtained. The preset image recognition model is trained based on key behaviors. The detection result information represents whether the business corresponding to the data to be inspected is compliant during the processing. Therefore, based on the context of the audio data, keyframe extraction in the video data significantly reduces the number of keyframes in the video data, improves the accuracy of quality inspection, increases the efficiency of video quality inspection, shortens the video quality inspection time, and solves the technical problem of low accuracy in audio and video data quality inspection. Simultaneously, by analyzing audio files, preprocessing audio and video files, and combining the "dual recording" scenario to classify audio files into roles, the false positive rate of text detection is reduced, the processing time of the image recognition model is reduced, and the accuracy of quality inspection is further improved.

[0103] Figure 3 A schematic diagram of the structure of another audio / video data quality detection device provided in the embodiments of this application is shown below. Figure 3As shown, in the audio and video dual-recording quality inspection of audio and video data, audio is extracted to obtain separate audio and video data; the audio data is preprocessed, including separating the audio data by timbre and confirming the roles, and sending the role-classified audio data to the speech recognition module for ASR speech recognition and role-based text detection. If the text is compliant, the audio data quality inspection ends; the video data is preprocessed, including extracting the corresponding time nodes according to the semantics of the audio data, extracting key frame nodes based on the time nodes, and using an image recognition model to identify whether it is compliant. If compliant, the quality inspection ends; if not compliant, it is manually reviewed.

[0104] Figure 4 This is a schematic diagram of the structure of an audio / video data quality detection device provided in an embodiment of this application, as shown below. Figure 4 As shown, the device includes:

[0105] The acquisition unit 31 is used to acquire a quality inspection request; wherein, the quality inspection request includes an identifier of the data to be inspected.

[0106] The first extraction unit 32 is used to extract and process the data to be inspected corresponding to the data identifier to be inspected based on the quality inspection request, so as to obtain audio data and video data.

[0107] The conversion unit 33 is used to convert and process audio data to obtain text information.

[0108] The matching unit 34 is used to perform key word matching on the text information according to the preset key word information, and determine the time set of the matched key words. The time set includes multiple time intervals in which the key words appear.

[0109] The second extraction unit 35 is used to extract video frames corresponding to each time interval in the video data based on multiple time intervals in the time set.

[0110] The detection unit 36 ​​is used to detect video frames in each time interval and obtain detection result information; wherein, the detection result information indicates whether the business corresponding to the data to be inspected is compliant in the process.

[0111] The apparatus in this embodiment can execute the technical solutions in the above method. Its specific implementation process and technical principles are the same, and will not be repeated here.

[0112] Figure 5 This is a schematic diagram of another audio / video data quality detection device provided in an embodiment of this application. Figure 4 Based on the illustrated embodiments, as Figure 5 As shown, the detection unit 36 ​​includes:

[0113] The generation module 361 is used to save the video frames of each time interval as images, and generate a set of keyframes for each time interval based on the images.

[0114] The determination module 362 is used to determine the target image recognition model corresponding to each time interval based on the preset mapping relationship between time nodes and image recognition models.

[0115] The detection module 363 is used to detect the keyframe set of the corresponding time interval based on the mapping relationship between the keyframe set of each time interval and the target image recognition model corresponding to each time interval, and obtain the detection result information; wherein, the preset image recognition model is trained based on key behaviors.

[0116] In one example, conversion unit 33 includes:

[0117] The extraction module 331 is used to extract audio data according to the timbre in the audio data, obtain the first audio with different timbres, and determine the role of the first audio with different timbres.

[0118] The conversion module 332 is used to convert the first audio of each character to obtain the text information of each character.

[0119] In one example, matching unit 34 is specifically used for:

[0120] Based on the preset key dialogue information, search for key dialogues that match the preset key dialogue information in the text information of each character, and determine the time set in which the matched key dialogues appear.

[0121] One example also includes:

[0122] The compliance unit 41 is used to perform compliance checks on the text information of each role according to the preset compliance rules after converting and processing the audio data to obtain the compliance check information of the text information of each role; wherein, the compliance check information indicates whether the text is compliant.

[0123] In one example, matching unit 34 is specifically used for:

[0124] If it is determined that the compliance detection information of the text information of each role represents text compliance, then for the text information of each role, according to the preset key dialogue information, the text information after compliance detection is matched with key dialogue information to determine the time set of the matched key dialogue.

[0125] The apparatus in this embodiment can execute the technical solutions in the above method. Its specific implementation process and technical principles are the same, and will not be repeated here.

[0126] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application, such as... Figure 6 As shown, the electronic device includes: a memory 51 and a processor 52.

[0127] The memory 51 stores a computer program that can run on the processor 52.

[0128] Processor 52 is configured to perform the methods provided in the embodiments described above.

[0129] The electronic device also includes a receiver 53 and a transmitter 54. The receiver 53 is used to receive instructions and data sent by an external device, and the transmitter 54 is used to send instructions and data to an external device.

[0130] Figure 7 This is a block diagram of an electronic device provided in an embodiment of this application. The electronic device may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, etc.

[0131] The device 600 may include one or more of the following components: a processing component 602, a memory 604, a power supply component 606, a multimedia component 608, an audio component 610, an input / output (I / O) interface 612, a sensor component 614, and a communication component 616.

[0132] Processing component 602 typically controls the overall operation of device 600, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 602 may include one or more processors 620 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 602 may include one or more modules to facilitate interaction between processing component 602 and other components. For example, processing component 602 may include a multimedia module to facilitate interaction between multimedia component 608 and processing component 602.

[0133] Memory 604 is configured to store various types of data to support the operation of device 600. Examples of such data include instructions for any application or method operating on device 600, contact data, phonebook data, messages, pictures, videos, etc. Memory 604 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0134] Power supply component 606 provides power to the various components of device 600. Power supply component 606 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to device 600.

[0135] Multimedia component 608 includes a screen that provides an output interface between device 600 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of touch or swipe actions but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 608 includes a front-facing camera and / or a rear-facing camera. When device 600 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0136] Audio component 610 is configured to output and / or input audio signals. For example, audio component 610 includes a microphone (MIC) configured to receive external audio signals when device 600 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 604 or transmitted via communication component 616. In some embodiments, audio component 610 also includes a speaker for outputting audio signals.

[0137] I / O interface 612 provides an interface between processing component 602 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0138] Sensor assembly 614 includes one or more sensors for providing state assessments of various aspects of device 600. For example, sensor assembly 614 may detect the on / off state of device 600, the relative positioning of components such as the display and keypad of device 600, changes in the position of device 600 or a component of device 600, the presence or absence of user contact with device 600, the orientation or acceleration / deceleration of device 600, and temperature changes of device 600. Sensor assembly 614 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 614 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 614 may also include an accelerometer, a gyroscope, a magnetometer, a pressure sensor, or a temperature sensor.

[0139] Communication component 616 is configured to facilitate wired or wireless communication between device 600 and other devices. Device 600 can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 616 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 616 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0140] In an exemplary embodiment, the apparatus 600 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0141] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 604 including instructions, which can be executed by a processor 620 of the device 600 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0142] This application also provides a non-transitory computer-readable storage medium, which, when the instructions in the storage medium are executed by the processor of an electronic device, enables the electronic device to perform the methods provided in the above embodiments.

[0143] This application also provides a computer program product, which includes: a computer program stored in a readable storage medium, at least one processor of an electronic device can read the computer program from the readable storage medium, and the at least one processor executes the computer program to cause the electronic device to perform the solution provided in any of the above embodiments.

[0144] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0145] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A method for quality detection of audio and video data, characterized in that, include: Obtain a quality inspection request; wherein, the quality inspection request includes an identifier for the data to be inspected; Based on the quality inspection request, the data to be inspected corresponding to the data identifier to be inspected is extracted and processed to obtain audio data and video data. The audio data is converted to obtain text information; and based on preset key dialogue information, the text information is matched with key dialogue information to determine the time set in which the matched key dialogue appears; wherein, the time set includes multiple time intervals in which the key dialogue appears; Based on multiple time intervals in the time set, video frames corresponding to each of the multiple time intervals are extracted from the video data, and the video frames of each time interval are detected to obtain detection result information; wherein, the detection result information indicates whether the business corresponding to the data to be inspected is compliant in the process. The video frames in each time interval are detected to obtain detection result information, including: saving the video frames in each time interval as images, and generating a set of keyframes for each time interval based on the images; Based on the preset mapping relationship between time nodes and image recognition models, the target image recognition model corresponding to each time interval is determined; Based on the mapping relationship between the keyframe set of each time interval and the target image recognition model corresponding to each time interval, the keyframe set of the corresponding time interval is detected by the target image recognition model corresponding to each time interval to obtain detection result information; wherein, the preset image recognition model is trained based on key behaviors.

2. The method according to claim 1, characterized in that, The audio data is converted to obtain text information, including: The audio data is extracted according to the timbre in the audio data to obtain the first audio with different timbres, and the role indicated by the first audio with different timbres is determined. The first audio of each character is converted to obtain the text information of each character.

3. The method according to claim 2, characterized in that, The step of matching the text information with key dialogue information based on preset key dialogue information and determining the time set of occurrence of the matched key dialogue information includes: Based on the preset key dialogue information, search for key dialogues that match the preset key dialogue information in the text information of each character, and determine the time set in which the matched key dialogues appear.

4. The method according to claim 2, characterized in that, After converting the audio data to obtain text information, the process further includes: Based on preset compliance rules, compliance checks are performed on the text information of each role to obtain compliance check information for the text information of each role; wherein, the compliance check information indicates whether the text is compliant.

5. The method according to claim 4, characterized in that, The step of matching the text information with key dialogue information based on preset key dialogue information and determining the time set of occurrence of the matched key dialogue information includes: If it is determined that the compliance detection information of the text information of each role represents text compliance, then for the text information of each role, according to the preset key dialogue information, the text information after compliance detection is matched with key dialogue information to determine the time set of the matched key dialogue.

6. A device for detecting the quality of audio and video data, characterized in that, include: An acquisition unit is used to acquire a quality inspection request; wherein the quality inspection request includes an identifier of the data to be inspected. The first extraction unit is used to extract and process the data to be inspected corresponding to the data identifier to be inspected based on the quality inspection request, so as to obtain audio data and video data. The conversion unit is used to convert the audio data to obtain text information; The matching unit is used to perform key word matching on the text information according to preset key word information, and determine the time set of the matched key words appearing; wherein, the time set includes multiple time intervals in which the key words appear. The second extraction unit is used to extract video frames corresponding to each of the multiple time intervals in the video data based on the multiple time intervals in the time set. The detection unit is used to detect video frames in each time interval and obtain detection result information; wherein, the detection result information indicates whether the business corresponding to the data to be inspected is compliant during the processing. The detection unit includes: The generation module is used to save the video frames of each time interval as images, and generate a set of keyframes for each time interval based on the images. The determination module is used to determine the target image recognition model corresponding to each time interval based on the preset mapping relationship between time nodes and image recognition models; The detection module is used to detect the keyframe set of the corresponding time interval based on the mapping relationship between the keyframe set of each time interval and the target image recognition model corresponding to each time interval, and obtain detection result information; wherein, the preset image recognition model is trained based on key behaviors.

7. An electronic device, characterized in that, The method includes a memory and a processor, wherein the memory stores a computer program that can run on the processor, and the processor executes the computer program to implement the method of any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-5.

9. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method of any one of claims 1-5.

Images