Double-recording quality inspection method, device and equipment and storage medium

Abstract

The invention discloses a double-recording quality inspection method, device and equipment and a storage medium, and relates to the technical field of quality inspection, and the method comprises the steps: responding to a double-recording trigger request initiated by a user, and determining a quality inspection video stream according to a double-recording quality inspection mode; performing key data extraction according to the quality inspection video stream, and determining data to be subjected to quality inspection; and performing identity quality inspection according to the data to be subjected to quality inspection and the quality inspection identity data, and determining an identity quality inspection result of the user. Through the above mode, the quality inspection video stream is determined based on the quality inspection mode, the corresponding data to be subjected to quality inspection is extracted, identity quality inspection is performed in combination with the quality inspection identity data, and video double recording and AI identification are fused, so that the double recording pre-verification process and manual auditing cost are reduced, whole-course double recording quality inspection is realized, the risk of personnel change is reduced, and the user experience is improved.

Description

Technical Field

[0001] The present application relates to the field of quality inspection technology, and in particular to a double-recording quality inspection method, device, equipment and storage medium. Background Art

[0002] With the development of science and technology and the popularization of terminal equipment, many business operations of financial institutions such as banks have gradually shifted from offline to online, such as account opening, account change, reporting loss, etc. These processes all involve user identity confirmation, which can be verified by audio and video; when financial institutions conduct sales business, they can also use audio and video dual recording for sales compliance supervision. At present, some essential functions of dual recording supervision, such as face verification, identity verification and other capabilities, and some AI capabilities that can improve business processing efficiency, need to be implemented by the business parties separately, which is inefficient. At the same time, the relevant AI identity verification functions need to be completed before entering the dual recording, which has a poor experience and a long process. At the same time, it is impossible to carry out full-process quality inspection, which does not meet regulatory requirements. Summary of the invention

[0003] The main purpose of this application is to provide a double recording quality inspection method, device, equipment and storage medium, aiming to solve the technical problems of low efficiency and inability to meet regulatory requirements in the prior art double recording quality inspection.

[0004] To achieve the above purpose, the present application proposes a dual recording quality inspection method, the dual recording quality inspection method comprising:

[0005] In response to a dual-recording trigger request initiated by a user, determining a quality inspection video stream according to a dual-recording quality inspection mode;

[0006] Extract key data according to the quality inspection video stream to determine the data to be inspected;

[0007] An identity quality check is performed based on the data to be quality checked and the quality checked identity data to determine the identity quality check result of the user.

[0008] In one embodiment, the step of determining the quality inspection video stream according to the dual recording quality inspection mode in response to the dual recording trigger request initiated by the user includes:

[0009] In response to a dual recording trigger request initiated by a user, when the dual recording quality inspection mode is the first quality inspection mode, face detection is performed on the current recording screen in a polling detection manner to obtain a face detection result;

[0010] When the face detection result indicates that a face exists in the current recording picture, video recording is performed according to a preset recording duration to obtain a quality inspection video stream.

[0011] In one embodiment, the step of determining the quality inspection video stream according to the dual recording quality inspection mode in response to the dual recording trigger request initiated by the user includes:

[0012] In response to a dual recording trigger request initiated by a user, when the dual recording quality inspection mode is the second quality inspection mode, video recording is performed according to the dual recording trigger request until a recording end instruction sent by the user is received and a quality inspection video stream is obtained.

[0013] In one embodiment, the step of extracting key data according to the quality inspection video stream and determining the data to be inspected includes:

[0014] When the dual-recording quality inspection mode is the second quality inspection mode, segmenting the quality inspection video stream according to a preset segmentation time to obtain a plurality of quality inspection video segments;

[0015] Extract key frames from each quality inspection video segment respectively to obtain key video frames corresponding to each quality inspection video segment;

[0016] Extract audio data from each quality inspection video segment to obtain key audio data corresponding to each quality inspection video segment;

[0017] The data to be quality inspected corresponding to the quality inspection video stream is determined according to the key audio data and key video frames corresponding to each quality inspection video segment.

[0018] In one embodiment, the step of performing identity quality check according to the data to be quality checked and the quality check identity data to determine the identity quality check result of the user includes:

[0019] Determine key video frames and key audio data according to the data to be inspected;

[0020] Comparing the area to be inspected in the key video frame with the user identity image in the quality inspection identity data to obtain a first comparison result;

[0021] Comparing the key audio data with the target user audio in the quality inspection identity data to obtain a second comparison result;

[0022] The identity quality inspection result of the user is determined according to the first comparison result and the second comparison result.

[0023] In one embodiment, before the step of performing identity quality check according to the data to be quality checked and the quality checked identity data to determine the identity quality check result of the user, the step further includes:

[0024] When receiving a service request initiated by a user, determine the target login user based on the user login information;

[0025] Determine the user identity image according to the user certificate information of the target logged-in user;

[0026] When the target logged-in user exists in the history, obtaining the target user audio recorded by the target logged-in user within the history time period;

[0027] The quality inspection identity data of the user is determined based on the historical recorded audio and the user identity image.

[0028] In one embodiment, after the step of performing identity quality check according to the data to be quality checked and the quality checked identity data to determine the identity quality check result of the user, the step further includes:

[0029] When the identity quality inspection result is a consistent identity verification result and the dual recording quality inspection mode is the first quality inspection mode, performing face in-frame detection on the current recording screen at a preset time interval to obtain an in-frame detection result;

[0030] When the in-frame detection result indicates that the user is on camera, dual-recording prompt information is generated and pushed, so that the user restarts the dual-recording according to the dual-recording prompt information.

[0031] In addition, to achieve the above purpose, the present application also proposes a dual recording quality inspection device, the dual recording quality inspection device comprising:

[0032] A response module, used to respond to a dual recording trigger request initiated by a user and determine a quality inspection video stream according to a dual recording quality inspection mode;

[0033] An extraction module, used to extract key data according to the quality inspection video stream and determine the data to be inspected;

[0034] The quality inspection module is used to perform identity quality inspection based on the data to be quality inspected and the quality inspection identity data, and determine the identity quality inspection result of the user.

[0035] In addition, to achieve the above-mentioned purpose, the present application also proposes a dual recording quality inspection device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, and the computer program is configured to implement the steps of the dual recording quality inspection method described above.

[0036] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by the processor, the steps of the double recording quality inspection method described above are implemented.

[0037] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the steps of the double recording quality inspection method as described above.

[0038] The method of the present application includes: responding to a dual-recording trigger request initiated by a user, determining a quality inspection video stream according to a dual-recording quality inspection mode; extracting key data according to the quality inspection video stream to determine the data to be inspected; performing identity quality inspection according to the data to be inspected and the quality inspection identity data, and determining the identity quality inspection result of the user. Through the above method, the quality inspection video stream is determined based on the quality inspection mode, and the corresponding data to be inspected is extracted, and the identity quality inspection is performed in combination with the quality inspection identity data. The video dual-recording and AI recognition are integrated, which reduces the dual-recording pre-verification process and manual review costs, realizes full-process dual-recording quality inspection, reduces the risk of personnel changes, and improves user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0040] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0041] Figure 1 A flowchart of the first embodiment of the double recording quality inspection method of this application is provided;

[0042] Figure 2 A flow chart of the second embodiment of the double recording quality inspection method of this application;

[0043] Figure 3 A brief flowchart of the double recording quality inspection method provided in Example 3 of the present application;

[0044] Figure 4 This is a schematic diagram of the module structure of the dual-recording quality inspection device according to an embodiment of the present application;

[0045] Figure 5 Schematic diagram of the device structure of the hardware operating environment involved in the dual recording quality inspection method in the embodiment of the present application.

[0046] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0047] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0048] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0049] The main solution of the embodiment of the present application is: in response to a dual recording trigger request initiated by a user, determine the quality inspection video stream according to the dual recording quality inspection mode; extract key data based on the quality inspection video stream to determine the data to be quality inspected; perform identity quality inspection based on the data to be quality inspected and the quality inspection identity data, and determine the identity quality inspection result of the user.

[0050] At present, some essential functions of dual recording supervision, such as face verification and identity authentication, are some AI capabilities that can improve business processing efficiency, but each business party needs to repeatedly connect to AI, which is inefficient; the AI ​​identity verification function needs to be pre-installed and completed before entering the dual recording, which results in poor user experience; there are personnel changes during the dual recording process, and risks cannot be identified in a timely manner and quality inspection cannot be carried out throughout the process, which does not meet regulatory requirements.

[0051] This application provides a solution, which determines the quality inspection video stream based on the quality inspection mode, extracts the corresponding data to be inspected, performs identity quality inspection in combination with the quality inspection identity data, integrates video dual recording and AI recognition, reduces the dual recording pre-verification process and manual review costs, realizes full-process dual recording quality inspection, reduces the risk of personnel changes, and improves user experience.

[0052] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or a dual recording quality inspection device capable of realizing the above functions, etc. The following takes the dual recording quality inspection device as an example of the execution subject to illustrate this embodiment and the following embodiments.

[0053] Based on this, the present application embodiment provides a dual recording quality inspection method, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the double recording quality inspection method of the present application.

[0054] In this embodiment, the method includes steps S10 to S30:

[0055] Step S10, in response to a dual recording trigger request initiated by a user, determining a quality inspection video stream according to a dual recording quality inspection mode.

[0056] It should be noted that when a user needs to handle a business, he or she will first enter the dual-recording video interface, and the user will initiate dual-recording through the dual-recording trigger control in the dual-recording video interface. At this time, the dual-recording front end in the dual-recording quality inspection device determines that it has received the dual-recording trigger request initiated by the user. In this embodiment, the dual-recording trigger control can be a virtual recording button on the dual-recording video interface, or a physical button on the device, and this embodiment does not limit this.

[0057] It is understandable that the dual-recording quality inspection mode includes the first quality inspection mode and the second quality inspection mode. The first quality inspection mode is also the in-process quality inspection mode, and the second quality inspection mode is also the post-process quality inspection mode. In the first quality inspection mode, the dual-recording front end will initiate polling to detect whether there is a face in the current video screen. If so, it will automatically initiate recording of a video of a preset length (for example, 10 seconds) to ensure that the video content includes the user's facial data; the dual-recording back end will push the video of the preset length in the form of a file to the AI ​​recognition system; after receiving the video file, the AI ​​recognition system will extract key frames based on the clarity of the video frame and whether the face is facing forward as conditions. The extracted keyframes are compared with the facial images that can authenticate the user's identity (such as the ID photo on the ID card), the user's identity is identified, and the result is returned to the dual recording front end; after receiving the identification result, if the dual recording front end determines that the user's identity is wrong, it will prompt that the identification failed and re-initiate the dual recording; if the user's identity is correct, the client's built-in face detection function is turned on, and the polling detection is performed at a regular interval of 1s. According to the coordinates of the returned face in the video frame, it is determined whether the current face has left the video camera lens. If the face is detected, a prompt will be issued and the dual recording will be re-initiated. Ensure that the face is in the frame throughout the process and no face is replaced.

[0058] In this embodiment, in the second quality inspection mode, the dual recording front end starts full video recording; after the recording is completed, the dual recording back end pushes the full recording video file to the AI ​​recognition system; after receiving the video file, the AI ​​recognition system will divide the video into video segments in units of preset duration (for example, 10s); for each video segment, the corresponding key frames are extracted in accordance with the first quality inspection mode as above; the extracted key frames are compared with the facial images that can authenticate the user's identity to identify the user's identity authentication, and if one of the identification results is inconsistent, the result is pushed to the dual recording front end; when the dual recording front end receives inconsistent identity verification results, it will prompt the user to re-initiate dual recording.

[0059] It can be understood that the quality inspection video stream refers to the video stream sent to the AI ​​recognition system for quality inspection. The dual-recording quality inspection mode can be determined according to the type of business handled by the user. After the dual-recording quality inspection device determines the business handled by the user, it selects the dual-recording quality inspection mode corresponding to the business type according to the business type-quality inspection mode mapping table to record the quality inspection video stream.

[0060] In a feasible implementation, step S10 may include steps A11 to A12:

[0061] Step A11, in response to a dual recording trigger request initiated by the user, when the dual recording quality inspection mode is the first quality inspection mode, face detection is performed on the current recording screen in a polling detection manner to obtain a face detection result.

[0062] It should be noted that the dual-recording front end of the dual-recording quality inspection device responds to the dual-recording trigger request initiated by the user, and selects the corresponding dual-recording quality inspection mode according to the type of business handled by the user. When the dual-recording quality inspection mode is the first quality inspection mode, the dual-recording front end will perform face detection on the current recording screen in a polling detection manner to determine whether there is a face in the current recording screen, thereby obtaining the corresponding face detection result. In this embodiment, the face detection result includes two results: the presence of a face in the current recording screen and the absence of a face in the current recording screen; when performing face detection, detection can be performed through a trained convolutional neural network, or through other detection methods, which is not limited in this embodiment.

[0063] Step A12, when the face detection result indicates that there is a face in the current recording image, video recording is performed according to a preset recording duration to obtain a quality inspection video stream.

[0064] It should be noted that when the face detection result shows that there is a face in the current recording screen, the dual recording front end will record the video according to the preset recording time, thereby obtaining a corresponding quality inspection video stream, which includes the user's face data. In this embodiment, the preset recording time can be set to 10s, or it can be set to other time lengths according to needs, which is not limited in this embodiment.

[0065] In a feasible implementation, step S10 may include step B11: in response to a dual recording trigger request initiated by a user, when the dual recording quality inspection mode is the second quality inspection mode, performing video recording according to the dual recording trigger request until a recording end instruction sent by the user is received and a quality inspection video stream is obtained.

[0066] It should be noted that the dual-recording front end of the dual-recording quality inspection device responds to the dual-recording trigger request initiated by the user and selects the corresponding dual-recording quality inspection mode according to the type of business handled by the user. When the dual-recording quality inspection mode is the second quality inspection mode, the dual-recording front end will start full-process video recording, that is, when the user handles the business, the video recording will continue until the business handling is completed or a specific handling node is set. This embodiment does not limit the specific time when the recording ends, and finally obtains a quality inspection video stream for quality inspection.

[0067] Step S20: extract key data based on the quality inspection video stream to determine the data to be quality inspected.

[0068] It should be noted that the key data of the quality inspection video stream is extracted in combination with different dual-recording quality inspection modes to obtain the data to be inspected. In this embodiment, the data to be inspected includes but is not limited to key video frames and key audio data in the quality inspection video stream. In the first quality inspection mode, after the dual-recording front end records the quality inspection video stream, it will send the quality inspection video stream to the AI ​​recognition system through the dual-recording back end. The AI ​​recognition system will extract key video frames based on the clarity of the video frame and whether the face is facing forward as conditions; at the same time, the AI ​​recognition system will first separate the audio stream from the quality inspection video stream, and then extract the audio data containing clear human voices, and perform audio enhancement operations on it, and finally obtain the key audio data.

[0069] In this embodiment, in the second quality inspection mode, after receiving the quality inspection video stream, the AI ​​recognition system will first segment the quality inspection video stream based on the preset segmentation duration to obtain multiple video segments, and extract the data to be quality inspected corresponding to each video segment in the same manner as the first quality inspection mode. The quality inspection video stream in the second quality inspection mode includes the data to be quality inspected of multiple video segments.

[0070] Step S30, performing identity quality inspection based on the data to be quality inspected and the quality inspection identity data, and determining the identity quality inspection result of the user.

[0071] It should be noted that the quality inspection identity data includes user identity images and target user audio. The user identity image refers to a face image that can authenticate the user's identity; the target user audio refers to the user audio that can authenticate the user's identity. The target user audio can be the user audio recorded by the user in the historical time period, or it can be the user audio recorded synchronously when the user initiates dual recording. This embodiment does not limit the method of obtaining the target user audio. In this embodiment, when a user has handled business in the historical time period, it means that the user has been subjected to dual recording quality inspection in the historical time period. At this time, the key audio data extracted when the user is subjected to dual recording quality inspection in the historical time period and the identity quality inspection result is a passing result is used as the target user audio of the user.

[0072] It is understandable that the key video frame in the data to be inspected is compared with the user identity image in the inspection identity data, and the key audio data in the data to be inspected is compared with the target user audio data in the inspection identity data, so as to obtain the user's identity inspection result. In this embodiment, the identity inspection result includes two results: identity verification consistent and identity verification inconsistent.

[0073] In the specific implementation, in the first quality inspection mode, when the key audio data is consistent with the target user audio and the face image in the key video frame is consistent with the user identity image, the identity quality inspection result is a consistent identity verification result. When any of the comparisons are inconsistent, the identity quality inspection result is an inconsistent identity verification result. When the identity quality inspection result is an inconsistent identity verification result, it is necessary to re-initiate dual recording.

[0074] It should be noted that in the second quality inspection mode, when the comparison results of all video segments are that the key audio data and the target user audio are consistent and the face image in the key video frame is consistent with the user identity image, the identity quality inspection result is a consistent identity verification result, otherwise, the identity quality inspection result is an inconsistent identity verification result. When the identity quality inspection result is an inconsistent identity verification result, it is necessary to re-initiate dual recording.

[0075] In a feasible implementation, step S30 may include steps C11 to C14:

[0076] Step C11, determining key video frames and key audio data according to the data to be quality inspected.

[0077] Step C12: Compare the area to be inspected in the key video frame with the user identity image in the inspection identity data to obtain a first comparison result.

[0078] It should be noted that the AI ​​recognition system preprocesses the key video frames, including but not limited to grayscale, normalization, and scaling operations, and uses a face detection algorithm to perform face detection on the preprocessed key video frames, and uses the area containing the face as the area to be inspected. The area to be inspected is compared with the user identity image. In this embodiment, a face comparison algorithm (such as FaceNet, ArcFace, etc.) can be used to calculate the similarity between the two. When the similarity is higher than or equal to a set threshold, the first comparison result is a consistent face result; when the similarity is lower than the set threshold, the first comparison result is an inconsistent face result.

[0079] Step C13, comparing the key audio data with the target user audio in the quality inspection identity data to obtain a second comparison result.

[0080] It should be noted that the AI ​​recognition system compares the audio features corresponding to the key audio data with the audio features corresponding to the target user audio. In this embodiment, an audio comparison algorithm (such as DTW (dynamic time warping), voiceprint recognition, etc.) can be used to calculate the similarity between the two audio segments. When the similarity is higher than or equal to a set threshold, the second comparison result is an audio consistency result; when the similarity is lower than the set threshold, the second comparison result is an audio inconsistency result.

[0081] Step C14: determining the identity verification result of the user according to the first comparison result and the second comparison result.

[0082] It should be noted that when the first comparison result and the second comparison result are both consistent results, the identity quality inspection result is a consistent identity verification result; otherwise, it is an inconsistent identity verification result.

[0083] It is understandable that, in addition to the above-mentioned method of using audio data and facial images for identity verification, identity verification can also be performed using only facial images, specifically: comparing the to-be-inspected area of ​​the key video frame with the user's identity image, and when the comparison is consistent, the identity verification result is an identity verification consistent result. In this embodiment, the specific method used for quality verification is not limited in this embodiment.

[0084] In a feasible implementation manner, before step S30, steps D11 to D14 may also be included:

[0085] Step D11, upon receiving a service processing request initiated by a user, determining a target login user according to the user login information.

[0086] It should be noted that when a user initiates a double recording quality inspection, he or she will first initiate a business processing request through the double recording quality inspection device. Before initiating the business processing request, he or she will first log in through the front-end interface and provide login information when logging in, including but not limited to any one or more of the user name, phone number, and password, so as to determine the target login user.

[0087] Step D12, determining the user identity image according to the user certificate information of the target login user.

[0088] It should be noted that the user certificate information of the target login user is queried in the database, including but not limited to the information corresponding to the resident identity card and the information corresponding to the passport, and the user certificate information is extracted to obtain the user identity image.

[0089] Step D13, when the target logged-in user exists in the history, obtain the target user audio recorded by the target logged-in user within the historical time period.

[0090] It should be noted that when the target login user has handled business within the historical time period, the target login user is a historical user, which means that the user has been double-recorded and quality-checked within the historical time period. At this time, the key audio data extracted when the user has been double-recorded and quality-checked within the historical time period and the identity quality check result is a passing result will be used as the target user audio of the user.

[0091] Step D14, determining the quality inspection identity data of the user based on the historical recorded audio and the user identity image.

[0092] It should be noted that the historical recorded audio and user identity image are summarized and used as the quality inspection identity data of the target login user.

[0093] In a feasible implementation manner, after step S30, steps E11 to E12 may be included:

[0094] Step E11, when the identity quality inspection result is a consistent identity verification result and the dual recording quality inspection mode is the first quality inspection mode, perform face in-frame detection on the current recording image at a preset time interval to obtain an in-frame detection result.

[0095] Step E12, when the in-frame detection result indicates that the user is on camera, dual-recording prompt information is generated and pushed, so that the user restarts the dual-recording according to the dual-recording prompt information.

[0096] It should be noted that when the identity quality inspection result is a consistent identity verification result and the dual-recording quality inspection mode is the first quality inspection mode, the face detection function of the client is turned on, and the face in-frame polling detection is performed on the current recording screen at a preset time interval to determine the coordinates of the face in the video frame and whether the face has left the camera, thereby obtaining the in-frame detection result. When the in-frame detection result is that the user appears on camera, it means that the face has left the camera, and a dual-recording prompt message is generated and pushed, so that the user can re-initiate the dual-recording according to the dual-recording prompt message. In this embodiment, the preset time interval can be set to 1s, or it can be set to other values, and this embodiment is not limited to this.

[0097] It is understandable that when the identity quality check result is an inconsistent result of identity verification, the user will be prompted to re-initiate double recording in both the first quality check mode and the second quality check mode.

[0098] The method of this embodiment includes: responding to a dual-recording trigger request initiated by a user, determining a quality inspection video stream according to a dual-recording quality inspection mode; extracting key data according to the quality inspection video stream to determine the data to be inspected; performing identity quality inspection according to the data to be inspected and the quality inspection identity data, and determining the identity quality inspection result of the user. Through the above method, the quality inspection video stream is determined based on the quality inspection mode, and the corresponding data to be inspected is extracted, and the identity quality inspection is performed in combination with the quality inspection identity data. The video dual-recording and AI recognition are integrated, the dual-recording pre-verification process and manual review costs are reduced, the full-process dual-recording quality inspection is realized, the risk of personnel changes is reduced, and the user experience is improved.

[0099] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. Figure 2In the dual recording quality inspection method, step S20 further includes steps S21 to S24:

[0100] Step S21, when the dual-recording quality inspection mode is the second quality inspection mode, the quality inspection video stream is segmented according to a preset segmentation time length to obtain a plurality of quality inspection video segments.

[0101] It should be noted that when the dual recording quality inspection mode is the second quality inspection mode, the quality inspection video stream is segmented according to the preset segmentation time, and the quality inspection video stream is divided into multiple quality inspection video segments. In this embodiment, the preset segmentation time can be set to 10s or other values, which is not limited in this embodiment.

[0102] Step S22: extract key frames from each quality inspection video segment to obtain key video frames corresponding to each quality inspection video segment.

[0103] Step S23: extract audio data from each quality inspection video segment to obtain key audio data corresponding to each quality inspection video segment.

[0104] Step S24, determining the to-be-checked data corresponding to the quality inspection video stream according to the key audio data and key video frames corresponding to each quality inspection video segment.

[0105] It should be noted that for each quality inspection video segment, the AI ​​recognition system will extract the key video frames of each quality inspection video segment based on the clarity of the video frames in the quality inspection video segment and whether the face is facing forward; and separate the audio stream from the quality inspection video segment, and then extract the audio data containing clear human voices, and perform audio enhancement operations on it, and finally obtain the key audio data of each quality inspection video segment, and summarize the key audio data and key video frames of each quality inspection video segment to obtain the data to be inspected for each quality inspection video segment. In the second quality inspection mode, the data to be inspected of the quality inspection video stream includes the data to be inspected of multiple quality inspection video segments.

[0106] In this embodiment, when the dual-recording quality inspection mode is the second quality inspection mode, the quality inspection video stream is segmented according to the preset segmentation time to obtain multiple quality inspection video segments; key frames are extracted from each quality inspection video segment to obtain key video frames corresponding to each quality inspection video segment; audio data is extracted from each quality inspection video segment to obtain key audio data corresponding to each quality inspection video segment; and the data to be inspected corresponding to the quality inspection video stream is determined according to the key audio data and key video frames corresponding to each quality inspection video segment. In the above manner, the data to be inspected corresponding to the quality inspection video stream can be accurately obtained.

[0107] For example, in order to help understand the implementation process of the dual recording quality inspection method obtained by combining this embodiment with the above-mentioned embodiment 1, please refer to Figure 3 , Figure 3A brief flow chart of a double recording quality inspection method is provided, specifically:

[0108] In this embodiment, in the in-process quality inspection mode: the user enters the dual-recording video interface, clicks the record button, and initiates dual-recording; the dual-recording front end will initiate polling to detect whether there is a face in the current video screen. If so, it will automatically initiate recording of a 10s video to ensure that the video content includes the user's face data; the dual-recording back end will push the 10s video to the AI ​​recognition system in the form of a file; after receiving the video file, the AI ​​recognition system will extract key frames based on the clarity of the video frame and whether the face is facing forward as conditions. The extracted key frames are compared with the system ID photo to identify the user's identity authentication, and the results are returned to the dual-recording front end; after receiving the recognition result, the front end determines that if the user identity is wrong, it prompts that the recognition failed and re-initiates dual-recording; if the user identity is correct, the client's built-in face detection function is turned on, and polling detection is performed at regular intervals of 1s. According to the coordinates of the returned face in the video frame, it is determined whether the current face has left the video camera lens. If the face is detected to be out of the frame, a prompt is given and dual-recording is re-initiated. Ensure that the face is in the frame throughout the process and no face is replaced.

[0109] In the post-event quality inspection mode: the user enters the dual-recording video interface and clicks the record button to initiate dual-recording; the dual-recording front end starts full-time video recording; after recording is completed, the dual-recording back end pushes the full-time recorded video file to the AI ​​recognition system; after receiving the video file, the AI ​​recognition system divides the video into video segments in units of 10 seconds; for each video segment, the corresponding key frames are extracted according to the above-mentioned in-event quality inspection method; the extracted key frames are compared with the system ID photo to identify the user's identity authentication. If one of the identification results is inconsistent, the result will be pushed to the dual-recording front end; if the dual-recording front end receives inconsistent identity verification results, it will prompt the user to re-initiate dual-recording.

[0110] The method of this embodiment can achieve the following effects: video dual recording provides AI quality inspection capabilities, reducing the need for business parties to separately connect to the AI ​​system; AI quality inspection is integrated into the dual recording process, reducing pre-verification and improving user experience; dual recording quality inspection is performed throughout the entire process to detect non-compliance issues in advance and reduce the cost of later manual review.

[0111] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the double-recording quality inspection method of the present application. More simple transformations based on this technical concept are all within the protection scope of the present application.

[0112] This application also provides a dual recording quality inspection device, please refer to Figure 4 , the dual recording quality inspection device comprises:

[0113] The response module 10 is used to respond to the dual-recording trigger request initiated by the user and determine the quality inspection video stream according to the dual-recording quality inspection mode.

[0114] The extraction module 20 is used to extract key data according to the quality inspection video stream and determine the data to be quality inspected.

[0115] The quality inspection module 30 is used to perform identity quality inspection based on the data to be quality inspected and the quality inspection identity data, and determine the identity quality inspection result of the user.

[0116] Optionally, the response module 10 is further configured to:

[0117] In response to a dual recording trigger request initiated by the user, when the dual recording quality inspection mode is the first quality inspection mode, face detection is performed on the current recording screen according to the polling detection method to obtain a face detection result; when the face detection result is that there is a face in the current recording screen, video recording is performed according to the preset recording time to obtain a quality inspection video stream.

[0118] Optionally, the response module 10 is further configured to:

[0119] In response to a dual recording trigger request initiated by a user, when the dual recording quality inspection mode is the second quality inspection mode, video recording is performed according to the dual recording trigger request until a recording end instruction sent by the user is received and a quality inspection video stream is obtained.

[0120] Optionally, the extraction module 20 is further used for:

[0121] When the dual-recording quality inspection mode is the second quality inspection mode, the quality inspection video stream is segmented according to the preset segmentation time to obtain multiple quality inspection video segments; key frames are extracted for each quality inspection video segment to obtain key video frames corresponding to each quality inspection video segment; audio data is extracted for each quality inspection video segment to obtain key audio data corresponding to each quality inspection video segment; and the data to be quality inspected corresponding to the quality inspection video stream is determined according to the key audio data and key video frames corresponding to each quality inspection video segment.

[0122] Optionally, the quality inspection module 30 is further used to:

[0123] Determine key video frames and key audio data based on the data to be quality inspected; compare the area to be quality inspected in the key video frame with the user identity image in the quality inspection identity data to obtain a first comparison result; compare the key audio data with the target user audio in the quality inspection identity data to obtain a second comparison result; determine the user's identity quality inspection result based on the first comparison result and the second comparison result.

[0124] Optionally, the quality inspection module 30 is further used to:

[0125] When a business processing request initiated by a user is received, the target login user is determined based on the user login information; the user identity image is determined based on the user ID information of the target login user; when the target login user exists in the history, the target user audio recorded by the target login user within the historical time period is obtained; the quality inspection identity data of the user is determined based on the historical recorded audio and the user identity image.

[0126] Optionally, the quality inspection module 30 is further used to:

[0127] When the identity quality inspection result is a consistent result of identity verification and the dual recording quality inspection mode is the first quality inspection mode, the face in-frame detection is performed on the current recording screen at a preset time interval to obtain an in-frame detection result; when the in-frame detection result is that the user appears on camera, a dual recording prompt message is generated and pushed, so that the user restarts the dual recording according to the dual recording prompt message.

[0128] The dual recording quality inspection device provided by the present application adopts the dual recording quality inspection method in the above embodiment, which can solve the technical problems of low efficiency and inability to meet regulatory requirements in the prior art dual recording quality inspection. Compared with the prior art, the beneficial effects of the dual recording quality inspection device provided by the present application are the same as the beneficial effects of the dual recording quality inspection method provided by the above embodiment, and the other technical features of the dual recording quality inspection device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0129] The present application provides a dual recording quality inspection device, which includes: at least one processor; and a memory that is communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the dual recording quality inspection method in the above-mentioned embodiment one.

[0130] Reference below Figure 5 , which shows a schematic diagram of the structure of a dual recording quality inspection device suitable for implementing the embodiment of the present application. The dual recording quality inspection device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 5 The dual-recording quality inspection device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0131] like Figure 5 As shown, the dual recording quality inspection device may include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. Various programs and data required for the operation of the dual recording quality inspection device are also stored in RAM1004. The processing device 1001, ROM1002, and RAM1004 are connected to each other via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 1003 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 1009. The communication device 1009 can allow the dual recording quality inspection device to communicate wirelessly or wired with other devices to exchange data. Although the figure shows a dual recording quality inspection device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems may be implemented or have alternatively.

[0132] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0133] The dual recording quality inspection device provided by the present application adopts the dual recording quality inspection method in the above embodiment, which can solve the technical problems of low efficiency and inability to meet regulatory requirements in the prior art dual recording quality inspection. Compared with the prior art, the beneficial effects of the dual recording quality inspection device provided by the present application are the same as the beneficial effects of the dual recording quality inspection method provided by the above embodiment, and the other technical features of the dual recording quality inspection device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0134] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0135] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0136] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the double-recording quality inspection method in the above-mentioned embodiment.

[0137] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0138] The computer-readable storage medium may be included in the dual-recording quality inspection device; or may exist independently without being assembled into the dual-recording quality inspection device.

[0139] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the dual-recording quality inspection device, the dual-recording quality inspection device: responds to the dual-recording trigger request initiated by the user, and determines the quality inspection video stream according to the dual-recording quality inspection mode; extracts key data based on the quality inspection video stream to determine the data to be quality inspected; performs identity quality inspection based on the data to be quality inspected and the quality inspection identity data, and determines the identity quality inspection result of the user.

[0140] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0141] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0142] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0143] The readable storage medium provided by the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned double-recording quality inspection method, and can solve the technical problems of low efficiency and inability to meet regulatory requirements during double-recording quality inspection in the prior art. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as those of the double-recording quality inspection method provided by the above-mentioned embodiment, and will not be elaborated here.

[0144] The present application also provides a computer program product, including a computer program, which implements the steps of the double-recording quality inspection method as described above when executed by a processor.

[0145] The computer program product provided by this application can solve the technical problems of low efficiency and inability to meet regulatory requirements in the prior art of double recording quality inspection. Compared with the prior art, the beneficial effects of the computer program product provided by this application are the same as those of the double recording quality inspection method provided by the above embodiment, and will not be elaborated here.

[0146] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A double recording quality inspection method, characterized in that: The dual recording quality inspection method comprises: In response to a dual-recording trigger request initiated by a user, determining a quality inspection video stream according to a dual-recording quality inspection mode; Extract key data according to the quality inspection video stream to determine the data to be inspected; An identity quality check is performed based on the data to be quality checked and the quality checked identity data to determine the identity quality check result of the user.

2. The method according to claim 1, characterized in that The step of determining the quality inspection video stream according to the dual recording quality inspection mode in response to the dual recording trigger request initiated by the user includes: In response to a dual recording trigger request initiated by a user, when the dual recording quality inspection mode is the first quality inspection mode, face detection is performed on the current recording screen in a polling detection manner to obtain a face detection result; When the face detection result indicates that a face exists in the current recording picture, video recording is performed according to a preset recording duration to obtain a quality inspection video stream.

3. The method according to claim 1, characterized in that The step of determining the quality inspection video stream according to the dual recording quality inspection mode in response to the dual recording trigger request initiated by the user includes: In response to a dual recording trigger request initiated by a user, when the dual recording quality inspection mode is the second quality inspection mode, video recording is performed according to the dual recording trigger request until a recording end instruction sent by the user is received and a quality inspection video stream is obtained.

4. The method according to claim 1, characterized in that The step of extracting key data according to the quality inspection video stream and determining the data to be inspected comprises: When the dual-recording quality inspection mode is the second quality inspection mode, segmenting the quality inspection video stream according to a preset segmentation time to obtain a plurality of quality inspection video segments; Extract key frames from each quality inspection video segment respectively to obtain key video frames corresponding to each quality inspection video segment; Extract audio data from each quality inspection video segment to obtain key audio data corresponding to each quality inspection video segment; The data to be quality inspected corresponding to the quality inspection video stream is determined according to the key audio data and key video frames corresponding to each quality inspection video segment.

5. The method according to claim 1, characterized in that The step of performing identity quality inspection based on the data to be quality inspected and the quality inspection identity data to determine the identity quality inspection result of the user comprises: Determine key video frames and key audio data according to the data to be inspected; Comparing the area to be inspected in the key video frame with the user identity image in the quality inspection identity data to obtain a first comparison result; Comparing the key audio data with the target user audio in the quality inspection identity data to obtain a second comparison result; The identity quality inspection result of the user is determined according to the first comparison result and the second comparison result.

6. The method according to any one of claims 1 to 5, characterized in that Before the step of performing identity quality inspection based on the data to be quality inspected and the quality inspection identity data to determine the identity quality inspection result of the user, the step further includes: When receiving a service request initiated by a user, determine the target login user based on the user login information; Determine the user identity image according to the user certificate information of the target logged-in user; When the target logged-in user exists in the history, obtaining the target user audio recorded by the target logged-in user within the history time period; The quality inspection identity data of the user is determined based on the historical recorded audio and the user identity image.

7. The method according to any one of claims 1 to 5, characterized in that After the step of performing identity quality inspection according to the data to be quality inspected and the quality inspection identity data to determine the identity quality inspection result of the user, the method further includes: When the identity quality inspection result is a consistent identity verification result and the dual recording quality inspection mode is the first quality inspection mode, performing face in-frame detection on the current recording screen at a preset time interval to obtain an in-frame detection result; When the in-frame detection result indicates that the user is on camera, dual-recording prompt information is generated and pushed, so that the user restarts the dual-recording according to the dual-recording prompt information.

8. A dual recording quality inspection device, characterized in that: The dual recording quality inspection device comprises: A response module, used to respond to a dual recording trigger request initiated by a user and determine a quality inspection video stream according to a dual recording quality inspection mode; An extraction module, used to extract key data according to the quality inspection video stream and determine the data to be inspected; The quality inspection module is used to perform identity quality inspection based on the data to be quality inspected and the quality inspection identity data, and determine the identity quality inspection result of the user.

9. A dual-recording quality inspection device, characterized in that: The dual recording quality inspection device includes: a memory, a processor, and a dual recording quality inspection program stored in the memory and executable on the processor, wherein the dual recording quality inspection program is configured to implement the dual recording quality inspection method according to any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium stores a dual-recording quality inspection program, which, when executed by a processor, implements the dual-recording quality inspection method according to any one of claims 1 to 7.

Images