Real-time double-recording method and device, computer equipment and storage medium

Abstract

The invention belongs to the field of artificial intelligence information security, is applied to the field of smart government affairs, and relates to a real-time double-recording method, which comprises the following steps: performing audio and video acquisition in real time based on a current link to obtain current double-recording information; according to a preset model configuration table, a target detection model corresponding to the current link and a model type corresponding to the target detection model are obtained, and the model type comprises a local model and a far-end model; determining a target port through the model type, calling a target detection model by adopting the target port, and performing quality inspection on the double-recording information to obtain a quality inspection result; and if the quality inspection result is qualified, executing the next link, and if the quality inspection result is unqualified, executing a corresponding re-recording measure. The invention further provides a real-time double-recording device, computer equipment and a storage medium. In addition, the invention also relates to a block chain technology, and the quality inspection result can be stored in a block chain. According to the invention, the double-recording quality inspection efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a real-time dual recording method, device, computer equipment and storage medium. Background Art

[0002] Double recording means that when banks or insurance companies sell financial products, they need to record audio and video of the entire process. Double recording quality inspection means that audio and video are used for manual or intelligent quality inspection.

[0003] Record and videotape the business handling of on-site personnel, use dual-recording equipment to capture dual-recording information, and save the dual-recording information in the cloud storage system to monitor the standardization of on-site personnel's work and business level, and preserve evidence of business-related operations and behaviors. After the recorded audio and video are uploaded to the business quality inspection platform, they need to be quality inspected to see if the quality of the recorded audio and video meets the specified requirements. If they meet the specified requirements, the recorded audio and video will be saved, otherwise they need to be re-recorded.

[0004] At present, the dual recording system in the industry only has basic functions such as simple audio and video recording, manual quality inspection, and background intelligent quality inspection. However, the quality inspection standards are inconsistent and the quality inspection process is long. The background intelligent quality inspection needs to wait for the entire video to be uploaded to the server, parse the video and perform intelligent analysis, and finally return the conclusion to the front end, which takes a long time and results in low quality inspection efficiency and accuracy. Summary of the invention

[0005] The purpose of the embodiments of the present application is to provide a real-time dual recording method, apparatus, computer equipment and storage medium to improve the efficiency of dual recording quality inspection.

[0006] In order to solve the above technical problems, the embodiment of the present application provides a real-time dual recording method, which adopts the following technical solution:

[0007] Based on the current link, audio and video are collected in real time to obtain the current dual recording information;

[0008] According to the preset model configuration table, obtain the target detection model corresponding to the current link and the model type corresponding to the target detection model, wherein the model type includes a local model and a remote model;

[0009] Determine the target port by model type, and use the target port to call the target detection model to perform quality inspection on the dual recording information to obtain a quality inspection result;

[0010] If the quality inspection result is qualified, the next step will be executed. If the quality inspection result is unqualified, the corresponding re-recording measures will be implemented.

[0011] Furthermore, the step of collecting audio and video in real time based on the current link to obtain the current dual recording information specifically includes:

[0012] Obtaining the dual recording rules of the current link, and parsing the obtained dual recording rules to obtain semantic information;

[0013] The semantic information is converted into voice guidance information by using a text-to-speech method;

[0014] Generate AI voice information according to the voice guidance information, and perform voice broadcast through the AI ​​voice information to guide audio and video collection to obtain current dual recording information.

[0015] Furthermore, the step of broadcasting the AI ​​voice information to guide the audio and video collection to obtain the current dual recording information specifically includes:

[0016] After the voice broadcast is finished, the collected information is divided into frames to obtain frame information;

[0017] Performing real-time detection on the frame information to determine whether the event corresponding to the current link has been recorded;

[0018] If the judgment result is that the recording is completed, the acquisition is ended, and the acquisition information obtained at this time is used as the current dual recording information.

[0019] Furthermore, the current link is identity authentication, and the step of using the target port to call the target detection model, performing quality inspection on the dual-recording information, and obtaining the quality inspection result specifically includes:

[0020] Extracting a number of video frame images from the dual-recording information at preset intervals;

[0021] Performing face recognition on the video frame image, and taking the video frame image containing the face image as the target image;

[0022] Performing liveness recognition based on the target image to obtain a liveness recognition result, and performing micro-expression recognition on the target image to obtain a micro-expression recognition result;

[0023] The quality inspection result is determined according to the face recognition result and the micro-expression recognition result.

[0024] Furthermore, the step of performing liveness recognition based on the target image to obtain a liveness recognition result specifically includes:

[0025] Sorting the target images in order of time frames to obtain an image sequence;

[0026] Extracting features of each of the target images in the image sequence to obtain a basic feature sequence;

[0027] After encoding the basic feature sequence, sequentially inputting it into a trained heart rhythm feature extraction network to obtain heart rate feature information, wherein the trained heart rhythm feature extraction network is a recurrent neural network;

[0028] The heart rate characteristic information is input into a trained liveness recognition model to perform liveness recognition to obtain the liveness recognition result.

[0029] Furthermore, the step of performing micro-expression recognition on the target image and obtaining the micro-expression recognition result specifically includes:

[0030] Using a preset convolutional neural network model, the target image is subjected to expression unit recognition to obtain a set of units to be recognized contained in each of the target images;

[0031] According to the mapping relationship between each preset expression unit combination and each micro-expression in the preset expression unit combination set, the micro-expression corresponding to the to-be-recognized unit set contained in the target image is determined as the target micro-expression.

[0032] Furthermore, the step of determining the micro-expression corresponding to the to-be-recognized unit set contained in the target image as the target micro-expression according to the mapping relationship between each preset expression unit combination and each micro-expression in the preset expression unit combination set specifically includes:

[0033] Obtain a preset expression unit combination set, wherein the expression unit combination set includes k preset expression unit combinations, each preset expression unit combination corresponds to a micro-expression, and k is a positive integer;

[0034] Calculating the similarity between the to-be-recognized unit set and each of the preset expression unit combinations respectively, to obtain k similarity values;

[0035] Selecting a preset expression unit combination corresponding to the largest similarity value from the k similarity values ​​as the target unit combination;

[0036] According to the mapping relationship, the micro-expression corresponding to the target unit combination is obtained as a target micro-expression.

[0037] In order to solve the above technical problems, the embodiment of the present application further provides a real-time dual recording device, which adopts the following technical solution:

[0038] The acquisition module is used to collect audio and video in real time based on the current link to obtain the current dual recording information;

[0039] A selection module is used to obtain the target detection model corresponding to the current link and the model type corresponding to the target detection model according to a preset model configuration table, wherein the model type includes a local model and a remote model;

[0040] A quality inspection module, used to determine the target port by model type, and use the target port to call the target detection model, perform quality inspection on the dual recording information, and obtain a quality inspection result;

[0041] The determination module is used to execute the next step if the quality inspection result is qualified, and to execute the corresponding re-recording measures if the quality inspection result is unqualified.

[0042] Furthermore, the acquisition module includes:

[0043] A parsing unit, used to obtain the dual recording rules of the current link, and parse the obtained dual recording rules to obtain semantic information;

[0044] A conversion unit, used to convert the semantic information into voice guidance information by using a text-to-speech method;

[0045] The guiding unit is used to generate AI voice information according to the voice guiding information, and perform voice broadcast through the AI ​​voice information to guide the audio and video collection to obtain the current dual recording information.

[0046] Furthermore, the guiding unit comprises:

[0047] The framing subunit is used to frame the collected information after the voice broadcast is finished to obtain frame information;

[0048] A detection subunit, used to perform real-time detection on the frame information to determine whether the event corresponding to the current link has been recorded;

[0049] The judging subunit is used to end the acquisition if the judgment result is that the recording is completed, and use the acquired information at this time as the current dual recording information.

[0050] Furthermore, the quality inspection module includes:

[0051] A video frame extraction unit, used to extract a number of video frame images from the dual recording information at preset intervals;

[0052] A face recognition unit, used to perform face recognition on the video frame image, and use the video frame image containing the face image as a target image;

[0053] A feature recognition unit, used to perform liveness recognition based on the target image to obtain a liveness recognition result, and perform micro-expression recognition on the target image to obtain a micro-expression recognition result;

[0054] A result determination unit is used to determine the quality inspection result according to the face recognition result and the micro-expression recognition result.

[0055] Furthermore, the feature recognition unit includes:

[0056] A sorting subunit, used for sorting the target images in the order of time frames to obtain an image sequence;

[0057] An image feature extraction subunit, used to extract features of each target image in the image sequence to obtain a basic feature sequence;

[0058] A heart rhythm feature extraction subunit, used for encoding the basic feature sequence and sequentially inputting it into a trained heart rhythm feature extraction network to obtain heart rate feature information, wherein the trained heart rhythm feature extraction network is a recurrent neural network;

[0059] The liveness recognition subunit is used to input the heart rate characteristic information into a trained liveness recognition model to perform liveness recognition and obtain the liveness recognition result.

[0060] Furthermore, the feature recognition unit also includes:

[0061] An expression recognition subunit, used to perform expression unit recognition on the target image through a preset convolutional neural network model to obtain a set of units to be recognized contained in each of the target images;

[0062] The micro-expression determination subunit is used to determine the micro-expression corresponding to the to-be-recognized unit set contained in the target image as the target micro-expression according to the mapping relationship between each preset expression unit combination and each micro-expression in the preset expression unit combination set.

[0063] Furthermore, the micro-expression determination subunit includes:

[0064] An acquisition component is used to acquire a preset expression unit combination set, wherein the expression unit combination set includes k preset expression unit combinations, each preset expression unit combination corresponds to a micro-expression, and k is a positive integer;

[0065] A calculation component is used to calculate the similarity between the set of units to be recognized and each preset expression unit combination, and obtain k similarity values;

[0066] A selection component is used to select a preset expression unit combination corresponding to the largest similarity value from the k similarity values ​​as a target unit combination;

[0067] The confirmation component is used to obtain the micro-expression corresponding to the target unit combination according to the mapping relationship as the target micro-expression.

[0068] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution:

[0069] Based on the current link, audio and video are collected in real time to obtain the current dual recording information;

[0070] According to the preset model configuration table, obtain the target detection model corresponding to the current link and the model type corresponding to the target detection model, wherein the model type includes a local model and a remote model;

[0071] Determine the target port by model type, and use the target port to call the target detection model to perform quality inspection on the dual recording information to obtain a quality inspection result;

[0072] If the quality inspection result is qualified, the next step will be executed. If the quality inspection result is unqualified, the corresponding re-recording measures will be implemented.

[0073] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution:

[0074] Based on the current link, audio and video are collected in real time to obtain the current dual recording information;

[0075] According to the preset model configuration table, obtain the target detection model corresponding to the current link and the model type corresponding to the target detection model, wherein the model type includes a local model and a remote model;

[0076] Determine the target port through the model type, and use the target port to execute the target detection model, perform quality inspection on the dual recording information, and obtain a quality inspection result;

[0077] If the quality inspection result is qualified, the next step will be executed. If the quality inspection result is unqualified, the corresponding re-recording measures will be implemented.

[0078] Compared with the prior art, the embodiments of the present application have the following beneficial effects: by splitting the double recording quality inspection process into quality inspections of different links, setting different quality inspection types and models for each link, ensuring the quality of quality inspection in each link, saving server resources, and at the same time, also helping to improve quality inspection efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0080] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;

[0081] Figure 2 A flowchart of an embodiment of a real-time dual recording method according to the present application;

[0082] Figure 3 is a structural schematic diagram of an embodiment of a real-time dual recording device according to the present application;

[0083] Figure 4 It is a structural diagram of an embodiment of a computer device according to the present application. DETAILED DESCRIPTION

[0084] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0085] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0086] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0087] like Figure 1 As shown, the system architecture 100 may include a terminal device 101, a network 102 and a server 103. The terminal device 101 may be a laptop 1011, a tablet computer 1012 or a mobile phone 1013. The network 102 is used to provide a medium for a communication link between the terminal device 101 and the server 103. The network 102 may include various connection types, such as wired, wireless communication links or optical fiber cables.

[0088] The user can use the terminal device 101 to interact with the server 103 through the network 102 to receive or send messages, etc. Various communication client applications can be installed on the terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0089] The terminal device 101 can be any electronic device with a display screen and supporting web browsing. In addition to a laptop computer 1011, a tablet computer 1012 or a mobile phone 1013, the terminal device 101 can also be an e-book reader, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 (Moving Picture Experts Group Audio Layer IV), a laptop computer, a desktop computer, etc.

[0090] The server 103 may be a server that provides various services, such as a background server that provides support for a web page displayed on the terminal device 101 .

[0091] It should be noted that the real-time dual recording method provided in the embodiment of the present application is generally executed by a server / terminal device, and accordingly, the real-time dual recording device is generally arranged in the server / terminal device.

[0092] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0093] Continue to refer Figure 2 , shows a flow chart of an embodiment of a real-time dual recording method according to the present application. The real-time dual recording method comprises the following steps:

[0094] Step S201, based on the current link, real-time audio and video collection is performed to obtain current dual recording information.

[0095] In this embodiment, the electronic device (eg Figure 1 shown Server / Terminal Terminal equipment ) can receive audio and video acquisition requests through a wired connection or a wireless connection. It should be noted that the above wireless connection methods may include but are not limited to 3G / 4G / 5G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultra wideband) connection, and other wireless connection methods currently known or to be developed in the future.

[0096] Specifically, the rule base pre-set on the server side includes at least two dual recording links. After the business identifier corresponding to each dual recording link obtains the current business identifier, the dual recording link corresponding to the current business identifier is selected from the rule base as the current link, so that the dual recording process can be performed according to the current link later.

[0097] Among them, each dual recording link has its own dual recording rules, including independent dual recording scenes and dual recording tasks. For example, the face authentication link uses a camera device to obtain the face image that needs to be authenticated, and transmits the image to the server to perform face verification processing.

[0098] It is easy to understand that each link has a unique service identifier. At the same time, in this embodiment, it often includes multiple links. Therefore, in the double recording link corresponding to each service identifier, a sequential ID is pre-set for each link.

[0099] For example, in a specific implementation, the steps included in the double recording process of a contract signing business are arranged in order of sequence ID: basic information recognition, face verification, business signing video collection, document confirmation and signature video collection, etc.

[0100] Step S202, according to the preset model configuration table, obtain the target detection model corresponding to the current link and the model type corresponding to the target detection model, and the model type includes a local model and a remote model.

[0101] Specifically, each dual recording link is pre-configured with a corresponding model according to its task. For example, basic information recognition is configured with a text recognition model, face verification is configured with a face recognition model, and business signing video collection is configured with a liveness recognition model.

[0102] Among them, the preset model configuration table can be set according to actual needs and is not limited here.

[0103] Step S203, determine the target port through the model type, and use the target port to call the target detection model to perform quality inspection on the dual recording information to obtain a quality inspection result.

[0104] Specifically, in this embodiment, in order to improve the quality inspection efficiency, a multi-terminal connection method is adopted to distribute different types of models on different ports, and the multi-terminals can be divided into at least a local end and a remote end.

[0105] In this embodiment, as a preferred method, in some links that require simple model processing, the local end can be used as the target port, and in some links that require complex model processing, the remote end can be used as the target port. The number of remote ends can be one or more, and is not specifically limited here.

[0106] It should be understood that the present embodiment uses different ports for quality inspection, which is beneficial to reduce the pressure on the service end and improve the efficiency of quality inspection.

[0107] Step S204: if the quality inspection result is qualified, the next step is executed; if the quality inspection result is unqualified, the corresponding re-recording measures are executed.

[0108] Specifically, for each link, when the quality inspection result is qualified, the next link is executed; otherwise, the corresponding re-recording measures are executed to avoid taking a long time for quality inspection after all links are recorded, and re-recording all links if any problems are found, resulting in long recording time and long quality inspection time, thereby improving quality inspection efficiency.

[0109] In this embodiment, by dividing the double recording quality inspection process into quality inspections of different links, different quality inspection types and models are set for each link to ensure the quality of quality inspection in each link, save server resources, and at the same time, help improve quality inspection efficiency.

[0110] In some optional implementations, in step S201, based on the current link, real-time audio and video acquisition is performed to obtain current dual recording information, including the following steps:

[0111] Obtain the dual recording rules of the current link, and parse the obtained dual recording rules to obtain semantic information;

[0112] The semantic information is converted into voice guidance information by using text-to-speech method;

[0113] Generate AI voice information according to the voice guidance information, and broadcast it through the AI ​​voice information to guide the audio and video collection to obtain the current dual recording information.

[0114] In this embodiment, by semantically understanding the set rules, generating text semantic information, and then converting it into voice, guiding through AI synthesized voice, it is beneficial for the client to respond quickly and accurately, and improves the efficiency and accuracy of the dual recording process.

[0115] In some optional implementations, the steps of broadcasting the AI ​​voice information to guide the audio and video collection and obtaining the current dual recording information specifically include:

[0116] After the voice broadcast is finished, the collected information is divided into frames to obtain frame information;

[0117] Perform real-time detection on the frame information to determine whether the recording of the items corresponding to the current link has been completed;

[0118] If the judgment result is that the recording is completed, the acquisition is terminated, and the acquisition information obtained at this time is used as the current dual recording information.

[0119] Specifically, after the voice broadcast is finished, it is necessary to determine whether the recording is completed, by framing the collected information to obtain framing information, and then performing image recognition detection on the framing information to obtain the detection result.

[0120] In this embodiment, by monitoring in real time whether the recording is completed after the voice broadcast is completed, the current method of waiting for a fixed time to detect is avoided (the recording is not completed if the time is too short, and the recording is not completed if the time is too long), which is conducive to accurately controlling the recording time and improving recording efficiency.

[0121] In some optional implementations, in step S203, the current link is identity authentication, and the target port is used to call the target detection model to perform quality inspection on the dual-recorded information to obtain the quality inspection result, including the following steps:

[0122] Extracting a number of video frame images from the dual-recording information at preset intervals;

[0123] Performing face recognition on the video frame image, and taking the video frame image containing the face image as the target image;

[0124] Perform liveness recognition based on the target image to obtain liveness recognition results, and perform micro-expression recognition on the target image to obtain micro-expression recognition results;

[0125] The quality inspection results are determined based on the facial recognition results and micro-expression recognition results.

[0126] The preset interval can be set according to actual needs, for example, 0.1 seconds.

[0127] In this embodiment, a combination of liveness recognition and micro-expression recognition is used to determine that the subject in the dual recording process is a real person and voluntarily participates in the recording, thereby ensuring the legality of the recording process.

[0128] In some optional implementations, performing liveness recognition based on the target image to obtain a liveness recognition result specifically includes the following steps:

[0129] Sort the target images in the order of time frames to obtain an image sequence;

[0130] Extract the features of each target image in the image sequence to obtain a basic feature sequence;

[0131] After encoding the basic feature sequence, the basic feature sequence is sequentially input into the trained heart rhythm feature extraction network to obtain heart rate feature information. The trained heart rhythm feature extraction network is a recurrent neural network.

[0132] The heart rate feature information is input into the trained liveness recognition model for liveness recognition to obtain the liveness recognition result.

[0133] In this embodiment, the heart rhythm information is extracted through a recurrent neural network, and living body recognition is performed based on the heart rhythm information, which is beneficial to improving the accuracy of living body recognition.

[0134] In some optional implementations, micro-expression recognition is performed on a target image, and the micro-expression recognition result specifically includes the following steps:

[0135] Through the preset convolutional neural network model, the expression unit recognition is performed on the target image to obtain the set of units to be recognized contained in each target image;

[0136] According to the mapping relationship between each preset expression unit combination and each micro-expression in the preset expression unit combination set, the micro-expression corresponding to the to-be-recognized unit set contained in the target image is determined as the target micro-expression.

[0137] Specifically, the preset expression unit combination set includes multiple preset expression unit combinations, each preset expression unit combination has a mapping relationship with a micro-expression, and the preset expression unit combination closest to the unit set to be identified contained in the facial image is selected from the preset expression unit combination set as the target expression unit combination, and then the micro-expression corresponding to the target expression unit combination is used as the target micro-expression corresponding to the unit set to be identified contained in the facial image.

[0138] Further, according to the mapping relationship between each preset expression unit combination and each micro-expression in the preset expression unit combination set, the step of determining the micro-expression corresponding to the to-be-recognized unit set contained in the target image as the target micro-expression specifically includes:

[0139] Obtain a preset expression unit combination set, wherein the expression unit combination set includes k preset expression unit combinations, each preset expression unit combination corresponds to a micro-expression, and k is a positive integer;

[0140] Calculate the similarity between the set of units to be recognized and each preset expression unit combination respectively, and obtain k similarity values;

[0141] From the k similarity values, select the preset expression unit combination corresponding to the largest similarity value as the target unit combination;

[0142] According to the mapping relationship, the micro-expression corresponding to the target unit combination is obtained as the target micro-expression.

[0143] Among them, the preset expression unit combination closest to the set of units to be recognized contained in the facial image is selected. Specifically, the similarity between the set of units to be recognized and each preset expression unit combination in the preset expression unit combination set is calculated, and then the similarity value with the largest value is selected from the obtained similarity values ​​as the target similarity value, and then the preset expression unit combination corresponding to the target similarity value is used as the preset expression unit combination closest to the set of units to be recognized.

[0144] In this embodiment, for each facial image in the image set, expression unit recognition is performed on the facial image through a preset convolutional neural network model to obtain a set of units to be recognized contained in each facial image, and then according to the mapping relationship between each preset expression unit combination and each micro-expression in the preset expression unit combination set, the micro-expression corresponding to the set of units to be recognized contained in the facial image is determined as the target micro-expression, thereby realizing intelligent recognition of each facial image in the basic image set through a neural network, obtaining the target micro-expression of each facial image, and improving the efficiency of obtaining the target micro-expression.

[0145] The blockchain referred to in this application is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, platform product service layer, and application service layer.

[0146] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0147] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0148] This application can be applied in the field of smart real-time dual recording, thereby promoting the construction of smart cities.

[0149] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0150] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0151] Further references Figure 3 , as a response to the above Figure 2 The present application provides an embodiment of a real-time dual recording device, and the device embodiment is similar to Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0152] like Figure 3 As shown, the real-time dual recording device 300 described in this embodiment includes: a collection module 301, a selection module 302, a quality inspection module 303 and a determination module 304. Among them:

[0153] The acquisition module 301 is used to acquire audio and video in real time based on the current link to obtain current dual recording information;

[0154] A selection module 302 is used to obtain the target detection model corresponding to the current link and the model type corresponding to the target detection model according to a preset model configuration table, and the model type includes a local model and a remote model;

[0155] The quality inspection module 303 is used to determine the target port by model type, and use the target port to call the target detection model to perform quality inspection on the dual recording information to obtain the quality inspection result;

[0156] The determination module 304 is used to execute the next step if the quality inspection result is qualified, and to execute the corresponding re-recording measures if the quality inspection result is unqualified.

[0157] In this embodiment, by dividing the double recording quality inspection process into quality inspections of different links, different quality inspection types and models are set for each link to ensure the quality of quality inspection in each link, save server resources, and at the same time, help improve quality inspection efficiency.

[0158] In some optional implementations of this embodiment, the acquisition module 301 includes:

[0159] A parsing unit is used to obtain the double-recording rules of the current link, and parse the obtained double-recording rules to obtain semantic information;

[0160] A conversion unit, used to convert the semantic information into voice guidance information by using a text-to-speech method;

[0161] The guidance unit is used to generate AI voice information according to the voice guidance information, and to perform voice broadcast through the AI ​​voice information to guide the audio and video collection and obtain the current dual recording information.

[0162] In this embodiment, by semantically understanding the set rules, generating text semantic information, and then converting it into voice, guiding through AI synthesized voice, it is beneficial for the client to respond quickly and accurately, and improves the efficiency and accuracy of the dual recording process.

[0163] In some optional implementations of this embodiment, the guiding unit includes:

[0164] The framing subunit is used to frame the collected information after the voice broadcast is finished to obtain frame information;

[0165] The detection subunit is used to perform real-time detection on the frame information to determine whether the recording of the event corresponding to the current link has been completed;

[0166] The judging subunit is used to end the acquisition if the judgment result is that the recording is completed, and use the acquired information obtained at this time as the current dual recording information.

[0167] In this embodiment, by monitoring in real time whether the recording is completed after the voice broadcast is completed, the current method of waiting for a fixed time to detect is avoided (the recording is not completed if the time is too short, and the recording is not completed if the time is too long), which is conducive to accurately controlling the recording time and improving recording efficiency.

[0168] In some optional implementations of this embodiment, the quality inspection module 403 includes:

[0169] A video frame extraction unit, used to extract a number of video frame images from the dual-recording information at preset intervals;

[0170] A face recognition unit, used to perform face recognition on a video frame image, and use the video frame image containing the face image as a target image;

[0171] A feature recognition unit is used to perform liveness recognition based on the target image to obtain a liveness recognition result, and to perform micro-expression recognition on the target image to obtain a micro-expression recognition result;

[0172] The result determination unit is used to determine the quality inspection result according to the face recognition result and the micro-expression recognition result.

[0173] In this embodiment, a combination of liveness recognition and micro-expression recognition is used to determine that the subject in the dual recording process is a real person and voluntarily participates in the recording, thereby ensuring the legality of the recording process.

[0174] In some optional implementations of this embodiment, the feature recognition unit includes:

[0175] A sorting subunit, used for sorting the target images in the order of time frames to obtain an image sequence;

[0176] An image feature extraction subunit is used to extract the features of each target image in the image sequence to obtain a basic feature sequence;

[0177] The heart rhythm feature extraction subunit is used to encode the basic feature sequence and then sequentially input it into the trained heart rhythm feature extraction network to obtain heart rate feature information. The trained heart rhythm feature extraction network is a recurrent neural network.

[0178] The liveness recognition subunit is used to input the heart rate characteristic information into the trained liveness recognition model to perform liveness recognition and obtain the liveness recognition result.

[0179] In this embodiment, the heart rhythm information is extracted through a recurrent neural network, and living body recognition is performed based on the heart rhythm information, which is beneficial to improving the accuracy of living body recognition.

[0180] In some optional implementations of this embodiment, the feature recognition unit further includes:

[0181] The expression recognition subunit is used to recognize expression units in the target image through a preset convolutional neural network model to obtain a set of units to be recognized contained in each target image;

[0182] The micro-expression determination subunit is used to determine the micro-expression corresponding to the set of units to be identified contained in the target image as the target micro-expression according to the mapping relationship between each preset expression unit combination and each micro-expression in the preset expression unit combination set.

[0183] Optionally, the micro-expression determination subunit includes:

[0184] An acquisition component is used to acquire a preset expression unit combination set, wherein the expression unit combination set includes k preset expression unit combinations, each preset expression unit combination corresponds to a micro-expression, and k is a positive integer;

[0185] A calculation component is used to calculate the similarity between the set of units to be recognized and each preset expression unit combination, and obtain k similarity values;

[0186] A selection component is used to select a preset expression unit combination corresponding to the largest similarity value from the k similarity values ​​as a target unit combination;

[0187] The confirmation component is used to obtain the micro-expression corresponding to the target unit combination according to the mapping relationship as the target micro-expression.

[0188] In this embodiment, for each facial image in the image set, expression unit recognition is performed on the facial image through a preset convolutional neural network model to obtain a set of units to be recognized contained in each facial image, and then according to the mapping relationship between each preset expression unit combination and each micro-expression in the preset expression unit combination set, the micro-expression corresponding to the set of units to be recognized contained in the facial image is determined as the target micro-expression, thereby realizing intelligent recognition of each facial image in the basic image set through a neural network, obtaining the target micro-expression of each facial image, and improving the efficiency of obtaining the target micro-expression.

[0189] To solve the above technical problems, the present application also provides a computer device. Figure 4 , Figure 4 This is a basic structural block diagram of the computer device in this embodiment.

[0190] The computer device 4 includes a memory 41, a processor 42, and a network interface 43 that are interconnected and communicated through a system bus. It should be noted that the figure only shows a computer device 4 with a memory 41, a processor 42, and a network interface 43, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable gate arrays (Field-Programmable Gate Array, FPGA), digital processors (Digital Signal Processor, DSP), embedded devices, etc.

[0191] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0192] The memory 41 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or D real-time dual recording memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 41 can be an internal storage unit of the computer device 4, such as a hard disk or memory of the computer device 4. In other embodiments, the memory 41 can also be an external storage device of the computer device 4, such as a plug-in hard disk equipped on the computer device 4, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Of course, the memory 41 can also include both the internal storage unit of the computer device 4 and its external storage device. In this embodiment, the memory 41 is generally used to store the operating system and various application software installed on the computer device 4, such as computer-readable instructions of the real-time dual recording method, etc. In addition, the memory 41 can also be used to temporarily store various types of data that have been output or are to be output.

[0193] The processor 42 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 42 is generally used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is used to run the computer-readable instructions stored in the memory 41 or process data, such as computer-readable instructions for running the real-time dual recording method.

[0194] The network interface 43 may include a wireless network interface or a wired network interface. The network interface 43 is generally used to establish a communication connection between the computer device 4 and other electronic devices.

[0195] By splitting the double-recording quality inspection process into different links of quality inspection, setting different quality inspection types and models for each link, we can ensure the quality of quality inspection in each link, save server resources, and at the same time, improve quality inspection efficiency.

[0196] The present application also provides another embodiment, namely, providing a computer-readable storage medium, wherein the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor to enable the at least one processor to perform the steps of the real-time dual recording method as described above.

[0197] By splitting the double-recording quality inspection process into different links of quality inspection, setting different quality inspection types and models for each link, we can ensure the quality of quality inspection in each link, save server resources, and at the same time, improve quality inspection efficiency.

[0198] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0199] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to replace some of the technical features therein with equivalents. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. A real-time dual recording method, characterized in that: The steps include: Based on the current link, audio and video are collected in real time to obtain the current dual recording information; According to the preset model configuration table, obtain the target detection model corresponding to the current link and the model type corresponding to the target detection model, wherein the model type includes a local model and a remote model; Determine the target port by model type, and use the target port to call the target detection model to perform quality inspection on the dual recording information to obtain a quality inspection result; If the quality inspection result is qualified, the next step will be executed. If the quality inspection result is unqualified, the corresponding re-recording measures will be implemented.

2. The real-time dual recording method according to claim 1, characterized in that: The step of collecting audio and video in real time based on the current link to obtain the current dual recording information specifically includes: Acquire the dual recording rules of the current link, and parse the acquired dual recording rules to obtain semantic information; The semantic information is converted into voice guidance information by using a text-to-speech method; Generate AI voice information according to the voice guidance information, and perform voice broadcast through the AI ​​voice information to guide the user to collect audio and video and obtain the current dual recording information.

3. The real-time dual recording method according to claim 2, characterized in that: The step of performing voice broadcasting through AI voice information to guide audio and video collection and obtain current dual recording information specifically includes: After the voice broadcast is finished, the collected information is divided into frames to obtain frame information; Performing real-time detection on the frame information to determine whether the event corresponding to the current link has been recorded; If the judgment result is that the recording is completed, the acquisition is ended, and the acquisition information obtained at this time is used as the current dual recording information.

4. The real-time dual recording method according to claim 1, characterized in that: The current link is identity authentication, and the step of using the target port to call the target detection model, performing quality inspection on the dual recording information, and obtaining the quality inspection result specifically includes: Extracting a number of video frame images from the dual-recording information at preset intervals; Performing face recognition on the video frame image, and taking the video frame image containing the face image as the target image; Performing liveness recognition based on the target image to obtain a liveness recognition result, and performing micro-expression recognition on the target image to obtain a micro-expression recognition result; The quality inspection result is determined according to the face recognition result and the micro-expression recognition result.

5. The real-time dual recording method according to claim 4, characterized in that: The step of performing liveness recognition based on the target image to obtain a liveness recognition result specifically includes: Sorting the target images in order of time frames to obtain an image sequence; Extracting features of each of the target images in the image sequence to obtain a basic feature sequence; After encoding the basic feature sequence, sequentially inputting it into a trained heart rhythm feature extraction network to obtain heart rate feature information, wherein the trained heart rhythm feature extraction network is a recurrent neural network; The heart rate characteristic information is input into a trained liveness recognition model to perform liveness recognition to obtain the liveness recognition result.

6. The real-time dual recording method according to claim 4, characterized in that: The step of performing micro-expression recognition on the target image and obtaining the micro-expression recognition result specifically includes: Using a preset convolutional neural network model, the target image is subjected to expression unit recognition to obtain a set of units to be recognized contained in each of the target images; According to the mapping relationship between each preset expression unit combination and each micro-expression in the preset expression unit combination set, the micro-expression corresponding to the to-be-recognized unit set contained in the target image is determined as the target micro-expression.

7. The real-time dual recording method according to claim 6, characterized in that: The step of determining the micro-expression corresponding to the to-be-recognized unit set contained in the target image as the target micro-expression according to the mapping relationship between each preset expression unit combination and each micro-expression in the preset expression unit combination set specifically includes: Obtain a preset expression unit combination set, wherein the expression unit combination set includes k preset expression unit combinations, each preset expression unit combination corresponds to a micro-expression, and k is a positive integer; Calculating the similarity between the to-be-recognized unit set and each of the preset expression unit combinations respectively, to obtain k similarity values; Selecting a preset expression unit combination corresponding to the largest similarity value from the k similarity values ​​as the target unit combination; According to the mapping relationship, the micro-expression corresponding to the target unit combination is obtained as a target micro-expression.

8. A real-time dual recording device, characterized in that: include: The acquisition module is used to collect audio and video in real time based on the current link to obtain the current dual recording information; A selection module is used to obtain the target detection model corresponding to the current link and the model type corresponding to the target detection model according to a preset model configuration table, wherein the model type includes a local model and a remote model; A quality inspection module, used to determine the target port by model type, and use the target port to call the target detection model, perform quality inspection on the dual recording information, and obtain a quality inspection result; The determination module is used to execute the next step if the quality inspection result is qualified, and to execute the corresponding re-recording measures if the quality inspection result is unqualified.

9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the real-time dual recording method according to any one of claims 1 to 7 when executing the computer-readable instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of the real-time dual recording method according to any one of claims 1 to 7 are implemented.

Images