Target object recognition method, apparatus, computer device, and computer program product

Abstract

The application relates to the technical field of artificial intelligence, and provides a target object identification method and device in a target scene, computer equipment, a storage medium and a computer program product, which can be applied to the fields of intelligent travel and smart city construction. The method comprises the following steps: when at least one face object is detected in a shooting visual field picture in a target scene, collecting a video frame comprising the face object; when a target video frame exists in the collected video frame, so that a face feature detection result of the target video frame meets a face feature condition adapted to the target scene, and a living body detection result meets a living body detection condition, determining a video clip formed by the collected video frame; performing face feature identification on the target video frame to determine a candidate object in the target video frame; and when a behavior feature of the candidate object in the video clip meets a behavior feature condition adapted to the target scene, determining that the candidate object is a target object in the target scene, so that the accuracy of the identified target object is improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method, apparatus, computer device, storage medium, and computer program product for identifying target objects in a target scenario. Background Technology

[0002] With the development of technology, facial recognition technology has been increasingly widely used. Facial recognition is a biometric identification technology that authenticates identity based on a person's facial features. It can perform identification in a contactless manner, avoiding the leakage of personal information, achieving fast and efficient identity verification, and shortening interaction time. Taking facial recognition in payment scenarios as an example, facial recognition payment is a new payment method centered on facial recognition. Through facial recognition, the user's payment process time can be effectively shortened.

[0003] However, existing facial recognition methods generally involve collecting facial images and comparing facial features. But in crowded places, the recognition results obtained based on the image information in the facial images may not accurately identify the target object. Summary of the Invention

[0004] Therefore, it is necessary to provide a method, apparatus, computer device, storage medium, and computer program product for identifying target objects in a target scenario, which can accurately determine the target objects in the target scenario.

[0005] A method for identifying target objects in a target scene, the method comprising:

[0006] When at least one human face is detected in the field of view of the target scene, video frames containing the at least one human face are captured.

[0007] When a target video frame exists in the acquired video frames, such that the face feature detection result of the target video frame meets the face feature conditions adapted to the target scene, and the liveness detection result corresponding to the target video frame meets the liveness detection conditions, the video segment composed of the acquired video frames is determined.

[0008] Facial feature recognition is performed on the target video frame to determine candidate objects in the target video frame;

[0009] When the behavioral characteristics of the candidate object in the video segment match the behavioral characteristic conditions adapted to the target scene, the candidate object is determined to be the target object in the target scene.

[0010] A target object recognition device in a target scene, the device comprising:

[0011] The video frame acquisition module is used to acquire video frames containing at least one face object when at least one face object is detected in the shooting field of view of the target scene.

[0012] The target video frame filtering module is used to determine the video segment composed of the acquired video frames when there is a target video frame in the acquired video frames, such that the face feature detection result of the target video frame meets the face feature conditions adapted to the target scene, and the liveness detection result corresponding to the target video frame meets the liveness detection conditions.

[0013] The candidate object determination module is used to perform facial feature recognition on the target video frame and determine the candidate objects in the target video frame;

[0014] The target object determination module is used to determine the candidate object as the target object in the target scene when the behavioral characteristics of the candidate object in the video segment meet the behavioral characteristic conditions adapted to the target scene.

[0015] In one embodiment, the target object recognition device in the target scene includes a sending module;

[0016] The sending model is used to send the video segment and the target video frame to the server, so that the server executes the processing steps of the candidate object determination module and the target object determination module.

[0017] In one embodiment, the target object recognition device in the target scene further includes a video frame buffer module;

[0018] The video frame caching module is used to cache the acquired video frames sequentially to the target queue according to the acquisition time, and the number of cacheable video frames in the target queue is not greater than a number threshold.

[0019] The target video frame filtering is further used to terminate video frame acquisition when a target video frame exists in the acquired video frames, such that the face feature detection result of the target video frame meets the face feature conditions adapted to the target scene, and the liveness detection result corresponding to the target video frame meets the liveness detection conditions; and to obtain video segments based on the video frames cached in the target queue.

[0020] In one embodiment, the target object recognition device in the target scene further includes a frame extraction processing module;

[0021] The frame extraction processing module is used to perform frame extraction processing on the video frames in the target queue according to the frame extraction parameters when the number of video frames cached in the target queue reaches the number threshold; to cache newly acquired video frames in the target queue; and to perform frame extraction processing on continuously acquired video frames in the target queue according to the frame extraction parameters.

[0022] In one embodiment, the target object recognition device in the target scene further includes a secondary frame extraction processing module;

[0023] The secondary frame extraction processing module is used to update the frame extraction parameters when the number of remaining video frames after frame extraction processing of the target queue reaches the number threshold; perform secondary frame extraction processing on the video frames in the target queue based on the updated frame extraction parameters; cache the newly acquired video frames in the target queue; and perform frame extraction processing on the newly cached video frames in the target queue based on the last video frame after secondary frame extraction processing and according to the updated frame extraction parameters.

[0024] In one embodiment, the frame-dropping parameter is the frame-dropping interval frame number corresponding to the frame-dropping method of discarding one video frame in every multi-frame video frame.

[0025] The frame extraction module is further configured to determine the initial inter-frame time interval of the video frames in the target queue based on the timestamps carried by the video frames in the target queue; determine the frame extraction interval time based on the initial inter-frame time interval and the frame extraction interval number; and perform frame extraction processing on the video frames in the target queue based on the frame extraction interval time.

[0026] In one embodiment, the target object recognition device in the target scene further includes a secondary frame extraction processing module;

[0027] The secondary frame extraction processing module is used to reduce the frame extraction interval time when the number of remaining video frames after frame extraction processing of the target queue reaches the number threshold; perform secondary frame extraction processing on the video frames in the target queue based on the reduced frame extraction interval time; cache the newly acquired video frames in the target queue; and perform frame extraction processing on the video frames in the target queue whose inter-frame interval time is the initial inter-frame interval time according to the reduced frame extraction interval time.

[0028] In one embodiment, the frame extraction parameter is the number of consecutive frame extraction frames corresponding to the frame extraction method of retaining one video frame in each multi-frame video frame.

[0029] The frame extraction module is further configured to determine the initial inter-frame time interval of video frames in the target queue based on the timestamps carried by the video frames cached in the target queue; obtain the updated inter-frame time interval based on the initial inter-frame time interval and the number of consecutive extracted frames; and perform frame extraction processing on video frames in the target queue that do not meet the updated inter-frame time interval.

[0030] In one embodiment, the target object recognition device in the target scene further includes a secondary frame extraction processing module;

[0031] The secondary frame extraction processing module is used to extend the update inter-frame time interval when the number of remaining video frames after frame extraction processing of the target queue reaches the number threshold; perform secondary frame extraction processing on the video frames in the target queue based on the extended inter-frame time interval; cache newly acquired video frames in the target queue; and perform frame extraction processing on video frames in the target queue that do not meet the extended inter-frame time interval.

[0032] In one embodiment, the target object recognition device in the target scene further includes a video frame acquisition process control module;

[0033] The video frame acquisition process control module is used to start the video frame acquisition process when at least one face object is detected in the shooting field of view of the target scene, and to simultaneously acquire color video frames, infrared video frames and depth video frames.

[0034] The target video frame filtering module is further configured to, when there is a target color video frame in the acquired color video frames whose face feature detection results meet the face feature conditions, determine the target infrared video frame and the target depth video frame based on the timestamp carried by the target color video frame; perform liveness detection based on the target color video frame, the target infrared video frame, and the target depth video frame; and when the liveness detection result meets the liveness detection conditions, end the video frame acquisition process and determine the video segments composed of the acquired color video frame, infrared video frame, and depth video frame.

[0035] In one embodiment, the video clips include color video clips, infrared video clips, and depth video clips;

[0036] The sending module is further configured to compress the acquired color video clips, infrared video clips, and depth video clips into video clips, and to compress the target color video frame, target infrared video frame, and target depth video frame into images; and to send the compressed video clips and compressed images to the server.

[0037] In one embodiment, the candidate object determination module is further configured to determine a target face region in the target video frame based on the area occupied by each face region in the target video frame; and to perform face feature recognition on the face features in the target face region to determine a candidate object.

[0038] In one embodiment, the target object determination module is further configured to obtain the behavioral characteristics of the candidate object based on the relative position change of the candidate object in each video frame of the video segment; when the behavioral characteristic analysis result corresponding to the behavioral characteristics matches the behavioral characteristic conditions of the user corresponding to the face recognition payment scenario, the candidate object is determined as the target payment object.

[0039] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program performing the following steps:

[0040] When at least one human face is detected in the field of view of the target scene, video frames containing the at least one human face are captured.

[0041] When a target video frame exists in the acquired video frames, such that the face feature detection result of the target video frame meets the face feature conditions adapted to the target scene, and the liveness detection result corresponding to the target video frame meets the liveness detection conditions, the video segment composed of the acquired video frames is determined.

[0042] Facial feature recognition is performed on the target video frame to determine candidate objects in the target video frame;

[0043] When the behavioral characteristics of the candidate object in the video segment match the behavioral characteristic conditions adapted to the target scene, the candidate object is determined to be the target object in the target scene.

[0044] A computer-readable storage medium having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0045] When at least one human face is detected in the field of view of the target scene, video frames containing the at least one human face are captured.

[0046] When a target video frame exists in the acquired video frames, such that the face feature detection result of the target video frame meets the face feature conditions adapted to the target scene, and the liveness detection result corresponding to the target video frame meets the liveness detection conditions, the video segment composed of the acquired video frames is determined.

[0047] Facial feature recognition is performed on the target video frame to determine candidate objects in the target video frame;

[0048] When the behavioral characteristics of the candidate object in the video segment match the behavioral characteristic conditions adapted to the target scene, the candidate object is determined to be the target object in the target scene.

[0049] A computer program product includes a computer program, characterized in that, when the computer program is executed by a processor, it performs the following steps:

[0050] When at least one human face is detected in the field of view of the target scene, video frames containing the at least one human face are captured.

[0051] When a target video frame exists in the acquired video frames, such that the face feature detection result of the target video frame meets the face feature conditions adapted to the target scene, and the liveness detection result corresponding to the target video frame meets the liveness detection conditions, the video segment composed of the acquired video frames is determined.

[0052] Facial feature recognition is performed on the target video frame to determine candidate objects in the target video frame;

[0053] When the behavioral characteristics of the candidate object in the video segment match the behavioral characteristic conditions adapted to the target scene, the candidate object is determined to be the target object in the target scene.

[0054] The aforementioned target object recognition method, apparatus, computer equipment, storage medium, and computer program product, when detecting at least one face object in the shooting field of view in the target scene, acquires video frames containing at least one face object. When a target video frame exists within the acquired video frames, such that the face feature detection result of the target video frame matches the face feature conditions adapted to the target scene, and the liveness detection result corresponding to the target video frame matches the liveness detection conditions, the video segment composed of the acquired video frames is determined. This ensures that at least one video frame in the acquired video segment meets both the face feature conditions and the liveness detection conditions. Based on the face feature recognition of the target video frame, candidate objects are determined, and the behavioral characteristics corresponding to the candidate objects in the video segment match the behavioral characteristic conditions adapted to the target scene. Therefore, the candidate objects can be accurately identified as target objects in the target scene. Attached Figure Description

[0055] Figure 1 This is an application environment diagram of a target object recognition method in a target scene in one embodiment;

[0056] Figure 2This is a flowchart illustrating a target object recognition method in a target scene in one embodiment;

[0057] Figure 3 This is a schematic diagram of the frame extraction results corresponding to different frame extraction parameters in one embodiment.

[0058] Figure 4 This is a schematic diagram illustrating the process of buffering video frames in the target queue when the frame extraction parameter is the frame extraction interval frame number in one embodiment.

[0059] Figure 5 This is a schematic diagram illustrating a secondary frame extraction process performed on cached video frames in the target queue by updating frame extraction parameters in one embodiment.

[0060] Figure 6 This is a flowchart illustrating a target object recognition method in a target scene, as shown in another embodiment.

[0061] Figure 7 This is a flowchart illustrating a target object recognition method in a target scene in one embodiment;

[0062] Figure 8 This is a schematic diagram of a terminal page for facial recognition payment in one embodiment;

[0063] Figure 9 This is a schematic diagram illustrating the process of frame extraction from a target queue in one embodiment.

[0064] Figure 10 This is a structural block diagram of a target object recognition device in a target scene according to one embodiment;

[0065] Figure 11 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0066] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0067] Artificial Intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to have perception, reasoning, and decision-making capabilities. AI technology is a comprehensive discipline involving a wide range of fields, encompassing both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly include computer vision, speech processing, natural language processing, as well as machine learning / deep learning, autonomous driving, and intelligent transportation.

[0068] Computer vision (CV) is the science that studies how to enable machines to "see." More specifically, it refers to machine vision, which uses cameras and computers to replace human eyes in tasks such as target recognition, tracking, and measurement, and then performs image processing to create images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision studies related theories and technologies, attempting to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include biometric recognition technologies such as facial recognition.

[0069] The solutions provided in this application relate to technologies such as artificial intelligence facial recognition, and are specifically illustrated through the following embodiments. The target object recognition method in the target scenario provided in this application can be applied to, for example... Figure 1The application environment shown is illustrated. Terminal 110 communicates with server 120 via a network. Terminal 110 is equipped with a camera device 102, such as a webcam. Terminal 110 can be a fixed terminal, permanently installed in a specific application scenario, such as a supermarket for facial recognition payment, or a high-speed rail station or train station for facial recognition gate access. Terminal 110 can also be a mobile terminal, corresponding to different application scenarios by launching different applications or different pages of applications. For example, the login page of an application corresponds to a facial recognition login scenario, and the payment page of an application corresponds to a facial recognition payment scenario. The terminal can also be an in-vehicle terminal, where users can identify drivers or passengers through applications installed on the in-vehicle terminal. Taking an in-vehicle terminal installed on a bus or rental car as an example, drivers can accurately identify the target object through a check-in application installed on the in-vehicle terminal, thus conveniently and quickly checking in for driving. Passengers can accurately identify the target object through a payment application installed on the in-vehicle terminal, quickly and conveniently paying for their fare.

[0070] Specifically, in a specific application, such as Figure 1 As shown, when terminal 110 detects at least one face object in the field of view of the target scene, it acquires video frames containing at least one face object. When a target video frame exists in the acquired video frames, such that the face feature detection result of the target video frame matches the face feature conditions adapted to the target scene, and the liveness detection result corresponding to the target video frame matches the liveness detection conditions, the video segment composed of the acquired video frames is determined. Terminal 110 sends the video segment and the target video frame to server 120. Server 120 performs face feature recognition on the target video frame to determine candidate objects in the target video frame. When the behavioral features corresponding to the candidate object in the video segment match the behavioral feature conditions adapted to the target scene, the candidate object is determined to be the target object in the target scene, thus achieving accurate identification of the target object in the target scene.

[0071] In another specific application, when the terminal detects at least one face in the field of view of the target scene, it captures video frames containing at least one face. If a target video frame is present in the captured video frames, and the face feature detection result of the target video frame matches the face feature conditions adapted to the target scene, and the liveness detection result of the target video frame matches the liveness detection conditions, then the video segment composed of the captured video frames is determined. The terminal, based on its built-in processor, performs face feature recognition on the target video frame to identify candidate objects within it. When the behavioral characteristics of a candidate object in the video segment match the behavioral feature conditions adapted to the target scene, the candidate object is determined to be the target object in the target scene, thus achieving accurate identification of the target object in the target scene.

[0072] The terminal 110 can be, but is not limited to, various devices set up in fixed locations, personal computers, laptops, smartphones, tablets, portable wearable devices, and vehicle terminals. The server 120 can be implemented using a standalone server or a server cluster composed of multiple servers.

[0073] In one embodiment, such as Figure 2 As shown, a method for target object recognition in a target scene is provided, which can be applied to... Figure 1 Taking the terminal in the example, the explanation includes the following steps:

[0074] Step 202: When at least one face object is detected in the shooting field of view of the target scene, video frames containing at least one face object are captured.

[0075] The target scenario refers to the specific situation in which target object recognition is required. The target scenario can be determined by the specific functions the terminal needs to perform. If the terminal's function is relatively simple, it is generally located in a specific space to perform data processing in that specific scenario, and its corresponding target scenario is usually fixed. For example, a terminal at a supermarket checkout is generally only used for payment processing, and the corresponding scenario is a facial recognition payment scenario; another example is a terminal at a train station or high-speed rail station gate used for security checks and gate control, and the corresponding scenario is a facial recognition security check scenario; yet another example is a terminal at a residential community or building access control point used for access control, and the corresponding scenario is a facial recognition access scenario. For mobile or portable terminals with multiple functions, the corresponding target scenario is determined by the functions that the currently running application on the terminal or the currently running page of the application can perform. For example, when a user launches a payment application installed on the terminal and enters the facial recognition payment page, the corresponding scenario is a facial recognition payment scenario; another example is when a user launches the application and enters the account login page, the corresponding scenario is a facial recognition login scenario.

[0076] The captured field of view refers to the image captured by the shooting device. The shooting device can be fixed hardware installed on the terminal, such as a terminal device with an embedded camera; or it can be a standalone device connected to the terminal, allowing the terminal to control the connected shooting device, such as a standalone camera and terminal. The terminal and shooting device can be configured according to the specific application scenario. Specifically, the terminal may include a display screen, on which the captured field of view can be displayed, allowing the user to adjust their angle or posture. Further, in the target scene, the terminal performs a preliminary analysis of the captured field of view. When a face is detected in the captured field of view, video frames containing at least one face are captured. Specifically, the preliminary analysis can be performed using a face detection algorithm to analyze whether a face exists in the captured field of view. A face detection algorithm is an algorithm that searches for any given image using a certain strategy to determine whether it contains a face.

[0077] A video frame is the smallest unit that makes up a video segment, and each video frame in a video segment is a still image. The acquisition of video frames by the terminal is the process of capturing video frames from the camera. It's important to note that this acquisition of video frames differs from capturing the field of view. For the field of view, the terminal discards the captured image after detection, a pipelined process of use and discard. For video frames containing at least one face, the video frame is extracted to facilitate further processing, a storage-based process of storing the data before use.

[0078] Specifically, in the target scenario, the terminal calls the shooting device to capture the current scene. The terminal detects the shooting field of view captured by the shooting device. When the terminal detects that there is at least one face object in the shooting field of view, it captures video frames containing at least one face object. By limiting the timing of video frame capture, it can avoid capturing video frames that do not contain face objects, effectively reducing the data processing pressure when processing video frames later.

[0079] Step 204: When a target video frame exists in the acquired video frames, such that the face feature detection result of the target video frame meets the face feature conditions adapted to the target scene, and the liveness detection result corresponding to the target video frame meets the liveness detection conditions, the video segment composed of the acquired video frames is determined.

[0080] The acquired video frames are those captured starting from the moment the terminal detects at least one human face in the field of view, and each captured video frame also includes at least one human face. The target video frame is one or more frames from the acquired video frames. Compared to other captured video frames, the target video frame must meet the following requirements: the face feature detection result must match the face feature conditions suitable for the target scene, and the liveness detection result corresponding to the target video frame must meet the liveness detection conditions.

[0081] Facial feature conditions refer to the conditions that a facial object must meet in a video frame. These conditions can be requirements for the displayed content of a facial object within the video frame, such as including key facial feature points. Specifically, facial feature conditions can be configured based on the target scene. For example, the facial shooting angle corresponding to each key facial feature point must be within a preset angle range. Alternatively, when the number of target objects in the target scene can only be one, the facial feature conditions can be configured such that the proportion of the facial region in the video frame reaches a preset ratio.

[0082] Liveness detection conditions are the criteria that a face must meet to be considered alive. These conditions can be specifically determined based on the liveness detection method. Liveness detection methods include static liveness detection and dynamic liveness detection. Static liveness detection uses a combination of multiple images captured simultaneously: depth maps, infrared maps, and color maps. Depth maps determine the three-dimensional distribution of the detected object, infrared maps determine its temperature distribution, and color maps determine its planar distribution. By comparing these multiple images, liveness can be accurately identified. Dynamic liveness detection uses one or more actions such as blinking, opening the mouth, shaking the head, or nodding, employing facial landmark localization and face tracking techniques across multiple consecutive frames to verify liveness.

[0083] A video clip consists of multiple consecutive video frames. When the number of consecutive video frames changes by more than 24 frames per second, according to the principle of visual persistence, the human eye cannot distinguish a single static video frame. The visual effect appears smooth and continuous, which is a video clip.

[0084] Specifically, the terminal performs face feature detection and liveness detection on each video frame acquired in chronological order. When the terminal acquires a target video frame in the video frame, such that the face feature detection result of the target video frame meets the face feature conditions suitable for the target scene, and the liveness detection result of the target video frame meets the liveness detection conditions, the video segment composed of the acquired video frames is determined. This ensures that at least one video frame in the obtained video segment simultaneously meets the face feature conditions and the liveness detection conditions, thereby improving the accuracy of target object recognition in subsequent processing.

[0085] Step 206: Perform facial feature recognition on the target video frame to determine the candidate objects in the target video frame.

[0086] Facial recognition is a biometric technology that uses facial feature information for identification. It involves using a camera or webcam to capture images or videos containing faces, automatically detecting and tracking faces within the images, and then performing facial recognition on the detected faces. Candidate objects are facial objects in the target video frame that need further confirmation to determine if they belong to the target object.

[0087] The number of candidate objects can be the same as the number of faces in the target video frame. Candidate objects are determined by performing facial feature recognition on each face region in the target video frame. Alternatively, a candidate object can be a unique one selected from face objects based on facial features. Specifically, facial feature recognition can filter facial features in the target video frame based on facial feature detection conditions to determine a unique face object. Furthermore, facial feature recognition can also identify the identity of a candidate object based on its facial features. By identifying candidate objects from the target video frame, it is possible to quickly and accurately obtain relevant information about the candidate objects from the video clip, and then perform behavioral analysis on the candidate objects.

[0088] Specifically, the process of performing facial feature recognition on the target video frame to identify candidate objects can be implemented on the terminal or on a server connected to the terminal. The specific configuration depends on the actual application requirements. Performing facial feature recognition on the target video frame on the terminal reduces the number of interactions between the terminal and the server, mitigating the impact of network instability. Alternatively, the terminal can send the target video frame to the server, where facial feature recognition is performed to identify candidate objects. This allows the data processing to be executed on the server, reducing the data processing load on the terminal and preventing delays, stuttering, or crashes caused by excessive data processing.

[0089] Step 208: When the behavioral characteristics of the candidate object in the video segment meet the behavioral characteristic conditions for the target scene, the candidate object is determined to be the target object in the target scene.

[0090] Among them, behavioral features are feature data obtained based on the action changes of a certain object in each video frame of a video segment. The terminal or server extracts the corresponding information of the candidate object from each video frame of the video segment based on the identified candidate object, and performs behavioral analysis on the candidate object based on the behavioral changes of the candidate object in each video frame to obtain the behavioral features of the candidate object in the video segment.

[0091] Behavioral feature conditions are used to determine whether a candidate object is the target object in a target scenario. These conditions can be configured based on the target scenario, and the behavioral features can differ across scenarios. Specifically, the behavioral feature conditions adapted to the target scenario can be determined based on a behavioral feature analysis algorithm adapted to that scenario. This algorithm can be implemented using a pre-trained behavioral feature analysis model. For example, in a facial recognition payment scenario, a payment intention analysis model can be trained using multiple sample video clips showing payment intention. The behavioral features of the candidate object in the video clips are input into the model. When the model outputs that the candidate object has a payment intention, it meets the behavioral feature conditions adapted to the target scenario; conversely, when it outputs that the candidate object does not have a payment intention, it does not meet the behavioral feature conditions adapted to the target scenario.

[0092] Specifically, the entity executing the process of determining whether the behavioral characteristics of a candidate object in a video segment meet the behavioral characteristic conditions suitable for the target scene can be the same as the entity executing step 206. This process can be implemented on the terminal or on the server connected to the terminal. The specific configuration depends on the actual application needs. Determining whether the behavioral characteristics of a candidate object in a video segment meet the behavioral characteristic conditions suitable for the target scene on the terminal reduces the number of interactions between the terminal and the server, mitigating the impact of unstable network conditions. The terminal sends the target video frame and video segment to the server, which then determines whether the behavioral characteristics of the candidate object in the video segment meet the behavioral characteristic conditions suitable for the target scene, thereby determining whether the candidate object is the target object. This allows the data processing to be executed on the server, reducing the amount of data processing on the terminal and preventing delays, stuttering, or crashes caused by excessive data processing.

[0093] In a specific embodiment, in the target scene, the terminal controls the shooting device to capture the image. The terminal detects the field of view of the shooting device. When at least one face object is detected in the field of view, the terminal captures video frames containing at least one face object. When the terminal detects a target video frame in the captured video frames, such that the face feature detection result of the target video frame meets the face feature conditions adapted to the target scene, and the liveness detection result corresponding to the target video frame meets the liveness detection conditions, the terminal determines the video segment composed of the captured video frames. The terminal performs face feature recognition on the target video frame to determine the candidate object in the target video frame and obtains the behavioral features corresponding to the candidate object in the video segment. When the behavioral features meet the behavioral feature conditions adapted to the target scene, the terminal determines the candidate object as the target object in the target scene, so as to perform subsequent processing in the target scene based on the target object. By centralizing the above processing in the terminal, the number of interactions between the terminal and the server can be reduced, and the impact caused by unstable network conditions can be reduced.

[0094] In one embodiment, facial feature recognition is performed on the target video frame to determine candidate objects in the target video frame; when the behavioral features corresponding to the candidate object in the video segment meet the behavioral feature conditions adapted to the target scene, the candidate object is determined to be the target object in the target scene, including: sending the video segment and the target video frame to the server so that the server performs facial feature recognition on the target video frame to determine the candidate objects in the target video frame; when the behavioral features corresponding to the candidate object in the video segment meet the behavioral feature conditions adapted to the target scene, the candidate object is determined to be the target object in the target scene.

[0095] In a specific embodiment, in the target scene, the terminal controls the shooting device to capture the image. The terminal detects the field of view of the shooting device. When at least one face object is detected in the field of view, the terminal captures video frames containing at least one face object. When the terminal detects a target video frame in the captured video frames, such that the face feature detection result of the target video frame meets the face feature conditions adapted to the target scene, and the liveness detection result corresponding to the target video frame meets the liveness detection conditions, the terminal determines the video segment composed of the captured video frames. The terminal sends the video segment and the target video frame to the server. The server performs face feature recognition on the target video frame, determines the candidate objects in the target video frame, and obtains the behavioral features corresponding to the candidate objects in the video segment. When the behavioral features meet the behavioral feature conditions adapted to the target scene, the server determines the candidate object as the target object in the target scene, so that the server can perform subsequent processing in the target scene based on the target object. This allows some data processing to be performed on the server, reducing the amount of data processing on the terminal and avoiding delays, lag, or crashes caused by excessive data processing on the terminal.

[0096] In this embodiment, when at least one face object is detected in the shooting field of view under the target scene, video frames containing at least one face object are acquired. When a target video frame exists in the acquired video frames, such that the face feature detection result of the target video frame meets the face feature conditions adapted to the target scene, and the liveness detection result corresponding to the target video frame meets the liveness detection conditions, the video segment composed of the acquired video frames is determined. This ensures that at least one video frame in the acquired video segment meets both the face feature conditions and the liveness detection conditions. Based on the face feature recognition of the target video frame, a candidate object is determined. Furthermore, if the behavioral characteristics of the candidate object in the video segment meet the behavioral characteristic conditions adapted to the target scene, the candidate object can be accurately determined to be the target object under the target scene.

[0097] In one embodiment, the target object recognition method in the target scene further includes: caching the acquired video frames sequentially to the target queue according to the acquisition time, wherein the number of cacheable video frames in the target queue is not greater than a number threshold.

[0098] The target queue is a container that stores video frames sequentially. The buffer location of video frames in the target queue is determined based on the capture time of the video frames. The capture time can be added to the video frames as timestamps, and each captured video frame carries a corresponding timestamp. Generally, the video frames captured earlier are buffered in the target queue first, thus ensuring that the video frames in the target queue are arranged in chronological order. By buffering video frames sequentially into the target queue according to their capture time, and buffering video frames based on the chronological order corresponding to the timestamps they carry, errors in buffer order caused by disordered transmission can be avoided. For example, when a camera captures N video frames, the nth frame is captured before the mth frame, and the timestamp of the nth frame is earlier than that of the mth frame. Since the nth frame arrives later than the mth frame when the camera transmits the captured video frames to the terminal's target queue for caching, the mth frame will be cached first. Then, based on the timestamp order, the nth frame can be inserted before the mth frame for caching, or the uncached nth frame or the cached mth frame can be discarded directly. This ensures that the video frames in the target queue are arranged in chronological order. The number of cacheable video frames in the target queue is fixed. Specifically, a threshold value can be set to determine the maximum number of cacheable video frames in the target queue, ensuring that the number of cached video frames in the target queue does not exceed this threshold. If the number of captured video frames exceeds the threshold, the cached video frames in the target queue can be discarded so that newly captured video frames can be cached. Discarding cached video frames can be based on a first-in, first-out (FIFO) caching order, discarding the earliest cached video frame in the target queue, or video frames can be extracted and discarded at certain intervals.

[0099] Furthermore, when a target video frame exists among the acquired video frames, such that the face feature detection result of the target video frame matches the face feature conditions adapted to the target scene, and the liveness detection result corresponding to the target video frame matches the liveness detection conditions, the video segment composed of the acquired video frames is determined, including: when a target video frame exists among the acquired video frames, such that the face feature detection result of the target video frame matches the face feature conditions adapted to the target scene, and the liveness detection result corresponding to the target video frame matches the liveness detection conditions, video frame acquisition is terminated; and a video segment is obtained based on the video frames cached in the target queue.

[0100] The video segment is formed by combining cached video frames arranged chronologically in the target queue. Terminating video frame acquisition can be achieved by the terminal controlling the shooting device to pause video capture. Specifically, when at least one face is detected in the shooting field of view of the target scene, a video acquisition thread is started to acquire video frames containing at least one face. When the target video frame is present in the acquired video frames, the video acquisition thread is terminated, and the video frames acquired from the start to the end of the acquisition thread are cached in the target queue. The cached video frames in the target queue constitute the video segment. By controlling the start and end times of video frame acquisition, the acquisition of unnecessary video frames can be avoided, reducing unnecessary resource consumption.

[0101] In this embodiment, by caching the acquired video frames into a target queue where the number of cacheable video frames does not exceed a certain threshold, and obtaining video segments based on the cached video frames in the target queue, the number of video frames contained in the target video can be effectively controlled, thereby reducing the number of video frames that need to be processed in subsequent processing and improving data processing efficiency.

[0102] In one embodiment, the target object recognition method in the target scene further includes: when the number of video frames cached in the target queue reaches a number threshold, performing frame extraction processing on the video frames in the target queue according to the frame extraction parameters; caching newly acquired video frames into the target queue, and performing frame extraction processing on the continuously acquired video frames in the target queue according to the frame extraction parameters.

[0103] The frame extraction parameter is a sampling parameter used to perform mean sampling on video frames. The frame extraction parameter can be the number of frames to discard out of every N frames. Specifically, it can be either extracting 1 frame and discarding it every N frames to retain N-1 frames, or extracting N-1 frames and discarding it to retain 1 frame. For example... Figure 3 As shown, for example, 1 frame is discarded and 5 frames are retained every 6 frames, or 5 frames are discarded and 1 frame is retained every 6 frames. After the video frames in the target queue are processed according to the frame extraction parameters, the number of cached video frames in the target queue will decrease, thus freeing up new cache space in the target queue. Since the last captured video frame may be the target video frame, in order to ensure that the last captured video frame is not discarded, the newly captured video frame will be cached in the target queue first. Then, frame extraction processing will be performed on the combination of the newly cached video frame and the video frame after frame extraction. This ensures that the last captured video frame is not discarded and that the newly cached video frame is also cached according to the frame extraction parameters.

[0104] Specifically, when the terminal detects that the number of video frames cached in the target queue has reached the number threshold, it performs frame extraction processing on the video frames in the target queue according to the frame extraction parameters, reorganizes the target queue, and caches the newly acquired video frames in the reorganized target queue. When the total number of newly cached video frames and the number of cached video frames after frame extraction reaches the number threshold, it performs frame extraction processing again according to the frame extraction parameters, reorganizes the target queue, and repeats the frame extraction and target queue reorganization process until the number of video frames cached in the reorganized target queue reaches the number threshold.

[0105] In a specific application, assuming the frame extraction parameter is a frame extraction interval of 3 frames (i.e., 1 frame is extracted and discarded every 4 frames), and the target queue can buffer a maximum of 30 video frames, then... Figure 4 As shown, when all 1-30 video frames acquired are cached in the target queue, the 4th, 8th, 12th...28th frames (shown in the dashed box in the figure) will be discarded based on the frame extraction parameters. Then, the 31st-37th video frames will be cached in the target queue. For the continuously acquired 31st-37th frames, the 32nd and 36th frames will be extracted and discarded based on the frame extraction parameters, and the 38th and 39th frames will be stored.

[0106] In this embodiment, by performing frame extraction processing on the cached video frames in the target queue according to the frame extraction parameters, and performing frame extraction processing on newly added cached consecutive video frames according to the same frame extraction parameters, the frame extraction method can ensure that the time span of the video is basically the same as the acquisition time span of the video frames under the premise of limiting the number of video frames in the video segment, thereby making the determination result of the target object based on the target video more accurate.

[0107] In one embodiment, the target object recognition method in the target scene further includes: updating the frame extraction parameters when the number of remaining video frames after frame extraction processing of the target queue reaches a number threshold; performing secondary frame extraction processing on the video frames in the target queue based on the updated frame extraction parameters; caching the newly acquired video frames to the target queue; and performing frame extraction processing on the newly cached video frames in the target queue based on the last video frame after secondary frame extraction processing and according to the updated frame extraction parameters.

[0108] When performing frame extraction on continuously acquired video frames according to the extraction parameters, if there are no video frames in the target queue that can be discarded according to the extraction parameters, the number of remaining video frames after extraction may reach a threshold. In this case, the extraction parameters need to be updated. Based on the refined extraction parameters, a second frame extraction process is performed on the video frames in the target queue to free up space for buffering newly acquired video frames. For newly acquired video frames, extraction also needs to be performed based on the updated extraction parameters to achieve average extraction.

[0109] Specifically, when the terminal performs frame extraction processing on continuously acquired video frames according to the frame extraction parameters, if there are no video frames in the target queue that can be discarded according to the frame extraction parameters, that is, when the number of remaining video frames after the frame extraction processing of the target queue reaches the number threshold, the frame extraction parameters are updated. Based on the updated frame extraction parameters, the terminal performs a second frame extraction processing on the video frames in the target queue to free up new space for caching newly acquired video frames. The terminal caches the newly acquired video frames in the target queue, and starting from the last video frame after the second frame extraction processing, performs the same frame extraction processing on the newly cached video frames in the target queue according to the updated frame extraction parameters.

[0110] In a specific application, video frames captured at different times can be distinguished by their frame numbers, such as frame 1, frame 2, etc. Figure 5 As shown, assuming the frame extraction parameter is a frame extraction interval of 3 (meaning 1 frame is extracted and discarded every 4 frames), and the target queue can cache a maximum of 30 video frames, when all 1-30 captured video frames are cached in the target queue, frames 4, 8, 12...28 (a total of 7 frames) will be discarded based on the frame extraction parameter (shown in the dashed box in the figure). Next, frames 31-37 will be cached in the target queue. For the consecutively acquired frames 31-37 (shown in the bold solid box in the figure), frames 32 and 36 will be extracted and discarded based on the frame extraction parameter, and frames 38 and 39 will be stored. At this point, the video frames stored in the target queue are 1, 2, 3, 5, 6, 7, 9, ... 33, 34, 35, 37, 38, 39. At this point, the frame extraction parameters are updated to a frame extraction interval of 1 frame, meaning 1 frame is dropped every 2 frames. The first round of frame extraction and the new cache result after updating the frame extraction parameters are frames 1, 3, 5, 7, 9, 11, 13, 15...33, 35, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49. The second round of frame extraction and the new cache result after updating the frame extraction parameters are frames 1, 3, 5...49, 50, 51, 52, 53, 54, 55. The final result after updating the frame extraction parameters is frames 1, 3, 5...53, 55, 57, 58. It should be noted that since frame 58 is the last cached video frame, frame dropping is not performed on frame 58 at this time. If a request to store the 59th video frame is detected, the 58th frame can be dropped, and the 59th frame can be cached in the target queue. Alternatively, the 58th frame can be dropped after the next update of the frame dropping parameters.

[0111] In this embodiment, by updating the frame extraction parameters and performing secondary frame extraction, it is possible to ensure that the time span of the video is basically the same as the acquisition time span of the video frames, while limiting the number of video frames in the video segment. This makes the determination of the target object based on the target video more accurate.

[0112] In one embodiment, the frame extraction parameter is the frame extraction interval frame number corresponding to the frame extraction method of discarding one video frame in every multi-frame video frame; according to the frame extraction parameter, the video frames in the target queue are subjected to frame extraction processing, including: determining the initial inter-frame time interval of the video frames in the target queue based on the timestamps carried by the video frames in the target queue; determining the frame extraction interval time based on the initial inter-frame time interval and the frame extraction interval frame number; and performing frame extraction processing on the video frames in the target queue based on the frame extraction interval time.

[0113] The frame extraction method is used to differentiate the frame extraction parameters. Frame extraction methods include discarding one frame per multiple video frames and retaining one frame per multiple video frames. The frame extraction parameter for discarding one frame per multiple video frames is the frame extraction interval (frames), while the frame extraction parameter for retaining one frame per multiple video frames is the number of consecutive frames extracted. Under certain conditions, the frame extraction method can be adjusted from discarding one frame per multiple video frames to retaining one frame per multiple video frames. For example, if the updated frame extraction parameter is a frame extraction interval of 1 (i.e., discarding one frame per two video frames), it yields the same result as retaining one frame per two video frames. In this case, the frame extraction method can be adjusted.

[0114] To easily distinguish video frames captured at different times, they can be differentiated by the timestamp corresponding to their capture time; each video frame carries a corresponding timestamp. The initial inter-frame interval is the time interval between the capture of two consecutive video frames. Before frame extraction, the initial inter-frame interval can be determined by the interval between any two adjacent video frames in the target queue. Alternatively, the average time interval of the video frames in the target queue can be calculated using the time interval between the timestamps of the first and last video frames in the target queue and the number of buffered video frames in the target queue; this average time interval is the initial inter-frame interval. By taking the average, the deviation caused by accidental frame drops before buffering can be reduced, improving the accuracy of the initial inter-frame interval.

[0115] The frame extraction interval time refers to the time interval condition that should be met between adjacent video frames in the target queue after frame extraction processing. The frame extraction interval time is calculated based on the initial inter-frame time interval and the number of frames for the frame extraction interval. The terminal performs frame extraction processing on the video frames in the target queue based on the frame extraction interval time.

[0116] Specifically, when the frame extraction parameter obtained by the terminal is the frame extraction interval (frame number), the frame extraction method can be determined to be discarding one video frame in every multi-frame video sequence. Correspondingly, the frame extraction process of discarding one video frame in every multi-frame video sequence is adopted. Specifically, the terminal determines the initial inter-frame time interval of the video frames in the target queue based on the ratio of the time difference between the timestamps carried by the first and last video frames in the target queue to the number of video frames, or the time difference between the timestamps carried by any two adjacent video frames in the target queue. The frame extraction interval time is obtained by calculating the initial inter-frame time interval * (frame extraction interval + 1). Then, according to the frame extraction interval time, the target queue is traversed from the beginning, and frame extraction processing is performed on the video frames in the target queue.

[0117] In this embodiment, for the frame extraction method of discarding one video frame in every multi-frame video frame, the frame extraction process is performed by determining the frame extraction interval time, which can accurately determine the video frames to be discarded in the target queue and achieve accurate frame extraction processing.

[0118] In one embodiment, the target object recognition method in the target scene further includes: when the number of remaining video frames after frame extraction processing of the target queue reaches a number threshold, reducing the frame extraction interval time; performing secondary frame extraction processing on the video frames in the target queue based on the reduced frame extraction interval time; caching the newly acquired video frames to the target queue, and performing frame extraction processing on the video frames in the target queue whose inter-frame interval time is the initial inter-frame time interval according to the reduced frame extraction interval time.

[0119] Since the frame extraction interval corresponds to the frame extraction method of discarding one video frame out of every multiple video frames, the updated frame extraction parameters can only be to discard one video frame out of every fewer video frames. For example, changing the method of discarding one video frame out of every 4 video frames to discarding one video frame out of every 2 video frames. Because the identification information carried by the video frames is a timestamp, the updated frame extraction parameters can be obtained directly by modifying the frame extraction interval. For example, if the frame extraction interval for discarding one video frame out of every 4 video frames is 1ms, then the frame extraction interval for discarding one video frame out of every 2 video frames is 0.5ms. Furthermore, the frame extraction interval before the update can be an integer multiple of the frame extraction interval after the update, so that the discarded video frames conform to the new frame extraction parameters, ensuring that the time intervals between the video frames remaining in the target queue are the same.

[0120] For newly acquired video frames, after caching them in the target queue, frame extraction processing is also required. Since the inter-frame interval between newly cached video frames in the target queue after frame extraction is the initial inter-frame interval, taking the last video frame after the second frame extraction as the starting point, and performing frame extraction processing on video frames in the target queue whose inter-frame interval is the initial inter-frame interval according to the reduced frame extraction interval, ensures that the inter-frame interval between all video frames in the target queue is the same.

[0121] In this embodiment, by reducing the frame extraction interval and performing secondary frame extraction, it can adapt to the frame extraction method of discarding one video frame in every multiple video frames. On the other hand, it can ensure that the time span of the video is basically the same as the acquisition time span of the video frame under the premise of limiting the number of video frames in the video segment, thereby making the determination result of the target object based on the target video more accurate.

[0122] In one embodiment, the frame extraction parameter is the number of consecutive frame extraction frames corresponding to the frame extraction method of retaining one video frame in every multi-frame video frame; according to the frame extraction parameter, the video frames in the target queue are subjected to frame extraction processing, including: determining the initial inter-frame time interval of the video frames in the target queue based on the timestamps carried by the video frames cached in the target queue; obtaining the updated inter-frame time interval based on the initial inter-frame time interval and the number of consecutive frame extraction frames; and performing frame extraction processing on video frames in the target queue that do not meet the updated inter-frame time interval.

[0123] The initial inter-frame interval is the time interval between two consecutively acquired video frames. Before frame extraction, the initial inter-frame interval can be determined by the interval between any two adjacent video frames in the target queue. Alternatively, the average time interval of the video frames in the target queue can be calculated using the time interval between the timestamps of the first and last video frames in the target queue and the number of buffered video frames in the target queue; this average time interval is the initial inter-frame interval. By taking the average, the deviation caused by accidental frame drops before buffering can be reduced, thus improving the accuracy of the calculated initial inter-frame interval.

[0124] The updated inter-frame interval refers to the time interval condition that should be met between adjacent video frames in the target queue after frame extraction. The updated inter-frame interval is calculated based on the product of the initial inter-frame interval and the number of consecutive extracted frames. The terminal then performs frame extraction on video frames in the target queue that do not meet the updated inter-frame interval.

[0125] Specifically, when the frame extraction parameter obtained by the terminal is the number of consecutive frame extractions, the frame extraction method can be determined to be retaining one video frame in every multi-frame video sequence. Correspondingly, the frame extraction process of retaining one video frame in every multi-frame video sequence is adopted. Specifically, the terminal determines the initial inter-frame time interval of the video frames in the target queue based on the timestamps carried by the video frames cached in the target queue. The updated inter-frame time interval is obtained by calculating the product of the initial inter-frame time interval and the number of consecutive frame extractions. Then, according to the frame extraction interval, the target queue is traversed from the beginning, and frame extraction processing is performed on video frames in the target queue that do not meet the updated inter-frame time interval.

[0126] In this embodiment, for the frame extraction method of retaining one video frame in every multi-frame video frame, the frame extraction process is performed by determining the inter-frame update time interval that the retained video frame needs to satisfy. This can accurately determine the video frame to be retained in the target queue and achieve accurate frame extraction processing.

[0127] In one embodiment, the target object recognition method in the target scene further includes: when the number of remaining video frames after frame extraction processing of the target queue reaches a number threshold, extending the update inter-frame time interval; performing secondary frame extraction processing on the video frames in the target queue based on the extended inter-frame time interval; caching newly acquired video frames to the target queue, and performing frame extraction processing on video frames in the target queue that do not meet the extended inter-frame time interval.

[0128] Since the consecutive frame extraction number corresponds to the frame extraction method of retaining one video frame out of every multiple video frames, the updated frame extraction parameter can only be one video frame retained out of every additional video frames. For example, changing the retention method from one video frame out of every 4 video frames to one video frame out of every 8 video frames. Because the identification information carried by the video frames is a timestamp, the updated frame extraction parameter can be directly obtained by updating the frame extraction interval time. For example, if the frame extraction interval time corresponding to retaining one video frame out of every 4 video frames is 1ms, then the frame extraction interval time corresponding to retaining one video frame out of every 8 video frames is 2ms. Furthermore, the frame extraction interval time before the update can be an integer multiple of the frame extraction interval time after the update, so that the retained video frames conform to the new frame extraction parameter, ensuring that the time intervals between the video frames retained in the target queue are the same.

[0129] For newly acquired video frames, after caching them to the target queue, frame extraction processing is also required. Since the inter-frame interval between newly cached video frames after frame extraction in the target queue is the initial inter-frame interval, starting from the last video frame after the second frame extraction processing, frame extraction processing is performed on video frames in the target queue that do not meet the extended inter-frame interval according to the extended frame extraction interval. This ensures that the inter-frame interval between all video frames in the target queue is the same.

[0130] In this embodiment, by extending the frame extraction interval and performing secondary frame extraction, it can adapt to the frame extraction method of retaining one video frame in every multiple video frames. On the other hand, it can ensure that the time span of the video is basically the same as the acquisition time span of the video frame under the premise of limiting the number of video frames in the video segment, thereby making the determination result of the target object based on the target video more accurate.

[0131] In one embodiment, the target object recognition method in the target scene further includes: when at least one face object is detected in the shooting field of view in the target scene, starting a video frame acquisition process and simultaneously acquiring color video frames, infrared video frames and depth video frames.

[0132] The color video frames, infrared video frames, and depth video frames can be obtained from different shooting devices. Color images are captured by a color video shooting device, and each pixel in the captured video frame is composed of R (red), G (green), and B (blue) components. Infrared video frames are captured by an infrared video shooting device, which uses a detector to measure the difference in infrared radiation between the target and the background to obtain infrared images formed by different thermal infrared radiation. Depth video frames are captured by a depth video shooting device, and each pixel value in the depth video frame represents the distance between a point in the scene and the depth video shooting device. Specifically, the acquisition and termination of color, infrared, and depth video frames are synchronized. When the video frame acquisition process starts, the acquisition of color, infrared, and depth video frames is performed simultaneously. When the video frame acquisition process ends, the acquisition of color, infrared, and depth video frames terminates simultaneously.

[0133] Furthermore, when there is a target video frame whose face feature detection result matches the face feature condition, and the liveness detection result corresponding to the target video frame matches the liveness detection condition, the video segments composed of the acquired video frames are determined, including: when there is a target color video frame whose face feature detection result matches the face feature condition among the acquired color video frames, the target infrared video frame and the target depth video frame are determined based on the timestamp carried by the target color video frame; liveness detection is performed based on the target color video frame, the target infrared video frame, and the target depth video frame; when the liveness detection result matches the liveness detection condition, the video frame acquisition process is terminated, and the video segments composed of the acquired color video frame, infrared video frame, and depth video frame are determined.

[0134] Facial feature detection can be based on any type of video frame, including color video frames, infrared video frames, and depth video frames. In a specific application, since facial features in color video frames are of higher clarity, the acquired color video frames are used as the detection objects. When a target color video frame exists whose facial feature detection results match the facial feature conditions, the target infrared video frame and the target depth video frame with the same timestamp are determined based on the timestamp carried by the target color video frame.

[0135] Specifically, when the terminal detects at least one face object in the field of view of the target scene, it initiates a video frame acquisition process, simultaneously acquiring color video frames, infrared video frames, and depth video frames. The terminal performs face feature detection based on the acquired color video frames. When a target color video frame exists whose face feature detection result matches the face feature conditions, the terminal determines the target infrared video frame and target depth video frame, each carrying the same timestamp, based on the timestamp carried by the target color video frame. Based on the target color video frame, the terminal determines the liveness detection region in the target infrared video frame and target depth video frame, and performs liveness detection based on the target infrared video frame and target depth video frame. When the liveness detection results corresponding to the target infrared video frame and target depth video frame both meet the corresponding liveness detection conditions, the video frame acquisition process ends, and the acquired color video segment, infrared video segment, and depth video segment are determined, facilitating subsequent identification of the target object in the target scene and execution of the corresponding processing flow for the target object.

[0136] In this embodiment, facial feature recognition and liveness detection are performed using different types of video frames. Based on the image characteristics of different categories, accurate target color video frames, target infrared video frames, and target depth video frames can be obtained, resulting in accurate video segments. This ensures that the subsequent determination of target objects in the target scene is more accurate.

[0137] In one embodiment, the video clip includes a color video clip, an infrared video clip, and a depth video clip; sending the video clip and the target video frame to the server includes: compressing the acquired color video clip, infrared video clip, and depth video clip into video clips respectively, and compressing the target color video frame, target infrared video frame, and target depth video frame into images respectively; and sending the compressed video clip and the compressed images to the server.

[0138] The video segments are composed of captured color video frames, infrared video frames are composed of captured infrared video frames, and depth video frames are composed of captured depth video frames. Because the server has different quality requirements for color, depth, and infrared image data during facial recognition and behavioral feature analysis, different image compression algorithms and compression ratios are used for different video frame types during the compression stage to ensure image compression effectiveness without affecting the server's algorithm.

[0139] Specifically, the terminal compresses the acquired color video clips, infrared video clips, and depth video clips separately according to different compression processes. In a specific application, for color video clips, infrared video clips, and depth video clips, based on the image compression algorithm and compression ratio corresponding to the image type, they can be compressed and transmitted to the server either as a video stream or as an image sequence. For target color video frames, target infrared video frames, and target depth video frames, images are compressed according to the corresponding image compression algorithm and compression ratio before being transmitted to the server.

[0140] In this embodiment, different image compression algorithms and compression ratios are used for target video frames and video segments of different image types to ensure image compression effect without affecting the server algorithm, thereby obtaining accurate target object recognition results.

[0141] In one embodiment, facial feature recognition is performed on a target video frame to determine candidate objects in the target video frame, including: determining a target face region in the target video frame based on the area occupied by each face region in the target video frame; and performing facial feature recognition on the facial features in the target face region to determine candidate objects.

[0142] In this context, a face region refers to the image area of ​​a face object within the target video frame. The closer the face object is to the shooting device, the larger the area occupied by its corresponding face region in the target video frame. Conversely, the farther the face object is from the shooting device, the smaller the area occupied by its corresponding face region in the target video frame. During the process of determining the target video frame, the terminal filters the video frames to ensure that at least one face object in the target video frame meets the facial feature criteria. Specifically, this means that at least one face object occupies an area in the target video frame that reaches a preset ratio. During the facial feature recognition process, the target face regions that reach the preset ratio are first determined based on the area occupied by each face region in the target video frame. Then, facial feature recognition is performed on the facial features of the target face regions, and candidate objects are determined based on the facial recognition results.

[0143] Specifically, the step of performing facial feature recognition on the target video frame to determine candidate objects within the target video frame can be performed by either a terminal or a server. Taking the server as the executing entity as an example, the server receives the target video frame sent by the terminal, determines the target face regions in the target video frame whose area reaches a preset ratio based on the area occupied by each face region in the target video frame, and then performs facial feature recognition on the facial features in the target face regions. Based on the facial recognition results, the server determines candidate objects so as to accurately determine the position of the candidate objects in each video frame of the video segment during subsequent processing.

[0144] In this embodiment, the target face region in the target video frame is first determined based on the area occupied by each face region in the target video frame; then, face feature recognition is performed on the face features in the target face region to determine candidate objects. This allows for face recognition to be performed only on the face features in the target face region, enabling targeted processing, reducing unnecessary data processing steps, and effectively improving the efficiency of data processing.

[0145] In one embodiment, when the behavioral characteristics of a candidate object in a video segment meet the behavioral characteristic conditions adapted to the target scene, the candidate object is determined to be the target object in the target scene, including: obtaining the behavioral characteristics of the candidate object based on the relative position change of the candidate object in each video frame of the video segment; when the behavioral characteristic analysis result corresponding to the behavioral characteristics meets the behavioral characteristic conditions of the user corresponding to the target scene, the candidate object is determined to be the target object in the target scene.

[0146] The relative position change is determined by comparing the positional movement of the corresponding feature points of the candidate object in each video frame of the video clip. These feature points can be facial or body features of the candidate object, and can be configured based on the behavioral feature conditions of the target scene. By fusing the positional changes of each feature point in each video frame of the video clip, the behavioral features of the candidate object in the video clip can be obtained. Behavioral feature analysis can then be performed on these behavioral features, based on a pre-trained behavioral feature analysis model.

[0147] Behavioral feature conditions are used to determine whether a candidate object is the target object in a target scenario. These conditions can be configured based on the target scenario, and the behavioral features can differ across scenarios. Specifically, the behavioral feature conditions adapted to the target scenario can be determined based on a behavioral feature analysis algorithm adapted to that scenario. This algorithm can be implemented using a pre-trained behavioral feature analysis model. For example, in a facial recognition payment scenario, a payment intention analysis model can be trained using multiple sample video clips showing payment intention. The behavioral features of the candidate object in the video clips are input into the model. When the model outputs that the candidate object has a payment intention, it meets the behavioral feature conditions adapted to the target scenario; conversely, when it outputs that the candidate object does not have a payment intention, it does not meet the behavioral feature conditions adapted to the target scenario.

[0148] Specifically, when the behavioral characteristics of a candidate object in a video segment match the behavioral characteristic conditions adapted to the target scene, the execution entity for the step of determining the candidate object as the target object in the target scene is the same as the execution entity for the step of determining the candidate object itself, which can be either a terminal or a server. Taking the execution entity as a server as an example, the server first identifies the candidate objects in each video frame of the video segment and obtains the behavioral characteristics of the candidate objects based on the changes in their relative positions in each video frame. By analyzing the behavioral characteristics, the server obtains the behavioral characteristic analysis results. Based on the relationship between the behavioral characteristic analysis results and the behavioral characteristic conditions of the user corresponding to the target scene, the server determines whether the candidate object is the target object in the target scene. When the behavioral characteristic analysis results match the behavioral characteristic conditions of the user corresponding to the target scene, the candidate object is determined to be the target object in the target scene; when the behavioral characteristic analysis results do not match the behavioral characteristic conditions of the user corresponding to the target scene, the candidate object is determined not to be the target object in the target scene.

[0149] In a specific application, the target scenario is a facial recognition payment scenario. The server obtains the behavioral features of candidate objects based on the relative position changes of the candidate objects in each video frame of a video segment. When the behavioral feature analysis result matches the behavioral feature conditions of the user corresponding to the facial recognition payment scenario, the candidate object is determined to be the target payment object. When the behavioral feature analysis result does not match the behavioral feature conditions of the user corresponding to the facial recognition payment scenario, the candidate object is determined not to be the target payment object, and the user is prompted to re-perform facial recognition.

[0150] In this embodiment, by analyzing the behavioral characteristics of candidate objects in video clips, it is possible to accurately determine whether a candidate object is a target object in the target scene based on the behavioral characteristics in the target scene, thereby effectively improving the accuracy of the target object recognition results in the target scene.

[0151] In a specific application, such as Figure 6 As shown, a method for identifying target objects in a target scene is provided, which specifically includes the following steps:

[0152] Step 602: In the target scene, when the terminal detects that there is at least one face object in the field of view, the video frame acquisition process is started to acquire video frames containing at least one face object. The acquired video frames include three types of video frames: color video frames, infrared video frames, and depth video frames.

[0153] Step 604: For each type of video frame, the acquired video frames are cached sequentially according to the acquisition time into a target queue where the number of cacheable video frames does not exceed a threshold, and face feature detection is performed on the acquired color video frames.

[0154] Step 606: When the number of video frames cached in the target queue reaches the number threshold, determine the initial inter-frame time interval of the video frames in the target queue based on the timestamps carried by the video frames in the target queue.

[0155] Step 608: When the frame extraction parameter is the number of frames corresponding to the frame extraction interval method of discarding one video frame in each multi-frame video frame, determine the frame extraction interval time based on the initial inter-frame time interval and the number of frames extracted; and perform frame extraction processing on the video frames in the target queue based on the frame extraction interval time.

[0156] Step 610: When the number of remaining video frames after frame extraction processing of the target queue reaches the number threshold, the frame extraction interval is reduced, and a second frame extraction process is performed on the video frames in the target queue based on the reduced frame extraction interval.

[0157] Step 612: Cache the newly acquired video frames into the target queue, and perform frame extraction processing on the video frames in the target queue whose inter-frame interval is the initial inter-frame interval, according to the reduced frame extraction interval time.

[0158] Step 614: When the frame extraction parameter is the number of consecutive frame extraction frames corresponding to the frame extraction method of retaining one video frame in each multi-frame video frame, the updated inter-frame time interval is obtained based on the initial inter-frame time interval and the number of consecutive frame extraction frames, and the video frames in the target queue that do not meet the updated inter-frame time interval are subjected to frame extraction processing.

[0159] Step 616: When the number of remaining video frames after frame extraction processing of the target queue reaches the number threshold, extend the inter-frame time interval and perform secondary frame extraction processing on the video frames in the target queue based on the extended inter-frame time interval.

[0160] Step 618: Cache the newly acquired video frames into the target queue, and perform frame extraction processing on video frames in the target queue that do not meet the extended inter-frame time interval.

[0161] Step 620: When there is a target color video frame in the acquired color video frames whose face feature detection results meet the face feature conditions, the target infrared video frame and the target depth video frame are determined based on the timestamp carried by the target color video frame.

[0162] Step 622: Perform liveness detection based on the target color video frame, target infrared video frame, and target depth video frame. When the liveness detection result meets the liveness detection conditions, end the video frame acquisition process.

[0163] Step 624: Based on the video frames cached in the target queue corresponding to each type of video frame, obtain color video clips, infrared video clips, and depth video clips.

[0164] Step 626: Compress the acquired color video clips, infrared video clips, and depth video clips respectively, and compress the target color video frame, target infrared video frame, and target depth video frame respectively. Send the compressed video clips and compressed images to the server.

[0165] Step 628: The server determines the target face region in the target video frame based on the area occupied by each face region in the target video frame, performs face feature recognition on the face features in the target face region, and determines the candidate object.

[0166] Step 630: Based on the relative position change of the candidate object in each video frame of the video segment, obtain the behavioral features of the candidate object.

[0167] Step 632: When the behavioral feature analysis result corresponding to the behavioral feature matches the behavioral feature conditions of the user in the face recognition payment scenario, the candidate object is determined as the target payment object.

[0168] This application also provides an application scenario for facial recognition payment, in which the target object recognition method described above is applied. Specifically, the target object recognition method in this application scenario is applied as follows:

[0169] Current solutions involve capturing facial images from the terminal and transmitting them to a server, where the server performs liveness detection, feature detection, and facial recognition. However, the captured images suffer from limited facial information. For example, because images are captured only at a specific moment, factors such as facial angle and occlusion can result in only partial information being captured. For highly similar individuals with limited differences, this can lead to the inability to identify the recipient for facial recognition payments. The inability to accurately analyze the user's payment intent can result in payment anomalies. For instance, if user A wants to use facial recognition, but users A and B are standing in front of the camera, and user A has the intention to use facial recognition, the server's algorithm might select user B for payment in a single frame, leading to an anomaly in the payment recipient's information.

[0170] In a specific application, such as Figure 7 As shown, the solution is implemented through interaction between the terminal and the server. It can be understood that in other embodiments, the server's processing can also be implemented on the terminal to reduce interaction between the terminal and the server, decrease dependence on the network, and thus reduce interference from network signals on the data processing process.

[0171] Taking facial recognition payment terminals in fixed locations such as supermarkets as an example, the corresponding target scenarios include facial recognition payment scenarios. Facial recognition payment terminals are equipped with cameras capable of simultaneously acquiring color images, infrared images, and depth images. For example... Figure 8 As shown, the facial recognition payment terminal is also equipped with a display screen that can display the camera's field of view and shooting guidance information on the video screen, so that users can adjust their position or posture based on the field of view and shooting guidance information.

[0172] When the terminal detects at least one human face in the field of view, it initiates a video frame acquisition process, capturing video frames containing at least one human face. These captured video frames include three different types: color video frames, infrared video frames, and depth video frames. The acquisition and frame extraction processes are identical for each type of video frame. Specifically, the terminal processes the captured video frames simultaneously in two ways: first, it sequentially caches the captured video frames according to their acquisition time into a target queue with a cacheable number of frames not exceeding a certain threshold; second, it performs facial feature detection on the captured color video frames.

[0173] Regarding the video frame caching process, since the three types of video frames are acquired synchronously and have the same timestamp, the target queue corresponding to one type of video frame can be used as the analysis object. The target queues corresponding to the other two types of video frames can be directly updated synchronously based on the timestamps carried by the actual cached video frames, effectively reducing the resources required for data processing.

[0174] Specifically, when the terminal detects that the number of video frames cached in the target queue has reached a threshold, it determines the initial inter-frame time interval of the video frames in the target queue based on the timestamps carried by the video frames. For frame extraction, there are two different extraction methods: the first is to discard one video frame in every set of multiple video frames, and the second is to retain one video frame in every set of multiple video frames. The extraction parameter corresponding to the method of discarding one video frame in every set of multiple video frames is the extraction interval frame number. The extraction parameter corresponding to the method of retaining one video frame in every set of multiple video frames is the consecutive extraction frame number.

[0175] When the frame extraction parameter is the number of frames corresponding to the frame extraction interval method that discards one video frame per multi-frame video frame, the frame extraction interval is determined based on the initial inter-frame time interval and the number of frames extracted. Based on this interval, frame extraction processing is performed on the video frames in the target queue. When the number of remaining video frames after frame extraction in the target queue reaches a threshold, the frame extraction interval is reduced, and a second frame extraction process is performed on the video frames in the target queue based on the reduced interval. Newly acquired video frames are buffered in the target queue, and frame extraction processing is performed on the video frames in the target queue whose inter-frame time interval is equal to the initial inter-frame time interval, according to the reduced interval.

[0176] When the frame extraction parameter is the number of consecutive frames extracted corresponding to the frame extraction method that retains one video frame in every multi-frame video frame, the updated inter-frame interval is obtained based on the initial inter-frame interval and the number of consecutive extracted frames. Frame extraction is then performed on video frames in the target queue that do not meet the updated inter-frame interval. When the number of remaining video frames after frame extraction in the target queue reaches a threshold, the updated inter-frame interval is extended, and a second frame extraction process is performed on the video frames in the target queue based on the extended inter-frame interval. Newly acquired video frames are buffered in the target queue, and frame extraction is then performed on video frames in the target queue that do not meet the extended inter-frame interval.

[0177] In a specific application, taking the acquisition and caching of color video frames as an example, the number of video frames that can be cached in the target queue is fixed. For example, the number of cacheable color, depth, and infrared video frames is N, to prevent the number of video frames in a video segment from becoming too large due to long user interaction time. The terminal acquires video frames captured by the camera. One copy of the acquired video frames is used for target video frame optimization and liveness detection, while another copy, containing a timestamp, is temporarily stored in the target queue. During the video frame acquisition process, in order to quickly obtain target video frames with suitable positions and angles, shooting guidance prompts can be displayed on the screen based on the analysis of the user's angle and position. This guides the user to adjust the angle or position according to the shooting guidance prompts, such as changing from a side view to a front view, or from a long distance to a close distance. When the terminal detects that the number of cached video frames in the target queue is greater than N, it discards some of the cached video frames in the target queue according to a certain discarding strategy, and repeats the video frame caching and video frame discarding process. When it obtains the message of the preferred target video frame and the successful liveness detection, it stops the acquisition of video frames, and uses FFMpeg (Fast Forward MPEG, an open source computer program used to record, convert digital audio and video, and convert them into streams) to compress the video frames in the color image queue, infrared image queue, and depth image queue respectively, to obtain three different types of video segments.

[0178] Furthermore, the discard strategy uses a mean sampling algorithm, such as... Figure 9 As shown, assuming the target queue length is N, and each video frame contains a timestamp, the average inter-frame interval t1 is calculated based on the corresponding time length of the target queue. After adding a new video frame, the target queue length will become N+1. At this time, a new sequence interval t2 is generated based on the frame extraction parameters and t1, where t2 = 2t1. Then, using the new t2 interval, the image sequence is traversed and detected, ensuring that the difference in timestamps between adjacent video frames in the target queue is ≥ t2. Video frames that do not meet this requirement are discarded. It should be noted that the discarding strategy ensures that the last cached video frame in the target queue is not discarded. After the first discard, the current implementation interval t2 needs to be recorded. Then, the caching of newly acquired video frames is repeated. When the remaining space in the target queue is 0 again, the target queue needs to be re-detected using the t2 interval, and the newly cached video frames that do not meet the interval requirement after the first reorganization are discarded. Then, the above caching and discarding process is repeated. Only when the target queue has no remaining space after being reorganized according to the t2 interval is the resampling logic triggered to update t2.

[0179] Regarding the video frame detection process, when a target color video frame whose face feature detection result matches the face feature condition exists among the acquired color video frames, the target infrared video frame and target depth video frame are determined based on the timestamp carried by the target color video frame. Liveness detection is then performed based on these three target video frames. When the liveness detection result meets the liveness detection condition, the video frame acquisition process ends, and no new video frames are acquired. No new video frames will be added to the target queue for caching. The terminal then obtains color video segments, infrared video segments, and depth video segments based on the cached video frames in the target queue corresponding to each type of video frame.

[0180] Because the server has different requirements for the quality of color, depth, and infrared image data, the terminal compresses the acquired color video clips, infrared video clips, and depth video clips using different image compression algorithms and compression ratios. It also compresses the target color video frame, target infrared video frame, and target depth video frame separately, and sends the compressed video clips and compressed images to the server.

[0181] The server determines the target face region in the target video frame based on the area occupied by each face region within the frame. It then performs facial feature recognition on the face features within this target face region to identify candidate payment targets. Next, based on the relative position changes of these candidate payment targets in each video frame, it obtains their behavioral characteristics. When the behavioral feature analysis results indicate a willingness to pay, the candidate payment target is designated as the target payment target. The server then returns the user identity information obtained through facial feature recognition matching to the terminal. Figure 8 The displayed membership information facilitates user identity verification and further confirmation of the goods to be paid and the amount to be paid. Based on the user's confirmation, facial recognition payment is completed.

[0182] It should be understood that, although Figure 2 and Figure 6 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Furthermore, the flowchart... Figure 2 and Figure 6 At least some of the steps in the process may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but may be executed at different times. The execution order of these steps or stages is not necessarily sequential, but may be executed in turn or alternately with other steps or at least some of the steps or stages in other steps.

[0183] In one embodiment, such as Figure 10 As shown, a target object recognition device 1000 in a target scene is provided. This device can be a software module, a hardware module, or a combination of both as part of a computer device. Specifically, the device includes: a video frame acquisition module 1002, a target video frame filtering module 1004, a candidate object determination module 1006, and a target object determination module 1008, wherein:

[0184] The video frame acquisition module 1002 is used to acquire video frames containing at least one face object when at least one face object is detected in the shooting field of view of the target scene.

[0185] The target video frame filtering module 1004 is used to determine the video segment composed of the acquired video frames when there is a target video frame in the acquired video frames, such that the face feature detection result of the target video frame meets the face feature conditions adapted to the target scene, and the liveness detection result corresponding to the target video frame meets the liveness detection conditions.

[0186] The candidate object determination module 1006 is used to perform facial feature recognition on the target video frame and determine the candidate objects in the target video frame.

[0187] The target object determination module 1008 is used to determine the candidate object as the target object in the target scene when the behavioral characteristics of the candidate object in the video segment meet the behavioral characteristic conditions adapted to the target scene.

[0188] In one embodiment, the target object recognition device in the target scene includes a sending module;

[0189] The sending model is used to send the video segment and the target video frame to the server, so that the server executes the processing steps of the candidate object determination module and the target object determination module.

[0190] In one embodiment, the target object recognition device in the target scene further includes a video frame caching module; the video frame caching module is used to cache the acquired video frames sequentially to the target queue according to the acquisition time, and the number of cacheable video frames in the target queue is not greater than a number threshold.

[0191] The target video frame filtering is further used to terminate video frame acquisition when a target video frame exists in the acquired video frames, such that the face feature detection result of the target video frame meets the face feature conditions adapted to the target scene, and the liveness detection result corresponding to the target video frame meets the liveness detection conditions; and to obtain video segments based on the video frames cached in the target queue.

[0192] In one embodiment, the target object recognition device in the target scene further includes a frame extraction processing module, which is used to perform frame extraction processing on the video frames in the target queue according to the frame extraction parameters when the number of video frames cached in the target queue reaches the number threshold; cache newly acquired video frames in the target queue; and perform frame extraction processing on the continuously acquired video frames in the target queue according to the frame extraction parameters.

[0193] In one embodiment, the target object recognition device in the target scene further includes a secondary frame extraction processing module, used to update the frame extraction parameters when the number of remaining video frames after frame extraction processing of the target queue reaches the number threshold; perform secondary frame extraction processing on the video frames in the target queue based on the updated frame extraction parameters; cache the newly acquired video frames in the target queue; and perform frame extraction processing on the newly cached video frames in the target queue based on the last video frame after secondary frame extraction processing and according to the updated frame extraction parameters.

[0194] In one embodiment, the frame extraction parameter is the frame extraction interval frame number corresponding to the frame extraction method of discarding one video frame in every multi-frame video frame; the frame extraction module is further configured to determine the initial inter-frame time interval of the video frames in the target queue based on the timestamps carried by the video frames in the target queue; determine the frame extraction interval time based on the initial inter-frame time interval and the frame extraction interval frame number; and perform frame extraction processing on the video frames in the target queue based on the frame extraction interval time.

[0195] In one embodiment, the target object recognition device in the target scene further includes a secondary frame extraction processing module, used to reduce the frame extraction interval time when the number of remaining video frames after frame extraction processing of the target queue reaches the number threshold; perform secondary frame extraction processing on the video frames in the target queue based on the reduced frame extraction interval time; cache the newly acquired video frames in the target queue; and perform frame extraction processing on the video frames in the target queue whose inter-frame interval time is the initial inter-frame time interval according to the reduced frame extraction interval time.

[0196] In one embodiment, the frame extraction parameter is the number of consecutive frame extraction frames corresponding to the frame extraction method of retaining one video frame in every multi-frame video frame; the frame extraction module is further configured to determine the initial inter-frame time interval of video frames in the target queue based on the timestamps carried by the video frames cached in the target queue; obtain the updated inter-frame time interval based on the initial inter-frame time interval and the number of consecutive frame extraction frames; and perform frame extraction processing on video frames in the target queue that do not meet the updated inter-frame time interval.

[0197] In one embodiment, the target object recognition device in the target scene further includes a secondary frame extraction processing module, which is used to extend the update inter-frame time interval when the number of remaining video frames after frame extraction processing of the target queue reaches the number threshold; perform secondary frame extraction processing on the video frames in the target queue based on the extended inter-frame time interval; cache newly acquired video frames in the target queue; and perform frame extraction processing on video frames in the target queue that do not meet the extended inter-frame time interval.

[0198] In one embodiment, the target object recognition device in the target scene further includes a video frame acquisition process control module, which is used to start the video frame acquisition process and simultaneously acquire color video frames, infrared video frames and depth video frames when at least one face object is detected in the shooting field of view in the target scene.

[0199] The target video frame filtering module is further configured to, when there is a target color video frame in the acquired color video frames whose face feature detection results meet the face feature conditions, determine the target infrared video frame and the target depth video frame based on the timestamp carried by the target color video frame; perform liveness detection based on the target color video frame, the target infrared video frame, and the target depth video frame; and when the liveness detection result meets the liveness detection conditions, end the video frame acquisition process and determine the video segments composed of the acquired color video frame, infrared video frame, and depth video frame.

[0200] In one embodiment, the video clips include color video clips, infrared video clips, and depth video clips;

[0201] The sending module is further configured to compress the acquired color video clips, infrared video clips, and depth video clips into video clips, and to compress the target color video frame, target infrared video frame, and target depth video frame into images; and to send the compressed video clips and compressed images to the server.

[0202] In one embodiment, the candidate object determination module is further configured to determine a target face region in the target video frame based on the area occupied by each face region in the target video frame; and to perform face feature recognition on the face features in the target face region to determine a candidate object.

[0203] In one embodiment, the target object determination module is further configured to obtain the behavioral characteristics of the candidate object based on the relative position change of the candidate object in each video frame of the video segment; when the behavioral characteristic analysis result corresponding to the behavioral characteristics matches the behavioral characteristic conditions of the user corresponding to the face recognition payment scenario, the candidate object is determined as the target payment object.

[0204] For specific embodiments of the target object recognition device in the target scene, please refer to the embodiments of the target object recognition method in the target scene described above, which will not be repeated here. Each module in the target object recognition device in the above-mentioned target scene can be implemented entirely or partially through software, hardware, or a combination thereof. Each module can be embedded in or independent of the processor in the computer device in hardware form, or it can be stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0205] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 11 As shown, the computer device includes a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements a target object recognition method in a target scenario. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0206] Those skilled in the art will understand that Figure 11 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0207] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0208] In one embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0209] In one embodiment, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and executes the computer instructions, causing the computer device to perform the steps in the above method embodiments.

[0210] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0211] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0212] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for identifying target objects in a target scene, characterized in that, The method includes: When at least one human face is detected in the field of view of the target scene, video frames containing the at least one human face are captured; the target scene is a face recognition payment scene. When a target video frame exists in the acquired video frames, such that the face feature detection result of the target video frame meets the face feature conditions adapted to the target scene, and the liveness detection result corresponding to the target video frame meets the liveness detection conditions, the video segment composed of the acquired video frames is determined. Facial feature recognition is performed on the target video frame to determine candidate objects in the target video frame; Based on the relative position change of the candidate object in each video frame of the video segment, the behavioral characteristics of the candidate object are obtained. When the behavioral feature analysis result corresponding to the behavioral feature matches the behavioral feature condition of the user corresponding to the face recognition payment scenario, the candidate object is determined as the target payment object.

2. The method according to claim 1, characterized in that, The method further includes: Send the video clip and the target video frame to the server; The candidate object and the target payment object are determined by the server after the video clip and the target video frame are sent to the server.

3. The method according to claim 1, characterized in that, The method further includes: The captured video frames are cached sequentially into the target queue according to the capture time, and the number of cacheable video frames in the target queue is no greater than the number threshold. When a target video frame exists among the acquired video frames, such that the facial feature detection result of the target video frame matches the facial feature conditions adapted to the target scene, and the liveness detection result corresponding to the target video frame matches the liveness detection conditions, the video segment composed of the acquired video frames is determined, including: When a target video frame is present in the acquired video frames, such that the face feature detection result of the target video frame matches the face feature conditions adapted to the target scene, and the liveness detection result corresponding to the target video frame matches the liveness detection conditions, the video frame acquisition is terminated. Video segments are obtained based on the video frames cached in the target queue.

4. The method according to claim 3, characterized in that, The method further includes: When the number of video frames cached in the target queue reaches the number threshold, the video frames in the target queue are extracted according to the extraction parameters. The newly acquired video frames are cached in the target queue, and the continuously acquired video frames in the target queue are processed by extracting frames according to the extraction parameters.

5. The method according to claim 4, characterized in that, The method further includes: When the number of remaining video frames after frame extraction processing of the target queue reaches the number threshold, the frame extraction parameters are updated. The video frames in the target queue are subjected to secondary frame extraction based on the updated frame extraction parameters. The newly acquired video frames are cached in the target queue. Based on the last video frame after the second frame extraction process, the newly cached video frames in the target queue are subjected to frame extraction processing according to the updated frame extraction parameters.

6. The method according to claim 4, characterized in that, The frame extraction parameter is the frame extraction interval number corresponding to the frame extraction method of discarding one video frame in each multi-frame video frame. The step of performing frame extraction processing on the video frames in the target queue according to the frame extraction parameters includes: Based on the timestamps carried by the video frames in the target queue, the initial inter-frame time interval of the video frames in the target queue is determined; The frame extraction interval time is determined based on the initial inter-frame time interval and the number of frames in the frame extraction interval. Based on the frame extraction interval, the video frames in the target queue are subjected to frame extraction processing.

7. The method according to claim 6, characterized in that, The method further includes: When the number of remaining video frames after frame extraction processing of the target queue reaches the number threshold, the frame extraction interval time is reduced. The video frames in the target queue are subjected to secondary frame extraction processing based on the reduced frame extraction interval time. The newly acquired video frames are cached in the target queue, and the video frames in the target queue with an inter-frame interval of the initial inter-frame interval are processed according to the reduced frame extraction interval time.

8. The method according to claim 4, characterized in that, The frame extraction parameter is the number of consecutive frame extraction frames corresponding to the frame extraction method of retaining one video frame in each multi-frame video frame. The step of performing frame extraction processing on the video frames in the target queue according to the frame extraction parameters includes: The initial inter-frame time interval of the video frames in the target queue is determined based on the timestamps carried by the video frames cached in the target queue. The updated inter-frame time interval is obtained based on the initial inter-frame time interval and the number of consecutive frame extractions; Frame extraction is performed on video frames in the target queue that do not meet the update inter-frame time interval.

9. The method according to claim 8, characterized in that, The method further includes: When the number of remaining video frames after frame extraction processing of the target queue reaches the number threshold, the time interval between updated frames is extended. The video frames in the target queue are subjected to secondary frame extraction based on the extended inter-frame time interval. The newly acquired video frames are cached in the target queue, and the video frames in the target queue that do not meet the extended inter-frame time interval are processed by frame extraction.

10. The method according to claim 1, characterized in that, The method further includes: When at least one human face is detected in the field of view of the target scene, the video frame acquisition process is started, and color video frames, infrared video frames and depth video frames are acquired simultaneously. When a target video frame exists whose face feature detection result matches the face feature condition, and the liveness detection result corresponding to the target video frame matches the liveness detection condition, the video segment composed of the acquired video frames is determined, including: When there is a target color video frame in the acquired color video frame whose face feature detection result matches the face feature condition, the target infrared video frame and the target depth video frame are determined based on the timestamp carried by the target color video frame. Liveness detection is performed based on the target color video frame, the target infrared video frame, and the target depth video frame. When the liveness detection result meets the liveness detection conditions, the video frame acquisition process ends, and the video segments composed of the acquired color video frame, infrared video frame, and depth video frame are determined.

11. The method according to claim 2, characterized in that, The video clips include color video clips, infrared video clips, and depth video clips; Sending the video segment and the target video frame to the server includes: The acquired color video clips, infrared video clips, and depth video clips are compressed separately, and the target color video frame, target infrared video frame, and target depth video frame are compressed separately. The compressed video clips and compressed images are sent to the server.

12. The method according to any one of claims 1 to 11, characterized in that, The step of performing facial feature recognition on the target video frame to determine candidate objects in the target video frame includes: The target face region in the target video frame is determined based on the area occupied by each face region in the target video frame. Facial features are identified in the target face region to determine candidate objects.

13. A target object recognition device in a target scene, characterized in that, The device includes: The video frame acquisition module is used to acquire video frames containing at least one face object when at least one face object is detected in the shooting field of view of the target scene; the target scene is a face recognition payment scene. The target video frame filtering module is used to determine the video segment composed of the acquired video frames when there is a target video frame in the acquired video frames, such that the face feature detection result of the target video frame meets the face feature conditions adapted to the target scene, and the liveness detection result corresponding to the target video frame meets the liveness detection conditions. The candidate object determination module is used to perform facial feature recognition on the target video frame and determine the candidate objects in the target video frame; The target object determination module is used to obtain the behavioral features of the candidate object based on the relative position change of the candidate object in each video frame of the video segment; when the behavioral feature analysis result corresponding to the behavioral features meets the behavioral feature conditions of the user corresponding to the face recognition payment scenario, the candidate object is determined as the target payment object.

14. The apparatus according to claim 13, characterized in that, The device further includes a sending module, which is used to send the video clip and the target video frame to the server. The candidate object and the target payment object are determined by the server after the video clip and the target video frame are sent to the server.

15. The apparatus according to claim 13, characterized in that, The device further includes a video frame caching module, which is used to cache the acquired video frames sequentially to a target queue according to the acquisition time. The number of cacheable video frames in the target queue is not greater than a number threshold. The target video frame filtering module is also used to terminate video frame acquisition when there is a target video frame among the acquired video frames, such that the face feature detection result of the target video frame meets the face feature conditions adapted to the target scene, and the liveness detection result corresponding to the target video frame meets the liveness detection conditions. Based on the video frames cached in the target queue, a video segment is obtained.

16. The apparatus according to claim 15, characterized in that, The device further includes a frame extraction processing module, which is used to perform frame extraction processing on the video frames in the target queue according to the frame extraction parameters when the number of video frames cached in the target queue reaches the number threshold; cache newly acquired video frames in the target queue; and perform frame extraction processing on continuously acquired video frames in the target queue according to the frame extraction parameters.

17. The apparatus according to claim 16, characterized in that, The device further includes a secondary frame extraction processing module, which is used to update the frame extraction parameters when the number of remaining video frames after frame extraction processing of the target queue reaches the number threshold; perform secondary frame extraction processing on the video frames in the target queue based on the updated frame extraction parameters; cache the newly acquired video frames in the target queue; and perform frame extraction processing on the newly cached video frames in the target queue based on the last video frame after secondary frame extraction processing and according to the updated frame extraction parameters.

18. The apparatus according to claim 16, characterized in that, The frame extraction parameter is the frame extraction interval number corresponding to the frame extraction method of discarding one video frame in each multi-frame video frame; the frame extraction processing module is also used to determine the initial inter-frame time interval of the video frames in the target queue based on the timestamps carried by the video frames in the target queue. Based on the initial inter-frame time interval and the number of frames to be extracted, the frame extraction interval is determined; based on the frame extraction interval, the video frames in the target queue are extracted.

19. The apparatus according to claim 18, characterized in that, The device further includes a secondary frame extraction processing module, which is used to reduce the frame extraction interval time when the number of remaining video frames after frame extraction processing of the target queue reaches the number threshold. The video frames in the target queue are subjected to secondary frame extraction based on the reduced frame extraction interval; the newly acquired video frames are cached in the target queue, and the video frames in the target queue with an inter-frame interval of the initial inter-frame interval are subjected to frame extraction according to the reduced frame extraction interval.

20. The apparatus according to claim 16, characterized in that, The frame extraction parameter is the number of consecutive frame extraction frames corresponding to the frame extraction method of retaining one video frame in each multi-frame video frame; the frame extraction processing module is also used to determine the initial inter-frame time interval of video frames in the target queue based on the timestamps carried by the video frames cached in the target queue; obtain the updated inter-frame time interval based on the initial inter-frame time interval and the number of consecutive frame extraction frames; and perform frame extraction processing on video frames in the target queue that do not meet the updated inter-frame time interval.

21. The apparatus according to claim 20, characterized in that, The device further includes a secondary frame extraction processing module, used to extend the update inter-frame time interval when the number of remaining video frames after frame extraction processing of the target queue reaches the number threshold; perform secondary frame extraction processing on the video frames in the target queue based on the extended inter-frame time interval; cache newly acquired video frames in the target queue; and perform frame extraction processing on video frames in the target queue that do not meet the extended inter-frame time interval.

22. The apparatus according to claim 13, characterized in that, The device further includes a video frame acquisition process control module, which is used to start the video frame acquisition process when at least one face object is detected in the shooting field of view of the target scene, and simultaneously acquire color video frames, infrared video frames, and depth video frames; the target video frame filtering module is also used to determine the target infrared video frame and the target depth video frame based on the timestamp carried by the target color video frame when there is a target color video frame whose face feature detection result meets the face feature condition; perform liveness detection based on the target color video frame, the target infrared video frame, and the target depth video frame; when the liveness detection result meets the liveness detection condition, end the video frame acquisition process, and determine the video segments composed of the acquired color video frames, infrared video frames, and depth video frames.

23. The apparatus according to claim 14, characterized in that, The video segments include color video segments, infrared video segments, and depth video segments; the sending module is also used to compress the acquired color video segments, infrared video segments, and depth video segments respectively, and to compress the target color video frame, target infrared video frame, and target depth video frame respectively; and to send the compressed video segments and compressed images to the server.

24. The apparatus according to any one of claims 13 to 23, characterized in that, The candidate object determination module is further configured to determine the target face region in the target video frame based on the area occupied by each face region in the target video frame; and to perform face feature recognition on the face features in the target face region to determine the candidate object.

25. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 12.

26. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 12.

27. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 12.

Images