A system and method for identifying head movement

Abstract

A method for identifying head movement includes receiving a series of images from an image acquisition device, wherein the images include visually presented images of a human face. The series of images may form an image sequence. Each image in the series of images has a previous image or a next image in the image sequence. The method further includes, for each image in the series of images, determining at least two feature points on the human face, determining the positions of the at least two feature points on the human face, and determining an asymmetry value based on the positions of the at least two feature points. The method further includes identifying a head shaking motion of the human face based on the asymmetry value of the series of images.

Description

[0001] This application is a divisional application of the invention application with application number 201880044545.9, application date April 25, 2018, and title “A system and method for identifying body movements”. Technical Field

[0002] The present application relates to image processing technology, and more specifically, to a system and method for identifying head movement. Background Art

[0003] In recent years, facial authentication, such as through cameras, has been widely used in a variety of fields (e.g., system login, security, finance) to prevent fraud. Examples of facial authentication, including recognition of movements of human body parts (e.g., blinking and nodding), may require the user to be very close to the camera. If the user is relatively far from the camera, the accuracy of recognizing such movements may be drastically reduced. Therefore, it may be desirable to provide a system and method for improving the accuracy of recognizing a user's head movements when he or she is at a certain distance from the camera. Summary of the invention

[0004] According to one aspect of the present application, a system may include an image acquisition device, a storage device including a set of instructions, and at least one processor in communication with the storage device. When executing the set of instructions, the at least one processor may be instructed to cause the system to: receive a series of images from the image acquisition device, the images including visually presented images of human faces. The series of images may form an image sequence. Each image in the series of images may have a previous image or a next image in the image sequence. For each image in the series of images, the at least one processor may be further instructed to cause the system to: determine at least two feature points on the human face; determine at least two feature points on the human face; and determine the positions of the at least two feature points on the human face. The at least one processor may be further instructed to cause the system to recognize a shaking motion of the human face based on the asymmetric value of the image sequence.

[0005] In some embodiments, the at least two feature points may include a first point, a second point, a third point, a fourth point, and a fifth point. In the front view of the face: the first point and the second point may be symmetrical with respect to a symmetry line passing through the third point; the fourth point and the fifth point may be symmetrical with respect to the symmetry line passing through the third point; and the third point may be located within a quadrilateral determined based on the first point, the second point, the fourth point, and the fifth point.

[0006] In some embodiments, the first point may be located at the central area of ​​the left eye of the face. The second point may be located at the central area of ​​the right eye of the face. The third point may be located at the tip area of ​​the nose on the face. The fourth point may be located at the left end area of ​​the lips on the face. The fifth point may be located at the right end area of ​​the lips on the face.

[0007] In some embodiments, in order to determine the asymmetry value based on the positions of the at least two feature points of the series of images, the processor can be further instructed to cause the system to: determine whether the third point in each image in the series of images is not located within the quadrilateral determined based on the first point, the second point, the fourth point and the fifth point; and based on the result of the determination that the third point in each image in the series of images is not located within the quadrilateral determined based on the first point, the second point, the fourth point and the fifth point, determine the asymmetry value according to the asymmetry value of the previous image in each image in the series of images.

[0008] In some embodiments, in order to determine the asymmetry value based on the positions of the at least two feature points of the series of images, the processor can be further instructed to cause the system to: determine a first angle based on the first point, the third point and the fourth point; determine a second angle based on the second point, the third point and the fifth point; and determine the asymmetry value based on the first angle and the second angle based on the result of determining that the third point in each image in the series of images is located within the quadrilateral determined based on the first point, the second point, the fourth point and the fifth point.

[0009] In some embodiments, the first angle may be an angle between a line segment connecting the first point and the third point and a line segment connecting the fourth point and the third point. The second angle may be an angle between a line segment connecting the second point and the third point and a line segment connecting the fifth point and the third point.

[0010] In some embodiments, the first angle may be an angle between a line segment connecting the first point and the fourth point and a line segment connecting the fourth point and the third point. The second angle may be an angle between a line segment connecting the second point and the fifth point and a line segment connecting the fifth point and the third point.

[0011] In some embodiments, the first angle may be an angle between a line segment connecting the first point and the fourth point and a line segment connecting the first point and the third point. The second angle may be an angle between a line segment connecting the second point and the fifth point and a line segment connecting the second point and the third point.

[0012] In some embodiments, in order to identify the head shaking motion of the human face based on the asymmetry value of the series of images, the processor can be further instructed to enable the system to: determine a first image in the series of images in the image sequence whose asymmetry value is less than a first asymmetry value threshold; determine a second image in the series of images in the image sequence whose asymmetry value is less than the first asymmetry value threshold, wherein the second image is a subsequent image of the first image in the image sequence; obtain asymmetry values ​​of a group of images, wherein the group of images includes the first image, the second image, and one or more images between the first image and the second image in the image sequence; and identify the head shaking motion of the human face based on the asymmetry value of the group of images and the number of images in the group of images.

[0013] In some embodiments, in order to identify the head shaking motion of the human face based on the asymmetry value of the group of images and the number of images in the group of images, the processor can be further instructed to enable the system to: identify the maximum asymmetry value among the asymmetry values ​​of the group of images; determine whether the maximum asymmetry value is greater than a second asymmetry value threshold; determine whether the number of images in the group of images is greater than a threshold number; and based on the result of determining that the maximum asymmetry value is greater than the second asymmetry value threshold and the result of determining that the number of images in the group of images is greater than the threshold number, identify the head shaking motion of the human face from the group of images.

[0014] According to another aspect of the present application, a method may include receiving a series of images from an image acquisition device, wherein the images include visually presented images of a human face. The series of images may form an image sequence. Each image in the series of images has a previous image or a next image in the image sequence. The method may also include determining at least two feature points on the human face; determining the positions of the at least two feature points on the human face; and determining an asymmetry value based on the positions of the at least two feature points. The method may also include identifying a shaking motion of the human face based on the asymmetry values ​​of the series of images.

[0015] According to another aspect of the present application, a non-transitory computer-readable medium may include at least one set of instructions. When executed by at least one processor of a computer device, the at least one set of instructions may instruct the at least one processor to: receive a series of images from an image acquisition device, the images including visually presented images of human faces. The series of images may form an image sequence. Each image in a series of images may have a previous image or a next image in the image sequence. For each image in a series of images, the at least one processor may be further instructed to: determine at least two feature points on the human face; determine the positions of the at least two feature points on the human face; and determine an asymmetry value based on the positions of the at least two feature points. The at least one processor may be further instructed to identify a shaking motion of the human face based on the asymmetry values ​​of the series of images. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The present application will be further described by exemplary embodiments. These exemplary embodiments will be described in detail by the accompanying drawings. These embodiments are non-limiting exemplary embodiments, in which the same numbers in each figure represent similar structures, characterized in that:

[0017] Figure 1 is a schematic diagram of an operating state of an exemplary system for identifying a head shaking action according to some embodiments of the present application;

[0018] Figure 2 is a schematic diagram of an exemplary system for identifying a head shaking action according to some embodiments of the present application;

[0019] Figure 3 is a schematic diagram of exemplary hardware components and / or software components of a computing device according to some embodiments of the present application;

[0020] Figure 4 is a schematic diagram of exemplary hardware and / or software components of a mobile device that can be implemented on the system 200 according to some embodiments of the present application;

[0021] Figure 5 is a module diagram of an exemplary processing engine according to some embodiments of the present application;

[0022] Figure 6 is a flowchart of an exemplary process of identifying a head shaking action according to some embodiments of the present application;

[0023] Figure 7 is a schematic diagram of an exemplary visual presentation image of a human face according to some embodiments of the present application;

[0024] Figure 8is a flowchart of an exemplary process of determining an asymmetric value based on positions of at least two feature points according to some embodiments of the present application;

[0025] Fig. 9 is a schematic diagram of exemplary feature points of a human face according to some embodiments of the present application;

[0026] Fig.10 is a schematic diagram of exemplary first angles and second angles according to some embodiments of the present application;

[0027] Fig.11 is a schematic diagram of exemplary first angles and second angles according to some embodiments of the present application;

[0028] Fig.12 is a schematic diagram of exemplary first angles and second angles according to some embodiments of the present application;

[0029] Fig.13 is a schematic diagram of an exemplary first angle and a second angle in the left direction of a human face according to some embodiments of the present application;

[0030] Fig.14 is a schematic diagram of an exemplary first angle and a second angle in the right direction according to some embodiments of the present application;

[0031] Fig.15 is a flowchart of an exemplary process of identifying a head shaking action according to some embodiments of the present application;

[0032] Fig.16 is a schematic diagram of the relationship between the asymmetric values ​​of the three rotations of the head shaking action and time (T) according to some embodiments of the present application; and

[0033] Fig.17 is a flowchart of an exemplary process of identifying a head shaking action according to some embodiments of the present application. DETAILED DESCRIPTION

[0034] The following description is intended to enable a person of ordinary skill in the art to implement and utilize the disclosure of the present application and is provided in the context of a specific application scenario and its requirements. It is apparent to a person of ordinary skill in the art that various changes may be made to the disclosed embodiments, and that the general principles defined in the present application may be applied to other embodiments and application scenarios without departing from the principles and scope of the present application. Therefore, the present application is not limited to the described embodiments, but should be given the broadest scope consistent with the claims.

[0035] The terms used herein are only used for the purpose of describing specific exemplary embodiments and are not restrictive. The singular forms "a", "an" and "the" used in this application may also include plural forms unless the context clearly indicates an exception. It should also be understood that, as in the specification disclosed in this application, the terms "include" and "comprise" only indicate the presence of the features, wholes, steps, operations, components and / or parts, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, components, parts and / or combinations thereof.

[0036] These and other features, characteristics and functions and methods of operation of the related structural elements disclosed in the present application, as well as the combination of parts and manufacturing economy, may become more apparent from the following description of the accompanying drawings, which constitute a part of the specification disclosed in the present application. However, it should be understood that the drawings are for illustration and description purposes only and are not intended to limit the scope of the present application. It should be understood that the drawings are not drawn to scale.

[0037] Flowcharts are used in this application to illustrate the operations performed by the systems of some embodiments disclosed in this application. It should be understood that the operations in the flow charts may not be performed in order. On the contrary, various steps may be processed in reverse order or simultaneously. At the same time, one or more other operations may also be added to these flow charts. One or more operations may also be deleted from the flow charts.

[0038] In addition, although the system and method disclosed in this application mainly relate to online services for facial authentication, it should also be understood that this is only an exemplary embodiment. The system or method disclosed in this application can be applied to any type of online service or offline service. For example, the system or method disclosed in this application can be applied to a security system for user login (e.g., a user login on-demand system), registration (e.g., a registration system for a hotel or airport), user verification (e.g., for user account verification at an automated teller machine (ATM)), etc., or any combination thereof. The on-demand system may include, for example, transportation / takeout / express delivery services. The application of the system or method disclosed in this application may include a web page, a plug-in for an Internet browser, a client terminal, a customization system, an internal analysis system, an artificial intelligence robot, etc., or any combination thereof.

[0039] The present application provides a system and method for identifying a head shaking motion. The system can collect at least two images of a face through an image acquisition device (e.g., a camera). The system can process the at least two images of the face and identify feature points of the face in the at least two images. The system can also determine an asymmetry value of the face in the image based on the identified feature points. The system can further determine whether there is a head shaking motion based on the asymmetry value.

[0040] Figure 1 1 is a schematic diagram of the operating state of an exemplary system for identifying a head shaking action according to some embodiments of the present application. The system 100 may include at least one image acquisition device, which may acquire an image in a field of view 107 of the image acquisition device. Figure 1 As shown, the field of view 107 may include a face 111. The system 100 may capture at least two images including the face 111. Then, the system 100 may determine whether there is a head shaking motion based on the at least two images. Details about the system 100 may be found elsewhere in this application (e.g., Figure 2 and its description).

[0041] Figure 2 2 is a schematic diagram of an exemplary system for identifying a head shaking action according to some embodiments of the present application. The system 200 may include a server 205, a memory 220, a network 230, and an image acquisition device 240. The system 200 may be an example of the system 100.

[0042] In some embodiments, the server 205 can be a single server or a server group. The server group can be centralized or distributed (for example, the server 205 can be a distributed system). In some embodiments, the server 205 can be local or remote. For example, the server 205 can access the information and / or data stored in the image acquisition device 240 and / or the memory 220 through the network 230. For another example, the server 205 can be directly connected to the memory 220 to access the stored information and / or data. In some embodiments, the server 205 can be implemented on a cloud platform. By way of example only, the cloud platform can include a private cloud, a public cloud, a hybrid cloud, a community cloud, a distributed cloud, an internal cloud, a multi-layer cloud, the like, or any combination thereof. In some embodiments, the server 205 can be implemented on a cloud platform having Figure 3 One or more components shown in FIG. 1 are implemented on a computing device 300 .

[0043] In some embodiments, the server 205 may include a processing engine 210. The processing engine 210 may process information and / or data related to identifying a head shaking motion to perform one or more functions described in the present application. For example, the processing engine 210 may determine one or more feature points on the face of each of the at least two images corresponding to the face, and determine the asymmetric value of each of the at least two images based on the feature points to identify a head shaking motion. In some embodiments, the processing engine 210 may include one or more processing engines (e.g., a single-chip processing engine or a multi-chip processing engine). The processing engine 210 may include a central processing unit (CPU), an application specific integrated circuit (ASIC), an application specific instruction set processor (ASIP), a graphics processing unit (GPU), a physical processing unit (PPU), a digital signal processor (DSP), a field programmable gate array (FPGA), a programmable logic device (PLD), a controller, a microcontroller unit, a reduced instruction set computer (RISC), a microprocessor, etc., or any combination thereof.

[0044] The memory 220 may store data and / or instructions related to identifying a shaking head motion. In some embodiments, the memory 220 may store data obtained from the image acquisition device 240. In some embodiments, the memory 220 may store data and / or instructions used by the server 205 to execute or use to complete the exemplary method described in the present application. In some embodiments, the memory 220 may include a large capacity memory, a removable memory, a volatile read-write memory, a read-only memory (ROM), etc., or any combination thereof. Exemplary large capacity memories may include disks, optical disks, solid-state disks, etc. Exemplary removable memories may include flash drives, floppy disks, optical disks, memory cards, compressed disks, tapes, etc. Exemplary volatile read-write memories may include random access memory (RAM). Exemplary RAM may include dynamic random access memory (DRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), static random access memory (SRAM), thyristor random access memory (T-RAM), zero capacitance random access memory (Z-RAM), etc. Exemplary read-only memory may include mask read-only memory (MROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), compact disk read-only memory (CD-ROM), digital versatile disk read-only memory, etc. In some embodiments, the memory 220 may be implemented on a cloud platform. By way of example only, the cloud platform may include a private cloud, a public cloud, a hybrid cloud, a community cloud, a distributed cloud, an internal cloud, a multi-layer cloud, etc., or any combination thereof.

[0045] In some embodiments, the memory 220 may be connected to the network 230 to communicate with one or more components of the system 200 (e.g., the server 205, the image acquisition device 240). One or more components of the system 200 may access the data and / or instructions stored in the memory 220 through the network 230. In some embodiments, the memory 220 may be directly connected or communicated with one or more components of the system 200 (e.g., the server 205, the image acquisition device 240). In some embodiments, the memory 220 may be part of the server 205.

[0046] In some embodiments, one or more components of the system 200 (e.g., the server 205, the image acquisition device 240) may have permission to access the memory 220. In some embodiments, when one or more conditions are met, one or more components of the system 200 may read and / or modify information related to the face and / or the public. For example, the server 205 may read and / or modify one or more of the at least two images corresponding to the face after completing the recognition.

[0047] The network 230 may facilitate the exchange of information and / or data. In some embodiments, one or more components of the system 200 (e.g., the server 205, the image acquisition device 240, or the memory 220) may transmit information and / or data to another component of the system 200 via the network 230. For example, the server 205 may obtain at least two images corresponding to the face from the image acquisition device 240 via the network 230. In some embodiments, the network 230 may be a wired network, a wireless network, or any combination thereof. By way of example only, the network 230 may include a cable network, a wired network, a fiber optic network, a telecommunications network, an intranet, the Internet, a local area network (LAN), a wide area network (WAN), a wireless local area network (WLAN), a metropolitan area network (MAN), a public switched telephone network (PSTN), a Bluetooth network, a ZigBee network, a near field communication (NFC) network, etc., or any combination thereof. In some embodiments, the network 230 may include one or more network access points. For example, the network 230 may include wired or wireless network access points, such as base stations and / or Internet exchange points 230-1, 230-2, ..., through which one or more components of the system 200 may be connected to the network 230 to exchange data and / or information.

[0048] In some embodiments, the image acquisition device 240 may include a portable computer 240-1 with a camera, a camera 240-2, a smartphone 240-3 with a camera, a smart watch with a camera, a vehicle-mounted computer with a camera, a vehicle-mounted TV with a camera, a wearable device with a camera, etc., or any combination thereof. In some embodiments, the image acquisition device 240 may acquire at least two images including a face in the field of view. In some embodiments, the image acquisition device 240 may be integrated with the server 205 and the memory 220. For example, Figure 4 As shown, the mobile device 400 can implement the operations of the image acquisition device 240, the server 205 and the memory 220 through the hardware and / or software components of the mobile device 400. In some embodiments, the image acquisition device 240 can be Figure 3 One or more components shown in FIG. 1 are implemented on a computing device 300 .

[0049] It will be understood by those skilled in the art that when the elements (or components) of the system 200 are executed, the elements may be executed via electrical signals and / or electromagnetic signals. For example, when the image acquisition device 240 transmits at least two images to the server 205, the image acquisition device 240 may generate an electrical signal encoding the at least two images. The image acquisition device 240 may then transmit the electrical signal to an output port. If the image acquisition device 240 communicates with the server 205 via a wired network, the output port may be physically connected to a cable, which may also transmit the electrical signal to an input port of the server 205. If the image acquisition device 240 communicates with the server 205 via a wireless network, the output port of the image acquisition device 240 may be one or more antennas, which may convert the electrical signal into an electromagnetic signal. In an electronic device, such as the image acquisition device 240 and / or the server 205, when its processor processes instructions, transmits instructions and / or performs actions, the instructions and / or actions are performed via electrical signals. For example, when the processor retrieves or saves data from a storage medium (e.g., the memory 220), the processor may transmit an electrical signal to a read / write device of the storage medium, and the read / write device may read or write structured data in the storage medium. The structured data may be transmitted to the processor via a bus of the electronic device in the form of an electrical signal. Here, an electrical signal may refer to one electrical signal, a series of electrical signals, and / or at least two discontinuous electrical signals.

[0050] Figure 33 is a schematic diagram of exemplary hardware components and / or software components of a computing device according to some embodiments of the present application. In some embodiments, the server 205 and the image acquisition device 240 can be implemented on the computing device 300. For example, the processing engine 210 can be implemented on the computing device 300, and the processing engine 210 is configured to perform the functions of the processing engine 210.

[0051] The computing device 300 may be used to implement any component of the system 200 as described herein. For example, the processing engine 210 may be implemented on the computing device 300 by hardware, software programs, firmware, or a combination thereof. Although only one such computer is shown, for convenience, the computer functions related to the recognition of head shaking motions as described herein may be implemented in a distributed manner on multiple similar platforms to distribute the processing load.

[0052] The computing device 300 may include, for example, a communication port 350 connected to a network to facilitate data communication. The computing device 300 may also include a processor 320 that executes program instructions in the form of one or more processors (e.g., logic circuits). For example, the processor 320 may include an interface circuit and a processing circuit therein. The interface circuit may be configured to receive an electrical signal from a bus 310, wherein the electrical signal encoding is used to process structured data and / or instructions of the processing circuit. The processing circuit may perform logical calculations and then determine the conclusion, result, and / or instruction encoding as an electrical signal. Then, the interface circuit may send the electrical signal from the processing circuit via the bus 310.

[0053] The computing device 300 may also include different forms of program storage and data storage, such as a disk 370, a read-only memory (ROM) 330, or a random access memory (RAM) 340, for storing various data files processed and / or transmitted by the computing device. The exemplary computer platform may also include program instructions stored in ROM 330, RAM 340, and / or other types of non-transitory storage media executed by the processor 320. The methods and / or processes disclosed in this application may be implemented in the form of program instructions. The computing device 300 may also include an I / O component 360, which may be used to support input / output between the computer and other components. The computing device 300 may also receive programming and data via network communications. The computing device 300 may also include a camera 380 for performing the functions of the image acquisition device 240.

[0054] For the sake of convenience, Figure 3Only one processor is described in the description. Multiple processors are also considered. Therefore, the operations and / or method steps performed by one processor in the description disclosed in the present application may also be performed jointly or independently by multiple processors. For example, if in the disclosure of the present application, the processor of the computing device 300 performs steps A and B, it should be understood that steps A and B may also be performed jointly or independently by two different CPUs and / or processors of the computing device 300 (for example, the first processor performs step A, the second processor performs step B, or the first and second processors perform steps A and B jointly).

[0055] Figure 4 2 is a schematic diagram of exemplary hardware and / or software components of a mobile device that can be implemented on system 200 according to some embodiments of the present application. Figure 4 As shown, the mobile device 400 may include a camera 405, a communication platform 410, a display 420, a graphics processing unit (GPU) 430, a central processing unit (CPU) 440, an I / O 450, a memory 460, a mobile operating system (OS) 470, and a storage 490. In some embodiments, any other suitable components, including but not limited to a system bus or a controller (not shown), may also be included in the mobile device 400.

[0056] In some embodiments, the mobile operating system 470 (e.g., iOS TM 、Android TM 、Windows Phone TM The application 480 may include a browser or any other suitable mobile application for receiving and presenting information related to the recognition of head shaking motion or other information from the system 200. The user's interaction with the information stream may be completed through the I / O 450 and provided to the processing engine 210 and / or other components of the system 200 through the network 230.

[0057] Figure 5 2 is a block diagram of an exemplary processing engine according to some embodiments of the present application. The processing engine 210 may include a collection module 510 , a determination module 520 and an identification module 530 .

[0058] The acquisition module 510 may be configured to receive image data. In some embodiments, the acquisition module 510 may receive a series of images from the image acquisition device 240, the images including visually presented images of a face. The series of images may form an image sequence, and each image in the series of images may have a previous image or a next image in the image sequence. In some embodiments, the sequence of the series of images may follow a time order. The sequence of the series of images may be determined based on a time point. A specific time point may have a previous time point and / or a next time point.

[0059] The determination module 520 may be configured to determine a plurality of information related to the image data received by the acquisition module 510. In some embodiments, the determination module 520 may determine at least two feature points on the face of the person for the series of images. The at least two feature points may include any point on the face of the person. According to the position of the feature points in the face of the person, the feature points may be classified into three different groups (e.g., a first group, a second group, and a third group). For example, the feature points in the first group may fall on the symmetry line of the face of the person. The determination module 520 may determine the position of the at least two feature points on the face of the person of the series of images. In some embodiments, for each image in a series of images, the processing engine 210 may first identify the relevant area of ​​the feature points. The determination module 520 may determine the asymmetry value of the series of images (or the asymmetry value of the visual presentation image including the face in the image, the series of images forming an image sequence) based on the position of the feature points of the series of images.

[0060] The recognition module 530 may be configured to recognize whether there is a head shaking motion based on the asymmetry value of the series of images (or the asymmetry value of the visually presented image including the face in the image, the series of images forming an image sequence). In some embodiments, the recognition module 530 may determine the head shaking motion based on the time sequence of the acquisition of the series of images and the change trend of the asymmetry value of the series of images. For example, the recognition module 530 may determine the first image in the series of images in the image sequence whose asymmetry value is less than the first asymmetry value threshold, and determine the second image in the series of images in the image sequence whose asymmetry value is less than the first asymmetry value threshold, the second image being the subsequent image of the first image in the image sequence. The recognition module 530 may also obtain the asymmetry value of a group of images, the group of images including the first image, the second image, and one or more images between the first image and the second image in the image sequence. The recognition module 530 may further recognize the head shaking motion of the face based on the asymmetry value of the group of images and the number of images in the group of images.

[0061] Figure 6 600 is a flowchart of an exemplary process of identifying a head shaking motion according to some embodiments of the present application. In some embodiments, the process 600 may be implemented as a set of instructions (e.g., an application program) stored in the storage ROM 330 or RAM 340. The processor 320 and / or Figure 5 The module in the embodiment may execute the set of instructions, and when executing the instructions, the processor 320 and / or the module may be configured to perform the process 600. The operations of the process shown below are for illustration purposes only. In some embodiments, the process 600 may be completed by one or more additional operations not described and / or one or more operations not discussed herein. In addition, Figure 6 The order in which the steps of the processes are shown and described below is not limiting.

[0062] In step 610, the processing engine 210 (e.g., the acquisition module 510) may receive a series of images from the image acquisition device 240, the images including visually presented images of human faces, the series of images forming an image sequence, each image in the series of images having a previous image or a next image in the image sequence. For example, the series of images may be video frames of a video acquired by the image acquisition device 240 (e.g., a camera), which may form an image sequence. Each of the video frames may have a previous video frame and a next video frame except for the first video frame and the last video frame, which may have only a next video frame and a previous video frame, respectively. In some embodiments, the sequence of the series of images may follow a time order. For example, the image acquisition device 240 may acquire images at least at two time points. The sequence of the series of images may be determined based on the time points. A specific time point may have a previous time point and / or a next time point. The previous video frame may be an image of the series of images acquired by the image acquisition device 240 at the previous time point. The next video frame may be an image of the series of images acquired by the image acquisition device 240 at the next time point. For example, when the image acquisition device 240 starts to acquire images, the processing engine 210 may start a timer. The image acquisition device 240 may acquire images every 0.03 seconds. For a specific image acquired at 0.06 seconds, the previous time point may be 0.03 seconds, and the next time point may be 0.09 seconds. Therefore, the previous image of the specific image may be acquired at a time point of 0.03 seconds, and the next image of the specific image may be acquired at a time point of 0.09 seconds.

[0063] In some embodiments, the image acquisition device 240 (e.g., the portable computer 240-1, the camera 240-2, the smartphone 240-3) may be configured to acquire images corresponding to one or more objects appearing in the field of view of the image acquisition device 240. For example, the image acquisition device 240 may detect the movement of an object (e.g., a person) appearing in its field of view and acquire images. The image acquisition device 240 may acquire a series of images including the person's face. The object in the field of view of the image acquisition device 240 may move or perform certain movements of his or her body parts (e.g., shaking head, bowing head).

[0064] In some embodiments, at least one of the series of images may include the visual presentation image of the face. The visual presentation image of the face may include at least two outlines of one or more feature components of the face. Exemplary feature components of a face may include eyes, nose, mouth, ears, lips, eyebrows, etc., or a combination thereof. The outlines of the feature components may include the outline of the face, the outline of the eyes, the outline of the nose, etc. In some embodiments, the processing engine 210 and / or the image acquisition device 240 may mark different parts of the face with different colors in the visual presentation image of the face, and the colored areas represent the feature components of the face. For example, the area of ​​the eyes may be marked with green, the area of ​​the pupils of the eyes may be marked with black, and the area of ​​the nose may be marked with blue, and so on.

[0065] In some embodiments, the processing engine 210 may transmit a request for an image to the image acquisition device 240, and the image acquisition device 240 may acquire a series of images and / or transmit a series of images to the processing engine 210 in response to the request. In some embodiments, the processing engine 210 may transmit a request to the image acquisition device 240, requesting a user to follow certain instructions. For example, after receiving the request from the processing engine 110, the image acquisition device 240 may display a message to the user, requesting the user to move or perform certain movements of his or her body parts (e.g., shaking head movements, head-down movements) in the field of view of the image acquisition device 240 to verify the user. The image acquisition device 240 may also acquire an image of the user.

[0066] In some embodiments, the processing engine 210 may receive the series of images from the image acquisition device 240 via the network 230. Alternatively, the processing engine 210 may communicate with the image acquisition device 240 (e.g., Figure 4 The CPU 440 and the camera 405 are integrated as shown in FIG. 2 . The processing engine 210 can be connected via a bus (e.g., Figure 3 The bus 310 shown in FIG. 3 receives the series of images from the image acquisition device 240 .

[0067] In step 620, the processing engine 210 (e.g., the determination module 520) may determine at least two feature points on the face for each image in a series of images. The at least two feature points may include any point on the face. The feature points may be classified into three groups, including a first group, a second group, and a third group. The feature points included in the first group may fall on a symmetry line of the face. For example, in a front view of the face, the symmetry line of the face may be a vertical line on the face, the vertical line divides the face into a left side and a right side, and the feature points included in the first group may fall on the vertical line. The left side of the face is substantially symmetrical to the right side of the face. The feature points included in the second group may be located in a left region of the face. The feature points included in the third group may be located in a right region of the face. For the feature points in the second group, there may be feature points in the third group that are symmetrical with respect to the symmetry line.

[0068] For example, the at least two feature points may include five points on the human face, namely, a first point, a second point, a third point, a fourth point and a fifth point. The third point may fall on a symmetry line of the human face (for example, a vertical line passing through the center of the nose and the center of the forehead) and may be in the first group. The first point and the fourth point may be in the second group. The second point and the fifth point may be in the third group. The first point and the second point may be symmetrical with respect to the symmetry line, and the third point may fall on the symmetry line. The fourth point and the fifth point may be symmetrical with respect to the symmetry line. In some embodiments, the third point may be located within a quadrilateral determined based on the first point, the second point, the fourth point and the fifth point (for example, a quadrilateral including the first, second, fourth and fifth points). The first point, the second point, the fourth point and the fifth point may include the four vertices of the quadrilateral. Figure 7 Take the face 111 shown in as an example; the feature points of the face 111 may include the first point 710-1, the second point 720-1, the third point 730-1, the fourth point 740-1 and the fifth point 740-2. The first point 710-1 may be located in the left eye area 710. The second point 720-1 may be located in the right eye area 720. In some embodiments, the first point 710-1 and the second point 720-1 may correspond to the central areas of the two eyes (for example, the pupils of the two eyes), respectively. The third point 730-1 may be located in the tip area of ​​the nose area 730. The fourth point 740-1 may be located in the left end area of ​​the lips 740. The fifth point 740-2 may be located in the right end area of ​​the lips 740. The symmetry line 705 may pass through the third point 730-1.

[0069] In step 630, the processing engine 210 (e.g., the determination module 520, the processor 320) may determine the positions of the at least two feature points on the face. In some embodiments, for each image in a series of images, the processing engine 210 may first identify a relevant region for the feature points. As used herein, the relevant region for a particular feature point refers to the region where the particular feature point is located. The position of the particular feature point may be substantially fixed in its relevant region. For example, Figure 7 As shown, the left eye region 710 is a related region of the feature point 710-1. The position of the feature point 710-1 may be located at the center point of the left eye region 710. The distance between the center point and the upper boundary of the outline of the left eye region 710 may be equal to the distance between the center point and the bottom boundary of the outline of the left eye region 710. The distance between the center point and the left boundary of the outline of the left eye region 710 may be equal to the distance between the center point and the right boundary of the outline of the left eye region 710.

[0070] In some embodiments, the processing engine 210 may determine the feature points based on the relevant areas. For example, the processing engine 210 may identify two eye areas, a nose area, and a lip area. The processing engine 210 may also determine five feature points based on the four areas. Taking the left eye area 710 as an example; the processing engine 210 may first identify the left eye contour from one of the series of images. The left eye contour may include at least two candidate points. The processing engine 210 may identify a candidate point that satisfies the features of the center point of the eye area. The processing engine 210 may designate the candidate point as the first point 710-1.

[0071] In step 640, the processing engine 210 (e.g., the determination module 520, the processor 320) may determine an asymmetry value based on the positions of the at least two feature points of the series of images (or include visually presented images including a human face in the images, the series of images forming an image sequence). For an image of a human face, if the image is not a frontal view of the human face, the left side of the human face and the right side of the human face may be asymmetric relative to the symmetry line. The asymmetry value is a parameter indicating the degree of asymmetry between the left and right sides of the human face. Each image in a series of images may have an asymmetry value (or include visually presented images including a human face in the images, the series of images forming an image sequence). Details on determining the asymmetry value of an image based on the positions of feature points on a human face may be found elsewhere in this application (e.g., Figure 8 and its description).

[0072] In step 650, the processing engine 210 (e.g., the recognition module 530, the processor 320) may identify a head shaking motion based on the asymmetry values ​​of the series of images (or the asymmetry values ​​of the visually presented images including the faces of the people in the images, the series of images forming an image sequence). In some embodiments, the processing engine 210 may determine the head shaking motion based on the time sequence of the acquisition of the series of images and the trend of changes in the asymmetry values ​​of the series of images. Details on identifying the head shaking motion may be found elsewhere in this application (e.g., Fig.13 and 17 and its description).

[0073] It should be understood that the above description is for illustrative purposes only and is not intended to limit the scope of the present application. For those of ordinary skill in the art, various changes and modifications can be made based on the description of the present application. However, these changes and modifications will not deviate from the scope disclosed in the present application. For example, the processing engine 210 can be part of a camera. The camera can perform the operations of the processing engine 210, for example, determining at least two feature points on the face of the person.

[0074] Figure 7 is a schematic diagram of an exemplary visual presentation image of a human face according to some embodiments of the present application. Figure 7 As shown, the visual representation 700 of the face 111 is in a front view. The visual representation 700 may include a left eye region 710, a right eye region 720, a nose region 730, and a lip region 740, etc. The visual representation 700 may also include a symmetry line 705 and at least two feature points, namely, the first point 710-1, the second point 720-1, the third point 730-1, the fourth point 740-1, and the fifth point 740-2. Figure 7 As shown, the first point 710-1 and the second point 720-1 may respectively correspond to pupils of the eyes of the face 111. The first point 710-1 and the second point 720-1 may be symmetrical with respect to the symmetry line 705. The fifth point 740-2 and the fourth point 740-1 may be symmetrical with respect to the symmetry line 705. The third point 730-1 may fall on the symmetry line 705.

[0075] Figure 8 800 is a flowchart of an exemplary process for determining an asymmetric value based on the positions of at least two feature points according to some embodiments of the present application. In some embodiments, the process 800 may implement a set of instructions (e.g., an application) stored in the storage ROM 330 or RAM 340. The processor 320 and / or Figure 5The modules in the embodiment may execute the set of instructions, and when executing the instructions, the processor 320 and / or the modules may be configured to perform the process 800. The operations of the process shown below are for illustration purposes only. In some embodiments, the process 800 may be completed using one or more additional operations not described and / or one or more operations not discussed herein. In addition, Figure 8 The order of the steps of the process shown and described below is not limiting. In some embodiments, step 640 of the process 600 can be performed based on the process 800 to determine the asymmetry value for each image in a series of images.

[0076] In step 810, for an image (e.g., each image in a series of images), the processing engine 210 (e.g., the acquisition module 510) may obtain the positions of the first point, the second point, the third point, the fourth point, and the fifth point. In some embodiments, the processing engine 210 may obtain the positions from the memory 220. The memory 220 may store the positions after determining the positions. The description of determining the positions of the first point, the second point, the third point, the fourth point, and the fifth point may be found elsewhere disclosed in this application (e.g., step 630 of the process 600).

[0077] The first point, the second point, the fourth point and the fifth point may be four vertices of a quadrilateral. In some images of the series of images, the third point may be located within the quadrilateral (e.g. Fig. 9 The third point 730-1 may be located within the quadrilateral. In some images of the series of images, the third point may be located outside the quadrilateral.

[0078] In step 820, the processing engine 210 (e.g., the determination module 520) may determine whether the third point is located within the quadrilateral formed by the first point, the second point, the fourth point, and the fifth point. If the processing engine 210 (e.g., the determination module 520) determines that the third point is not located within the quadrilateral, the process 800 may proceed to step 860; if the processing engine 210 (e.g., the determination module 520) determines that the third point is located within the quadrilateral, the process 800 may proceed to step 830.

[0079] In step 830 , the processing engine 210 (eg, the determination module 520 ) may determine a first angle based on the first point, the third point, and the fourth point.

[0080] In step 840 , the processing engine 210 (eg, the determination module 520 ) may determine a second angle based on the second point, the third point, and the fifth point.

[0081] In step 850, the processing engine 210 (e.g., the determination module 520) may determine the asymmetry value based on the first angle and the second angle in each image in the series of images. In some embodiments, the asymmetry value may be determined according to the following equation (1):

[0082]

[0083] Among them, α represents the first angle, and β represents the second angle. Compared with the asymmetry value corresponding to the left direction / right direction of the face, the asymmetry value corresponding to the front direction of the face may be a minimum value (for example, equal to 1). The front direction of the face means that the person's head is directly facing the camera. The left direction of the face means that the person's head turns from the front direction of the face to the left. The right direction of the face means that the person's head turns from the front direction of the face to the right. For example, when the face is in the front direction, the first angle α may be equal to the second angle β. The asymmetry value in this case may be approximately equal to 1 (ignoring the minimum error caused by the asymmetric position of the feature points on the face). For example Fig.13 As shown in FIG. 1 , when the face is in the left direction, the first angle α may be greater than the second angle β. In this case, the corresponding asymmetric value may be greater than 1. Fig.14 As shown, when the face is in the right direction, the first angle α may be smaller than the second angle β. In this case, the corresponding asymmetry value may be greater than 1.

[0084] In step 860, the processing engine 210 (e.g., the determination module 520) may determine the asymmetry value based on the asymmetry value of the previous image in each image in the series of images. In some embodiments, the processing engine 210 may determine the previous image in each image in the series of images based on the image sequence of the series of images. The processing engine 210 may determine the asymmetry value of the previous image in each image in the series of images by operating the process 800. Then, the processing engine 210 may specify the asymmetry value of the previous image in each image in the series of images as the asymmetry value of each image in the series of images.

[0085] It should be understood that the above description is for illustrative purposes only and is not intended to limit the scope of the present application. For those of ordinary skill in the art, various changes and modifications can be made according to the description of the present application. However, these changes and modifications will not deviate from the scope disclosed by the present application. For example, the steps 830 and 840 can be operated in one step. For another example, a storage operation can be added in the process 800, for example, between the steps 840 and 850.

[0086] Fig. 9 7 is a schematic diagram of exemplary feature points of a human face according to some embodiments of the present application. The feature points, including the first point 710-1, the second point 720-1, the third point 730-1, the fourth point 740-1 and the fifth point 740-2, may correspond to the following: Figure 7 The feature points on the face 111 are shown. The first point 710 - 1 , the second point 720 - 1 , the fourth point 740 - 1 , and the fifth point 740 - 2 may be four vertices of a quadrilateral 910 , and the third point 730 - 1 may be within the quadrilateral 910 .

[0087] Fig.10 Schematic diagram of exemplary first angle and second angle according to some embodiments of the present application. Fig.10 shown.

[0088] Fig.11 is a schematic diagram of exemplary first angles and second angles shown according to some embodiments of the present application.

[0089] Fig.12 is a schematic diagram of exemplary first angles and second angles shown according to some embodiments of the present application.

[0090] Fig.13 is a schematic diagram of exemplary first angles and second angles in a downward and left direction of a human face according to some embodiments.

[0091] Fig.14 is a schematic diagram of exemplary first angles and second angles in a right-down direction according to some embodiments.

[0092] Fig.15 1500 is a flowchart of an exemplary process of identifying a head shaking action according to some embodiments of the present application. In some embodiments, the process 1500 may be implemented as a group. The processor 320 and / or Figure 5The modules in the embodiment may execute the set of instructions, and when executing the instructions, the processor 320 and / or the modules may be configured to perform the process 1500. The operations of the process shown below are for illustration purposes only. In some embodiments, the process 1500 may be completed using one or more additional operations not described and / or one or more operations not discussed herein. In addition, Fig.15 The order of the steps of the process shown and described below is not limiting.In some embodiments, step 650 of the process 600 may be performed based on the process 1500 for identifying a head shaking motion based on the asymmetry values ​​of the series of images.

[0093] In step 1510, the processing engine 210 (e.g., the recognition module 530) may determine a first image in the series of images in the image sequence whose asymmetry value is less than a first asymmetry value threshold. If the asymmetry value of a specific image in the series of images is less than the first asymmetry value threshold, the face on the specific image may be considered to be in the frontal view. In some embodiments, the first asymmetry value threshold may be selected based on a detection error of the feature points of the face and / or a tolerance of a physiological asymmetry of the face. For example, the first asymmetry value threshold may be greater than and close to 1, for example, 1.02, 1.05. In some embodiments, the first asymmetry value threshold may be in the range of 1.01 to 1.20. In some embodiments, the first asymmetry value threshold may be limited to sub-ranges of 1.01 to 1.02, 1.02 to 1.04, 1.04 to 1.06, 1.06 to 1.10, 1.10 to 1.15, and 1.15 to 1.20. The processing engine 210 can retrieve the asymmetry value of the series of images based on their sequence.Once the processing engine 210 first determines a specific image whose asymmetry value is less than the first asymmetry value threshold, the processing engine 210 can designate the specific image as the first image.

[0094] In step 1520, the processing engine 210 (e.g., the recognition module 530) may determine a second image in the series of images in the image sequence whose asymmetry value is less than a first asymmetry value threshold, the second image being a subsequent image of the first image in the image sequence. After determining the first image, the processing engine 210 may further retrieve the series of images based on the image sequence. When the processing engine 210 determines another image having an asymmetry value less than the first asymmetry value threshold, the processing engine 210 may designate the other image as the second image. Both the first image and the second image may include a human face in a frontal orientation. In a head shaking motion, the first image may correspond to an initial state in which the human face is in a frontal orientation. The second image may correspond to a second state in which the human face subsequently turns back to the frontal orientation after the initial state.

[0095] In step 1530, the processing engine 210 (e.g., the recognition module 530) may obtain an asymmetry value of a set of images, the set of images including the first image, the second image, and one or more images between the first image and the second image in the image sequence. In some embodiments, the set of images may correspond to a rotation of the human face. The rotation of the human face refers to an action of shaking the head of the person from the initial state to the second state. In some embodiments, the processing engine 210 may obtain an asymmetry value of the set of images from the memory 220, the set of images including storing the asymmetry values ​​of the series of images in the image sequence.

[0096] In step 1540, the processing engine 210 (e.g., the recognition module 530) may recognize the head shaking motion of the human face based on the asymmetry value of the group of images and the number of images in the group of images. In some embodiments, the processing engine 210 may determine the number of images in the group of images based on the serial number of the group of images. In some embodiments, the processing engine 210 may recognize the head shaking motion when both the asymmetry value of the group of images and the number of images in the group of images satisfy certain conditions. The conditions may be related to a threshold value of the asymmetry value and / or the number. The details on recognizing the head shaking motion may be found elsewhere disclosed in this application (e.g., Fig.16 and its description).

[0097] It should be understood that the above description is for illustrative purposes only and is not intended to limit the scope of the present application. For those of ordinary skill in the art, various changes and modifications can be made according to the description of the present application. However, these changes and modifications will not deviate from the scope disclosed in the present application. For example, steps can be added in the process 800, for example, between step 1530 and step 1540, to determine the number of images in the group of images.

[0098] Fig.16 FIG. 1 is a schematic diagram showing the relationship between the asymmetric values ​​of the three rotations of the head shaking action and time (T) according to some embodiments of the present application. Fig.16 As shown, the time point when the face is in the first frontal orientation can be designated as T=0. The asymmetry value at this time point can be close to 1 and less than the first asymmetry value threshold 1640. During the movement of the person shaking his or her head to the left or right from the first frontal orientation 1620, the asymmetry value increases over time until the third point leaves the quadrilateral. The asymmetry value corresponds to the third point being outside the quadrilateral (e.g. Fig.16 The direction of the face (shaded area 1610 in the figure) can be equal to the maximum asymmetry value where the third point is located on one side of the quadrilateral. During the movement of the person shaking his or her head from the second frontal direction 1630 to the left or right direction, the asymmetry value decreases over time and is less than the first asymmetry value threshold 1640. The asymmetry value corresponding to the rotation of the face (e.g., from a forward direction to a next forward direction) can be as follows: Fig.16 As shown, it increases and then decreases from the first frontal direction to the second frontal direction. The second asymmetry value threshold 1650 represents the amplitude of the movement. In the rotation of the movement, if the maximum asymmetry value is greater than the second asymmetry value threshold 1650, the movement can be considered as a head shaking motion. The second asymmetry value threshold can be specified as greater than 1. In some embodiments, the second asymmetry value threshold can be in the range of 1 to 1.3. Optionally, the second asymmetry value threshold can be limited in a sub-range of 1.0 to 1.1, 1.1 to 1.2, or 1.2 to 1.3. In some embodiments, the second asymmetry value threshold can be 1.2.

[0099] Fig.17 1700 is a flowchart of an exemplary process of identifying a head shaking motion according to some embodiments of the present application. In some embodiments, the process 1700 may implement a set of instructions (e.g., an application) stored in the storage ROM 330 or RAM 340. The processor 320 and / or Figure 5The modules in the process 1700 may execute the set of instructions, and when executing the instructions, the processor 320 and / or the modules may be configured to perform the process 1700. The operations of the process shown below are for illustration purposes only. In some embodiments, the process 1700 may be completed using one or more additional operations not described and / or one or more operations not discussed herein. In addition, Fig.17 The order of the steps of the process shown and described below is not limiting. In some embodiments, step 1540 of the process 1500 can be performed based on the process 1700 for identifying a head shaking motion of a face based on the asymmetry value of the set of images and the number of images in the set of images.

[0100] In step 1710, the processing engine 210 (eg, the identification module 530) may identify the maximum asymmetry value among the asymmetry values ​​of the set of images.

[0101] In step 1720, the processing engine 210 (e.g., the identification module 530) may determine whether the maximum asymmetry value is greater than or equal to the second asymmetry value threshold. If the processing engine 210 (e.g., the identification module 530) determines that the maximum asymmetry value is greater than or equal to the second asymmetry value threshold, the process 1700 may proceed to step 1730; if the processing engine 210 (e.g., the identification module 530) determines that the maximum asymmetry value is less than the second asymmetry value threshold, the process 1700 may proceed to step 1750.

[0102] At step 1730, the processing engine 210 (e.g., the identification module 530) may determine whether the number of the images in the set of images is greater than or equal to a threshold number. The threshold number may be a predetermined threshold based on the frame rate of the image acquisition device 240. The frame rate refers to the number of frames acquired by the image acquisition device 240 per second. The greater the frame rate, the greater the threshold number. For example, if the frame rate of the image acquisition device 240 is 30 frames per second (fps), the threshold number may be specified as 10. For another example, if the frame rate of the image acquisition device 240 is 35 frames per second (fps), the threshold number may be specified as 12. If the processing engine 210 (e.g., the identification module 530) determines that the number of the images in the set of images is greater than or equal to the threshold number, the process 1700 may proceed to step 1740. If the processing engine 210 (eg, the identification module 530 ) determines that the number of images in the set of images is less than the threshold number, the process 1700 may proceed to step 1750 .

[0103] In step 1740, the processing engine 210 (e.g., the identification module 530) may identify a head shaking motion from the set of images. The processing engine 210 may identify the head shaking motion based on two conditions, namely, determining that the maximum asymmetry value is greater than or equal to the second asymmetry value threshold and determining that the number of the images in the set of images is greater than or equal to the threshold number.

[0104] At step 1750, the processing engine 210 (e.g., the recognition module 530) may recognize a non-shaking motion of the face from the set of images. The processing engine 210 may recognize a non-shaking motion of the face based on determining that the maximum asymmetry value is less than the second asymmetry value threshold or determining that the number of the images in the set of images is less than the threshold number.

[0105] It should be understood that the above description is for illustrative purposes only and is not intended to limit the scope of the present application. For those of ordinary skill in the art, various changes and modifications can be made according to the description of the present application. However, these changes and modifications will not deviate from the scope disclosed by the present application. For example, the steps 1720 and 1730 can be operated in one step.

[0106] The basic concepts have been described above. Obviously, for those of ordinary skill in the art who have read this application, the above invention disclosure is only for example and does not constitute a limitation of this application. Although not explicitly stated here, those of ordinary skill in the art may make various modifications, improvements and amendments to this application. Such modifications, improvements and amendments are suggested in this application, so such modifications, improvements and amendments still belong to the spirit and scope of the exemplary embodiments of this application.

[0107] At the same time, the present application uses specific words to describe the embodiments of the present application. For example, "one embodiment", "an embodiment", and / or "some embodiments" refer to a certain feature, structure or characteristic related to at least one embodiment of the present application. Therefore, it should be emphasized and noted that "one embodiment" or "an embodiment" or "an alternative embodiment" mentioned twice or more in different positions in this specification does not necessarily refer to the same embodiment. In addition, some features, structures or characteristics in one or more embodiments of the present application can be appropriately combined.

[0108] In addition, it will be appreciated by those skilled in the art that various aspects of the present application may be illustrated and described by a number of patentable categories or situations, including any new and useful process, machine, product, or combination of substances, or any new and useful improvements thereto. Accordingly, various aspects of the present application may be performed entirely by hardware, entirely by software (including firmware, resident software, microcode, etc.), or by a combination of hardware and software. The above hardware or software may all be referred to as "units", "modules" or "systems". In addition, various aspects of the present application may be represented as computer products located in one or more computer-readable media, the product including computer-readable program code.

[0109] A computer readable signal medium may include a propagated data signal containing computer program code, for example, in baseband or as part of a carrier wave. Such propagated signals may be in a variety of forms, including electromagnetic, optical, etc., or any suitable combination. A computer readable signal medium may be any computer readable medium other than a computer readable storage medium that can be connected to an instruction execution system, device or apparatus to communicate, propagate or transmit a program for use. The program code on a computer readable signal medium may be propagated via any suitable medium, including radio, cable, fiber optic cable, RF, etc., or any combination of the above.

[0110] The computer program coding required for the operation of each part of the application can be written in any one or more programming languages, including subject-oriented programming languages ​​such as Java, Scala, Smalltalk, Eiffel, JADE, Emerald, C++, C#, VB.NET, Python, etc., conventional procedural programming languages ​​such as C language, Visual Basic, Fortran 2003, Perl, COBOL 2002, PHP, ABAP, dynamic programming languages ​​such as Python, Ruby, and Groovy, or other programming languages, etc. The program code can be run completely on the user's computer, or run on the user's computer as an independent software package, or run partially on the user's computer and partially on the remote computer, or run completely on the remote computer or server. In the latter case, the remote computer can be connected to the user's computer through any network form, for example, a local area network (LAN) or a wide area network (WAN), or connected to an external computer (for example, via the Internet), or in a cloud computing environment, or used as a service such as software as a service (SaaS).

[0111] In addition, unless explicitly stated in the claims, the order of the processing elements and sequences described in the present application, the use of numbers and letters, or the use of other names, are not intended to limit the order of the processes and methods of the present application. Although the above disclosure discusses some invention embodiments that are currently considered useful through various examples, it should be understood that such details are only for illustrative purposes, and the attached claims are not limited to the disclosed embodiments. On the contrary, the claims are intended to cover all modifications and equivalent combinations that are consistent with the essence and scope of the embodiments of the present application. For example, although the system components described above can be implemented by hardware devices, they can also be implemented only by software solutions, such as installing the described system on an existing server or mobile device.

[0112] Similarly, it should be noted that in order to simplify the description disclosed in this application and thus facilitate the understanding of one or more embodiments of the invention, in the foregoing description of the embodiments of the application, multiple features are sometimes grouped into one embodiment, figure, or description thereof. However, this method of the application should not be interpreted as reflecting the intention that the claimed object material to be scanned requires more features than those explicitly stated in each claim. In fact, the features of the embodiments are less than all the features of the single embodiment disclosed above.

Claims

1. A system for identifying head movement, comprising: Image acquisition equipment; A storage device including a set of instructions; as well as at least one processor in communication with the storage device, wherein when executing the set of instructions, the at least one processor is instructed to cause the system to: receiving a series of images from the image acquisition device, the images including visually presented images of a human face, the series of images forming an image sequence, each image in the series of images having a previous image or a next image in the image sequence; For each image in a sequence of images: Determine at least two feature points on the face of the person; Determining the positions of the at least two feature points on the face of the person; and determining an asymmetry value based on the positions of the at least two feature points; and recognizing a shaking motion of the face based on the asymmetry values ​​of the series of images; Wherein, the at least two characteristic points include a first point, a second point, a third point, a fourth point and a fifth point; and In the front view of the face: The first point and the second point are symmetrical with respect to a symmetry line passing through the third point, and the fourth point and the fifth point are symmetrical with respect to the symmetry line passing through the third point, and The third point is located within a quadrilateral determined based on the first point, the second point, the fourth point and the fifth point; Wherein, the asymmetry value is determined based on the positions of the at least two feature points of the series of images, and the processor is further instructed to cause the system to: determining whether the third point in each image in the series of images is not located within the quadrilateral determined based on the first point, the second point, the fourth point, and the fifth point; and Based on the result of the determination that the third point in each image in the series of images is not located within the quadrilateral determined based on the first point, the second point, the fourth point and the fifth point, the asymmetry value is determined according to the asymmetry value of the previous image in each image in the series of images.

2. The system according to claim 1, characterized in that Based on the asymmetry values ​​of the series of images, the processor is further instructed to cause the system to: Determine a first image in the series of images in the image sequence whose asymmetry value is less than a first asymmetry value threshold; Determine a second image in the series of images in the image sequence whose asymmetry value is less than a first asymmetry value threshold, the second image being a subsequent image of the first image in the image sequence; acquiring an asymmetry value of a set of images, the set of images comprising the first image, the second image, and one or more images between the first image and the second image in the image sequence; as well as The shaking motion of the face is identified based on the asymmetry value of the group of images and the number of images in the group of images.

3. The system according to claim 2, characterized in that Based on the asymmetry value of the set of images and the number of the images in the set of images, the processor is further instructed to cause the system to: identifying a maximum asymmetry value among the asymmetry values ​​of the set of images; determining whether the maximum asymmetry value is greater than a second asymmetry value threshold; determining whether a number of images in the set of images is greater than a threshold number; as well as Based on a result of determining that the maximum asymmetry value is greater than the second asymmetry value threshold and a result of determining that the number of images in the group of images is greater than the threshold number, the head shaking motion of the human face is identified from the group of images.

4. The system according to claim 3, characterized in that Determine a time point when the human face is in a first frontal direction, the asymmetry value corresponding to the time point being close to 1 and less than the first asymmetry value threshold; While the face is shaking from the first frontal direction, the corresponding asymmetry value changes over time.

5. The system according to claim 1, characterized in that: The first point is located in the central area of ​​the left eye of the face; The second point is located in the central area of ​​the right eye of the face; The third point is located at the tip area of ​​the nose of the face; The fourth point is located in the left end area of ​​the lips of the face; and The fifth point is located in the right end area of ​​the lips of the face.

6. A method for identifying head movement, comprising: receiving a series of images from an image acquisition device, the images including visually presented images of a human face, the series of images forming an image sequence, each image in the series of images having a previous image or a next image in the image sequence; For each image in a sequence of images: Determine at least two feature points on the face of the person; Determining the positions of the at least two feature points on the face of the person; and determining an asymmetry value based on the positions of the at least two feature points; Wherein, the at least two characteristic points include a first point, a second point, a third point, a fourth point and a fifth point; and In the front view of the face: The first point and the second point are symmetrical with respect to a symmetry line passing through the third point, The fourth point and the fifth point are symmetric with respect to the symmetry line passing through the third point, and The third point is located within a quadrilateral determined based on the first point, the second point, the fourth point and the fifth point; Wherein, determining the asymmetry value based on the positions of the at least two feature points of the series of images comprises: determining whether the third point in each image in the series of images is not located within the quadrilateral determined based on the first point, the second point, the fourth point, and the fifth point; and Based on the result of the determination that the third point in each image in the series of images is not located within the quadrilateral determined based on the first point, the second point, the fourth point and the fifth point, the asymmetry value is determined according to the asymmetry value of the previous image in each image in the series of images.

7. The method according to claim 6, characterized in that The step of identifying the shaking motion of the face based on the asymmetric value of the series of images comprises: Determine a first image in the series of images in the image sequence whose asymmetry value is less than a first asymmetry value threshold; determine a second image in the series of images in the image sequence whose asymmetry value is less than the first asymmetry value threshold, wherein the second image is a subsequent image of the first image in the image sequence; acquiring an asymmetry value of a set of images, the set of images comprising the first image, the second image, and one or more images between the first image and the second image in the image sequence; and The shaking motion of the face is identified based on the asymmetry value of the group of images and the number of images in the group of images.

8. The method according to claim 7, characterized in that The step of recognizing the shaking motion of the face based on the asymmetry value of the group of images and the number of the images in the group of images includes: identifying a maximum asymmetry value among the asymmetry values ​​of the set of images; determining whether the maximum asymmetry value is greater than a second asymmetry value threshold; determining whether a number of images in the set of images is greater than a threshold number; and Based on a result of determining that the maximum asymmetry value is greater than the second asymmetry value threshold and a result of determining that the number of images in the group of images is greater than the threshold number, the head shaking motion of the human face is identified from the group of images.

9. The method according to claim 8, characterized in that Determine a time point when the human face is in a first frontal direction, the asymmetry value corresponding to the time point being close to 1 and less than the first asymmetry value threshold; While the face is shaking from the first frontal direction, the corresponding asymmetry value changes over time.

10. The method according to claim 6, characterized in that: The first point is located in the central area of ​​the left eye of the face; The second point is located in the central area of ​​the right eye of the face; The third point is located at the tip area of ​​the nose of the face; The fourth point is located in the left end area of ​​the lips of the face; and The fifth point is located in the right end area of ​​the lips of the face.

11. A device for identifying head movement, comprising: Memory for storing instructions; as well as at least one processor; An acquisition module, configured to receive a series of images, wherein the images include visually presented images of human faces, wherein the series of images form an image sequence, and each image in the series of images has a previous image or a next image in the image sequence; A determination module, used to determine at least two feature points on the face of the person; Determining positions of the at least two feature points on the face; and determining an asymmetry value based on the positions of the at least two feature points; A recognition module, used for recognizing the shaking motion of the face based on the asymmetric value of the series of images; Wherein, the at least two characteristic points include a first point, a second point, a third point, a fourth point and a fifth point; and In the front view of the face: The first point and the second point are symmetrical with respect to a symmetry line passing through the third point, The fourth point and the fifth point are symmetric with respect to the symmetry line passing through the third point, and The third point is located within a quadrilateral determined based on the first point, the second point, the fourth point and the fifth point; Wherein, the determination module is further used for: determining whether the third point in each image in the series of images is not located within the quadrilateral determined based on the first point, the second point, the fourth point, and the fifth point; and Based on the result of the determination that the third point in each image in the series of images is not located within the quadrilateral determined based on the first point, the second point, the fourth point and the fifth point, the asymmetry value is determined according to the asymmetry value of the previous image in each image in the series of images.

12. A non-transitory computer-readable medium comprising at least one set of instructions, characterized in that: When executed by at least one processor of a computer device, the at least one set of instructions instructs the at least one processor to: receiving a series of images from an image acquisition device, the images including visually presented images of a human face, the series of images forming an image sequence, each image in the series of images having a previous image or a next image in the image sequence; For each image in a sequence of images: Determine at least two feature points on the face of the person; Determining the positions of the at least two feature points on the face of the person; as well as determining an asymmetry value based on the positions of the at least two feature points; as well as recognizing a shaking motion of the face based on the asymmetry values ​​of the series of images; Wherein, the at least two characteristic points include a first point, a second point, a third point, a fourth point and a fifth point; and In the front view of the face: The first point and the second point are symmetrical with respect to a symmetry line passing through the third point, The fourth point and the fifth point are symmetric with respect to the symmetry line passing through the third point, and The third point is located within a quadrilateral determined based on the first point, the second point, the fourth point and the fifth point; Wherein, determining the asymmetry value based on the positions of the at least two feature points of the series of images comprises: determining whether the third point in each image in the series of images is not located within the quadrilateral determined based on the first point, the second point, the fourth point, and the fifth point; and Based on the result of the determination that the third point in each image in the series of images is not located within the quadrilateral determined based on the first point, the second point, the fourth point and the fifth point, the asymmetry value is determined according to the asymmetry value of the previous image in each image in the series of images.

Images