Object recognition method and device, electronic device and storage medium

Abstract

The object recognition method and device, electronic device and storage medium provided by the present application relate to the field of face recognition technology. In the present application, firstly, multiple frames of target images obtained by photographing the target object are obtained, wherein each frame of the target image includes facial information of the target object. Secondly, similarity information between the multiple frames of target images is determined based on the facial information. Then, based on the similarity information, it is determined whether the target object is a living object. Based on the above method, the problem of low accuracy of recognition results existing in the existing face recognition technology can be improved.

Description

Technical Field

[0001] The present application relates to the field of face recognition technology, and in particular to an object recognition method and device, an electronic device, and a storage medium. Background Art

[0002] As the application scope of face recognition technology becomes wider and wider, the accuracy of face recognition results is required to be higher. In order to improve the accuracy of face recognition results, the object to be recognized is generally required to make a specified action, such as blinking, or a depth image sensor is required to be configured on the recognition device to collect the depth information of the face of the object to be recognized.

[0003] However, the inventors have discovered that in many applications, it may not be possible to require the identified object to perform a specified action (for example, to provide a better user experience), or the recognition device may not be equipped with a depth image sensor (for example, considering the cost of the device). As a result, the recognition results obtained based on face recognition technology still have the problem of low accuracy. Summary of the invention

[0004] In view of this, the purpose of the present application is to provide an object recognition method and device, an electronic device and a storage medium to improve the problem of low recognition result accuracy in existing face recognition technology.

[0005] To achieve the above purpose, the embodiment of the present application adopts the following technical solution:

[0006] An object recognition method, comprising:

[0007] Obtaining multiple frames of target images obtained by photographing the target object, wherein each frame of the target image includes facial information of the target object;

[0008] Determining similarity information between multiple frames of the target image based on the facial information;

[0009] It is determined whether the target object is a living object based on the similarity information.

[0010] In a preferred embodiment of the present application, in the above-mentioned object recognition method, the step of determining similarity information between multiple frames of the target image based on the facial information includes:

[0011] Based on the pre-trained face recognition model, feature extraction processing is performed on the facial information in each frame of the target image to obtain target feature information of each frame of the target image;

[0012] Based on the target feature information, similarity information between multiple frames of the target images is obtained.

[0013] In a preferred embodiment of the present application, in the above object recognition method, the step of determining whether the target object is a living object based on the similarity information includes:

[0014] Comparing at least one of the target feature information with a plurality of pre-formed contrast feature information through the face recognition model to obtain confidence information that the target object belongs to a living object, wherein the plurality of contrast feature information is obtained based on a plurality of frames of images including facial information of a plurality of different objects, and the objects include living objects and non-living objects;

[0015] It is determined whether the target object is a living object based on the confidence information and the similarity information.

[0016] In a preferred embodiment of the present application, in the above-mentioned object recognition method, the step of comparing at least one of the target feature information with a plurality of pre-formed comparison feature information through the face recognition model to obtain confidence information that the target object belongs to a living object includes:

[0017] Performing identity category recognition processing on at least one of the target feature information by using the face recognition model to obtain identity category information of the at least one target feature information;

[0018] Determine, among the multiple feature spaces included in the face recognition model, a target feature space of the at least one target feature information based on the identity category information, wherein different feature spaces have contrast feature information of different objects, and each of the contrast feature information has first label information identifying the identity category of the corresponding object and second label information whether the object is alive;

[0019] The at least one target feature information is compared with the contrast feature information in the target feature space corresponding to the target feature information through the face recognition model to obtain confidence information that the target object belongs to a living object.

[0020] In a preferred embodiment of the present application, in the above-mentioned object recognition method, the step of comparing at least one of the target feature information with a plurality of pre-formed comparison feature information through the face recognition model to obtain confidence information that the target object belongs to a living object includes:

[0021] For each frame of target feature information corresponding to the target image, the target feature information is compared with a plurality of pre-formed comparison feature information by the face recognition model to determine the confidence that the target object in the frame of the target image belongs to a living object, thereby obtaining a plurality of confidences;

[0022] Confidence information that the target object belongs to a living object is obtained based on the multiple confidences.

[0023] In a preferred embodiment of the present application, in the above-mentioned object recognition method, the method further includes a step of training to obtain the face recognition model, the step including:

[0024] Performing feature extraction processing on multiple sample images through a feature extraction layer in a preset neural network model to obtain multiple sample feature information;

[0025] Determine, through the loss determination layer in the neural network model, a first loss value of each sample feature information based on first label information pre-configured for each sample image, and a second loss value of each sample feature information based on second label information pre-configured for each sample image, wherein the first label information is used to identify the identity category of the object in the corresponding sample image, and the second label information is used to identify whether the object in the corresponding sample image is a living body;

[0026] Based on the first loss value and the second loss value, the neural network model is trained to obtain the face recognition model.

[0027] In a preferred embodiment of the present application, in the above object recognition method, the step of determining whether the target object is a living object based on the similarity information includes:

[0028] Comparing at least one target image of the multiple target images with a pre-acquired multiple image frames to obtain confidence information that the target object belongs to a living object, wherein the multiple image frames include facial information of multiple different objects, and the objects include living objects and non-living objects;

[0029] It is determined whether the target object is a living object based on the confidence information and the similarity information.

[0030] The present application also provides an object recognition device, including:

[0031] A target image acquisition module, used to acquire multiple frames of target images obtained by photographing a target object, wherein each frame of the target image includes facial information of the target object;

[0032] A similarity information determination module, used to determine similarity information between the multiple frames of target images based on the facial information;

[0033] A living object determination module is used to determine whether the target object is a living object based on the similarity information.

[0034] Based on the above, an embodiment of the present application further provides an electronic device, including:

[0035] Memory for storing computer programs;

[0036] The processor connected to the memory is used to execute the computer program stored in the memory to implement the above-mentioned object recognition method.

[0037] Based on the above, an embodiment of the present application further provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed, the above-mentioned object recognition method is implemented.

[0038] The object recognition method and device, electronic device and storage medium provided in the present application obtain multiple frames of target images obtained by photographing the target object, and then determine whether the target object is a living object based on the similarity information between the facial information in the multiple frames of target images. In this way, without relying on the target object to perform a specified action and without relying on the device that photographs the target object to have a depth image sensor, it is also possible to achieve a high degree of accuracy in identifying whether the target object is a living object (the inventors of the present application have found that even if a living object does not deliberately perform a specified action on its face, it will show slight changes with random time changes, that is, there will be micro-expression changes, so that there will be at least slight differences between the multiple frames of target images captured, while for non-living objects, such as photos or three-dimensional models, there will be no changes), thereby improving the problem of low accuracy of recognition results in existing face recognition technology in applications where the identified object cannot be required to perform a specified action (such as for a better user experience, etc.) or the shooting device is not equipped with a depth image sensor (such as considering the cost of the equipment, etc.), thereby effectively guaranteeing the accuracy of face recognition, improving the experience of the identified object (without performing a specified action), and reducing the cost of the equipment, making it more applicable and having a higher practical value.

[0039] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 A structural block diagram of an electronic device provided in an embodiment of the present application.

[0041] Figure 2 A flowchart of an object recognition method provided in an embodiment of the present application.

[0042] Figure 3 A schematic diagram of the effect of obtaining multiple frames of target images provided in an embodiment of the present application.

[0043] Figure 4for Figure 2 Schematic diagram of the flow chart of the sub-steps included in step S120.

[0044] Figure 5 for Figure 2 Schematic diagram of the flow chart of the sub-steps included in step S130.

[0045] Figure 6 for Figure 2 Schematic diagram of the flow chart of other sub-steps included in step S130.

[0046] Figure 7 for Figure 6 Schematic diagram of the flow chart of the sub-steps included in step S133.

[0047] Figure 8 for Figure 6 Schematic diagram of the flow chart of other sub-steps included in step S133.

[0048] Fig. 9 A flowchart of other steps of the object recognition method provided in an embodiment of the present application.

[0049] Fig.10 A block diagram of the functional modules of the object recognition device provided in an embodiment of the present application.

[0050] Icons: 10 - electronic device; 12 - memory; 14 - processor; 100 - object recognition device; 110 - target image acquisition module; 120 - similarity information determination module; 130 - living object determination module. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0052] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0053] like Figure 1 As shown, an embodiment of the present application provides an electronic device 10. The electronic device 10 may include a memory 12, a processor 14 and an object recognition device 100.

[0054] In detail, the memory 12 and the processor 14 are electrically connected directly or indirectly to realize data transmission or interaction. For example, they can be electrically connected to each other through one or more communication buses or signal lines. The memory 12 can store at least one software function module that can exist in the form of software or firmware, such as the object recognition device 100. The processor 14 can be used to execute the executable computer program stored in the memory 12, such as the object recognition device 100, so as to implement the object recognition method provided in the embodiment of the present application (as described later) to determine whether the target object is a living object.

[0055] Optionally, the memory 12 can be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable read-only memory (EEPROM), etc.

[0056] Furthermore, the processor 14 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), a system on chip (SoC), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0057] It can be understood that the electronic device 10 can be a terminal device (such as a mobile phone, a computer, etc.) or a server with data processing capabilities.

[0058] and, Figure 1 The structure shown is for illustration only. The electronic device 10 may also include Figure 1 More or fewer components as shown, or with Figure 1 The different configurations shown, for example, may also include a communication unit for exchanging information with other devices. For example, when the electronic device 10 is a server, the communication unit can be used to communicate with a photographing device of the target object to obtain a target image or feedback a recognition result, etc.

[0059] Combination Figure 2 The present application also provides an object recognition method, which can be applied to the electronic device 10. The method steps defined in the process related to the object recognition method can be implemented by the electronic device 10. Figure 2 The specific process shown is explained in detail.

[0060] Step S110, obtaining multiple frames of target images obtained by photographing the target object.

[0061] In this embodiment, when it is necessary to identify the target object to determine whether the target object is a living object, the electronic device 10 may first obtain multiple frames of target images.

[0062] The multiple frames of target images may be obtained based on photographing the target object, and each frame of the target image may include facial information of the target object.

[0063] Step S120: determining similarity information between multiple frames of target images based on the facial information.

[0064] In this embodiment, after obtaining the multiple frames of target images based on step S110, the electronic device 10 may determine similarity information between the multiple frames of target images based on facial information in the multiple frames of target images.

[0065] Step S130: determining whether the target object is a living object based on the similarity information.

[0066] In this embodiment, after obtaining the similarity information between the multiple frames of the target image based on step S120, the electronic device 10 can determine whether the target object is a living object based on the similarity information.

[0067] Based on the above method, without relying on the target object to perform a specified action and without relying on the device photographing the target object to have a depth image sensor, it is also possible to achieve a high degree of accuracy in identifying whether the target object is a living object, thereby improving the problem of low accuracy of recognition results in existing face recognition technologies in applications where the identified object cannot be required to perform a specified action (such as for a better user experience) or the shooting device is not equipped with a depth image sensor (such as considering the cost of the equipment, etc.), thereby effectively ensuring the accuracy of face recognition, improving the experience of the identified object (without having to perform a specified action), and reducing the cost of the equipment.

[0068] Among them, even if a living object (such as a real person) does not deliberately make a specified movement on the face, there will be slight changes on the face with random changes in time, that is, there will be micro-expression changes, so that there will be at least slight differences between the multiple frames of target images captured. However, for non-living objects, such as photos or three-dimensional models, no changes are possible.

[0069] It is based on this discovery that the inventors of the present application, after long-term research, proposed a technical solution for determining whether a target object is a living object based on the similarity information between images, namely, the object recognition method improved by the embodiment of the present application.

[0070] First, it should be noted that the specific method of obtaining multiple frames of target images is not limited and can be selected according to actual application requirements.

[0071] For example, in an alternative example, if the electronic device 10 is a terminal device, such as a mobile phone or a computer, the electronic device 10 can photograph the target object based on a carried image acquisition device (such as a camera) to obtain multiple frames of target images with facial information of the target object.

[0072] That is to say, when facial recognition of the target object is required, the electronic device 10 can control the image acquisition device to turn on to photograph the target object. Then, after the image acquisition device captures multiple frames of target images, the image acquisition device can send the multiple frames of target images to the electronic device 10, so that the electronic device 10 can obtain multiple frames of target images.

[0073] For another alternative example, if the electronic device 10 is a server, the electronic device 10 can photograph the target object based on an image acquisition device (such as a camera) on a connected terminal device to obtain multiple frames of target images with facial information of the target object.

[0074] That is, when the target object needs to be recognized by the face, the electronic device 10 is connected to the terminal device, and the image acquisition device can be controlled to start to shoot the target object. Then, the terminal device can obtain multiple frames of target images captured by the image acquisition device, and send the multiple frames of target images to the electronic device 10, so that the electronic device 10 can obtain multiple frames of target images.

[0075] It should be further explained that, for step S110, the obtained multiple frames of target images may be all target images obtained by photographing the target object, or may be partial target images among all target images obtained by photographing the target object.

[0076] For example, in an alternative example, if the recognition accuracy requirement is particularly high, or the number of all target images captured is not particularly large, the obtained multiple frames of target images can be all target images obtained by capturing the target object.

[0077] For another alternative example, in order to reduce the amount of data processing by the electronic device 10, the obtained multiple frames of target images may be partial target images among all target images obtained by photographing the target object.

[0078] Among them, based on different application requirements, the method of obtaining a partial target image from all the captured target images is not limited and can be selected according to actual application requirements.

[0079] For example, in an alternative example, the first N target image frames can be obtained from all the target images captured (research has found that living objects are more likely to show larger changes in expression when they are first captured. In this way, while reducing the data processing volume of the electronic device 10, larger changes in expression are easier to recognize, which can fully avoid the problem of misidentification); the last N target image frames can also be obtained from all the target images captured; and the middle N target image frames can also be obtained from all the target images captured.

[0080] For another alternative example, the inventors of the present application have discovered through research that, since the time interval between shooting of target images in adjacent frames is extremely short, the difference in expression (facial information) is extremely small and difficult to be effectively recognized. Thus, based on a comprehensive consideration of data processing volume and recognition accuracy, one frame of target image can be obtained from all the target images shot at intervals of a preset number of frames, thereby obtaining multiple frames of target images.

[0081] In combination with the aforementioned finding (generally, living objects are more likely to show greater changes in expression when they are first photographed), the inventors of the present application also provide a solution, that is, the value of the above-mentioned preset number of frames can be gradually increased.

[0082] That is to say, among the obtained multiple frames of target images, the time difference between two adjacent frames of target images whose time information is closer to the front can be smaller.

[0083] In detail, in a specific application example, combined with Figure 3, if the multiple frames of target images obtained by shooting can be, in chronological order, the first frame of target image, the second frame of target image, the third frame of target image, the first four frame of target image, the fifth frame of target image, the sixth frame of target image, the seventh frame of target image, the eighth frame of target image, the ninth frame of target image and the tenth frame of target image. In this way, the multiple frames of target images obtained can be, in chronological order, the first frame of target image, the third frame of target image (one frame interval), the sixth frame of target image (two frames interval) and the tenth frame of target image (three frames interval).

[0084] Furthermore, it is necessary to further explain step S110 that, in order to further improve the accuracy of identifying whether the target object is a living object, the total frame length of the obtained multiple frames of target images may be less than a preset time length, for example, 1 s or 0.5 s.

[0085] In this way, by limiting the total frame length, the problem of misidentification caused by forging a living object by replacing a photo or using a video can be fully avoided.

[0086] Secondly, it should be noted that for step S120, the specific method of determining the similarity information between the multiple frames of the target image is not limited and can be selected according to actual application requirements.

[0087] For example, in an alternative example, facial information in multiple frames of the target image can be compared based on some image processing algorithms, such as extracting facial contours in the target image based on a contour extraction algorithm, and then comparing the extracted facial contours to determine similarity information between the multiple frames of the target image.

[0088] For example, in another alternative example, in order to improve the accuracy of the determined similarity information, the determination can be performed based on a neural network. Figure 4 Step S120 may include step S121 and step S122, the specific contents of which are described as follows.

[0089] Step S121, performing feature extraction processing on the facial information in each frame of the target image based on the pre-trained face recognition model to obtain target feature information of each frame of the target image.

[0090] In this embodiment, after obtaining multiple frames of target images based on step S110, the multiple frames of target images can be input into a pre-trained face recognition model, and feature extraction processing is performed on the facial information in the multiple frames of target images based on the face recognition model. In this way, target feature information of the multiple frames of target images can be obtained.

[0091] Step S122, obtaining similarity information between multiple frames of target images based on the target feature information.

[0092] In this embodiment, after the target feature information of the multiple frames of target images is obtained based on step S121, the similarity information between the multiple frames of target images can be obtained based on the target feature information.

[0093] In this way, since the face recognition model, that is, the neural network model, has a higher information processing capability, the target feature information extracted is richer, so that the similarity information determined based on the target feature information has a higher accuracy, thereby improving the recognition accuracy of living objects.

[0094] Furthermore, it is necessary to further explain step S120 that the multiple frames of target images for which similarity information is determined may be all the target images in the multiple frames of target images obtained in step S110, or may be part of the target images in the multiple frames of target images obtained in step S110.

[0095] Thirdly, it should be noted that for step S130, the specific method of determining whether the target object is a living object is not limited and can be selected according to actual application requirements.

[0096] For example, in an alternative example, it may be determined whether the target object is a living object based only on the similarity information.

[0097] In detail, in a specific example, if the similarity information is less than a preset similarity, it can be determined that the target object is a non-living object, that is, it may be a photo or a three-dimensional model, etc. If the similarity information is greater than the preset similarity, it can be determined that the target object is a living object, that is, it may be a real person, etc.

[0098] Alternatively, in another specific application example, if the similarity information is 100%, that is, they are exactly the same, it can be determined that the target object is a non-living object, that is, it may be a photo or a three-dimensional model, etc. If the similarity information is not 100%, it can be determined that the target object is a living object.

[0099] For another example, in order to further improve the accuracy of identifying whether it is a living object, other information can be combined to make a comprehensive judgment based on the similarity information. Among them, the inventor of the present application found through research that when a target object is photographed to obtain a first image, and then a photo or a three-dimensional model formed based on the first image is photographed to form a second image, there are actually some differences between the first image and the second image.

[0100] Based on this, the inventor of the present application provides a technical solution: based on the similarity information, combined with the confidence information of belonging to a living object obtained by comparing the target image with some preset images, it is determined whether the target object is a living object.

[0101] Among them, based on different requirements, the implementation of the above technical solution may also be different, that is, there may be different options for the specific method of determining the confidence information.

[0102] For example, in an alternative example, in order to make the determination of the confidence information not rely on a complex neural network and reduce the demand for the data processing performance of the electronic device 10, the Figure 5 Step S130 may include step S131 and step S132, the specific contents of which are described as follows.

[0103] Step S131 : Compare at least one frame of the target image in the multiple frames of target images with the multiple frames of images obtained in advance to obtain confidence information that the target object is a living object.

[0104] In this embodiment, after multiple frames of target images are obtained based on step S110, at least one frame of the target image in the multiple frames of target images may be compared with multiple frames of images obtained in advance.

[0105] The multiple frames of images may include facial information of multiple different objects, and the objects include living objects and non-living objects. Thus, by comparing the target image with the multiple frames of images, confidence information that the target object is a living object can be obtained.

[0106] For example, the target image is an image obtained by photographing the front of the target object during recognition, the first image is an image obtained by photographing the front of a living object (such as a person with the same characteristics as the target object) in advance, and the second image is an image obtained by photographing a photo formed based on the first image. In this way, if the similarity between the target image and the first image is higher than the similarity between the target image and the second image, a higher confidence level can be obtained, that is, the target object is more likely to be a living object; if the similarity between the target image and the first image is lower than the similarity between the target image and the second image, a lower confidence level can be obtained, that is, the target object is more likely to be a non-living object.

[0107] Step S132: determining whether the target object is a living object based on the confidence information and the similarity information.

[0108] In this embodiment, after the confidence information is obtained based on step S131, it can be determined whether the target object is a living object based on the confidence information and in combination with the similarity information obtained in step S120.

[0109] For example, in another alternative example, in order to improve the accuracy of the determined confidence information, thereby ensuring that the accuracy of identifying whether it is a living object is also high, Figure 6 , step S130 may also include step S133 and step S134, the specific contents of which are described as follows.

[0110] Step S133: Compare at least one of the target feature information with a plurality of pre-formed comparison feature information through the face recognition model to obtain confidence information that the target object is a living object.

[0111] In this embodiment, after feature extraction processing is performed on multiple frames of the target image based on the pre-trained face recognition model, such as after obtaining the target feature information based on step S121, the target feature information (at least one) can be further processed by the face recognition model, that is, the target feature information is compared with multiple pre-formed comparison feature information.

[0112] The plurality of contrast feature information can be obtained based on a plurality of frames of images including facial information of a plurality of different objects, and the objects include living objects and non-living objects. Thus, confidence information that the target object is a living object can be obtained.

[0113] Step S134: determining whether the target object is a living object based on the confidence information and the similarity information.

[0114] In this embodiment, after the confidence information is obtained based on step S133, it can be determined whether the target object is a living object based on the confidence information and in combination with the similarity information obtained in step S120.

[0115] Optionally, the specific method of obtaining the confidence information in step S133 is not limited and can be selected according to actual application requirements.

[0116] For example, in an alternative example, the obtained at least one target feature information may be directly compared with all pre-formed feature information to determine the confidence information that the target object belongs to a living object.

[0117] Among them, the entire feature information can be formed based on images obtained by photographing multiple different objects, the different objects include objects of different identity categories, and objects of the same identity category include living objects and non-living objects. For example, a first image is obtained by photographing a living object A, and a second image is obtained by photographing a photo B formed by the first image. At this time, the photo B is actually a non-living object and has the same identity category as the living object A, such as belonging to the same person.

[0118] For example, in another alternative example, in order to improve the accuracy of comparative analysis when comparing information, Figure 7 Step S133 may include step S133a, step S133b and step S133c, the specific contents of which are described as follows.

[0119] Step S133a, performing identity category recognition processing on at least one of the target feature information through the face recognition model to obtain identity category information of the at least one target feature information.

[0120] In this embodiment, after feature extraction processing is performed on multiple frames of the target image based on the face recognition model, such as after obtaining the target feature information based on step S121, identity category recognition processing can be performed on at least one target feature information obtained through the face recognition model to determine the identity category information corresponding to the target feature information.

[0121] That is to say, based on the face recognition model, it can be determined first to which person the target feature information belongs, that is, it can be determined to which person the target object belongs.

[0122] Step S133b, determining a target feature space of the at least one target feature information based on the identity category information in a plurality of feature spaces included in the face recognition model.

[0123] In this embodiment, after the identity category information of the target feature information is determined based on step S133a, a target feature space of the target feature information can be determined in a plurality of feature spaces included in the face recognition model based on the identity category information.

[0124] Different feature spaces have contrast feature information of different objects, and each contrast feature information has first label information for identifying the identity category of the corresponding object and second label information for identifying whether the object is alive.

[0125] That is, the comparison feature information in the same feature space may have the same first label information, but may have different second label information.

[0126] Step S133c: Compare the at least one target feature information with the contrast feature information in the target feature space corresponding to the target feature information through the face recognition model to obtain confidence information that the target object belongs to a living object.

[0127] In this embodiment, after determining the target feature space based on step S133b, the target feature information can be compared with the contrast feature information in the target feature space through the face recognition model, and then the confidence information that the target object belongs to a living object can be determined based on the comparison result and the second label information of the contrast feature information.

[0128] That is to say, in the above example, by first determining the identity category of the target feature information (ie, the target object), and then determining whether it is a living object, a more refined comparison process can be performed when judging a living object, so that the comparison result can be more accurate.

[0129] Furthermore, in step S133, the specific method of obtaining the confidence information may be different based on the different specific quantities of the compared target feature information, and may be selected according to actual application requirements.

[0130] For example, in an alternative example, in order to reduce the data processing amount of the electronic device 10, so as to improve the recognition efficiency and reduce the performance requirements of the electronic device 10, after obtaining the target feature information of multiple frames of target images based on step S121, the target feature information of one frame of the target image can be selected, and then the corresponding confidence information can be obtained based on the target feature information.

[0131] For example, in another alternative example, in order to improve the accuracy of recognition, Figure 8 Step S133 may include step S133d and step S133e, the specific contents of which are described as follows.

[0132] Step S133d, for each frame of target feature information corresponding to the target image, the target feature information is compared with a plurality of pre-formed contrast feature information through the face recognition model to determine the confidence that the target object in the frame of target image belongs to a living object, and obtain a plurality of confidences.

[0133] In this embodiment, for each frame of the target image obtained, the target feature information corresponding to the frame of the target image can be compared with multiple pre-formed comparison feature information through the face recognition model, so as to determine the confidence that the target object in the frame of the target image belongs to a living object (for the specific method, refer to the previous explanation of step S133a, step S133b and step S133c and the parallel scheme). In this way, multiple confidence levels can be obtained.

[0134] Step S133e: obtaining confidence information that the target object is a living object based on the multiple confidences.

[0135] In this embodiment, after a plurality of confidences are obtained based on step S133d, confidence information that the target object is a living object can be obtained based on the plurality of confidences.

[0136] For example, in an alternative example, a minimum confidence level may be determined from multiple confidence levels as confidence information that the target object is a living object.

[0137] For another alternative example, a maximum confidence level may be determined from a plurality of confidence levels as confidence information that the target object is a living object.

[0138] For another example, in another alternative example, an average value may be obtained based on multiple confidence calculations as the confidence information that the target object is a living object.

[0139] Optionally, in step S132 and step S134, the specific method of determining whether the target object is a living object based on the confidence information and the similarity information is not limited and can also be selected according to actual application requirements.

[0140] For example, in an alternative example, a larger information can be selected from the confidence information and the similarity information (wherein, if the confidence information and the similarity information are information in different value ranges, they can be normalized before selection) as a basis for judgment to determine whether the target object is a living object, such as comparing the larger information with the preset information. If the larger information is greater than the preset information, it is determined that the target object is a living object.

[0141] For example, in another alternative example, a smaller information can be selected from the confidence information and the similarity information as a basis for judgment to determine whether the target object is a living object, such as comparing the smaller information with the preset information. If the smaller information is greater than the preset information, it is determined that the target object is a living object.

[0142] For another example, in order to further improve the accuracy of determining whether the target object is a living object, different weight coefficients can be configured for the confidence information and the similarity information respectively, and then the weighted sum of the confidence information and the similarity information is calculated, and then whether the target object is a living object is determined based on the weighted sum.

[0143] In detail, in a specific application example, the weight coefficient corresponding to the similarity information can be greater than the weight coefficient corresponding to the confidence information, so that the basis for determining whether it belongs to a living object is more focused on the similarity between the target images, that is, the subtle changes in facial information.

[0144] Furthermore, in the above example, feature extraction processing can be performed based on the face recognition model to obtain target feature information. Therefore, in order to extract richer target feature information based on the face recognition model so that the target feature information includes more detailed information, in this embodiment, combined with Fig. 9 The object recognition method may further include a model training step, which may specifically include step S140, step S150 and step S160, the specific contents of which are described as follows.

[0145] Step S140, performing feature extraction processing on multiple sample images through a feature extraction layer in a preset neural network model to obtain multiple sample feature information.

[0146] In this embodiment, after obtaining a plurality of sample images, feature extraction processing may be performed on the plurality of sample images based on a feature extraction layer in a preset neural network model, so that a plurality of sample feature information may be obtained.

[0147] Step S150, through the loss determination layer in the neural network model, determine the first loss value of each sample feature information based on the first label information pre-configured for each sample image, and determine the second loss value of each sample feature information based on the second label information pre-configured for each sample image.

[0148] In this embodiment, after obtaining multiple sample feature information based on step S140, for each sample feature information, a first loss value of the sample feature information can be determined based on the first label information pre-configured for each sample image, and a second loss value of the sample feature information can be determined based on the second label information pre-configured for each sample image through the loss determination layer in the neural network model. In this way, multiple first loss values ​​and multiple second loss values ​​can be obtained.

[0149] The first tag information may be used to identify the identity category of the object in the corresponding sample image, such as which person it is. The second tag information may be used to identify whether the object in the corresponding sample image is a living object, such as whether it is a real person.

[0150] Step S160: training the neural network model based on the first loss value and the second loss value to obtain the face recognition model.

[0151] In this embodiment, after obtaining the first loss value and the second loss value based on step S150, the neural network model can be trained based on the first loss value and the second loss value (such as updating the parameters of the neural network model) to obtain the face recognition model.

[0152] Fourthly, it should be noted that for step S140, the specific architecture of the neural network model is not limited and can be selected according to actual application requirements.

[0153] For example, in an alternative example, the neural network model may be a residual model, such as a deep residual network model (DRN).

[0154] Furthermore, the specific structure of the feature extraction layer is not limited. For example, in an alternative example, the feature extraction layer may be an encoder.

[0155] Fifthly, it should be noted that for step S150, the specific composition of the loss determination layer is not limited and can also be selected according to actual application requirements.

[0156] For example, in an alternative example, the loss determination layer may include an image classification network (such as fully connected layers, FC), which is used to perform feature classification processing on each sample feature information to obtain multiple feature vectors. In this way, the first loss value can be obtained by calculating the feature vector and the first label vector formed based on the first label information; and the second loss value can be obtained by calculating the feature vector and the second label vector formed based on the second label information.

[0157] In the sixth aspect, it should be noted that for step S160, the specific method of training based on the first loss value and the second loss value is not limited and can also be selected according to actual application requirements.

[0158] For example, in an alternative example, the sum of the first loss value and the second loss value can be calculated first, and the sum can be used as the total loss value. Then, the neural network model can be trained based on the total loss value.

[0159] For another example, in an alternative example, a weighted sum of the first loss value and the second loss value may be calculated first, and the weighted sum may be used as the total loss value. Then, the neural network model may be trained based on the total loss value.

[0160] There is no restriction on the specific method of training.

[0161] For example, in an alternative example, the neural network model can be updated based on the calculated total loss value through a backpropagation algorithm (BP algorithm, a supervised learning algorithm) to obtain the face recognition model.

[0162] Combination Fig.10 The present application also provides an object recognition device 100, which can be applied to the above-mentioned electronic device 10. The object recognition device 100 may include a target image acquisition module 110, a similarity information determination module 120 and a living object determination module 130.

[0163] The target image acquisition module 110 is used to obtain multiple frames of target images obtained by photographing the target object, wherein each frame of the target image includes the facial information of the target object. In this embodiment, the target image acquisition module 110 can be used to perform Figure 2 As shown in step S110 , the relevant contents of the target image obtaining module 110 may refer to the above description of step S110 .

[0164] The similarity information determination module 120 is used to determine the similarity information between the multiple frames of the target image based on the facial information. In this embodiment, the similarity information determination module 120 can be used to perform Figure 2 As shown in step S120 , the relevant contents of the similarity information determination module 120 may refer to the above description of step S120 .

[0165] The living object determination module 130 is used to determine whether the target object is a living object based on the similarity information. In this embodiment, the living object determination module 130 can be used to perform Figure 2 As shown in step S130 , the relevant contents of the living object determination module 130 may refer to the above description of step S130 .

[0166] In an embodiment of the present application, corresponding to the above-mentioned object recognition method, a computer-readable storage medium is also provided, in which a computer program is stored, and when the computer program is run, each step of the above-mentioned object recognition method is executed.

[0167] Among them, the steps executed when the aforementioned computer program is running will not be described one by one here, and reference can be made to the explanation of the object recognition method in the previous text.

[0168] It can be understood that in the foregoing description, multiple or multiple frames, etc., refer to two or more, such as multiple frames of target images refer to two or more frames of target images.

[0169] In summary, the object recognition method and device, electronic device and storage medium provided by the present application obtain multiple frames of target images obtained by photographing the target object, and then determine whether the target object is a living object based on the similarity information between the facial information in the multiple frames of target images. In this way, without relying on the target object to perform a specified action and without relying on the device that photographs the target object to have a depth image sensor, it is also possible to achieve a high degree of accuracy in identifying whether the target object is a living object (the inventors of the present application have found that even if a living object does not deliberately perform a specified action on its face, it will show slight changes with random time changes, that is, there will be micro-expression changes, so that there will be at least slight differences between the multiple frames of target images captured, while for non-living objects, such as photos or three-dimensional models, there will be no changes), thereby improving the problem of low accuracy of recognition results in existing face recognition technology in applications where the identified object cannot be required to perform a specified action (such as for a better user experience, etc.) or the shooting device is not equipped with a depth image sensor (such as considering the cost of the equipment, etc.), thereby effectively guaranteeing the accuracy of face recognition, improving the experience of the identified object (without performing a specified action), and reducing the cost of the equipment, making it more applicable and having a higher practical value.

[0170] In several embodiments provided in the embodiments of the present application, it should be understood that the disclosed device and method can also be implemented in other ways. The device and method embodiments described above are merely schematic, for example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the device, method and computer program product according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0171] In addition, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0172] If the function is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, an electronic device, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a disk or an optical disk. It should be noted that, in this article, the term "include", "include" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements that are not explicitly listed, or also includes elements inherent to such a process, method, article or device. Without more constraints, an element defined by the phrase "comprising a..." does not exclude the existence of other identical elements in the process, method, article or apparatus comprising the element.

[0173] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. An object recognition method, characterized in that: include: Shooting a target object, extracting frames based on a preset number of frames that gradually increase in chronological order, to obtain multiple frames of target images, wherein the total frame length of the multiple frames of the target images is less than a preset time length, and each frame of the target image includes facial information of the target object; Determining similarity information between multiple frames of the target image based on the facial information includes: performing feature extraction processing on the facial information in each frame of the target image based on a pre-trained face recognition model to obtain target feature information of each frame of the target image; obtaining similarity information between multiple frames of the target image based on the target feature information; the similarity information is used to characterize subtle changes in the facial information; Determining whether the target object is a living object based on the similarity information includes: comparing at least one of the target feature information with a plurality of pre-formed comparison feature information through the face recognition model to obtain confidence information that the target object is a living object, wherein the plurality of comparison feature information is obtained based on a plurality of frames of images including facial information of a plurality of different objects, and the objects include living objects and non-living objects; determining whether the target object is a living object based on the confidence information and the similarity information.

2. The object recognition method according to claim 1, characterized in that: The step of comparing at least one of the target feature information with a plurality of pre-formed comparison feature information by the face recognition model to obtain confidence information that the target object belongs to a living object comprises: Performing identity category recognition processing on at least one of the target feature information by using the face recognition model to obtain identity category information of at least one of the target feature information; Determine, among the multiple feature spaces included in the face recognition model, a target feature space of the at least one target feature information based on the identity category information, wherein different feature spaces have contrast feature information of different objects, and each of the contrast feature information has first label information identifying the identity category of the corresponding object and second label information whether the object is alive; The at least one target feature information is compared with the contrast feature information in the target feature space corresponding to the target feature information through the face recognition model to obtain confidence information that the target object belongs to a living object.

3. The object recognition method according to claim 1, characterized in that: The step of comparing at least one of the target feature information with a plurality of pre-formed comparison feature information by the face recognition model to obtain confidence information that the target object belongs to a living object comprises: For each frame of target feature information corresponding to the target image, the target feature information is compared with a plurality of pre-formed comparison feature information by the face recognition model to determine the confidence that the target object in the frame of the target image belongs to a living object, thereby obtaining a plurality of confidences; Confidence information that the target object belongs to a living object is obtained based on the multiple confidences.

4. The object recognition method according to claim 2, characterized in that: The method also includes a step of training to obtain the face recognition model, the step comprising: Performing feature extraction processing on multiple sample images through a feature extraction layer in a preset neural network model to obtain multiple sample feature information; Determine, through the loss determination layer in the neural network model, a first loss value of each sample feature information based on first label information pre-configured for each sample image, and a second loss value of each sample feature information based on second label information pre-configured for each sample image, wherein the first label information is used to identify the identity category of the object in the corresponding sample image, and the second label information is used to identify whether the object in the corresponding sample image is a living body; Based on the first loss value and the second loss value, the neural network model is trained to obtain the face recognition model.

5. An object recognition method, characterized in that: include: Shooting a target object, extracting frames based on a preset number of frames that gradually increase in chronological order, to obtain multiple frames of target images, wherein the total frame length of the multiple frames of the target images is less than a preset time length, and each frame of the target image includes facial information of the target object; Determining similarity information between multiple frames of the target image based on the facial information includes: performing feature extraction processing on the facial information in each frame of the target image based on a pre-trained face recognition model to obtain target feature information of each frame of the target image; obtaining similarity information between multiple frames of the target image based on the target feature information; the similarity information is used to characterize subtle changes in the facial information; Determining whether the target object is a living object based on the similarity information includes: comparing at least one frame of the multiple frames of target images with multiple frames of images obtained in advance to obtain confidence information that the target object is a living object, wherein the multiple frames of images include facial information of multiple different objects, and the objects include living objects and non-living objects; determining whether the target object is a living object based on the confidence information and the similarity information.

6. An object recognition device, characterized in that: include: A target image acquisition module is used to shoot a target object, extract frames based on a preset number of frames that gradually increase in chronological order, and obtain multiple frames of target images, wherein the total frame length of the multiple frames of the target image is less than a preset time length, and each frame of the target image includes facial information of the target object; A similarity information determination module is used to determine similarity information between multiple frames of the target image based on the facial information, including: performing feature extraction processing on the facial information in each frame of the target image based on a pre-trained face recognition model to obtain target feature information of each frame of the target image; obtaining similarity information between multiple frames of the target image based on the target feature information; the similarity information is used to characterize subtle changes in the facial information; A living object determination module, used to determine whether the target object is a living object based on the similarity information, comprising: comparing at least one of the target feature information with a plurality of pre-formed contrast feature information through the face recognition model to obtain confidence information that the target object is a living object, wherein the plurality of contrast feature information is obtained based on a plurality of frames of images including facial information of a plurality of different objects, and the objects include living objects and non-living objects; and determining whether the target object is a living object based on the confidence information and the similarity information.

7. An electronic device, characterized in that: include: Memory for storing computer programs; A processor connected to the memory is used to execute the computer program stored in the memory to implement the object recognition method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed, the object recognition method described in any one of claims 1 to 5 is implemented.

Images