Identity recognition method and device

Abstract

The embodiment of the present application provides an identity recognition method and device, which relates to the field of identity recognition. The terminal device trains to obtain a sub-model corresponding to a template sample group, and then uses the sub-model to determine whether the image sample to be processed matches the template image in the template sample group, thereby improving the identity recognition accuracy on the terminal device side. The specific scheme is: the terminal device trains to obtain a first sub-model based on multiple reference image samples, a first model, and a first template sample group of the collected identity recognition data; collects image samples to be processed; according to the first sub-model, compares the image samples to be processed with the template images in the first template sample group one by one to obtain a first matching result; according to the first model, compares the image samples to be processed with the template images outside the target template sample group one by one to obtain a second matching result; and determines the target matching result based on the first matching result and the second matching result. The embodiment of the present application is used in the process of matching identity recognition data.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of identity recognition technology, and in particular to an identity recognition method and device. Background Art

[0002] Fingerprints are a common biometric feature used for identification, and fingerprint recognition has become increasingly common on mobile devices in recent years. Fingerprint recognition technology first captures a fingerprint image, then processes and extracts features from the image. Finally, the feature comparison is performed against a template stored in the mobile device, and the result is output.

[0003] With the development of deep learning, terminals are able to use neural networks to extract and compare features. However, existing technologies do not allow terminals to self-learn based on user-specific data and adaptively adjust the parameters of the current neural network. As a result, terminal-side identity recognition accuracy is limited, and continuous improvement in identity recognition accuracy is impossible. Summary of the Invention

[0004] An embodiment of the present application provides an identity recognition method and device, whereby a terminal device can be trained to obtain a sub-model corresponding to a template sample group of the terminal device, so that the sub-model can be used to determine whether an image sample to be processed matches a template image in a template sample group corresponding to the sub-model, thereby improving the identity recognition accuracy on the terminal device side.

[0005] To achieve the above objectives, the present invention adopts the following technical solutions:

[0006] In a first aspect, an embodiment of the present application provides an identity recognition method. A terminal device is configured with a first model, and the terminal device has at least one template sample group, the at least one template sample group includes a first template sample group, and each template sample group includes at least one template image. The method includes: the terminal device collects multiple reference image samples of identity recognition data; the terminal device obtains a first sub-model corresponding to the first template sample group based on multiple reference image samples, the first model, and the first template sample group training; the terminal device collects image samples to be processed; the terminal device compares the image samples to be processed with the template images in the first template sample group one by one according to the first sub-model to obtain a first matching result; the terminal device compares the image samples to be processed with template images other than the target template sample group one by one according to the first model to obtain a second matching result; wherein the target template sample group is a template sample group in at least one template sample group that has been used to train the sub-model, and the target template sample group includes the first template sample group; the terminal device determines a target matching result between the image sample to be processed and the template images in at least one template sample group based on the first matching result and the second matching result.

[0007] In this solution, the terminal device first trains a first sub-model for the first template sample group. Afterwards, when the terminal device collects an image sample to be processed, it polls all template images. For the template images in the trained first template sample group, the terminal device uses the first sub-model for comparison to obtain a first matching result. Since the terminal device has trained the corresponding first sub-model for the first template sample group, the terminal device uses the first sub-model to determine whether the image sample to be processed matches the template image in the first template sample group, which can improve the identity recognition accuracy on the terminal device side. For template images in other untrained template sample groups, the terminal device uses the first model for comparison to obtain a second matching result. Since the terminal device retains the first model, it can solve the anti-forgetting problem and ensure the security of identity recognition.

[0008] In one possible design, the terminal device obtains a first sub-model through training based on multiple reference image samples, the first model and the first template sample group, including: the terminal device compares each reference image sample with the template image in the first template sample group one by one according to the first model to determine a reference matching result; the terminal device obtains a first sub-model corresponding to the first template sample group based on the multiple reference image samples, the first model, the first template sample group and the reference matching result.

[0009] In this solution, the terminal device specifically trains a first sub-model based on multiple reference image samples, a first model, a first template sample group and a reference matching result.

[0010] In one possible design, the terminal device trains a first sub-model corresponding to the first template sample group based on multiple reference image samples, the first template sample group described in the first model, and a reference matching result, including: the terminal device trains the model parameters of the first sub-model corresponding to the first template sample group based on multiple reference image samples, the model parameters of the first model, the first template sample group, and the reference matching result; or, the terminal device trains the model parameters of the first sub-model corresponding to the first template sample group based on multiple reference image samples, the samples used when training the first model, the first template sample group, and the reference matching result.

[0011] That is to say, the terminal device can use the parameters of the first model when training the first sub-model, and can also use the samples used when training the first model.

[0012] In one possible design, the first sub-model and the first model both have multiple model parameters, and the multiple model parameters include a first model parameter group and a second model parameter group; wherein the model parameters in the first model parameter group are used for feature extraction between the image sample to be processed and the template image, and the model parameters in the second model parameter group are used for similarity comparison between the features of the image sample to be processed and the features of the template image. The model parameters in the first model parameter group of the first sub-model and the first model are the same, and at least one model parameter in the second model parameter group of the first sub-model and the first model is different; or, at least one model parameter in the first model parameter group of the first sub-model and the first model is different, and the model parameters in the second model parameter group of the first sub-model and the first model are the same; or, at least one model parameter in the first model parameter group of the first sub-model and the first model is different, and at least one model parameter in the second model parameter group of the first sub-model and the first model is different.

[0013] In this solution, the first set of model parameters is applied to the feature extraction network, and the second set of model parameters is applied to the decision output network. The model parameters of the first sub-model and the first model differ at least in part. Training the first sub-model can also be considered the process of adjusting the model parameters of the first model.

[0014] In one possible design, if the target comparison result indicates that the image sample to be processed matches the first template image in at least one template sample group, and the target comparison result indicates that the confidence value of the match between the image sample to be processed and the first template image is greater than a preset value, the terminal updates the template sample group to which the first template image belongs based on the image sample to be processed.

[0015] In this solution, the collected high-quality image samples to be processed can be used to update the template sample group, thereby improving the subsequent comparison accuracy by updating the template sample group.

[0016] In one possible design, after the terminal device collects multiple reference image samples of identity recognition data, the method further includes: the terminal device trains to obtain a second sub-model based on the multiple reference image samples, the first model, and the second template sample group. Wherein, at least one template sample group also includes a second template sample group, and the target template sample group also includes a second sub-model. After the terminal device collects the image sample to be processed, the method further includes: the terminal device compares the image sample to be processed with the template images in the second template sample group one by one according to the second sub-model to obtain a third matching result. Determine the target matching result between the image sample to be processed and the template image in at least one template sample group based on the first matching result and the second matching result, including: determine the target matching result between the image sample to be processed and the template image in at least one template sample group based on the first matching result, the second matching result, and the third matching result.

[0017] In this solution, the terminal device can also train a second sub-model corresponding to the second template sample group, that is, the terminal device can also train multiple sub-models. The terminal device uses different sub-models for different template sample groups, thereby improving the accuracy of identity recognition.

[0018] In one possible design, the method also includes: the terminal device correcting the first sub-model based on the image sample to be processed, the first template sample group and the first matching result; and / or, the terminal device correcting the second sub-model based on the image sample to be processed, the second template sample group and the third matching result.

[0019] In this solution, the terminal device corrects each sub-model, so that the terminal device can obtain more accurate comparison results through the corrected sub-models.

[0020] In one possible design, the method also includes: the terminal device trains a third sub-model corresponding to the first template sample group based on the first model, the image sample to be processed, the first template sample group and the first matching result, and replaces the first sub-model with the third sub-model; and / or, the terminal device trains a fourth sub-model corresponding to the second template sample group based on the first model, the image sample to be processed, the second template sample group and the third matching result, and replaces the second sub-model with the fourth sub-model.

[0021] In this solution, the terminal device can train a new sub-model for each sub-model to replace the old sub-model, so that the terminal device can obtain more accurate comparison results through the new sub-model.

[0022] In one possible design, the first model comes from a cloud server or other device. That is, the cloud server or other device can train to obtain the first model and send it to the terminal device.

[0023] In a second aspect, embodiments of the present application provide a terminal device. The terminal device is configured with a first model and has at least one template sample group, the at least one template sample group including a first template sample group, and each template sample group including at least one template image. The terminal device includes an acquisition module and a processing module. The acquisition module is configured to acquire multiple reference image samples of identity recognition data. The processing module is configured to train a first sub-model corresponding to the first template sample group based on the multiple reference image samples, the first model, and the first template sample group. The acquisition module is further configured to acquire image samples to be processed. The processing module is further configured to perform a one-to-one comparison of the image samples to be processed with template images in the first template sample group based on the first sub-model to obtain a first matching result. The processing module is further configured to perform a one-to-one comparison of the image samples to be processed with template images outside of a target template sample group based on the first model to obtain a second matching result. The target template sample group is a template sample group in the at least one template sample group that has been used to train the sub-model, and the target template sample group includes the first template sample group. The processing module is further configured to determine a target matching result between the image samples to be processed and the template images in the at least one template sample group based on the first matching result and the second matching result.

[0024] In one possible design, the processing module is also used to: compare each reference image sample with the template image in the first template sample group one by one according to the first model to determine the reference matching result; and train a first sub-model corresponding to the first template sample group based on multiple reference image samples, the first model, the first template sample group and the reference matching results.

[0025] In one possible design, the processing module is also used to: train and obtain the model parameters of the first sub-model corresponding to the first template sample group based on multiple reference image samples, the model parameters of the first model, the first template sample group and the reference matching results; or, train and obtain the model parameters of the first sub-model corresponding to the first template sample group based on multiple reference image samples, the samples used when training the first model, the first template sample group and the reference matching results.

[0026] In one possible design, the first sub-model and the first model both have multiple model parameters, and the multiple model parameters include a first model parameter group and a second model parameter group; wherein the model parameters in the first model parameter group are used for feature extraction between the image sample to be processed and the template image, and the model parameters in the second model parameter group are used for similarity comparison between the features of the image sample to be processed and the features of the template image. The model parameters in the first model parameter group of the first sub-model and the first model are the same, and at least one model parameter in the second model parameter group of the first sub-model and the first model is different; or, at least one model parameter in the first model parameter group of the first sub-model and the first model is different, and the model parameters in the second model parameter group of the first sub-model and the first model are the same; or, at least one model parameter in the first model parameter group of the first sub-model and the first model is different, and at least one model parameter in the second model parameter group of the first sub-model and the first model is different.

[0027] In one possible design, the processing module is further configured to: train a second sub-model based on multiple reference image samples, the first model, and the second template sample group; wherein at least one template sample group also includes the second template sample group, and the target template sample group also includes the second sub-model. The processing module is further configured to: perform a one-to-one comparison of the image sample to be processed with the template images in the second template sample group based on the second sub-model to obtain a third matching result. The processing module is further configured to: determine a target matching result between the image sample to be processed and the template images in the at least one template sample group based on the first matching result, the second matching result, and the third matching result.

[0028] In one possible design, the processing module is also used to: modify the first sub-model based on the image sample to be processed, the first template sample group and the first matching result; and / or modify the second sub-model based on the image sample to be processed, the second template sample group and the third matching result.

[0029] In one possible design, the processing module is also used to: train a third sub-model corresponding to the first template sample group based on the first model, the image sample to be processed, the first template sample group and the first matching result, and replace the first sub-model with the third sub-model; and / or, train a fourth sub-model corresponding to the second template sample group based on the first model, the image sample to be processed, the second template sample group and the third matching result, and replace the second sub-model with the fourth sub-model.

[0030] In one possible design, the first model comes from a cloud server.

[0031] In a third aspect, an embodiment of the present application provides an identity recognition system. The identity recognition system includes a cloud server and a terminal device. The terminal device includes at least one template sample group, the at least one template sample group includes a first template sample group, and each template sample group includes at least one template image. The terminal device is configured with a first model and a first sub-model, wherein the first model comes from the cloud server, and the first sub-model is trained based on multiple reference image samples, the first model, and the first template sample group. The first sub-model is used to compare the image sample to be processed with the template images in the first template sample group one by one; the first model is used to compare the image sample to be processed with the template images outside the target template sample group one by one. The target template sample group is a template sample group in at least one template sample group that has been used to train the sub-model, and the target template sample group includes the first template sample group.

[0032] In one possible design, the terminal device is further configured with a second sub-model, and the at least one template sample group also includes the second template sample group. The second sub-model is trained based on multiple reference image samples, the first model, and the second template sample group. The second sub-model is configured to perform a one-to-one comparison between the image sample to be processed and the template images in the second template sample group. The target template sample group also includes the second template sample group.

[0033] In one possible design, the terminal device is used to execute the identity identification method in any possible design of the first aspect.

[0034] In a fourth aspect, embodiments of the present application provide a terminal device. The terminal device includes a processor and a memory. The memory is configured to store computer instructions. When the terminal device is in operation, the processor executes the computer instructions stored in the memory to implement the identity recognition method of any possible design described in the first aspect.

[0035] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium. The computer-readable storage medium includes computer instructions, which, when executed on a computer or processor, cause the computer or processor to execute the identity recognition method in any possible design of the first aspect.

[0036] In a sixth aspect, embodiments of the present application provide a computer program product that, when executed on a computer or processor, causes the computer or processor to execute the identity recognition method in any possible design of the first aspect.

[0037] For the beneficial effects corresponding to the above-mentioned other aspects, please refer to the description of the beneficial effects of the method, which will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1A schematic diagram of a neural network system provided in an embodiment of the present application;

[0039] Figure 2 A schematic diagram of the structure of a terminal device provided in an embodiment of the present application;

[0040] Figure 3 A flowchart of an identity recognition method provided in an embodiment of the present application;

[0041] Figure 4 A schematic diagram of the basic structure of a first model provided in an embodiment of the present application;

[0042] Figure 5 A schematic diagram of a training sub-model process provided in an embodiment of the present application;

[0043] Figure 6 A flowchart of a method for training a sub-model provided in an embodiment of the present application;

[0044] Figure 7 A schematic diagram of a neural network structure provided in an embodiment of the present application;

[0045] Figure 8 A flowchart of another identity recognition method provided in an embodiment of the present application;

[0046] Figure 9 Another structural diagram of a terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0047] The technical solutions in the embodiments of the present application will be described below in conjunction with the accompanying drawings in the embodiments of the present application. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is merely a description of the association relationship of associated objects, indicating that three relationships can exist, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0048] The present invention provides an identification method that can be applied to Figure 1 In the neural network system shown in Figure 1, the neural network system includes a terminal device 101, a cloud server 102, a public model 103 and a private model 104, etc. Among them, the terminal device 101 can be a mobile phone, a wearable device (such as a watch or a bracelet, etc.), a tablet computer, a car-mounted device, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook or a personal digital assistant (PDA), etc., which can be matched according to user identity recognition data. The cloud server 102 can train the public model and send the public model to the terminal device. The public model 103 is a neural network model trained by the cloud server in the prior art. The private model 104 is a specific neural network model for a template sample group obtained by end-side self-learning using user-side data.

[0049] For example, Figure 2 The figure shows a schematic diagram of the structure of the terminal device 101 in an embodiment of the present application. The terminal device 101 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display 194, and a subscriber identification module (SIM) card interface 195. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0050] It should be understood that the structures illustrated in the embodiments of the present application do not constitute a specific limitation on the terminal device 101. In other embodiments of the present application, the terminal device 101 may include more or fewer components than shown, or may combine or separate certain components, or may have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0051] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0052] The controller may be the nerve center and command center of the terminal device 101. The controller may generate an operation control signal based on the instruction operation code and the timing signal to complete the control of instruction fetching and execution.

[0053] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the same instruction or data again, it can directly access the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0054] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface.

[0055] It is understood that the interface connection relationship between the modules illustrated in the embodiments of the present application is merely an illustrative illustration and does not constitute a structural limitation on the terminal device 101. In other embodiments of the present application, the terminal device 101 may also adopt a different interface connection method from the above embodiments, or a combination of multiple interface connection methods.

[0056] The wireless communication function of the terminal device 101 can be implemented through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor.

[0057] Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in terminal device 101 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In other embodiments, the antennas can be used in conjunction with a tuning switch.

[0058] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to the terminal device 101. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0059] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc. applied to the terminal device 101. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, frequency modulate it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0060] In some embodiments, antenna 1 of terminal device 101 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, so that terminal device 101 can communicate with a network and other devices via wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0061] Terminal device 101 implements display functionality through a GPU, display screen 194, and an application processor. A GPU is a microprocessor for image processing that connects display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0062] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-oLed, or a quantum dot light-emitting diode (QLED). In some embodiments, terminal device 101 may include one or N display screens 194, where N is a positive integer greater than 1.

[0063] The terminal device 101 can realize the shooting function through the ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0064] The ISP processes data fed back by camera 193. For example, when taking a photo, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, which is then passed to the ISP for processing and converted into a visible image. The ISP can also perform algorithmic optimization on image noise, brightness, and skin tone. It can also optimize parameters such as exposure and color temperature of the captured scene. In some embodiments, the ISP can be located within camera 193.

[0065] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then passes the electrical signal to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the terminal device 101 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0066] The NPU is a neural network (NN) computing processor. Drawing on the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it rapidly processes input information and can continuously self-learn. The NPU enables intelligent cognitive applications in terminal device 101, such as image recognition, face recognition, speech recognition, and text comprehension.

[0067] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the terminal device 101. The external memory card communicates with the processor 110 via the external memory interface 120 to implement data storage functions. For example, files such as music and videos can be stored on the external memory card.

[0068] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the terminal device 101 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area can store data created during the use of the terminal device 101 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0069] For example, the neural network processor trains a private model for the template sample group based on multiple reference image samples, public models, and template sample groups stored in the internal memory by executing instructions in the internal memory 121. The processor 110 may also compare the image sample to be processed with the template image based on the public and private models stored in the internal memory by executing instructions in the internal memory 121 to obtain a matching result.

[0070] The terminal device 101 can implement audio functions such as music playback and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0071] The fingerprint sensor 180H is used to collect fingerprints. The terminal device 101 can use the fingerprint sensor to collect fingerprint data. If the collected fingerprint data successfully matches the stored template image, fingerprint unlocking is achieved, allowing access to application locks, fingerprint photography, fingerprint answering calls, etc.

[0072] The touch sensor 180K is also called a "touch panel." The touch sensor 180K can be provided on the display screen 194. The touch sensor 180K and the display screen 194 form a touch screen, also called a "touch screen." The touch sensor 180K is used to detect touch operations applied thereto or in the vicinity thereof. The touch sensor can transmit the detected touch operations to the application processor to determine the type of touch event. Visual output related to the touch operations can be provided via the display screen 194. In other embodiments, the touch sensor 180K can also be provided on the surface of the terminal device 101, at a location different from that of the display screen 194.

[0073] The buttons 190 include a power button, a volume button, etc. The buttons 190 may be mechanical buttons or touch buttons. The terminal device 101 may receive key inputs and generate key signal inputs related to user settings and function control of the terminal device 101.

[0074] Motor 191 can generate vibration prompts. Motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, audio playback, etc.) can correspond to different vibration feedback effects. For touch operations acting on different areas of the display screen 194, motor 191 can also correspond to different vibration feedback effects. Different application scenarios (for example: time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

[0075] The indicator 192 may be an indicator light, which may be used to indicate the charging status, power level changes, messages, missed calls, notifications, etc.

[0076] The SIM card interface 195 is used to connect a SIM card. The SIM card can be connected to and disconnected from the terminal device 101 by inserting or removing it from the SIM card interface 195. The terminal device 101 can support one or N SIM card interfaces, where N is a positive integer greater than one. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, and the like. Multiple cards can be inserted into the same SIM card interface 195 at the same time. The multiple cards can be of the same or different types. The SIM card interface 195 can also be compatible with different types of SIM cards. The SIM card interface 195 can also be compatible with external memory cards. The terminal device 101 interacts with the network through the SIM card to implement functions such as calls and data communications. In some embodiments, the terminal device 101 uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the terminal device 101 and cannot be separated from the terminal device 101.

[0077] In an embodiment of the present application, a fingerprint sensor is used to collect fingerprints, thereby obtaining reference image samples or image samples to be processed. The neural network processor (NPU) trains a private model for the template sample group based on multiple reference image samples, public models, and template sample groups stored in the internal memory by executing instructions in the internal memory. The processor can also compare the image sample to be processed with the template image based on the public and private models stored in the internal memory by executing instructions in the internal memory to obtain a matching result.

[0078] The following will describe the identity recognition method provided by the embodiment of the present application based on the above terminal device. An identity recognition method provided by the embodiment of the present application includes a sub-model training process and a sub-model application process.

[0079] In an embodiment of the present application, a terminal device has at least one template sample group, wherein a template sample group includes at least one template image, which represents a template image of identity recognition data.

[0080] The identification data refers to data that can identify the user. For example, the identification data includes biometric data, such as fingerprint data, facial data, or other biometric data that can represent the user's identity. The embodiments of this application do not limit the specific type of the identification data.

[0081] In some embodiments, at least one template sample in a template sample group may be from a single subject or multiple subjects. For example, if the template image in the template sample group is a fingerprint image, the template image in the template sample group may be multiple fingerprint data recorded for a single finger, multiple fingerprint data recorded for five fingers on a hand, or multiple fingerprint data recorded for any number of fingers. This application does not impose any limitations on this.

[0082] The terminal device can obtain at least one template sample group through various methods. For example, when a user initially enters a fingerprint on the terminal device, the terminal device can obtain a set of high-quality, high-definition, or high-completeness fingerprint data as a template sample group. For another example, the terminal device can pre-configure at least one set of fingerprint data of the user corresponding to the account from the cloud based on the user information corresponding to the account as at least one template sample group.

[0083] Taking a mobile phone as an example, when a user initially enters fingerprint data on the mobile phone, the multiple fingerprint data of the user's left index finger entered are recorded as the first template sample group, and the multiple fingerprint data of the user's left middle finger entered are recorded as the second template sample group; alternatively, the multiple fingerprint data of the user's left thumb, the multiple fingerprint data of the left index finger, the multiple fingerprint data of the left middle finger, the multiple fingerprint data of the left ring finger, and the multiple fingerprint data of the left pinky finger entered are collectively recorded as the first template sample group; the multiple fingerprint data of the user's right thumb, the multiple fingerprint data of the right index finger, the multiple fingerprint data of the right middle finger, the multiple fingerprint data of the right ring finger, and the multiple fingerprint data of the right pinky finger entered are collectively recorded as the second template sample group.

[0084] In some embodiments, each template image in the same template sample group has the same template label, and template images in different template sample groups have different template labels, so that different template sample groups can be distinguished by the template labels.

[0085] For example, multiple fingerprint data of the left index finger entered by the user are recorded as the first template sample group, and the template labels of the template images in the first template sample group are all the left index finger or the first finger or the number a; multiple fingerprint data of the left middle finger entered by the user are recorded as the second template sample group, and the template labels of the template images in the second template sample group are all the left middle finger or the second finger or the number b. The embodiments of the present application do not limit the names of the template labels, as long as they can distinguish different template sample groups. Template images with different template labels belong to different template sample groups.

[0086] The template tag allows the terminal device to identify the template sample group to which the template image belongs. During the subsequent comparison process, the image sample to be processed and the template sample are compared based on the neural network model corresponding to the template sample group to which the template image belongs. This isolates the models, achieving strict anti-forgetting effects and improving recognition accuracy.

[0087] The following will be combined Figure 3 The identity recognition method provided in the embodiments of the present application is described in detail.

[0088] For example, see Figure 3 ,The process of sub-model training on the terminal device includes:

[0089] 301. The terminal device collects multiple reference image samples of identity recognition data.

[0090] After the terminal device obtains at least one template sample group, during normal use of the terminal device by the user, the terminal device will collect multiple reference image samples of identity recognition data, wherein the identity recognition data corresponding to the multiple reference image samples is consistent with the identity recognition data represented by the template image.

[0091] During normal use of a terminal device, the terminal device will collect multiple reference image samples of the user's identification data. This means that the terminal device will collect reference image samples of the user's identification data in response to various user operations. For example, when the terminal device needs to match the user's identification data with stored identification data in response to a user operation such as unlocking, payment, or identification, the terminal device will collect multiple reference image samples of the user's identification data.

[0092] For example, let's use fingerprint data as the identity identification data and a mobile phone as the terminal device. When using a mobile phone, users typically use their fingerprint data to unlock the phone. Specifically, to unlock the phone, the user places a finger, whose fingerprint data has been previously recorded, on the corresponding fingerprint recognition area of ​​the phone. The phone's fingerprint sensor then collects the user's fingerprint data as a reference image sample and compares this reference image sample with the previously recorded template image. Upon a successful match, the phone unlocks. Users may need to unlock their phone multiple times, and each time, the phone collects at least one reference image sample of the user's fingerprint data. Alternatively, when unlocking the phone, the phone may fail due to incomplete fingerprint data or impurities on the screen, necessitating re-collection of fingerprint data. Consequently, the phone collects multiple reference image samples of fingerprint data. Furthermore, when the user needs to make a payment, the phone may also collect the user's fingerprint data as a reference image sample and compare this reference image sample with the previously recorded template image. Upon a successful match, the phone completes the payment in response to the user's payment operation. In summary, in various scenarios when a user uses a mobile phone, the mobile phone will collect the user's fingerprint data as a reference image sample. In other words, when the user uses the terminal device, the terminal device will collect multiple reference image samples of identity recognition data.

[0093] In some embodiments, when a terminal device collects reference image samples for identity recognition data, it can use the reference image samples to update the template sample group. Specifically, if the terminal device collects a new template image that meets the requirements during the reference image sample collection process, it can add the template image to the corresponding template sample group and update the template sample group.

[0094] For example, consider fingerprint data and a mobile phone as the identification data. When comparing a reference image sample of the fingerprint data collected by the mobile phone with stored template images, if the reference image sample closely matches one of the stored template images, the finger from which the reference image sample originated can be clearly determined based on the template label of the matching template image. This allows the reference image sample to be added to the corresponding template sample group, and the template sample group is updated.

[0095] 302. The terminal device obtains a sub-model corresponding to the template sample group through training based on multiple reference image samples, the first model, and the template sample group.

[0096] In the embodiments of the present application, the first model may also be referred to as a public model or an initial model. The first model is a neural network model initially deployed to a terminal device. For example, the first model is trained in the cloud, i.e., the first model is from a cloud server. The training method of the first model is the same as in the prior art and will not be further described here.

[0097] Usually, see Figure 4 The first model includes an image input module, a feature extraction network, a decision output network and a result output module. The image input module inputs a template image and a reference image sample. Then, the feature extraction network extracts features from the input template image and the reference image sample respectively. Afterwards, the decision output network compares the features of the template image and the reference image sample to obtain a matching result. Finally, the result output module outputs the matching result. It should be noted that after the image input module or after the feature extraction network, the template image and the reference image sample need to be spliced ​​together so that the subsequent decision output network can compare the features of the two. Exemplarily, the feature extraction network can be a convolutional neural network (CNN), which includes a convolutional layer, a pooling layer, a nonlinear layer (such as a ReLU layer), etc.; the decision output network can be a fully connected neural network (FCC), which includes a fully connected layer, a nonlinear layer (such as a ReLU layer), etc.

[0098] When the terminal device has not trained a sub-model, it uses the first model to compare the reference image sample and template image of the identification data collected by the terminal device. Based on the description of the first model above, after any reference image sample and any template image are input into the image input module, they pass through the feature extraction network, the decision output network, and the result output module in sequence to obtain a matching result. The matching result can also be referred to as the true label. If the reference image sample and the template image match successfully, the true label is 1; if the reference image sample and the template image do not match successfully, the true label is 0.

[0099] See also Figure 5 For example, using fingerprint data as identification data, before training a sub-model, the terminal device continuously accumulates sample pairs of {template image, reference image sample} and compares each template image with the reference image sample to obtain the true label. When the number of sample pairs accumulated by the terminal device reaches a preset value, the terminal device trains the sub-model.

[0100] It should be noted that the true label can also be obtained with the assistance of face-ID / personal identification number (PIN). For example, sometimes the collected fingerprint and the template image are actually matched, but because the fingerprint collected by the mobile phone is offset, or because the environment is humid and the mobile phone screen is wet, the collected fingerprint cannot be successfully matched with the template image. In this case, if it can be confirmed through face-ID or PIN code that the collected fingerprint does belong to the owner, the true label of this match can also be recorded as 1. With the assistance of face-ID or PIN code, it is possible to avoid missing training data used in subsequent training sub-models.

[0101] When the number of sample pairs accumulated by the terminal device reaches a preset value, the terminal device will train the sub-model. The following describes the sub-model training process in detail, taking the training of the first sub-model as an example. The first sub-model is the sub-model corresponding to the first template sample group, which is any one of the at least one template sample group acquired by the terminal device.

[0102] The sub-model may also be referred to as a private model or a branch model, etc. The first sub-model is a neural network branch corresponding to the first template sample group obtained by the terminal device through training.

[0103] For example, see Figure 6 In the above step 302, the terminal device obtains a first sub-model corresponding to the first template sample group based on the multiple reference image samples, the first model and the first template sample group through training, including:

[0104] 501. The terminal device compares each reference image sample with the template image in the first template sample group one by one according to the first model to determine a reference matching result.

[0105] This step is the same as the process of comparing the reference image sample and the template image using the first model before the terminal device trains the sub-model. If the reference image sample successfully matches the template image, the true label is 1; if the reference image sample does not successfully match the template image, the true label is 0.

[0106] The reference matching results include matching results with a true label of 1 and matching results with a true label of 0.

[0107] It is understandable that there is a mapping relationship between the true label and the reference image sample and template image pair. In other words, if the true label is 1, it can be obtained that the template image corresponding to the reference image sample is matched successfully.

[0108] Exemplarily, the fingerprint data of the first finger entered by the user when registering the fingerprint is recorded as the first template sample group, and the fingerprint data of the first finger collected by the mobile phone when the user uses the mobile phone later is recorded as the corresponding reference image sample. Assume that there are 20 template images and 700 reference image samples in the first template sample group, and there are 20*700 matching methods between the template images and the reference image samples, that is, there are 14,000 pairs of {template images, reference image samples}. If the first template image matches the first reference image sample, or the face-ID or PIN code can be used to determine that the first reference image sample matches the first template image, the true label is 1; if the first template image does not match the first reference image sample, the true label is 0. Afterwards, the reference image samples that successfully match the template image can be used as positive samples, and the reference image samples that fail to match the template image can be used as negative samples to participate in the training process of the first sub-model corresponding to the first template sample group.

[0109] 502. The terminal device trains a first sub-model corresponding to the first template sample group based on multiple reference image samples, the first model, the first template sample group, and the reference matching result.

[0110] Since the reference matching results include matching results with a true label of 1 and matching results with a true label of 0, it can be considered that when the terminal device trains the first sub-model corresponding to the first template sample group based on multiple reference image samples, the first model, the first template sample group and the first reference matching results, the samples used include positive samples and negative samples, that is, reference image samples that match the template image and reference image samples that do not match the template image.

[0111] It should be noted that negative samples that do not match the template image can also be obtained through other methods. For example, negative samples can also be samples preset in the terminal device that do not belong to the device owner, or negative samples can also be samples obtained from a server that do not belong to the device owner. This embodiment of the application does not limit the method by which the terminal device obtains negative samples.

[0112] The terminal device uses a continuous learning method to train the first sub-model. The continuous learning needs to be based on the first model, that is, in the process of continuous learning, one must learn new things but not forget the old ones. Therefore, the terminal device trains the first sub-model corresponding to the first template sample group based on multiple reference image samples, the first model, the first template sample group, and the reference matching results, including: the terminal device trains the model parameters of the first sub-model corresponding to the first template sample group based on multiple reference image samples, the model parameters of the first model, the first template sample group, and the reference matching results; or, the terminal device trains the model parameters of the first sub-model corresponding to the first template sample group based on multiple reference image samples, the samples used when training the first model, the first template sample group, and the reference matching results. That is, the first model used by the terminal device when training the first sub-model can be the parameters of the first model or the samples used when training the first model.

[0113] Exemplarily, the terminal device may train the first sub-model by using a regularized continuous learning method. The embodiment of the present application does not limit the specific continuous learning method used by the terminal device.

[0114] For example, the terminal device can use the regularized continuous learning method shown below to train the first sub-model. In the embodiment of the present application, the sub-model is trained in a continuous learning manner, which can resist overfitting caused by insufficient data on the terminal side. In this method, each time the terminal device learns a task, it estimates the Fisher information matrix of the loss function for the parameters on the task data set. In this embodiment of the present application, the task refers to comparing the collected reference image samples with each template image.

[0115] l T =l cross +λ||w- w * || F (1)

[0116] As shown in formula (1), where l T To learn the loss function for the T-th task, the first term l cross is the error loss function of the T-th task itself, and the second term is the regularization term, which can reduce the forgetfulness of old tasks. Among them, w is the updated parameter value, w *is the initial parameter value, F is the mean of the Fisher information matrix corresponding to the 0th to T-1th tasks, which represents the importance of the neural network parameters relative to the previous 0th to T-1th tasks, and λ is a hyperparameter for adjusting the regularization strength.

[0117] During the training of the first sub-model, comparing a reference image sample to a template image is considered a task. Due to the regularization term, the modification of important weights during training of the first sub-model is reduced, thereby preserving the neural network's performance on the previous task as much as possible.

[0118] The sign of successful training of the first sub-model is that the first sub-model converges. The convergence of the first sub-model can be that the training of the first sub-model reaches a preset number of times, or that the loss function during the training process is less than a preset value.

[0119] It should be noted that both the first sub-model and the first model have multiple model parameters. The multiple model parameters include a first model parameter group and a second model parameter group. Specifically, the model parameters in the first model parameter group are used to extract features of the image sample to be processed and the template image; the model parameters in the second model parameter group are used to compare the similarity between the features of the image sample to be processed and the features of the template image. Figure 4 The basic structure of the first model is shown, with the first model parameter group applied to the feature extraction network and the second model parameter group applied to the decision output network. In some embodiments of the present application, the basic structure of the first sub-model is the same as the basic structure of the first model. The first sub-model also includes a feature extraction network and a decision output network.

[0120] The model parameters of the first sub-model and the first model are at least partially different. For example, the model parameters in the first model parameter group of the first sub-model and the first model are the same, and at least one model parameter in the second model parameter group of the first sub-model and the first model is different; or, at least one model parameter in the first model parameter group of the first sub-model and the first model is different, and the model parameters in the second model parameter group of the first sub-model and the first model are the same; or, at least one model parameter in the first model parameter group of the first sub-model and the first model is different, and at least one model parameter in the second model parameter group of the first sub-model and the first model is different. Therefore, it can also be considered that the process of training the first sub-model is the process of adjusting the model parameters of the first model.

[0121] For example, see Figure 7 , the first model and the first sub-model only differ in the model parameters in the feature extraction network, that is, the first model parameters are different; the model parameters of the decision output network of the first model and the first sub-model are the same, that is, the second model parameters are the same.

[0122] Through the above steps, the terminal device trains the first sub-model for the first template sample group. Subsequently, when the terminal device captures an image sample to be processed, it polls all template images. If a match is performed against a template image in the first template sample group, the terminal device uses the first sub-model to output a matching result. If a match is performed against a template image in an untrained template sample group, the terminal device uses the first model to output a matching result.

[0123] The identity recognition method provided in the embodiment of the present application also includes a sub-model application process. The sub-model application process will be specifically described below by taking the case where the terminal device has only successfully trained the first sub-model and the terminal device has applied the trained first sub-model as an example.

[0124] Continue to see Figure 3 After step 302, the process of applying the first sub-model on the terminal device specifically includes:

[0125] 303. The terminal device collects image samples to be processed.

[0126] The process by which the terminal device collects image samples to be processed is similar to the process by which the terminal device collects reference image samples for identity recognition data in step 301. After the terminal device has trained the first sub-model, the image samples of identity recognition data collected by the terminal device in response to user operations during the user's use of the terminal device are the image samples to be processed.

[0127] For example, after the first sub-model is successfully trained, when a user needs to unlock their phone, they still place their finger on the corresponding unlocking area of ​​the phone. At this point, the phone will collect the user's fingerprint data and match it with the stored template image to determine whether the fingerprint data can unlock the phone. Here, the fingerprint data collected by the phone is the image sample to be processed.

[0128] That is to say, the image samples to be processed are image samples of identity recognition data collected by the terminal device after the first sub-model is successfully trained.

[0129] 304. The terminal device compares the image sample to be processed with the template images in the first template sample group one by one according to the first sub-model to obtain a first matching result.

[0130] Since the terminal device has trained the first sub-model for the first template sample group, the terminal device compares the image sample to be processed with the template images in the first template sample group one by one according to the first sub-model to obtain a first matching result.

[0131] It can be understood that since the first sub-model is a sub-model trained for the first template sample group, when the terminal device compares the image sample to be processed with the template image in the first template sample group, the matching result obtained according to the first sub-model is more accurate than the matching result obtained according to the first model.

[0132] 305. The terminal device compares the image sample to be processed with the template images outside the target template sample group one by one according to the first model to obtain a second matching result.

[0133] The target template sample group is a template sample group in at least one template sample group that has been used to train the sub-model, and the target template sample group includes the first template sample group.

[0134] During the comparison process of step 304 and step 305, the terminal device compares the image sample to be processed with the template images in all template sample groups one by one to determine the target matching result.

[0135] The terminal device will compare the image sample to be processed with the template images in all template sample groups one by one in a polling manner. In other words, the terminal device will compare the image sample to be processed with all template images respectively.

[0136] For example, if the terminal device stores 1000 template images in total, the terminal device compares the image sample to be processed with the 1000 template images one by one, for a total of 1000 comparisons.

[0137] The terminal device will use different models to compare the image sample to be processed with the template image according to whether the template image to be compared belongs to the trained first template sample group.

[0138] For the template images in the first template sample group, the terminal device compares the image sample to be processed with the template images in the first template sample group one by one according to the first sub-model corresponding to the first template sample group to obtain a first matching result.

[0139] Because the template image belongs to the first template sample group, and the terminal device has completed self-learning of the first template sample group and trained the first sub-model corresponding to the first template sample group, the terminal device uses the corresponding first sub-model to compare the image sample to be processed with the template image for the template image in the learned first template sample group.

[0140] After the terminal device compares the image sample to be processed with the template images in the first template group one by one, it obtains a first matching result. The first matching result indicates whether the image sample to be processed successfully matches or fails to match the template images in the first template sample group. It may also indicate the degree of matching between the image sample to be processed and the template images in the first template sample group. This embodiment of the present application is not limited to this.

[0141] For template images outside the target template group, the terminal device compares the image sample to be processed with the template images outside the target template group based on the first model, one by one, to obtain a second matching result. The target template sample group is a template sample group in at least one template sample group that has been used to train the sub-model, and the target template sample group includes the first template sample group.

[0142] Because the template image does not belong to the first template sample group or any other template sample group in the target template group, the template image has not been learned, and the terminal device has not trained a sub-model corresponding to the template image. For template images outside the target template sample group that have not been learned, the terminal device uses the first model to compare the image sample to be processed with the template image.

[0143] After the terminal device compares the image sample to be processed with the template images outside the target template sample group one by one, it obtains a second matching result. The second matching result indicates whether the image sample to be processed successfully matches or fails to match the template images outside the target template sample group. It may also indicate the degree of matching between the image sample to be processed and the template images outside the target template sample group. This embodiment of the present application is not limited to this.

[0144] 306. The terminal device determines a target matching result between the image sample to be processed and a template image in at least one template sample group according to the first matching result and the second matching result.

[0145] Based on the above, the terminal device uses a round-robin approach to compare the image sample to be processed with all template images one by one. Specifically, for template images in the first template sample group, the terminal device uses the first sub-model to compare the image sample to be processed with the template images in the first template sample group one by one. For template images outside the target template sample group, the terminal device uses the first model to compare the image sample to be processed with the template images outside the target template sample group one by one. In this way, the terminal device uses the first sub-model or the first model to complete a one-to-one comparison of the image sample to be processed with all template images, thereby obtaining all matching results, namely, the first matching result and the second matching result.

[0146] The terminal device determines the target matching result between the image sample to be processed and the template image in at least one template sample group based on the above-mentioned first matching result and the second matching result, including that the terminal device determines the matching result with the highest matching degree as the target matching result based on the above-mentioned first matching result and the second matching result.

[0147] If the target matching result is greater than or equal to the preset value, it means that the matching degree between the image sample to be processed and the corresponding template image is high, and it can be considered that the image sample to be processed and the corresponding template image match. If the target matching result is less than the preset value, it means that the matching degree between the image sample to be processed and the corresponding template image is low, and it can be considered that the image sample to be processed and the corresponding template image do not match.

[0148] In addition, if the target matching result indicates that the image sample to be processed matches the first template image in at least one template sample group, and the target matching result indicates that the confidence value of the match between the image sample to be processed and the first template image is greater than a preset value, the terminal device updates the template sample group to which the first template image belongs based on the image sample to be processed.

[0149] An embodiment of the present application provides an identity recognition method, in which a terminal device first trains a first sub-model for a first template sample group. Afterwards, when the terminal device collects an image sample to be processed, it polls the template image. For the template image in the trained first template sample group, the terminal device uses the first sub-model for comparison and outputs a matching result. For the template image in other untrained template sample groups, the terminal device uses the first model for comparison and outputs a matching result. Since the terminal device trains a corresponding first sub-model for the first template sample group, that is, the first sub-model is specifically used to compare the image to be processed with the template image in the first template sample group, the terminal device uses the first sub-model to determine whether the image sample to be processed matches the template image in the first template sample group corresponding to the first sub-model, which can improve the identity recognition accuracy on the terminal device side.

[0150] The above content is explained by taking the example of only successfully training one sub-model, namely the first sub-model. The identity recognition method provided in the embodiment of the present application can also train multiple sub-models after the terminal device collects multiple reference samples of identity recognition data. Therefore, see Figure 8 After step 302, the identity recognition method provided in the embodiment of the present application further includes:

[0151] 302a. The terminal device obtains a second sub-model through training based on multiple reference image samples, the first model, and the second template sample group.

[0152] The second template sample group also belongs to at least one template sample group possessed by the terminal device. The second sub-model is a sub-model corresponding to the second template sample group.

[0153] The specific process for the terminal device to train the second sub-model based on multiple reference image samples, the first model, and the second template sample group is similar to the specific process for the terminal device to train the first sub-model, except that the first template sample group in the sub-model training process is replaced by the second template sample group. Therefore, the specific process of this step is not repeated here.

[0154] It is understandable that the terminal device can also be trained to obtain more sub-models, each sub-model corresponding to the template sample group used for training. The embodiment of the present application does not limit the number of sub-models.

[0155] After the second sub-model is successfully trained, the terminal device can also apply the second sub-model. Therefore, after step 304, steps 305-306 can be replaced by the following steps:

[0156] 305a. The terminal device compares the image sample to be processed with the template images in the second template sample group one by one according to the second sub-model to obtain a third matching result.

[0157] Because the template image belongs to the second template sample group, and the terminal device has completed self-learning of the second template sample group and trained the second sub-model corresponding to the second template sample group, the terminal device uses the corresponding second sub-model to compare the image sample to be processed with the template image for the template image in the learned second template sample group.

[0158] After the terminal device compares the image sample to be processed with the template images in the second template set one by one, it obtains a third matching result. The third matching result indicates whether the image sample to be processed successfully matches or fails to match the template images in the second template set. It may also indicate the degree of matching between the image sample to be processed and the template images in the second template set. This embodiment of the present application is not limited to this.

[0159] 306a. The terminal device compares the image sample to be processed with the template images outside the target template sample group one by one according to the first model to obtain a second matching result.

[0160] The target template sample group is a template sample group that has been used to train the sub-model in at least one template sample group. Here, the target template sample group specifically includes a first template sample group and a second template sample group.

[0161] 307a. The terminal device determines a target matching result between the image sample to be processed and a template image in at least one template sample group according to the first matching result, the second matching result, and the third matching result.

[0162] It is understandable that if the terminal device has successfully trained more sub-models for more template sample groups, the template sample groups corresponding to all the successfully trained sub-models belong to the target template sample group.

[0163] When the terminal device uses polling to compare the image sample to be processed with all template images one by one, see Figure 7 If the template image belongs to the target template sample group, it means that the template image has been learned. For the template image that has been learned, the terminal device uses the corresponding sub-model to compare the image sample to be processed with the learned template image, thereby improving the comparison accuracy. For example, the terminal device can use Figure 7 At the same time, if the template image does not belong to the target template sample, it means that the template image has not been learned. For the template image that has not been learned, the terminal device uses the initial first model to compare the image sample to be processed and the template image that has not been learned. For example, the terminal device can use Figure 7 By using different sub-models to compare different template sample groups, the mutual influence between different template sample groups can be avoided. At the same time, the first model, that is, the public model, is retained in the embodiment of the present application to ensure security and to achieve a continuous learning method that is strictly resistant to forgetting. In addition, the embodiment of the present application adopts a continuous learning method to train the sub-model, which can resist overfitting caused by insufficient data on the terminal side. Therefore, the identity recognition method provided in the embodiment of the present application can improve the recognition accuracy and ensure security during the identity recognition process.

[0164] Furthermore, as the number of image samples to be processed increases, existing sub-models may become less accurate. In some embodiments, after successfully training a sub-model, the terminal device can modify each sub-model based on the image samples to be processed and the matching results output by each sub-model. For example, the model parameters of each sub-model can be modified to achieve more accurate comparison results.

[0165] Taking the example of a terminal device successfully training the first sub-model and the second sub-model, the process of the terminal device correcting the sub-model includes:

[0166] 307. The terminal device modifies the first sub-model based on the image sample to be processed, the first template sample group and the first matching result; and / or, the terminal device modifies the second sub-model based on the image sample to be processed, the second template sample group and the third matching result.

[0167] Furthermore, after the terminal device successfully trains a sub-model, as the terminal device collects more and more image samples to be processed, the existing sub-model may no longer be accurate enough, requiring more model parameters to be corrected, or the correction range to be large. In this case, in some embodiments, the terminal device can also train a new sub-model for each template sample group to replace the old sub-model, so that the new sub-model can obtain more accurate comparison results.

[0168] Taking the example of a terminal device successfully training the first sub-model and the second sub-model, the process of replacing the sub-model on the terminal device includes:

[0169] 308. The terminal device trains a third sub-model corresponding to the first template sample group based on the first model, the image sample to be processed, the first template sample group and the first matching result, and replaces the first sub-model with the third sub-model; and / or, the terminal device trains a fourth sub-model corresponding to the second template sample group based on the first model, the image sample to be processed, the second template sample group and the third matching result, and replaces the second sub-model with the fourth sub-model.

[0170] It is understood that the first and third matching results are similar to the aforementioned true labels and can represent the matching results of the image sample to be processed in the first sub-model and the second sub-model. For example, if the terminal device determines that the image sample to be processed successfully matches the template sample in the first template sample group using the first sub-model, the first matching result can be recorded as 1; if the terminal device determines that the image sample to be processed does not successfully match the template sample in the first template sample group using the first sub-model, the first matching result can be recorded as 0.

[0171] As the terminal device collects more and more image samples to be processed, the terminal device can obtain a more suitable sub-model by correcting the sub-model or generating a new sub-model to replace the original sub-model to adapt to the increasing number of image samples to be processed, thereby obtaining more accurate recognition results and avoiding the problem of reduced recognition accuracy caused by the original sub-model being unsuitable for the newly added image samples to be processed.

[0172] It is understandable that in order to implement the above functions, the terminal device includes hardware and / or software modules corresponding to the execution of each function. In combination with the algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in combination with the embodiments, but such implementation should not be considered to be beyond the scope of this application.

[0173] In this embodiment, the terminal device can be divided into functional modules according to the above-mentioned method example. For example, each functional module can be divided according to each function, or two or more functions can be integrated into a single processing module. The above-mentioned integrated modules can be implemented in the form of hardware. It should be noted that the module division in this embodiment is illustrative and only represents a logical functional division. In actual implementation, other division methods may be used.

[0174] In the case of dividing each functional module into corresponding functional modules, Figure 9 A possible schematic diagram of the terminal device 900 involved in the above embodiment is shown. Figure 9 As shown, the terminal device 900 may include: a collection module 901 and a processing module 902.

[0175] The acquisition module 901 can be used to support the terminal device 900 to execute the above embodiment. Figure 3 The processing module 902 can be used to support the terminal device 900 in performing the above-mentioned steps 301 and 303, and / or other steps or functions performed by the terminal device in the above-mentioned method embodiment. Figure 3 Steps 302, 304 to 308 are shown. Figure 5 Steps 501 and 502 shown, Figure 8 Steps 305a to 307a, and / or other steps or functions performed by the terminal device in the above method embodiment.

[0176] It should be noted that all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module and will not be repeated here.

[0177] In the embodiments of the present application, the terminal device 900 is presented in the form of various functional modules divided in an integrated manner. Here, "module" can refer to a specific ASIC, circuit, processor and memory that executes one or more software or firmware programs, integrated logic circuit, and / or other devices that can provide the above functions.

[0178] Optionally, an embodiment of the present application further provides an identity recognition system, comprising a cloud server and a terminal device. The terminal device includes at least one template sample group, such as a first template sample group and / or a second template sample group. The terminal device is configured with a first model and a sub-model, such as a first sub-model and a second sub-model.

[0179] The first model comes from the cloud server; the sub-model is trained based on multiple reference image samples, the first model and the corresponding template sample group.

[0180] It can also be considered that, in this identity recognition system, the terminal device is capable of executing the identity recognition method executed by the terminal device in each of the above method embodiments.

[0181] Optionally, an embodiment of the present application further provides a computer-readable storage medium, which stores computer instructions. When the computer instructions are executed on a terminal device, the terminal device executes the above-mentioned related method steps to implement the identity recognition method in the above-mentioned embodiment.

[0182] Optionally, an embodiment of the present application further provides a computer program product, which, when executed on a computer, enables the computer to execute the above-mentioned related steps to implement the identity recognition method executed by the terminal device in the above-mentioned embodiment.

[0183] Optionally, embodiments of the present application further provide a terminal device, which may be a chip, component, module, or system-on-chip. The terminal device may include a connected processor and memory; wherein the memory is used to store computer instructions. When the device is running, the processor may execute the computer instructions stored in the memory to cause the chip to perform the identity recognition method performed by the terminal device in each of the above method embodiments.

[0184] Among them, the terminal device, computer-readable storage medium, computer program product, chip or system on chip provided in the embodiments of the present application are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0185] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using a software program, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more media integrated therewith. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a DVD), or a semiconductor medium (eg, a solid state disk (SSD)).

[0186] Although the present application is described herein in conjunction with various embodiments, in the process of implementing the claimed application, those skilled in the art may understand and implement other variations of the disclosed embodiments by reviewing the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude multiple situations. A single processor or other unit may implement several functions listed in the claims. Certain measures are recorded in different dependent claims, but this does not mean that these measures cannot be combined to produce good results.

[0187] Although the present application has been described with reference to specific features and embodiments thereof, it is apparent that various modifications and combinations may be made thereto without departing from the spirit and scope of the present application. Accordingly, this specification and the drawings are merely illustrative of the present application as defined by the appended claims and are deemed to cover any and all modifications, variations, combinations or equivalents within the scope of the present application. Obviously, those skilled in the art may make various modifications and variations to the present application without departing from the spirit and scope of the present application. Thus, the present application is intended to include such modifications and variations as fall within the scope of the claims of the present application and their equivalents.

Claims

1. An identity recognition method, characterized in that: The terminal device is configured with a first model, the terminal device includes at least one template sample group, the at least one template sample group includes a first template sample group and a second template sample group, each of the template sample groups includes at least one template image; the method includes: The terminal device collects multiple reference image samples of identity recognition data; The terminal device obtains a first sub-model corresponding to the first template sample group through training according to the multiple reference image samples, the first model and the first template sample group; The terminal device obtains a second sub-model corresponding to the second template sample group by training according to the multiple reference image samples, the first model and the second template sample group; the first sub-model and the second sub-model are different; The terminal device collects image samples to be processed; The terminal device compares the image sample to be processed with the template images in the first template sample group one by one according to the first sub-model to obtain a first matching result; The terminal device compares the image sample to be processed with the template images in the second template sample group one by one according to the second sub-model to obtain a third matching result; The terminal device compares the image sample to be processed with template images outside the target template sample group one by one according to the first model to obtain a second matching result; wherein the target template sample group is a template sample group in the at least one template sample group that has been used to train the sub-model, and the target template sample group includes the first template sample group and the second template sample group; The terminal device determines a target matching result between the image sample to be processed and the template image in the at least one template sample group according to the first matching result, the second matching result, and the third matching result.

2. The method according to claim 1, characterized in that The terminal device obtains a first sub-model through training according to the multiple reference image samples, the first model, and the first template sample group, including: The terminal device compares each of the reference image samples with the template images in the first template sample group one by one according to the first model to determine a reference matching result; The terminal device trains and obtains the first sub-model corresponding to the first template sample group based on the multiple reference image samples, the first model, the first template sample group and the reference matching result.

3. The method according to claim 2, characterized in that The terminal device trains, based on the multiple reference image samples, the first model, the first template sample group, and the reference matching result, to obtain the first sub-model corresponding to the first template sample group, including: The terminal device trains the model parameters of the first sub-model corresponding to the first template sample group based on the multiple reference image samples, the model parameters of the first model, the first template sample group and the reference matching result; or The terminal device trains and obtains model parameters of the first sub-model corresponding to the first template sample group based on the multiple reference image samples, the samples used when training the first model, the first template sample group and the reference matching results.

4. The method according to any one of claims 1 to 3, characterized in that The first sub-model and the first model each have a plurality of model parameters, the plurality of model parameters including a first model parameter group and a second model parameter group; wherein the model parameters in the first model parameter group are used for feature extraction between the image sample to be processed and the template image, and the model parameters in the second model parameter group are used for similarity comparison between the features of the image sample to be processed and the features of the template image; The model parameters in the first model parameter group of the first sub-model and the first model are the same, and at least one model parameter in the second model parameter group of the first sub-model and the first model is different; or At least one model parameter in the first model parameter group of the first sub-model and the first model is different, and the model parameters in the second model parameter group of the first sub-model and the first model are the same; or, At least one model parameter in the first model parameter group of the first sub-model and the first model is different, and at least one model parameter in the second model parameter group of the first sub-model and the first model is different.

5. The method according to claim 1, wherein The method further comprises: The terminal device modifies the first sub-model based on the image sample to be processed, the first template sample group and the first matching result; and / or, The terminal device modifies the second sub-model based on the image sample to be processed, the second template sample group and the third matching result.

6. The method according to claim 1, wherein The method further comprises: The terminal device trains a third sub-model corresponding to the first template sample group based on the first model, the image sample to be processed, the first template sample group, and the first matching result, and replaces the first sub-model with the third sub-model; and / or, The terminal device trains a fourth sub-model corresponding to the second template sample group based on the first model, the image sample to be processed, the second template sample group and the third matching result, and replaces the second sub-model with the fourth sub-model.

7. The method according to any one of claims 1 to 3, characterized in that The first model comes from a cloud server.

8. A terminal device, characterized in that: The terminal device is configured with a first model, the terminal device includes at least one template sample group, the at least one template sample group includes a first template sample group and a second template sample group, each template sample group includes at least one template image, and the terminal device includes a collection module and a processing module; Wherein, the acquisition module is used to: acquire multiple reference image samples of identity recognition data; The processing module is configured to: obtain a first sub-model corresponding to the first template sample group through training according to the multiple reference image samples, the first model and the first template sample group; The processing module is further configured to: obtain a second sub-model corresponding to the second template sample group through training according to the plurality of reference image samples, the first model and the second template sample group; the first sub-model and the second sub-model are different; The acquisition module is further used to: acquire image samples to be processed; The processing module is further configured to: compare the image sample to be processed with the template images in the first template sample group one by one according to the first sub-model to obtain a first matching result; The processing module is further configured to: perform a one-to-one comparison between the image sample to be processed and the template images in the second template sample group according to the second sub-model to obtain a third matching result; The processing module is further configured to: perform a one-to-one comparison of the image sample to be processed with template images other than a target template sample group according to the first model to obtain a second matching result; wherein the target template sample group is a template sample group in the at least one template sample group that has been used to train the sub-model, and the target template sample group includes the first template sample group and the second template sample group; The processing module is further configured to determine a target matching result between the image sample to be processed and a template image in the at least one template sample group according to the first matching result, the second matching result, and the third matching result.

9. The device according to claim 8, characterized in that The processing module is further configured to: According to the first model, each of the reference image samples is compared with the template images in the first template sample group one by one to determine a reference matching result; Based on the multiple reference image samples, the first model, the first template sample group and the reference matching result, the first sub-model corresponding to the first template sample group is obtained through training.

10. The device according to claim 9, characterized in that The processing module is further configured to: Based on the multiple reference image samples, the model parameters of the first model, the first template sample group and the reference matching result, the model parameters of the first sub-model corresponding to the first template sample group are obtained by training; or Model parameters of the first sub-model corresponding to the first template sample group are obtained through training based on the multiple reference image samples, the samples used when training the first model, the first template sample group, and the reference matching results.

11. The device according to any one of claims 8 to 10, characterized in that The first sub-model and the first model each have a plurality of model parameters, the plurality of model parameters including a first model parameter group and a second model parameter group; wherein the model parameters in the first model parameter group are used for feature extraction between the image sample to be processed and the template image, and the model parameters in the second model parameter group are used for similarity comparison between the features of the image sample to be processed and the features of the template image; The model parameters in the first model parameter group of the first sub-model and the first model are the same, and at least one model parameter in the second model parameter group of the first sub-model and the first model is different; or At least one model parameter in the first model parameter group of the first sub-model and the first model is different, and the model parameters in the second model parameter group of the first sub-model and the first model are the same; or, At least one model parameter in the first model parameter group of the first sub-model and the first model is different, and at least one model parameter in the second model parameter group of the first sub-model and the first model is different.

12. The device according to claim 8, characterized in that The processing module is further configured to: modifying the first sub-model based on the image sample to be processed, the first template sample group, and the first matching result; and / or, The second sub-model is modified based on the image sample to be processed, the second template sample group and the third matching result.

13. The device according to claim 8, characterized in that The processing module is further configured to: Based on the first model, the image sample to be processed, the first template sample group, and the first matching result, training a third sub-model corresponding to the first template sample group, and replacing the first sub-model with the third sub-model; and / or, Based on the first model, the image sample to be processed, the second template sample group and the third matching result, a fourth sub-model corresponding to the second template sample group is trained and the fourth sub-model is used to replace the second sub-model.

14. The device according to any one of claims 8 to 10, characterized in that The first model comes from a cloud server.

15. An identity recognition system, characterized in that: Including cloud servers and terminal devices, The terminal device includes at least one template sample group, the at least one template sample group includes a first template sample group and a second template sample group, and each template sample group includes at least one template image; the terminal device is configured with a first model, a first sub-model, and a second sub-model, the first model is from the cloud server, the first sub-model is trained based on multiple reference image samples, the first model, and the first template sample group, the second sub-model is trained based on the multiple reference image samples, the first model, and the second template sample group, and the first sub-model and the second sub-model are different; The first sub-model is used for comparing the image sample to be processed with the template images in the first template sample group one by one; The second sub-model is used for comparing the image sample to be processed with the template images in the second template sample group one by one; The first model is used to compare the image sample to be processed with template images other than the target template sample group one by one; wherein the target template sample group is a template sample group in the at least one template sample group that has been used to train the sub-model, and the target template sample group includes the first template sample group; The terminal device is used to execute the identity recognition method as described in any one of claims 1-7.

16. A terminal device, characterized in that: include: A processor and a memory; wherein the memory is used to store computer instructions, and when the terminal device is running, the processor executes the computer instructions stored in the memory to implement the identity recognition method as described in any one of claims 1-7.

17. A computer-readable storage medium, characterized in that The method comprises computer instructions, which, when executed on a computer or a processor, enable the computer or the processor to execute the identity recognition method according to any one of claims 1 to 7.

18. A computer program product, characterized in that When the computer program product runs on a computer or a processor, the computer or the processor is enabled to execute the identity recognition method according to any one of claims 1 to 7.

Images