Identity recognition method and device in virtual reality scene, equipment and storage medium

Abstract

The invention discloses an identity recognition method and device in a VR scene, equipment and a storage medium, and relates to the technical field of artificial intelligence, the method fuses inclination angle information as auxiliary information in an identity recognition process, inclination angle features can be used for representing feature weights of image areas in an original iris image, and the feature weights of the image areas in the original iris image can be used for representing the feature weights of the image areas in the original iris image. Therefore, different feature weights can be given to each image area during feature extraction, iris features can be better recognized, and the iris recognition accuracy is improved; meanwhile, an image generator model is trained by collecting low-quality iris image training samples in advance, and then super-resolution reconstruction is performed on the collected low-image-quality iris images by using the image generator model obtained by training to obtain high-resolution iris images, so that the problem that in the iris recognition process, the iris recognition efficiency is high is solved. And the identification effect is not good due to the problem of image quality.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to the field of artificial intelligence (AI), and provides an identity recognition method, device, equipment and storage medium in a virtual reality (VR) scene. Background Art

[0002] Iris recognition technology is a technology that uses the iris in the eye to identify people. The structure of the human eye is composed of the sclera, iris, pupil, lens, retina and other parts. The iris is the circular part between the black pupil and the white sclera. It contains many interlaced spots, filaments, crowns, stripes, crypts and other detailed features. After the iris is formed during the fetal development stage, it will remain unchanged throughout the life course. These features determine the uniqueness of the iris features and also determine the uniqueness of identity recognition. Therefore, the iris features of the eye can be used as the object of identity recognition for each person.

[0003] With the popularization of VR technology, iris recognition technology is increasingly used in VR devices. Iris images are usually collected through image acquisition devices on VR devices, but in actual applications, VR devices will move with the wearer, and the collected iris images may be affected by factors such as device shaking and angle tilt, resulting in blurry or distorted iris images, which ultimately leads to low iris recognition accuracy. Summary of the invention

[0004] The embodiments of the present application provide a method, apparatus, device and storage medium for identity recognition in a VR scene, which are used to improve the accuracy of identity recognition results based on iris recognition technology in a VR scene.

[0005] On the one hand, a method for identity recognition in a VR scene is provided, the method comprising:

[0006] Obtaining an original iris image of a person wearing a VR device, and obtaining information about the tilt angle of the iris in the original iris image; wherein the original iris image is collected when a target application running on the VR device needs to identify the identity of the person wearing the VR device;

[0007] Performing feature mapping on the tilt angle information to obtain a tilt angle feature of a target feature dimension, wherein the tilt angle feature is used to characterize: a feature weight of each image area in the original iris image;

[0008] Performing shallow feature extraction based on the original iris image to obtain shallow iris image features of the target feature dimension;

[0009] Performing feature fusion on the tilt angle feature and the shallow iris image feature, and performing deep feature extraction on the obtained fused image feature, so as to obtain the deep iris image feature corresponding to the original iris image;

[0010] The wearable object is identified based on the deep iris image features, and the target application is run based on the obtained identification result.

[0011] On the one hand, a device for identifying an identity in a VR scene is provided, the device comprising:

[0012] A data acquisition unit, used to obtain an original iris image of a person wearing a VR device, and to obtain information about the tilt angle of the iris in the original iris image; wherein the original iris image is collected when a target application running on the VR device needs to identify the identity of the person wearing the VR device;

[0013] A feature mapping unit, used for performing feature mapping on the tilt angle information to obtain a tilt angle feature of a target feature dimension, wherein the tilt angle feature is used to characterize: a feature weight of each image area in the original iris image;

[0014] A feature extraction unit, configured to perform shallow feature extraction based on the original iris image to obtain shallow iris image features of the target feature dimension;

[0015] A feature fusion unit, used for performing feature fusion on the tilt angle feature and the shallow iris image feature, and performing deep feature extraction on the obtained fused image feature, so as to obtain a deep iris image feature corresponding to the original iris image;

[0016] The identity recognition unit is used to perform identity recognition on the wearable object based on the deep iris image features, and run the target application based on the obtained identity recognition result.

[0017] In a possible implementation manner, the data obtaining unit is specifically used to:

[0018] Extracting the tilt angle of the iris in the original iris image to obtain the tilt angle information; or,

[0019] The shooting angle information when the image acquisition device acquires the original iris image is obtained, and the tilt angle information is determined based on the position information of the image acquisition device in the VR device and the shooting angle information.

[0020] In a possible implementation, the device further includes a quality assessment unit and a super-resolution reconstruction unit;

[0021] The quality assessment unit is used to perform image quality assessment on the original iris image based on the trained image quality assessment model to obtain a corresponding image quality assessment value;

[0022] The super-resolution reconstruction unit is used to perform super-resolution reconstruction on the original iris image based on the trained image generator model to obtain a reconstructed iris image if the image quality evaluation value is less than a preset evaluation value threshold; wherein the second resolution of the reconstructed iris image is higher than the first resolution of the original iris image;

[0023] The feature extraction unit is specifically used to perform shallow feature extraction on the original iris image to obtain the shallow iris image features.

[0024] In a possible implementation manner, the super-resolution reconstruction unit is further used to:

[0025] Obtaining a sample iris image set, wherein the sample iris image set includes sample iris images of the second resolution acquired at a plurality of tilt angles;

[0026] performing image degradation processing on each sample iris image based on the degradation model to obtain a degraded iris image set, wherein the degraded iris image set includes degraded iris images of a first resolution;

[0027] Based on the sample iris images and the degraded iris images corresponding to the sample iris images, construct a plurality of sample image pairs, each sample image pair including a sample iris image and a degraded iris image;

[0028] Based on the multiple sample image pairs, the image generator model is iteratively trained multiple times to obtain the trained image generator model.

[0029] In a possible implementation manner, the super-resolution reconstruction unit is specifically used to:

[0030] In each training iteration, the following operations are performed:

[0031] Through the feature extraction network of the image generator model used this time, feature extraction is performed on each of the input degraded iris images to obtain image feature sets corresponding to each of the degraded iris images, each of which includes: image features of the corresponding degraded iris image in multiple feature dimensions;

[0032] Performing image feature restoration based on each obtained image feature set through the image restoration network of the image generator model used this time, and generating a reconstructed iris image corresponding to the second resolution for each of the degraded iris images;

[0033] Based on the similarity between each reconstructed iris image and the corresponding sample iris image, the parameters of the image generator model used this time are adjusted.

[0034] In a possible implementation manner, the super-resolution reconstruction unit is specifically used to:

[0035] Performing feature extraction on each of the degraded iris images step by step to obtain a corresponding image feature set; wherein the feature extraction of each level of each degraded iris image includes the following steps:

[0036] Through the convolutional feature extraction sub-network, the image features obtained by the upper-level feature extraction are subjected to convolutional feature extraction to obtain convolutional image features;

[0037] Through the downsampling sub-network, the image features obtained by the previous level feature extraction are downsampled to obtain downsampled image features;

[0038] Based on the convolution image features and the down-sampled image features, image features output by feature extraction at this level are obtained.

[0039] In a possible implementation manner, the super-resolution reconstruction unit is specifically used to:

[0040] For each image feature set, starting from the image features obtained at the last level, image feature restoration is performed level by level until the reconstructed iris image is obtained; wherein image feature restoration at each level includes the following steps:

[0041] Through the upsampling subnetwork, the image features outputted at the previous level are upsampled to obtain upsampled image features;

[0042] The up-sampled image features are superimposed with the image features corresponding to the current stage to obtain the image features output by the current stage.

[0043] In a possible implementation manner, the super-resolution reconstruction unit is specifically used to:

[0044] Superimposing the upsampled image features with the image features corresponding to the current level to obtain superimposed image features;

[0045] Based on the attention matrix set, the superimposed image features are interactively processed based on the self-attention mechanism to obtain the image features of the current level output.

[0046] In a possible implementation manner, the feature fusion unit is specifically used to:

[0047] The tilt angle feature and the shallow iris image feature are fused by any of the following processing methods:

[0048] Performing feature superposition processing on the tilt angle feature and the shallow iris image feature;

[0049] Taking each feature value in the tilt angle feature as a weight, weighted processing is performed on the image feature value at the corresponding position in the shallow iris image feature to obtain the fused image feature;

[0050] Taking each feature value in the tilt angle feature as a weight, weighting the image feature value at the corresponding position in the shallow iris image feature is performed, and then the weighted image feature value is added to the image feature value at the corresponding position in the shallow iris image feature to obtain the fused image feature;

[0051] The tilt angle feature and the shallow iris image feature are cross-correlated to obtain the attention weights corresponding to the respective image feature values ​​in the shallow iris image feature, and based on the attention weights corresponding to the respective image feature values, the shallow iris image feature is subjected to feature weighting processing based on the attention mechanism to obtain the fused image feature; wherein each attention weight represents the correlation between the tilt angle feature and the corresponding image feature value.

[0052] In a possible implementation manner, the feature mapping unit is specifically used to:

[0053] Performing identity recognition on the wearer based on the original iris image to obtain a confidence score of the corresponding identity recognition result;

[0054] If the confidence score is within the suspected score interval, feature mapping is performed on the tilt angle information to obtain the tilt angle feature of the target feature dimension.

[0055] On the one hand, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any one of the above methods when executing the computer program.

[0056] In one aspect, a computer storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the steps of any of the above methods are implemented.

[0057] In one aspect, a computer program product is provided, the computer program product comprising a computer program, the computer program being stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the computer device performs the steps of any of the above methods.

[0058] In the method provided in the embodiment of the present application, in addition to the original iris image of the wearer, the tilt angle information of the iris in the original iris image is also obtained. Then, when performing identity recognition, the shallow iris image features extracted from the original iris image and the tilt angle features obtained by mapping the tilt angle information are fused, and then deep features are extracted from the fused image features, and identity recognition is performed based on the obtained deep iris image features.

[0059] That is to say, in the recognition process, the tilt angle information is incorporated as auxiliary information in the embodiment of the present application. When the tilt angle of the iris is different, the shape of the iris in the collected iris image will also be different accordingly. Therefore, the tilt angle feature can be used to characterize the feature weights of each image area in the original iris image, so that different feature weights can be given to each image area during feature extraction, which helps to better identify iris features and thus improve the accuracy of iris recognition. Then, when running the target application, corresponding control can be more effectively performed based on the identity recognition results. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0061] Figure 1a and Figure 1b A schematic diagram of an application scenario provided for an embodiment of the present application;

[0062] Figure 2 An architectural diagram of an identity recognition system in a VR scene provided in an embodiment of the present application;

[0063] Figure 3 A schematic diagram of the training process of the iris recognition model provided in the embodiment of the present application;

[0064] Figure 4 A schematic diagram of a flow chart of an identity recognition method in a VR scene provided in an embodiment of the present application;

[0065] Figure 5 Schematic diagram of different iris shapes provided in the embodiments of the present application;

[0066] Figure 6 A schematic diagram of the structure of the feature extraction layer provided in the embodiment of the present application;

[0067] Figure 7 A schematic diagram of feature fusion provided in an embodiment of the present application;

[0068] Figure 8 Another schematic diagram of a flow chart of an identity recognition method in a VR scene provided in an embodiment of the present application;

[0069] Fig. 9 A flowchart of a training method for an image generator model provided in an embodiment of the present application;

[0070] Fig.10a and Fig.10b A schematic diagram of the operation process provided by the embodiment of the present application;

[0071] Fig.11 A schematic diagram of the structure of the image generator model provided in the embodiment of the present application;

[0072] Fig.12a and Figure 12b A schematic diagram of a characteristic operation process provided by an embodiment of the present application;

[0073] Fig.13 A schematic diagram of another flow chart of the identity recognition method in a VR scene provided in an embodiment of the present application;

[0074] Fig.14 A schematic diagram of a structure of an identity recognition device in a VR scene provided in an embodiment of the present application;

[0075] Fig.15 A schematic diagram of the structure of a computer device provided in an embodiment of the present application;

[0076] Fig.16 A schematic diagram of the composition structure of another computer device to which an embodiment of the present application is applied. DETAILED DESCRIPTION

[0077] In order to make the purpose, technical scheme and advantages of the present application clearer, the technical scheme in the embodiment of the present application will be clearly and completely described below in conjunction with the drawings in the embodiment of the present application. Obviously, the described embodiment is only a part of the embodiment of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application. In the absence of conflict, the embodiments in the present application and the features in the embodiments can be combined with each other arbitrarily. In addition, although the logical order is shown in the flow chart, in some cases, the steps shown or described can be performed in an order different from that here.

[0078] It is understandable that in the following specific embodiments of the present application, data related to the wearable object, such as iris images, is involved. When the various embodiments of the present application are applied to specific products or technologies, relevant permissions or consents need to be obtained, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0079] To facilitate understanding of the technical solutions provided in the embodiments of the present application, some key terms used in the embodiments of the present application are explained here:

[0080] Tilt angle information: Tilt angle information represents the relative position relationship between the iris and the image acquisition device when collecting the iris image, or represents the viewing direction of the eye in the iris image. When the eye looks in different directions, the tilt angle of the iris is different, so that the image presented by the iris in the collected iris image will also be different accordingly. The tilt angle information helps to better identify the iris features and improve the accuracy of iris recognition.

[0081] Wearable object: refers to the user who wears the VR device. Generally speaking, VR devices mostly use VR head-mounted display devices, which are worn on the user's head.

[0082] Shallow iris image features and deep iris image features: Iris image features refer to the features of the iris extracted from the iris image. The difference between shallow and deep lies in the different dimensions of feature extraction. When extracting features, it is usually from shallow to deep, that is, shallow features are extracted first, and then deeper features are gradually extracted. Therefore, shallow iris image features and deep iris image features are relative.

[0083] Generally speaking, low-level features come from shallow networks, which are rich in spatial information and have a relatively high feature resolution of spatial information. The receptive field of shallow networks is small, and the overlapping area of ​​receptive fields is also small, so the network can capture more details. Therefore, the features extracted by shallow networks are closer to the input and contain more pixel information, mainly some fine-grained information, such as color, texture, edge, and angular information; high-level features come from deep networks, which are rich in semantic information and have a relatively low feature resolution of semantic information. The receptive field of deep networks increases, the overlapping area between receptive fields increases, and the image information is compressed. The purpose is to obtain some information about the overall image. The extracted features are closer to the output and contain more abstract information, namely semantic information, which is mainly some coarse-grained information.

[0084] Or it can also be called local information and global information. Local information comes from the shallow network, that is, fine-grained information. At this time, the receptive field is relatively small, so the feature map obtained by the shallow network has richer local information. The resolution of the feature map at this level is relatively high, and the receptive field of a single pixel is relatively small, which can capture more information about small targets; global information comes from the deep network. At this time, as the network deepens, the receptive field becomes larger, so the feature map obtained by the deep network has richer global information. The resolution of the feature map at this level is relatively low, and the receptive field of a single pixel is relatively large, which can capture more information about medium and large targets.

[0085] Image Quality Assessment (IQA): It is one of the basic techniques in image processing. It mainly analyzes the characteristics of the image and then evaluates the quality of the image, such as the degree of image distortion. Image quality assessment plays an important role in algorithm analysis and comparison, system performance evaluation, etc. in image processing systems. Objective image quality evaluation can be divided into three types: full-reference (FR), reduced-reference (RR) and no-reference (NR). Full-reference image quality evaluation refers to comparing the difference between the image to be evaluated and the reference image when the ideal image is selected as the reference image, analyzing the degree of distortion of the image to be evaluated, and thus obtaining the quality evaluation of the image to be evaluated. Commonly used full-reference image quality objective evaluation is mainly based on pixel statistics, information theory, and structural information. For example, among the methods based on image pixel statistics, Peak-Signal to Noise Ratio (PSNR) and Mean Square Error (MSE) are two common quality evaluation methods. They measure the quality of the image to be evaluated from a statistical perspective by calculating the difference between the grayscale values ​​of the corresponding pixels of the image to be evaluated and the reference image. The no-reference method is also called the first evaluation method. The no-reference method is generally based on the statistical characteristics of the image, such as the average value of the image pixels, which reflects the average brightness of the image. The larger the average brightness, the better the image quality. Or, for example, the standard deviation of the image pixels, which represents the degree of dispersion of the grayscale values ​​of the image pixels relative to the mean. The larger the standard deviation, the more dispersed the grayscale levels in the image, and the better the image quality.

[0086] Image degradation: During the formation, recording, processing and transmission of images, the image quality deteriorates due to the imperfections of the imaging system, recording equipment, transmission medium and processing methods. This phenomenon is called image degradation. After the input image passes through a certain degradation system, the output is a degraded image. Usually, a degradation model is proposed to simulate this degradation system. The degradation model is used to process high-resolution images and generate corresponding low-resolution images. The degradation model usually includes image degradation processing methods such as blur convolution kernel, downsampling operator and random noise.

[0087] Attention mechanism: It is a method that uses high-level information to measure the intermediate features of the network, so that the network focuses on some information in the image that assists judgment and ignores irrelevant information. The essence of the attention mechanism comes from the human visual attention mechanism. When people perceive things, they generally do not watch a scene from beginning to end every time, but often observe and pay attention to a specific part according to their needs. When people find that a scene often has something they want to observe in a certain part, they will learn to pay attention to that part when similar scenes appear in the future. Therefore, the attention mechanism is essentially a means of filtering out high-value information from a large amount of information. In a large amount of information, the importance of different information to the result is different. This importance can be reflected by assigning weights of different sizes. In other words, the attention mechanism can be understood as a rule for assigning weights when synthesizing multiple sources. It can usually be used to solve the problem that it is difficult to obtain a reasonable final vector representation when the model input sequence is long. The approach is to retain the intermediate results of the model, learn it with a new model, and associate it with the output to achieve the purpose of information screening.

[0088] The embodiments of the present application relate to artificial intelligence and machine learning (ML) technology, and are mainly designed based on machine learning and computer vision (CV) technology in artificial intelligence.

[0089] Artificial intelligence is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0090] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. Basic artificial intelligence technologies generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, pre-trained model technology, operation / interaction systems, mechatronics, etc. Among them, pre-trained models are also called large models and basic models. After fine-tuning, they can be widely used in downstream tasks in various major directions of artificial intelligence. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0091] Computer vision technology Computer vision is a science that studies how to make machines "see". To put it more specifically, it refers to machine vision such as using cameras and computers to replace human eyes to identify and measure targets, and further perform graphic processing to make computer processing into images that are more suitable for human eye observation or transmission to instruments for detection. As a scientific discipline, computer vision studies related theories and technologies, and attempts to establish an artificial intelligence system that can obtain information from images or multidimensional data. Large model technology has brought important changes to the development of computer vision technology. Pre-trained models in the visual field such as swin-transformer, ViT, V-MOE, and MAE can be quickly and widely applied to downstream specific tasks after fine tuning. Computer vision technology usually includes image processing, image recognition, image semantic understanding, image retrieval, optical character recognition (OCR), video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous positioning and mapping, and other technologies, as well as common biometric recognition technologies such as face recognition and fingerprint recognition.

[0092] Machine Learning (ML) is a multi-disciplinary interdisciplinary subject involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and learning by teaching. The pre-trained model is the latest development of deep learning, which integrates the above technologies.

[0093] Artificial Neural Network (ANN) abstracts the human brain neural network from the perspective of information processing, establishes a simple model, and forms different networks according to different connection methods. A neural network is a computing model composed of a large number of nodes (or neurons) connected to each other. Each node represents a specific output function, called an activation function. The connection between each two nodes represents a weighted value for the signal passing through the connection, called a weight, which is equivalent to the memory of the artificial neural network. The output of the network varies depending on the network connection method, weight value and activation function. The network itself is usually an approximation of a certain algorithm or function in nature, or it may be an expression of a logical strategy.

[0094] With the research and advancement of artificial intelligence technology, artificial intelligence technology has been studied and applied in many fields, such as common smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, driverless cars, autonomous driving, drones, digital twins, virtual humans, robots, artificial intelligence generated content (AIGC), conversational interaction, smart medical care, smart customer service, game AI, etc. It is believed that with the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role.

[0095] The solution provided in the embodiment of the present application involves artificial intelligence CV technology, specifically involves the identification of the wearable object based on the collected iris image, so as to provide the identity verification data basis for the operation of the target application in the VR device. When the identity of the wearable object is identified, it is necessary to adopt an artificial neural network model based on deep learning, that is, the embodiment of the present application adopts a machine learning method to obtain an iris recognition model for the identification of the wearable object based on the iris image. The iris recognition model is based on machine learning to process and understand the deep semantic features in the iris image combined with the tilt angle information, so that the input iris image can be combined with the tilt angle information for feature understanding, so as to output the deep iris image features of the wearable object, so as to compare with each feature in the identity library to identify the identity of the current wearable object, thereby determining the operation logic of the target application.

[0096] Specifically, the identification of the wearable object based on the collected iris image in the embodiment of the present application can be divided into two parts, including a training part and an application part. Among them, the training part involves the technical field of machine learning. The training part includes the training of the image generator model for generating high-resolution iris images and the training of the iris image recognition model. For the training of the image generator model, the high-resolution iris image is collected in advance and degraded by the degradation model to obtain the corresponding low-resolution image, which is used as the model training data to train the image generator model. After the training is completed, the image generator model can be used to perform super-resolution reconstruction on the low-resolution image, and the generated high-resolution iris image can be used as the input of the iris recognition model to train the iris recognition model, and the model parameters are continuously adjusted by the optimization algorithm until the model converges; the application part is used to use the artificial neural network model trained in the training part to perform iris recognition on the iris image input during actual use, and output the corresponding recognition result. In addition, it should be noted that the artificial neural network model in the embodiment of the present application can be either online training or offline training, which is not specifically limited here. In this article, offline training is used as an example for illustration.

[0097] The following is a brief introduction to the technical ideas of the embodiments of the present application.

[0098] Iris recognition technology is increasingly used in VR devices. Iris images are usually collected by image acquisition devices on VR devices, but in actual applications, VR devices will move with the wearer, and the collected iris images may be affected by factors such as device shaking and angle tilt, resulting in blur or distortion of the iris images. In related technologies, the collected iris images are usually used directly for recognition, resulting in low iris recognition accuracy.

[0099] Based on this, an embodiment of the present application provides an identity recognition method in a VR scene. In this method, in addition to the original iris image of the wearer, the tilt angle information of the iris in the original iris image is also obtained. Then, when performing identity recognition, the shallow iris image features extracted from the original iris image and the tilt angle features obtained by mapping the tilt angle information are fused, and then deep features are extracted for the fused image features, and identity recognition is performed based on the obtained deep iris image features. That is, in this recognition process, the tilt angle information is integrated into the embodiment of the present application as auxiliary information. When the tilt angle of the iris is different, the shape of the iris in the collected iris image will also be different accordingly. Therefore, the tilt angle feature can be used to characterize the feature weights of each image area in the original iris image, so that different feature weights can be given to each image area when performing feature extraction, which is helpful to better identify the iris features, thereby improving the accuracy of iris recognition. Then, when the target application is run, corresponding control can be more effectively performed based on the identity recognition result. For example, when entering a VR game, the user is allowed to enter the game when it is recognized that the user belongs to a legal identity.

[0100] In addition, considering the influence of various factors during shooting, the image quality of the collected iris image may be poor. Therefore, in the embodiment of the present application, low-quality iris image training samples are collected in advance to train the image generator model, and then the trained image generator model is used to perform super-resolution reconstruction on the collected low-quality iris image to obtain a high-resolution iris image, so as to solve the problem of poor recognition effect caused by image quality problems during the iris recognition process.

[0101] The following briefly introduces the application scenarios to which the technical solutions of the embodiments of the present application can be applied. It should be noted that the application scenarios introduced below are only used to illustrate the embodiments of the present application and are not limited. In the specific implementation process, the technical solutions provided by the embodiments of the present application can be flexibly applied according to actual needs.

[0102] The solution provided in the embodiments of the present application can be applied to VR-based identity authentication scenarios, such as VR games, VR videos, or VR offices.

[0103] In a possible implementation, Figure 1a As shown, it is a schematic diagram of an application scenario provided by an embodiment of the present application. In this scenario, a VR device 101 can be included. The VR device 101 can be worn by a wearable object to realize the VR function. The VR device 101 is any device that can realize the VR function, for example, it can be one or more combinations of VR glasses, VR handles, VR head-mounted display devices (referred to as VR helmets or VR head displays for short), etc.

[0104] Among them, the VR device 101 may include one or more processors, memories, and interactive I / O interfaces, etc. The interactive I / O interface is used to implement data interaction between the various components (such as glasses, handles, or helmets, etc.) included in the VR device 101. The memory of the VR device 101 can store the program instructions required to be executed in the identity recognition method in the VR scene provided by the embodiment of the present application, and these program instructions can be used to implement the identity recognition process in the VR scene provided by the embodiment of the present application when executed by the processor.

[0105] In a possible implementation, Figure 1b As shown, it is a schematic diagram of an application scenario provided by an embodiment of the present application. In this scenario, a VR device 101 and a VR server 102 may be included.

[0106] Among them, VR device 101 can be used with Figure 1b The implementation method of the VR device 101 shown is similar, that is, it can be any device that can realize the VR function, for example, it can be one or a combination of VR glasses, VR helmets, etc.

[0107] The VR device 101 may be installed with a target application, which may be any application involving identity recognition, etc. It may be an application built into the operating system of the VR device, or a third-party application installed in the VR device, etc. In actual application, the target application may be configured to implement the identity recognition process through the identity recognition method in the VR scene of the embodiment of the present application. For example, the identity recognition function in the VR scene may be implemented as a plug-in embedded in the application, or it may be a function possessed by the target application itself. The application involved in the embodiment of the present application may be a software client, or a client such as a web page or a small program.

[0108] The VR server 102 is used to provide background services for the target application to meet the computing needs of the target application. The VR server 102 can be, for example, an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, i.e., content delivery networks (CDNs), and big data and artificial intelligence platforms, but is not limited thereto.

[0109] It should be noted that the identity recognition method in the VR scene in the embodiment of the present application can be executed by the VR device 101 or the VR server 102 alone, or can be executed by the VR server 102 and the VR device 101 together. For example, when the VR device 101 is executed alone, the VR device 101 can collect the original iris image and the tilt angle information, and perform feature mapping on the tilt angle information to obtain the tilt angle feature of the target feature dimension, and perform shallow feature extraction based on the original iris image to obtain the shallow iris image feature of the target feature dimension, and then perform feature fusion on the tilt angle feature and the shallow iris image feature, and perform deep feature extraction on the obtained fused image feature to obtain the deep iris image feature corresponding to the original iris image, and finally perform identity recognition on the wearer based on the deep iris image feature, and run the target application based on the obtained identity recognition result; or, when the VR server 102 is executed alone, the VR server 102 performs the above process. When the VR server 102 and the VR device 101 are jointly executed, the process with large computational complexity can be implemented through the VR server 102, so as to achieve a faster computational process based on the stronger computing power of the VR server 102, thereby improving the efficiency of iris recognition.

[0110] Among them, both the VR device 101 and the VR server 102 may include one or more processors, memories, and I / O interfaces for interacting with each other. In addition, the VR server 102 may also be configured with a database, which may be used to store various model-related parameters. Among them, the VR device 101 and the VR server 102 may also store program instructions required to be executed in the identity recognition in the VR scene provided in the embodiment of the present application, and these program instructions, when executed by the processor, can be used to implement the identity recognition process in the VR scene provided in the embodiment of the present application.

[0111] In the embodiment of the present application, the VR device 101 and the VR server 102 can be directly or indirectly connected to each other through one or more networks 103. The network 103 can be a wired network or a wireless network, for example, the wireless network can be a mobile cellular network, or a wireless fidelity (Wireless-Fidelity, WIFI) network, and of course, it can be other possible networks, which are not limited in the embodiment of the present application.

[0112] The following describes the identity recognition method in the VR scene provided by the exemplary embodiment of the present application in combination with the application scenario described above and with reference to the accompanying drawings. It should be noted that the above application scenario is only shown to facilitate understanding of the spirit and principles of the present application, and the implementation of the present application is not limited in this respect.

[0113] In a possible implementation, the method of the embodiment of the present application can be applied to the identity authentication scenario of VR games. Since the collection and analysis of iris images involve the user's privacy data, in order to realize the collection and use of relevant data, the wearer can be requested to authorize the collection and use rights of the relevant data. If the wearer allows it, when the wearer is ready to play VR games and needs to perform iris recognition to verify its identity, the user's iris image can be collected, and the corresponding tilt angle information can be obtained at the same time. Then, the deep features of the iris image can be extracted in combination with the tilt angle information, and identity recognition can be performed based on the extracted deep iris image features, and then whether the user is allowed to enter the VR game can be determined based on the identity recognition result. By using the tilt angle information as supplementary information, the expression ability of the features extracted from the iris image is enhanced, thereby improving the accuracy of iris recognition.

[0114] like Figure 2 As shown, it is an architecture diagram of an identity recognition system in a VR scene provided by an embodiment of the present application. In this architecture, the following modules may be included:

[0115] (1) Image acquisition and storage module

[0116] The image acquisition and storage module is used to capture clear iris images at different angles, and record and store the acquired images to provide basic data for subsequent iris recognition and verification processes. In VR devices, iris image acquisition mainly relies on built-in cameras and optical systems.

[0117] (2) Iris image preprocessing module

[0118] The iris image preprocessing module is mainly responsible for constructing a data set based on high-resolution iris images, including low-resolution image and high-resolution image pairs. Specifically, it can provide training data for subsequent super-resolution reconstruction by simulating the degradation process of iris images in reality.

[0119] (3) Generator module

[0120] The main task of the generator module is to convert low-resolution images into high-resolution images, retaining more details and clarity, improving image quality, enhancing the ability to express features during subsequent iris recognition, and improving the accuracy of subsequent iris recognition.

[0121] In one possible implementation, see Figure 2 As shown, the generator module may include:

[0122] Input layer, which receives low-resolution images.

[0123] Multiple dense blocks, each of which contains multiple convolutional layers and activation functions for feature extraction.

[0124] The residual network is a short-circuit connection that connects dense blocks. It solves the gradient vanishing problem by adding short-circuit connections. These short-circuit connections allow low-level feature information to be combined with high-level feature information, improving the learning ability of the overall network. At the same time, the residual network enables the network to learn new features based on the input features, thereby achieving higher quality image reconstruction.

[0125] The self-attention mechanism allows the model to establish dependencies between input sequences at different positions in order to focus on high-level information, which helps capture long-range contextual information and thus generate finer image details.

[0126] Output layer, used to output high-resolution images.

[0127] (4) Discriminator module

[0128] The main task of the discriminator module is to determine whether the high-resolution image generated by the generator module is close to the original high-resolution image. The architecture of the discriminator module usually adopts a convolutional neural network, which can extract features from the image generated by the generator and the original high-resolution image. By calculating the similarity between these features, the discriminator module can evaluate the performance of the generator module.

[0129] (5) Iris recognition module

[0130] The main task of the iris recognition module is to perform identity recognition based on the input high-resolution iris image and iris tilt angle information. This module usually uses deep learning methods, such as convolutional neural networks, to extract and classify iris features.

[0131] In one possible implementation, see Figure 2 As shown, the iris recognition module may include the following parts:

[0132] The input layer receives the iris image and iris tilt angle information, such as the high-resolution iris image and tilt angle information from the generator module.

[0133] The feature extraction layer is used to extract features that are helpful for iris recognition from the input image. For example, it can be composed of multiple convolutional layers and pooling layers.

[0134] The angle supplement layer is used to fuse the tilt angle information with the features output by the feature extraction layer to enhance the accuracy of iris recognition.

[0135] The fully connected layer is used to flatten the data after feature fusion, and then perform further feature fusion through the fully connected layer.

[0136] The output layer is responsible for generating classification results based on the extracted features. For example, the Softmax activation function can be used to output the probability distribution of each category.

[0137] In combination with the application scenarios described above, the following describes the relevant processes of the identity recognition method in the VR scenario provided by the exemplary embodiment of the present application with reference to the accompanying drawings. It should be noted that the above application scenarios are only shown to facilitate understanding of the spirit and principles of the present application, and the implementation of the present application is not limited in this respect.

[0138] The identity recognition method in the VR scene in the embodiment of the present application is based on a trained iris recognition model. Therefore, before introducing the method flow, the training process of the iris recognition model is first introduced here.

[0139] See also Figure 3 FIG. 1 is a schematic diagram of a training process of an iris recognition model provided in an embodiment of the present application. The method can be executed by a computer device, which can be, for example, Figure 1a or Figure 1b The VR device or VR server shown in . The method flow may include the following steps:

[0140] Step 301: obtaining a plurality of iris image training samples, each of which includes a sample iris image and its iris tilt angle information, and is annotated with corresponding actual identity information.

[0141] In an embodiment of the present application, a supervised training method can be used, and the actual identity information will be labeled for each iris image training sample, that is, for each sample iris image, it can be labeled with its corresponding actual identity information, that is, the identity recognition result of the iris in the iris image training sample, which is mainly used for subsequent verification of the accuracy of the model.

[0142] In a possible implementation, considering that the number of existing sample iris images is limited, while the number of sample iris images required for actual training is huge, image transformation processing can be performed on the sample iris images of each user already in the system, such as image processing methods such as enlarging a local area or increasing image noise, so as to obtain different sample iris images, and each sample iris image can be labeled according to its image source. For example, sample iris images derived from the original iris image of user A can all be labeled as user A.

[0143] In a possible implementation, sample iris images may be collected based on actual application scenarios. For example, the identity recognition scenario of the embodiment of the present application is mainly identity recognition in a VR scenario. Then, sample iris images may be collected from VR scenarios to better meet the characteristics of the scenario and improve the accuracy of the trained model. In addition, considering that sample iris images in a single VR scenario may not meet the large number of sample requirements for training, in addition to the above-mentioned image derivation method, sample iris images of other scenarios, such as non-VR scenarios, may be collected and converted into sample iris images in VR scenarios through certain processing methods. For example, the sample iris images in VR scenarios are mainly characterized by insufficient light, and the main light comes from the VR display area and related fill light devices. Therefore, the light is very characteristic of VR scenarios. Therefore, the image characteristics in VR scenarios can be simulated, and the sample iris images in non-VR scenarios can be processed. For example, VR scene features can be added to the images to obtain sample iris images that can be used for VR scene training and expand the sample size.

[0144] Step 302: using an iris recognition model, perform identity recognition based on the sample iris image and the iris tilt angle information in the input iris image training sample to obtain the corresponding predicted identity information.

[0145] In the embodiment of the present application, the iris recognition model is iteratively trained through multiple iris image training samples until it satisfies the iteration termination condition. During each training, some or all of the iris image training samples can be extracted from the multiple iris image training samples and input into the current iris recognition model, for example, a random selection method can be adopted, or the multiple iris image training samples can be pre-divided into batches, and a batch of iris image training samples is input each time. Then, this training is performed based on the input iris image training samples, that is, the current iris recognition model can be used for model forward reasoning to obtain the predicted identity information corresponding to each iris image training sample.

[0146] It should be noted that since the training process is similar each time, a training process is taken as an example here for introduction, and the processing process of the iris recognition model will be introduced in detail in the subsequent process, so I will not go into details here.

[0147] Step 303: Generate a model loss value of the iris recognition model based on the difference between the predicted identity information and the actual identity information of the iris image training sample.

[0148] In one possible implementation, the model loss value can be calculated using any possible loss function, such as a cross entropy loss function (Cross Entropy Loss Function), a mean squared error (MSE) loss function, a squared absolute error loss function, a maximum likelihood loss (Likelihood Loss, LHL) function, etc. Of course, other possible loss functions may also be used, and the embodiments of the present application are not limited to this.

[0149] Step 304: Determine whether the current iris recognition model meets the iteration termination condition.

[0150] In the embodiment of the present application, the iteration termination condition may include at least one of the following conditions:

[0151] (1) The number of iterations reaches the set threshold.

[0152] (2) The model loss value is less than the set loss threshold.

[0153] Step 305: If the determination process of step 304 is no, the model parameters are adjusted based on the model loss value.

[0154] In the embodiment of the present application, when the number of iterations does not exceed the preset number threshold, and the model loss value is not less than the set loss threshold, the determination process of step 304 is no, that is, it is considered that the current iris recognition model does not meet the iteration termination condition, and the model parameters need to be adjusted to continue training. After the parameter adjustment, the next iterative training process is entered, that is, jumping to step 302.

[0155] In a possible implementation, when the model still does not meet the convergence conditions, the model weight parameters can be updated by the gradient descent method, and the training can continue with the updated model weight parameters. When the number of iterations has exceeded the preset number threshold, or the model loss value is less than the set loss threshold, the determination process of step 304 is yes, that is, it is considered that the current iris recognition model has met the convergence conditions, the model training is completed, and the trained iris recognition model is obtained. In order to verify the effect of the trained iris recognition model, the embodiment of the present application tests the trained iris recognition model using iris image test samples. If the evaluation index reaches the preset evaluation effect, the iris recognition model can be used in the actual identity recognition process. If the model evaluation fails, it is necessary to continue training and testing until the iris recognition model reaches the preset evaluation effect.

[0156] In addition, considering that the goal of the iris recognition module is to achieve accurate classification of high-resolution iris images and use the iris tilt angle information to improve recognition accuracy, the performance of the iris recognition module can also be measured through evaluation indicators such as accuracy, recall rate and F1 score. Generally speaking, the iris recognition module should have a high accuracy and recall rate, which means that the module can accurately identify and distinguish the iris features of different individuals, while making full use of the tilt angle information to improve the accuracy of iris recognition.

[0157] After the iris recognition model is trained, it can be used in the identity recognition process in the actual VR scene. The processing of the sample iris image in the training stage is the same as the processing of the original iris image described below. The above process can refer to the following processing process.

[0158] See also Figure 4 As shown, it is a flow chart of the identity recognition method in the VR scene provided by the embodiment of the present application. The method can be executed by a computer device, and the computer device can be, for example, Figure 1a or Figure 1b The VR device or VR server shown in . The specific implementation process of this method is as follows:

[0159] Step 401: obtaining an original iris image of a person wearing a VR device, and obtaining tilt angle information of the iris in the original iris image.

[0160] In the present application examples, see Figure 2 As shown, the iris recognition related process is performed by the iris recognition module, and the original iris image and the tilt angle information can be obtained by the input layer included in the iris recognition module.

[0161] In the embodiment of the present application, the original iris image is collected when the target application running on the VR device needs to identify the identity of the wearer. For example, if the target application is a VR game application, when the wearer needs to enter the VR game application, it is necessary to verify whether the wearer has the right to enter. At this time, identity recognition and verification are required, and the iris recognition method can be selected, and the original iris image of the wearer will be collected based on this method; or, if the target application is a VR payment application, when the wearer makes a payment in the VR scene, identity recognition and verification are required, and the iris recognition payment method can be selected, and the original iris image of the wearer will be collected based on this method.

[0162] In a possible implementation, the above method may be executed by a VR server, which may be a manufacturer server corresponding to the VR device, or may refer to a background server corresponding to the target application. In actual application, when the target application on the VR device needs to identify the identity of the wearer, the collection of the iris image of the wearer is triggered, and the obtained original iris image is uploaded to the VR server. After the VR server obtains the identity recognition result of the wearer through the method of the embodiment of the present application, the VR device controls the operation of the target application based on the identity recognition result.

[0163] In a possible implementation, the above method can be executed by a VR device, that is, the entire process of the method of the embodiment of the present application can be executed by a VR device. When the target application on the VR device needs to identify the identity of the worn object, the collection of the iris image of the worn object is triggered, and the VR device obtains the identity recognition result of the worn object through the method of the embodiment of the present application, and controls the operation of the target application based on the identity recognition result.

[0164] In actual application, the VR device is equipped with an image acquisition device, and when the acquisition of the iris image is triggered, the image acquisition device can be called to acquire the image. The tilt angle information refers to the angle information of the iris in the acquired iris image. When the eye's viewing angle is at different angles, the iris in the presented iris image has different shapes, so the angle information of the iris should be considered when extracting iris features.

[0165] In a possible implementation, the tilt angle information may be obtained by extracting the tilt angle of the iris in the original iris image. For example, the angle information may be extracted from the original iris image by an image processing method that analyzes the eye perspective or iris angle in the image; or a dedicated tilt angle extraction model may be pre-trained, and the original iris image may be input into the tilt angle extraction model to obtain the tilt angle information outputted by the model.

[0166] In a possible implementation, the tilt angle information can be obtained based on the conversion of the shooting angle of the image acquisition device. This is because generally speaking, in order to capture a better quality iris image, the image acquisition device can convert the shooting angle according to the viewing angle when capturing the iris image, so as to better follow the iris image. Therefore, the shooting angle is usually convertible with the angle information of the iris in the iris image. Therefore, when the image acquisition device outputs the original iris image, it can also output the shooting angle information corresponding to the original iris image when the original iris image is captured, and then the tilt angle information can be obtained by performing angle conversion based on the shooting angle information.

[0167] For example, the position of the image acquisition device in the VR device is usually fixed, so a coordinate system can be constructed in advance, and then the corresponding tilt angle information can be determined based on the position information of the image acquisition device in the VR device and the shooting angle information.

[0168] Step 402: Perform feature mapping on the tilt angle information to obtain the tilt angle feature of the target feature dimension. The tilt angle feature is used to characterize: the feature weight of each image area in the original iris image.

[0169] In the embodiment of the present application, it is used to assist the iris recognition process in paying more attention to the content in a specific direction of the image.

[0170] In order to facilitate the subsequent fusion with the iris image features, feature mapping can be performed on the tilt angle information first to obtain a tilt angle feature that can characterize the tilt angle information. The tilt angle feature can include the feature weights of each image area in the original iris image. The feature weight is closely related to the tilt angle. When the tilt angle is different, the feature weights of each image area are correspondingly different.

[0171] For example, when the shooting angle is frontal, that is, the viewing direction is looking straight ahead, the shape of the iris in the iris image taken at this time is the most standard, so the feature weight can be assigned according to the standard shape of the iris. When the shooting angle is upward, it means that the viewing angle of the wearer is looking down, and the photographed iris is deformed to a certain extent, and is not in the standard shape. In this case, the feature weight can be assigned according to the iris shape corresponding to the actual angle.

[0172] In a possible implementation, the tilt angle feature may also be the tilt angle itself, that is, a direction or area in the image may be indicated by an angle, and then the weight of the direction or area may be increased in the subsequent feature extraction process, thereby enhancing the importance of the area for iris recognition.

[0173] In a possible implementation, the assignment of feature weights to each image region may be performed through a mapping relationship between different angles and iris shapes.

[0174] Specifically, the iris photographed at different tilt angles presents different shapes, see Figure 5 As shown in , when the iris is photographed from the front, it presents a standard shape, and the feature weight can be assigned according to the standard shape. When there is a certain shooting angle, such as Figure 5When shooting from an upward angle as shown, the iris presented has a certain deformation compared to the standard shape, and the feature weight can be assigned according to the shape corresponding to the current angle. This deformation may cause errors in the subsequent extraction of iris features. Therefore, the embodiment of the present application can compensate for the recognition inaccuracy caused by this error by adding tilt angle information.

[0175] In a possible implementation, the assignment of feature weights of each image region may also be performed through a feature mapping network, in which a plurality of nodes exist corresponding to the size of the input iris image, and the parameter value of each node may be obtained through a model training process.

[0176] For example, the number of nodes in the feature mapping network can be consistent with the number of pixels in the original iris image, and the parameters of each node are obtained through the model training process. Then, after the tilt angle information is input, the tilt angle information can be mapped to a feature weight value through each node, thereby obtaining a tilt angle feature that is consistent with the size of the original iris image, and each feature value in the tilt angle feature represents the feature weight corresponding to the corresponding pixel point in the original iris image.

[0177] In a possible implementation, each tilt angle information can be converted into a unique corresponding tilt angle feature, that is, the corresponding relationship between the tilt angle information and the tilt angle feature can be pre-configured, and then after the tilt angle information is obtained, feature mapping can be performed through the corresponding relationship to obtain the corresponding tilt angle feature. The corresponding relationship can be configured based on experience, or it can be initialized and obtained by training during the iris recognition model recognition process.

[0178] In a possible implementation, each tilt angle information may be converted into a unique corresponding tilt angle feature matrix, in which each eigenvalue may represent a feature weight corresponding to a corresponding pixel point or an image region in the original iris image.

[0179] Step 403: Perform shallow feature extraction based on the original iris image to obtain shallow iris image features of target feature dimensions.

[0180] See also Figure 2 As shown, the iris recognition module is the iris recognition model mentioned above, and the process of performing shallow feature extraction on the original iris image can be implemented by the feature extraction layer included therein.

[0181] Specifically, the feature extraction layer can be implemented using any possible feature extraction network, such as CNN or a derivative network combined with CNN.

[0182] It should be noted that the target feature dimension refers to the shallow feature dimension, which is relative to the deep layer. Generally speaking, the shallow feature dimension pays more attention to the local features of the image, and its definition can also be configured according to the preset dimension threshold. For example, a feature dimension sequence can be preset, and each feature dimension in the sequence is a feature dimension that may be obtained by feature extraction for iris images, and each feature dimension can be sorted according to the size of the feature size. Then, the shallow feature dimension can set a dimension threshold from it, and the dimension larger than the dimension threshold is the shallow feature dimension.

[0183] In order to facilitate the subsequent feature fusion of the tilt angle feature and the shallow iris image feature, the tilt angle feature and the shallow iris image feature can be configured as the same feature dimension, that is, the target feature dimension mentioned above. Of course, in actual applications, they can also be configured as different feature dimensions, and then certain conversion operations can be performed during the subsequent feature fusion.

[0184] In a possible implementation, the feature extraction layer may be composed of a plurality of convolutional layers and a pooling layer, which is responsible for extracting features that are helpful for iris recognition from the input image. The convolutional layer is used to perform convolution processing on the original iris image or the iris image features output by the previous pooling layer through a preset convolution kernel and step size, and the pooling layer is used to perform pooling processing on the iris image features output by the previous convolutional layer to perform dimensionality reduction processing on the iris image features. The pooling layer may be implemented by a max pooling layer, an average pooling layer, an overlapping pooling layer, or a spatial pyramid pooling layer.

[0185] See also Figure 6 As shown in the figure, taking the feature extraction layer including two convolution layers and two pooling layers as an example, the first convolution layer is used to extract features from the original iris image, and the obtained image features are input to the pooling layer for pooling processing to reduce the image feature scale, and then input to the next convolution layer for convolution feature extraction, and then input to the next pooling layer for pooling processing to further reduce the image feature scale, and finally obtain the shallow iris image features. However, it should be noted that Figure 6 The number and order of convolutional layers and pooling layers in the feature extraction layer shown are only a possible structural illustration. In actual scenarios, they can be reasonably configured according to needs, and the embodiments of the present application do not impose any restrictions on this.

[0186] Step 404: Fusing the tilt angle feature with the shallow iris image feature to obtain a fused image feature.

[0187] In the embodiment of the present application, in order to enhance the expressive power of iris features and improve the amount of information contained in the extracted iris features, the shallow iris image features are enhanced by tilt angle information to obtain enhanced fused image features.

[0188] In a possible implementation manner, the manner of fusing the tilt angle feature with the shallow iris image feature includes but is not limited to the following two manners:

[0189] (1) Perform feature superposition processing on the tilt angle feature and the shallow iris image feature, that is, superimpose the tilt angle feature and the shallow iris image feature in the feature channel dimension. For example, if the number of feature channels of the tilt angle feature is 1 and the number of feature channels of the shallow iris image feature is 127, then the number of feature channels of the fused image feature obtained by superimposing the two is 128.

[0190] (2) Taking each eigenvalue in the tilt angle feature as a weight, weighted processing is performed on the image eigenvalues ​​at the corresponding position in the shallow iris image feature to obtain the fused image feature.

[0191] See also Figure 7 As shown, it is a schematic diagram of feature fusion provided in an embodiment of the present application, wherein: Figure 7 In the figure, only one feature channel is used as an example. Each feature value in the tilt angle feature can be used as the weight of the feature value at the corresponding position in the shallow iris image feature. The two can be multiplied to obtain the fused image feature, such as Figure 7 The first eigenvalue in the upper left corner is multiplied by 0.2 to obtain the fused image feature. It can be seen that the features of some areas can be amplified through weights. Then, in subsequent predictions, more attention will be paid to the features of these areas, thereby improving the accuracy of iris recognition.

[0192] (3) Using each feature value in the tilt angle feature as a weight, weight the image feature value at the corresponding position in the shallow iris image feature, and then add it to the shallow iris image feature to obtain the fused image feature. Figure 7 The processing shown is similar, the two are multiplied in position and then added to the shallow iris image features, so as to retain more feature details and avoid feature loss.

[0193] (4) Adopting a feature fusion method based on the attention mechanism. Specifically, the tilt angle feature and the shallow iris image feature can be cross-correlated to obtain the attention weights corresponding to each image feature value in the shallow iris image feature. Each attention weight represents the correlation between the tilt angle feature and the corresponding image feature value. Then, based on the attention weights corresponding to each image feature value, the shallow iris image feature is subjected to feature weighting processing based on the attention mechanism to obtain the fused image feature.

[0194] Step 405: Perform deep feature extraction on the fused image features to obtain deep iris image features corresponding to the original iris image.

[0195] In an embodiment of the present application, deep feature extraction can adopt a feature extraction method similar to shallow feature extraction, and it can also be implemented using any possible feature extraction network, such as CNN or a derivative network combined with CNN.

[0196] It should be noted that the definition of deep layer is similar to that of shallow layer mentioned above, which is a relative concept. Generally speaking, deep feature dimension pays more attention to the global features of the image, and its definition can also be configured according to a preset dimension threshold. As mentioned above, after the feature dimensions of the feature dimension sequence are sorted according to the feature size, the deep feature dimension can set a dimension threshold from it, and the dimension with a size smaller than the dimension threshold is the deep feature dimension.

[0197] Step 406: Identify the wearer based on the deep iris image features, and run the target application based on the obtained identification result.

[0198] Specifically, the identity recognition process can be implemented using a fully connected layer and an output layer. The fully connected layer fuses the previously fused feature data, and the output layer generates a classification result, which represents the probability distribution of the input iris image belonging to each category.

[0199] In a possible implementation, the identity recognition process can be achieved by comparing iris features. That is, an identity information library is pre-built to store the identity information and iris features of each wearer. When the deep iris image features are obtained, they can be compared with the identity information library in the identity information library, that is, the similarity between the iris features in the library and the deep iris image features is calculated, and the wearer with the highest similarity and meeting the similarity threshold requirement is found, that is, the wearer can be considered as the final identity recognition result.

[0200] In a possible implementation, the identity recognition process can also be implemented by classification. For example, the iris features in the identity information database and the deep iris image features can be fused to predict the probability of whether the two belong to the same wearer. When the probability meets the requirements, for example, the probability value is greater than a preset threshold and it is the maximum probability value in the identity information database, the wearer corresponding to the probability value is considered to be the final identity recognition result.

[0201] In a possible implementation, each wearing object in the identity information database can also be considered as a category, so that the probability of the current wearing object belonging to each category can be predicted based on the deep iris image features, and the category with the highest probability can be determined, and the corresponding wearing object can be considered as the final identity recognition result.

[0202] In an embodiment of the present application, the operation of the target application can be controlled based on the obtained identity recognition result. For example, when the identity information of the current worn object is recognized from the identity information library, that is, the identity of the current worn object is legal, the target application is run with its identity information; or when the identity information of the current worn object is not recognized from the identity information library, that is, the identity of the current worn object is illegal, the current worn object is denied to use the target application.

[0203] In the embodiment of the present application, the use of the tilt angle information can be determined based on the confidence of the identity recognition. When the confidence of the identity recognition is high, it indicates that the credibility of the current identity recognition result is high, and there is no need to combine the tilt angle information for additional verification. Figure 8 As shown, it is a schematic diagram of another method flow provided in an embodiment of the present application, which may include the following steps.

[0204] Step 801: obtaining an original iris image of a person wearing a VR device, and obtaining tilt angle information of the iris in the original iris image.

[0205] Step 802: performing identity recognition on the wearer based on the original iris image, and obtaining a confidence score of the corresponding identity recognition result.

[0206] Step 803: Determine the interval range in which the confidence score is located, that is, it is necessary to determine whether the confidence score meets the identity recognition certainty condition. The identity recognition certainty condition means that the identity of the wearer can be determined more accurately, that is, to verify whether the current identity recognition result is sufficiently reliable. When the confidence score is high, it is considered that the current identity recognition result is sufficiently reliable.

[0207] Step 804: If the confidence score is within the suspected score interval, feature mapping is performed on the tilt angle information to obtain the tilt angle feature of the target feature dimension.

[0208] Among them, the suspected score interval refers to the ability to identify the identity of the current wearer, but the credibility of the current identity recognition result is slightly low. In order to improve the accuracy of identity recognition, the embodiment of the present application provides a logic based on the tilt angle information for verification, that is, identity recognition is performed again by combining the tilt angle information in the above manner to improve the confidence of identity recognition.

[0209] Step 805: Perform shallow feature extraction based on the original iris image to obtain shallow iris image features of target feature dimensions.

[0210] Step 806: Fusing the tilt angle feature with the shallow iris image feature to obtain a fused image feature.

[0211] Step 807: Perform deep feature extraction on the fused image features to obtain deep iris image features corresponding to the original iris image.

[0212] Step 808: Identify the wearer based on the deep iris image features, and run the target application based on the obtained identification result.

[0213] Step 809: If the confidence score is greater than the upper limit of the suspected score interval, indicating that the credibility of the current identity recognition result is high, the target application is run with the identity recognition result.

[0214] Step 810: If the confidence score is less than the lower limit of the suspected score interval, indicating that the credibility of the current identity recognition result is extremely low, the wearer is denied access to the function of the target application, for example, the wearer may be prompted that the identity is illegal or requires secondary verification.

[0215] The above steps and Figure 4 The implementation methods of the corresponding processes in the shown processes are the same, so they will not be repeated here. Please refer to the introduction of the above processes.

[0216] In the embodiment of the present application, considering the environmental influence when the VR device collects the iris image, such as shaking, poor light, etc., the image quality of the original iris image collected may be poor. Therefore, in order to avoid the problem of inaccurate identity recognition caused by poor image quality, the embodiment of the present application also provides an identity recognition method based on super-resolution reconstructed images. This method requires super-resolution reconstruction of the original iris image based on the trained image generator model, so here we first introduce the training process of the model. Fig. 9 FIG. 1 is a flow chart of a training method for an image generator model provided in an embodiment of the present application. The method may include the following steps:

[0217] Step 901: Obtain a sample iris image set, where the sample iris image set includes sample iris images of the second resolution acquired at various tilt angles.

[0218] In the embodiment of the present application, the second resolution refers to high resolution, and the collection of the sample iris image can be performed by Figure 2The image acquisition and storage module shown in the figure is used to realize the image acquisition and storage module. The main task of the image acquisition and storage module is to capture clear iris images at different angles and record and store the acquired images to provide basic data for the subsequent iris recognition and verification process. Among them, the acquisition of iris images in VR devices mainly depends on the built-in high-resolution camera and optical system. In order to acquire clear iris images, the following points need to be considered:

[0219] (1) Lighting conditions

[0220] Lighting has a great impact on the quality of iris images, so the camera in VR devices should have good low-light performance. For example, during the acquisition process, infrared light sources or other light sources with specific wavelengths can be used to fill in the light to ensure image clarity.

[0221] (2) Focus and stabilization conditions

[0222] Camera focus and stability are the key to capturing clear iris images. Therefore, during the acquisition process, the camera needs to automatically adjust the focus distance to ensure that the iris is always clear. In addition, the camera should also be equipped with optical image stabilization or electronic image stabilization to reduce the impact of device shaking on image quality.

[0223] (3) Iris image collection at different tilt angles

[0224] In order to fully utilize the tilt angle verification function, iris images need to be captured at different angles. This can be achieved by using multiple cameras or a single camera to capture multiple frames of images. During the acquisition process, the VR device can automatically adjust the camera's shooting angle according to the user's head posture and eye position to capture iris images at different angles.

[0225] In practical applications, the collected iris images need to be stored for subsequent processing. VR devices can save image data in built-in memory or upload it to a cloud server via a wireless network. During the storage process, a data compression algorithm can be used to compress the image to save storage space. At the same time, in order to protect user privacy, iris image data should be encrypted.

[0226] Step 902: performing image degradation processing on each sample iris image based on the degradation model to obtain a degraded iris image set, wherein the degraded iris image set includes degraded iris images of a first resolution.

[0227] In the embodiment of the present application, the process of step 902 may be performed by Figure 2The iris image preprocessing module shown in the figure is used to construct a test data set from the previously collected sample iris images of the second resolution, including low-resolution image and high-resolution image pairs. The goal of this module is to provide training data for subsequent super-resolution reconstruction by simulating the degradation process of iris images in reality.

[0228] The iris image preprocessing module performs image degradation processing on each sample iris image based on the degradation model to obtain a degraded iris image of a first resolution, where the first resolution is lower than the second resolution. The image degradation methods used by the degradation model include but are not limited to blur convolution kernels, downsampling operators, and random noise. A sample iris image can obtain more degraded iris images through different degradation models or their combination, so as to provide a rich training data foundation for subsequent training.

[0229] For example, suppose there is a high-resolution iris image A, and the corresponding low-resolution image B is generated through the degradation model. Fig.10a As shown, a blur convolution kernel H, such as a Gaussian blur kernel, can be selected to simulate the blur in the iris image; then a downsampling operator D, such as bilinear interpolation, can be selected to reduce the image size; finally, random noise n, such as Gaussian noise, can be added to the image to simulate the noise in the image acquisition process. After degradation processing, a low-resolution image can be obtained, which can be expressed as:

[0230] B=H(D(A))+n

[0231] Take the following sampling calculation as an example, see Fig.10b As shown, for a 4x4 pixel original image A, it is downsampled to a 2x2 pixel image B. Here, bilinear interpolation is used as the downsampling operator. Fig.10b The left image in the figure is the original image A of 4x4 pixels. To reduce the size of the original image by half and the target size is 2x2 pixels, the image can be divided into small 2x2 blocks, and then the corresponding interpolation in each small block is calculated. The block 1 in the upper left corner (1, 2, 5, 6) calculates the interpolation (1+2+5+6) / 4, that is, 3.5, the block 2 in the upper right corner (3, 4, 7, 8) calculates the interpolation (3+4+7+8) / 4, that is, 5.5, the block 3 in the lower left corner (9, 10, 13, 14) calculates the interpolation (9+10+13+14) / 4, that is, 11.5, and the block 4 in the upper right corner (11, 12, 15, 16) calculates the interpolation (11+12+15+16) / 4, that is, 13.5. The interpolation obtained by the above calculation is put into the 2x2 pixel image B to get the downsampling result. Of course, this is a simplified example of the downsampling process, and in real applications the image size will be much larger and more elaborate interpolation calculations may be required.

[0232] Step 903: construct a plurality of sample image pairs based on each sample iris image and the degraded iris image corresponding to each sample iris image, where each sample image pair includes a sample iris image and a degraded iris image.

[0233] Then, the image generator model can be iteratively trained multiple times based on the constructed multiple sample image pairs until the iteration termination condition is met, thereby obtaining a trained image generator model. It should be noted that since the training process is similar each time, a training process is taken as an example for introduction here.

[0234] In an embodiment of the present application, an image generator model is used to convert a low-resolution image into a high-resolution image. The model can adopt any model architecture based on super-resolution reconstruction technology, such as a generative adversarial network (GAN) architecture, a convolutional network architecture, and a derivative network architecture based on a convolutional network.

[0235] See also Fig.11 As shown, it is a structural schematic diagram of an image generator model provided by an embodiment of the present application, wherein the model includes multiple dense blocks, each of which may include multiple convolutional layers and activation functions for feature extraction and image restoration, and the residual network establishes short-circuit connections between dense blocks, thereby solving the gradient vanishing problem by adding short-circuit connections, and these short-circuit connections allow low-level feature information to be combined with high-level feature information, thereby improving the learning ability of the overall network, and at the same time, the residual network enables the network to learn new features based on the input features, thereby achieving higher quality image reconstruction, and the model also uses a self-attention mechanism, allowing the model to establish dependencies between input sequences at different positions in order to focus on high-level information. In the generator module, the self-attention mechanism helps to capture long-distance contextual information, thereby generating finer image details.

[0236] The following is an introduction to the processing of the image generator model using a possible network architecture. Here, an image generator model with an encoder-decoder architecture is taken as an example, which can first extract features from the input degraded iris image and restore the extracted features to obtain the corresponding reconstructed iris image.

[0237] Step 904: Perform feature extraction on each of the input degraded iris images through the feature extraction network of the image generator model used this time, and obtain image feature sets corresponding to each degraded iris image. Each image feature set includes: image features of the corresponding degraded iris image in multiple feature dimensions.

[0238] Specifically, feature extraction can be implemented in combination with the residual network mentioned above, and can also be implemented in combination with the self-attention mechanism when performing feature extraction. The self-attention mechanism can refer to an attention mechanism based on the current image features themselves, or it can refer to a cross-level attention mechanism, that is, the attention mechanism between the current image features and the upper-level image features.

[0239] In a possible implementation, the feature extraction network extracts features by performing feature extraction on each degraded iris image, and then obtaining a corresponding image feature set through a multi-level feature extraction process. Since the feature extraction process of each level of each degraded iris image is similar, the first-level extraction is specifically used as an example for introduction.

[0240] See also Fig.12a As shown, through the convolution feature extraction subnetwork, the image features obtained by the upper feature extraction are subjected to convolution feature extraction to obtain convolution image features. When this is the first extraction, the image features obtained by the upper feature extraction are the degraded iris image, and when this is not the first extraction, the image features obtained by the upper feature extraction are the previous output. In addition, the image features obtained by the upper feature extraction can be subjected to downsampling processing through the downsampling subnetwork to obtain downsampled image features, and then the image features output by the feature extraction of this level can be obtained based on the convolution image features and the downsampled image features.

[0241] Among them, the fusion of convolution image features and down-sampled image features can be performed by operations such as pooling, superposition or concat, and the embodiments of the present application do not limit this.

[0242] Step 905: Perform image feature restoration based on each image feature set obtained through the image restoration network of the image generator model used this time, and generate a reconstructed iris image corresponding to the second resolution for each degraded iris image.

[0243] Through the above-mentioned step-by-step feature extraction, each level can output corresponding image features, thereby forming an image feature set. Then, when performing image feature restoration, correspondingly, for each image feature set, the image features obtained at the last level can be restored step by step until a reconstructed iris image is obtained. Since the feature restoration process of each level of each degraded iris image is similar, the first-level restoration process is specifically introduced here as an example.

[0244] For details, see Figure 12bAs shown in FIG. 1 , the image features outputted at the previous level can be upsampled by the upsampling subnetwork to obtain the upsampled image features. When this is the first feature restoration process, the image features outputted at the previous level are the image features obtained by the last feature extraction, and when this is not the first extraction, the image features outputted at the previous level are the outputs of the last feature restoration process. Then, the upsampled image features are superimposed with the image features corresponding to this level to obtain the image features outputted at this level.

[0245] In the embodiment of the present application, the superposition processing process can be implemented in combination with the attention mechanism.

[0246] Specifically, the upsampled image features are superimposed with the image features corresponding to the current level to obtain the superimposed image features, and then the superimposed image features can be interactively processed based on the self-attention mechanism based on the attention matrix set to obtain the image features output by the current level. The attention matrix set can be obtained during the training process, and in the training process, it can be assigned values ​​randomly or based on experience at the beginning, and continuously updated iteratively during the training process.

[0247] Among them, the attention matrix set is usually composed of a key parameter matrix, a value parameter matrix and a query parameter matrix. When processing the superimposed image features, the corresponding query matrix, key matrix and value matrix can be obtained by multiplying the three matrices with the superimposed image features respectively, and then the attention matrix is ​​obtained based on the similarity between the query matrix and the key matrix, and the image features of the output at this level are obtained by weighting the attention matrix and the value matrix.

[0248] Step 906: Generate a model loss value of the image generator model based on the similarity between each obtained reconstructed iris image and the corresponding sample iris image.

[0249] Theoretically, when the reconstructed iris image is accurate enough, it should be the same as the sample iris image. Therefore, the accuracy of the image generator model can be measured by the similarity between the two. This process can be achieved by Figure 2 The discriminator module shown in the figure is implemented. The main task of the discriminator module is to determine whether the high-resolution image generated by the image generator model is close to the original high-resolution image. Its architecture is usually constructed using a convolutional neural network. For example, a convolutional neural network can be used to extract features from the reconstructed iris image and the corresponding sample iris image, respectively, and then by calculating the differences between these features, the discriminator module can evaluate the performance of the image generator model.

[0250] Specifically, the discriminator module uses a loss function to measure the difference between the reconstructed iris image generated by the image generator model and the corresponding sample iris image, that is, the discriminator module calculates the model loss value and updates the parameters of the image generator model and the discriminator through back propagation to minimize the loss. The model loss value can be calculated using any possible loss function, such as the cross entropy loss function (Cross Entropy Loss Function), the mean squared error (MSE) loss function, the squared absolute error loss function, the maximum likelihood loss (Likelihood Loss, LHL) function, etc. Of course, it can also be other possible loss functions, which are not limited in the embodiments of the present application.

[0251] In a possible implementation, the peak signal-to-noise ratio (PSNR) may be used to measure the image restoration quality. PSNR is a commonly used image quality evaluation index used to measure the similarity between the original image and the reconstructed image. The higher the PSNR value, the closer the quality of the reconstructed image is to the original image. PSNR may be calculated using the following formula:

[0252] PSNR=10*log 10 (MAX 2 / MSE)

[0253] Wherein, MAX represents the maximum value of the image pixel value, for example, for an 8-bit image, MAX=255, and MSE is the mean square error, which represents the mean value of the square of the difference between each pixel value of the original image and the reconstructed image.

[0254] Exemplarily, when there is an original iris image A (high resolution) and an iris image B (high resolution) reconstructed by super-resolution, the PSNR values ​​of the two images can be calculated by the following process.

[0255] First, when A and B are both 8-bit grayscale images with a size of 512x512, traverse each pixel position, calculate the square of the difference between the pixel values, and then find the average:

[0256] MSE = (1 / (512*512))*Σ(Σ(A ij -B ij ) 2 )

[0257] Here, i and j represent the row and column coordinates of the pixel, respectively.

[0258] Then, the PSNR value is calculated using the above PSNR formula. For an 8-bit image, MAX = 255, which can be expressed as:

[0259] PSNR=10*log 10 (255 2 / MSE)

[0260] Then, after obtaining the PSNR value, it can be used to evaluate the quality of the reconstructed iris image B. Generally speaking, a PSNR value above 30 dB indicates good image quality, while a PSNR value above 40 dB indicates very good image quality.

[0261] Step 907: Determine whether the iteration termination condition is met.

[0262] In the embodiment of the present application, the iteration termination condition may include at least one of the following conditions:

[0263] (1) The number of iterations reaches the set threshold.

[0264] (2) The model loss value is less than the set loss threshold, or the PSNR index meets the preset requirements.

[0265] Step 908: If the determination result of step 907 is no, the model parameters are adjusted based on the model loss value.

[0266] In the embodiment of the present application, when the number of iterations does not exceed the preset number threshold, and the model loss value is not less than the set loss threshold, the determination result of step 907 is no, that is, it is considered that the current image generator model does not meet the iteration termination condition, and the model parameters need to be adjusted to continue training. After the parameter adjustment, the next iterative training process is entered, that is, jumping to step 904.

[0267] Once the image generator model is trained, it can be used in iris recognition in real scenarios. Fig.13 As shown, it is another flow chart of an identity recognition method in a VR scene provided in an embodiment of the present application, which may include the following steps.

[0268] Step 1301: Obtain an original iris image of a person wearing a VR device, and obtain tilt angle information of the iris in the original iris image.

[0269] Step 1302: Based on the trained image quality assessment model, perform image quality assessment on the original iris image to obtain a corresponding image quality assessment value.

[0270] Among them, the image quality assessment model is used to evaluate the quality of the iris image. It can be trained using training samples with labeled image quality, or a general image quality assessment model can be used, or image quality can be assessed in a statistical manner for the image, such as by statistically analyzing image quality indicators. The embodiments of the present application do not limit this.

[0271] Step 1303: If the image quality evaluation value is less than a preset evaluation value threshold, based on the trained image generator model, super-resolution reconstruction is performed on the original iris image to obtain a reconstructed iris image; wherein a second resolution of the reconstructed iris image is higher than a first resolution of the original iris image.

[0272] That is, in actual scenarios, the image quality of the collected original iris image can be evaluated. If the image quality is poor, the image generator model can be used to first perform super-resolution reconstruction on the original iris image, and then the subsequent recognition process can be performed based on the reconstructed iris image, thereby improving the image quality of the iris image and further improving the recognition accuracy.

[0273] It should be noted that, in practical applications, it is also possible not to evaluate the image quality, but to directly use the image generator model to perform super-resolution reconstruction, that is, super-resolution reconstruction is performed on all original iris images by default.

[0274] Step 1304: Perform feature mapping on the tilt angle information to obtain the tilt angle feature of the target feature dimension.

[0275] Step 1305: Perform shallow feature extraction on the original iris image to obtain shallow iris image features.

[0276] Step 1306: Fusing the tilt angle feature with the shallow iris image feature to obtain a fused image feature.

[0277] Step 1307: Perform deep feature extraction on the fused image features to obtain deep iris image features corresponding to the original iris image.

[0278] Step 1308: Identify the wearer based on the deep iris image features, and run the target application based on the obtained identification result.

[0279] The above steps and Figure 4 The implementation methods of the corresponding processes in the shown processes are the same, so they will not be repeated here. Please refer to the introduction of the above processes.

[0280] Below, the scheme of the embodiment of the present application is introduced by taking the access scenario of VR games as an example. For example, in a VR game scenario, iris recognition is required to verify the identity of the current wearer. Due to lighting conditions and angle problems, the quality of the collected iris image is poor, resulting in the inability of traditional iris recognition methods to obtain reliable recognition results. At this time, the user's iris image can be captured in advance through a VR device, and the iris tilt angle information can be recorded, and the collected high-resolution iris image can be pre-processed, including downsampling and test set construction, etc., and then the generator module is used to convert the low-resolution iris image into a high-resolution image. The discriminator module can evaluate the quality of the high-resolution image output by the generator module, and calculate the loss function, and then reversely optimize the generator module.

[0281] When identity recognition is required, the iris recognition module can perform identity recognition based on the high-resolution iris image output by the generator module. If the recognition score is not enough, the tilt angle information can be used as supplementary information to enhance the specific recognition score. Finally, the system determines the user's identity based on the iris recognition results and decides whether to allow the user to enter the game.

[0282] In summary, the embodiments of the present application provide a method for iris recognition verification for VR devices based on super-resolution and shallow features of iris images at tilted angles. The method aims to solve the problem of poor recognition effect caused by image quality and angle problems during iris recognition. By using super-resolution technology and supplementary information of iris images at tilted angles, the features of iris images can be enhanced when the recognition score is insufficient, thereby improving the accuracy of iris recognition.

[0283] See also Fig.14 Based on the same inventive concept, the embodiment of the present application further provides an identity recognition device 140 in a VR scene, the device comprising:

[0284] The data acquisition unit 1401 is used to obtain an original iris image of a person wearing a VR device, and to obtain the tilt angle information of the iris in the original iris image; wherein the original iris image is collected when the target application running on the VR device needs to identify the identity of the person wearing the VR device;

[0285] The feature mapping unit 1402 is used to perform feature mapping on the tilt angle information to obtain the tilt angle feature of the target feature dimension, where the tilt angle feature is used to characterize: the feature weight of each image area in the original iris image;

[0286] A feature extraction unit 1403 is used to perform shallow feature extraction based on the original iris image to obtain shallow iris image features of target feature dimensions;

[0287] The feature fusion unit 1404 is used to fuse the tilt angle feature with the shallow iris image feature, and perform deep feature extraction on the obtained fused image feature to obtain the deep iris image feature corresponding to the original iris image;

[0288] The identity recognition unit 1405 is used to perform identity recognition on the wearer based on the deep iris image features, and run the target application based on the obtained identity recognition result.

[0289] In a possible implementation manner, the data obtaining unit 1401 is specifically configured to:

[0290] Extract the tilt angle of the iris in the original iris image to obtain tilt angle information; or,

[0291] The shooting angle information when the image acquisition device acquires the original iris image is obtained, and the tilt angle information is determined based on the position information of the image acquisition device in the VR device and the shooting angle information.

[0292] In a possible implementation, the apparatus further includes a quality assessment unit 1406 and a super-resolution reconstruction unit 1407;

[0293] The quality assessment unit 1406 is used to perform image quality assessment on the original iris image based on the trained image quality assessment model to obtain a corresponding image quality assessment value;

[0294] The super-resolution reconstruction unit 1407 is used to perform super-resolution reconstruction on the original iris image based on the trained image generator model to obtain a reconstructed iris image if the image quality evaluation value is less than a preset evaluation value threshold; wherein the second resolution of the reconstructed iris image is higher than the first resolution of the original iris image;

[0295] The feature extraction unit 1403 is specifically used to perform shallow feature extraction on the original iris image to obtain shallow iris image features.

[0296] In a possible implementation manner, the super-resolution reconstruction unit 1407 is further configured to:

[0297] Obtaining a sample iris image set, the sample iris image set comprising sample iris images of the second resolution acquired at a variety of tilt angles;

[0298] performing image degradation processing on each sample iris image based on the degradation model to obtain a degraded iris image set, wherein the degraded iris image set includes degraded iris images of a first resolution;

[0299] Based on each sample iris image and the degraded iris image corresponding to each sample iris image, construct a plurality of sample image pairs, each sample image pair including a sample iris image and a degraded iris image;

[0300] Based on multiple sample image pairs, the image generator model is iteratively trained multiple times to obtain a trained image generator model.

[0301] In a possible implementation manner, the super-resolution reconstruction unit 1407 is specifically configured to:

[0302] In each training iteration, the following operations are performed:

[0303] Through the feature extraction network of the image generator model used this time, feature extraction is performed on each input degraded iris image to obtain an image feature set corresponding to each degraded iris image, each image feature set including: image features of the corresponding degraded iris image in multiple feature dimensions;

[0304] By using the image restoration network of the image generator model used this time, image feature restoration is performed based on each image feature set obtained, so as to generate a reconstructed iris image corresponding to the second resolution for each degraded iris image;

[0305] Based on the similarity between each reconstructed iris image and the corresponding sample iris image, the parameters of the image generator model used this time are adjusted.

[0306] In a possible implementation manner, the super-resolution reconstruction unit 1407 is specifically configured to:

[0307] Performing feature extraction on each degraded iris image level by level to obtain a corresponding image feature set; wherein the feature extraction of each level of each degraded iris image includes the following steps:

[0308] Through the convolutional feature extraction sub-network, the image features obtained by the upper-level feature extraction are subjected to convolutional feature extraction to obtain convolutional image features;

[0309] Through the downsampling sub-network, the image features obtained by the previous level feature extraction are downsampled to obtain downsampled image features;

[0310] Based on the convolution image features and the down-sampled image features, the image features output by the feature extraction at this level are obtained.

[0311] In a possible implementation manner, the super-resolution reconstruction unit 1407 is specifically configured to:

[0312] For each image feature set, starting from the image features obtained at the last level, image feature restoration is performed level by level until a reconstructed iris image is obtained; wherein image feature restoration at each level includes the following steps:

[0313] Through the upsampling subnetwork, the image features outputted at the previous level are upsampled to obtain upsampled image features;

[0314] The upsampled image features are superimposed with the image features corresponding to this level to obtain the image features output by this level.

[0315] In a possible implementation manner, the super-resolution reconstruction unit 1407 is specifically configured to:

[0316] Superimpose the upsampled image features with the image features corresponding to the current level to obtain superimposed image features;

[0317] Based on the attention matrix set, the superimposed image features are interactively processed based on the self-attention mechanism to obtain the image features of the current level output.

[0318] In a possible implementation manner, the feature fusion unit is specifically configured to:

[0319] The tilt angle feature and the shallow iris image feature are fused by any of the following processing methods:

[0320] Perform feature superposition processing on the tilt angle feature and the shallow iris image feature;

[0321] Taking each feature value in the tilt angle feature as a weight, weighted processing is performed on the image feature value at the corresponding position in the shallow iris image feature to obtain a fused image feature;

[0322] Taking each feature value in the tilt angle feature as a weight, weighting the image feature value at the corresponding position in the shallow iris image feature is performed, and then added to the image feature value at the corresponding position in the shallow iris image feature to obtain a fused image feature;

[0323] The tilt angle feature and the shallow iris image feature are cross-correlated to obtain the attention weights corresponding to each image feature value in the shallow iris image feature, and based on the attention weights corresponding to each image feature value, the shallow iris image feature is subjected to feature weighting processing based on the attention mechanism to obtain the fused image feature; wherein each attention weight represents the correlation between the tilt angle feature and the corresponding image feature value.

[0324] In a possible implementation, the feature mapping unit 1402 is specifically configured to:

[0325] Perform identity recognition on the wearer based on the original iris image and obtain a confidence score of the corresponding identity recognition result;

[0326] If the confidence score is within the suspected score interval, feature mapping is performed on the tilt angle information to obtain the tilt angle feature of the target feature dimension.

[0327] The above device solves the problem of poor recognition effect caused by image quality and angle problems during iris recognition. By using super-resolution technology and supplementary information of iris images with tilted angles, the features of iris images can be enhanced when the recognition score is insufficient, thereby improving the accuracy of iris recognition.

[0328] The device can be used to execute the methods shown in the various embodiments of the present application. Therefore, for the functions that can be implemented by the various functional modules of the device, reference can be made to the description of the aforementioned embodiments and no further details will be given.

[0329] See also Fig.15 Based on the same technical concept, the present application also provides a computer device. In one embodiment, the computer device can be Figure 1b The VR server shown in FIG. Fig.15 As shown, it includes a memory 1501 , a communication module 1503 and one or more processors 1502 .

[0330] The memory 1501 is used to store computer programs executed by the processor 1502. The memory 1501 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and programs required to run various functions of the embodiment of the present application, etc.; the data storage area may store various instant messaging information and operation instruction sets, etc.

[0331] The memory 1501 may be a volatile memory, such as a random-access memory (RAM); the memory 1501 may also be a non-volatile memory, such as a read-only memory, a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD); or the memory 1501 may be any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 1501 may be a combination of the above memories.

[0332] The processor 1502 may include one or more central processing units (CPU) or a digital processing unit, etc. The processor 1502 is used to implement the identity recognition method in the above VR scene when calling the computer program stored in the memory 1501.

[0333] The communication module 1503 is used to communicate with terminal devices and other servers.

[0334] The specific connection medium between the memory 1501, the communication module 1503 and the processor 1502 is not limited in the embodiment of the present application. Fig.15 In the embodiment, the memory 1501 and the processor 1502 are connected via a bus 1504. The bus 1504 is connected to the processor 1502 via a bus 1504. Fig.15 The connections between the other components are described with bold lines, which are only for illustration and are not intended to be limiting. The bus 1504 can be divided into an address bus, a data bus, a control bus, etc. For ease of description, Fig.15 The diagram shows that only one thick line is used, but this does not mean that there is only one bus or only one type of bus.

[0335] The memory 1501 stores a computer storage medium, and the computer storage medium stores computer executable instructions. The computer executable instructions are used to implement the identity recognition method in the VR scene of the embodiment of the present application. The processor 1502 is used to execute the identity recognition method in the VR scene of the above-mentioned embodiments.

[0336] In another embodiment, the computer device may also be a terminal device, such as Figure 1a or Figure 1b In this embodiment, the structure of the computer device can be as follows: Fig.16 As shown, it includes: a communication component 1610, a memory 1620, a display unit 1630, a camera 1640, a sensor 1650, an audio circuit 1660, a Bluetooth module 1670, a processor 1680 and other components.

[0337] The communication component 1610 is used to communicate with the server. In some embodiments, a circuit wireless fidelity (Wireless Fidelity, WiFi) module may be included. The WiFi module belongs to a short-range wireless transmission technology. The computer device can help the user to send and receive information through the WiFi module.

[0338] The memory 1620 can be used to store software programs and data. The processor 1680 executes various functions and data processing of the terminal device by running the software programs or data stored in the memory 1620. The memory 1620 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device. The memory 1620 stores an operating system that enables the terminal device to run. In the present application, the memory 1620 can store an operating system and various application programs, and can also store code for executing the identity recognition method in the VR scene of the embodiment of the present application.

[0339] The display unit 1630 can also be used to display information input by the user or information provided to the user and a graphical user interface (GUI) of various menus of the terminal device. Specifically, the display unit 1630 uses a display screen 1632 based on VR technology, and can display the relevant display pages in the embodiment of the present application based on VR display technology.

[0340] Camera 1640 can be used to capture iris images. Camera 1640 can be one or more. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then transmits the electrical signal to the processor 1680 to convert it into a digital image signal.

[0341] The terminal device may further include at least one sensor 1650, such as an acceleration sensor 1651, a distance sensor 1652, a fingerprint sensor 1653, and a temperature sensor 1654. The terminal device may also be configured with other sensors such as a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, a light sensor, and a motion sensor.

[0342] The audio circuit 1660, the speaker 1661, and the microphone 1662 can provide an audio interface between the user and the terminal device. The audio circuit 1660 can transmit the electrical signal converted from the received audio data to the speaker 1661, which is converted into a sound signal for output. The terminal device can also be equipped with a volume button for adjusting the volume of the sound signal. On the other hand, the microphone 1662 converts the collected sound signal into an electrical signal, which is received by the audio circuit 1660 and converted into audio data, and then the audio data is output to the communication component 1610 to be sent to, for example, another terminal device, or the audio data is output to the memory 1620 for further processing.

[0343] The Bluetooth module 1670 is used to exchange information with other Bluetooth devices having Bluetooth modules through the Bluetooth protocol. For example, the terminal device can establish a Bluetooth connection with a wearable computer device (such as a smart watch) that also has a Bluetooth module through the Bluetooth module 1670 to exchange data.

[0344] The processor 1680 is the control center of the terminal device. It uses various interfaces and lines to connect various parts of the entire terminal. It executes various functions of the terminal device and processes data by running or executing software programs stored in the memory 1620 and calling data stored in the memory 1620. In some embodiments, the processor 1680 may include one or more processing units; the processor 1680 may also integrate an application processor and a baseband processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the baseband processor mainly processes wireless communications. It is understandable that the above-mentioned baseband processor may not be integrated into the processor 1680. In the present application, the processor 1680 can run the operating system, application programs, user interface display and touch response, as well as the identity recognition method in the VR scene of the embodiment of the present application. In addition, the processor 1680 is coupled to the display unit 1630.

[0345] Based on the same inventive concept, an embodiment of the present application also provides a storage medium, which stores a computer program. When the computer program runs on a computer, the computer executes the steps of the identity recognition method in a VR scene according to various exemplary embodiments of the present application described above in this specification.

[0346] In some possible implementations, various aspects of the identity recognition method in a VR scene provided by the present application may also be implemented in the form of a computer program product, which includes a computer program. When the program product is run on a computer device, the computer program is used to enable the computer device to execute the steps of the identity recognition method in a VR scene according to various exemplary embodiments of the present application described above in this specification. For example, the computer device may execute the steps of each embodiment.

[0347] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0348] The program product of the embodiment of the present application may adopt a portable compact disk read-only memory (CD-ROM) and include a computer program, and can be run on a computer device. However, the program product of the present application is not limited thereto. In the present application, the readable storage medium may be any tangible medium containing or storing a program, and the computer program included therein may be used by or in combination with a command execution system, apparatus or device.

[0349] A readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, wherein a readable computer program is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. A readable signal medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with a command execution system, apparatus, or device.

[0350] The computer program embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0351] Computer programs for performing the operations of the present application may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" language or similar programming languages.

[0352] It should be noted that, although several units or subunits of the device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more units described above can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided into multiple units to be embodied.

[0353] In addition, although the operations of the method of the present application are described in a specific order in the drawings, this does not require or imply that the operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0354] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0355] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0356] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for identity recognition in a virtual reality (VR) scene, characterized in that: The method comprises: Obtaining an original iris image of a person wearing a VR device, and obtaining information about the tilt angle of the iris in the original iris image; wherein the original iris image is collected when a target application running on the VR device needs to identify the identity of the person wearing the VR device; Performing feature mapping on the tilt angle information to obtain a tilt angle feature of a target feature dimension, wherein the tilt angle feature is used to characterize: a feature weight of each image area in the original iris image; Performing shallow feature extraction based on the original iris image to obtain shallow iris image features of the target feature dimension; Performing feature fusion on the tilt angle feature and the shallow iris image feature, and performing deep feature extraction on the obtained fused image feature, so as to obtain the deep iris image feature corresponding to the original iris image; The wearable object is identified based on the deep iris image features, and the target application is run based on the obtained identification result.

2. The method according to claim 1, characterized in that Obtaining the tilt angle information of the iris in the original iris image, comprising: Extracting the tilt angle of the iris in the original iris image to obtain the tilt angle information; or, The shooting angle information when the image acquisition device acquires the original iris image is obtained, and the tilt angle information is determined based on the position information of the image acquisition device in the VR device and the shooting angle information.

3. The method according to claim 1, characterized in that Based on the shallow feature extraction, shallow iris image features of the target feature dimension are obtained, including: Based on the trained image quality assessment model, performing image quality assessment on the original iris image to obtain a corresponding image quality assessment value; If the image quality evaluation value is less than a preset evaluation value threshold, based on the trained image generator model, super-resolution reconstruction is performed on the original iris image to obtain a reconstructed iris image; wherein a second resolution of the reconstructed iris image is higher than a first resolution of the original iris image; Perform shallow feature extraction on the original iris image to obtain the shallow iris image features.

4. The method according to claim 3, characterized in that Before performing super-resolution reconstruction on the original iris image based on the trained image generator model to obtain the reconstructed iris image, the method further includes: Obtaining a sample iris image set, wherein the sample iris image set includes sample iris images of the second resolution acquired at a plurality of tilt angles; performing image degradation processing on each sample iris image based on the degradation model to obtain a degraded iris image set, wherein the degraded iris image set includes degraded iris images of a first resolution; Based on the sample iris images and the degraded iris images corresponding to the sample iris images, construct a plurality of sample image pairs, each sample image pair including a sample iris image and a degraded iris image; Based on the multiple sample image pairs, the image generator model is iteratively trained multiple times to obtain the trained image generator model.

5. The method according to claim 4, characterized in that In each training iteration, the following operations are performed: Through the feature extraction network of the image generator model used this time, feature extraction is performed on each of the input degraded iris images to obtain image feature sets corresponding to each of the degraded iris images, each of which includes: image features of the corresponding degraded iris image in multiple feature dimensions; Performing image feature restoration based on each obtained image feature set through the image restoration network of the image generator model used this time, and generating a reconstructed iris image corresponding to the second resolution for each of the degraded iris images; Based on the similarity between each reconstructed iris image and the corresponding sample iris image, the parameters of the image generator model used this time are adjusted.

6. The method according to claim 5, characterized in that Through the feature extraction network of the image generator model used this time, feature extraction is performed on each of the input degraded iris images to obtain image feature sets corresponding to each of the degraded iris images, including: Performing feature extraction on each of the degraded iris images step by step to obtain a corresponding image feature set; wherein the feature extraction of each level of each degraded iris image includes the following steps: Through the convolutional feature extraction sub-network, the image features obtained by the upper-level feature extraction are subjected to convolutional feature extraction to obtain convolutional image features; Through the downsampling sub-network, the image features obtained by the previous level feature extraction are downsampled to obtain downsampled image features; Based on the convolution image features and the down-sampled image features, image features output by feature extraction at this level are obtained.

7. The method according to claim 5, characterized in that The image feature restoration is performed based on each image feature set obtained by using the image restoration network of the image generator model used this time, so as to generate a reconstructed iris image corresponding to the second resolution for each of the degraded iris images, including: For each image feature set, starting from the image features obtained at the last level, image feature restoration is performed level by level until the reconstructed iris image is obtained; wherein image feature restoration at each level includes the following steps: Through the upsampling subnetwork, the image features outputted at the previous level are upsampled to obtain upsampled image features; The up-sampled image features are superimposed with the image features corresponding to the current stage to obtain the image features output by the current stage.

8. The method according to claim 5, characterized in that The up-sampled image features are superimposed with the image features corresponding to the current stage to obtain the image features output by the current stage, including: Superimposing the upsampled image features with the image features corresponding to the current level to obtain superimposed image features; Based on the attention matrix set, the superimposed image features are interactively processed based on the self-attention mechanism to obtain the image features of the current level output.

9. The method according to any one of claims 1 to 8, characterized in that: The tilt angle feature and the shallow iris image feature are fused by any of the following processing methods: Performing feature superposition processing on the tilt angle feature and the shallow iris image feature; Taking each feature value in the tilt angle feature as a weight, weighted processing is performed on the image feature value at the corresponding position in the shallow iris image feature to obtain the fused image feature; Taking each feature value in the tilt angle feature as a weight, weighting the image feature value at the corresponding position in the shallow iris image feature is performed, and then the weighted image feature value is added to the image feature value at the corresponding position in the shallow iris image feature to obtain the fused image feature; The tilt angle feature and the shallow iris image feature are cross-correlated to obtain the attention weights corresponding to the respective image feature values ​​in the shallow iris image feature, and based on the attention weights corresponding to the respective image feature values, the shallow iris image feature is subjected to feature weighting processing based on the attention mechanism to obtain the fused image feature; wherein each attention weight represents the correlation between the tilt angle feature and the corresponding image feature value.

10. The method according to any one of claims 1 to 8, characterized in that: Before performing feature mapping on the tilt angle information to obtain the tilt angle feature of the target feature dimension, the method further includes: Performing identity recognition on the wearer based on the original iris image to obtain a confidence score of the corresponding identity recognition result; If the confidence score is within the suspected score interval, feature mapping is performed on the tilt angle information to obtain the tilt angle feature of the target feature dimension.

11. An identity recognition device in a VR scene, characterized in that: The device comprises: A data acquisition unit, used to obtain an original iris image of a person wearing a VR device, and to obtain information about the tilt angle of the iris in the original iris image; wherein the original iris image is collected when a target application running on the VR device needs to identify the identity of the person wearing the VR device; A feature mapping unit, used for performing feature mapping on the tilt angle information to obtain a tilt angle feature of a target feature dimension, wherein the tilt angle feature is used to characterize: a feature weight of each image area in the original iris image; A feature extraction unit, configured to perform shallow feature extraction based on the original iris image to obtain shallow iris image features of the target feature dimension; A feature fusion unit, used for performing feature fusion on the tilt angle feature and the shallow iris image feature, and performing deep feature extraction on the obtained fused image feature, so as to obtain a deep iris image feature corresponding to the original iris image; The identity recognition unit is used to perform identity recognition on the wearable object based on the deep iris image features, and run the target application based on the obtained identity recognition result.

12. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 10 are implemented.

13. A computer storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 10 are implemented.

14. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 10 are implemented.

Images