Data acquisition method and device

By acquiring and processing TOF data using a TOF camera, and combining trusted execution environment and exposure parameter optimization, the security, accuracy, and speed issues in facial recognition technology have been resolved, achieving higher security and faster recognition speed.

CN117014727BActive Publication Date: 2024-11-26HONOR DEVICE CO LTD
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
CN202310804077.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-08-12
Publication Date
2024-11-26
Estimated Expiration
2041-08-12

AI Technical Summary

Technical Problem

Existing facial recognition technology has shortcomings in terms of security, accuracy, and speed, especially in the face of malicious attacks, poor image quality, and excessive latency.

Method used

A TOF camera is used to acquire TOF data. By controlling the switch of the TOF light source, projection off data and infrared data are collected. The difference between the infrared data and the projection off data is used to correct overexposure. Combined with TOF image processing, face recognition is performed in a trusted execution environment to optimize exposure parameters and light intensity adaptation, thereby improving data quality and processing speed.

Benefits of technology

It improves the security and accuracy of facial recognition, reduces processing time, and enhances the user experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117014727B_ABST
    Figure CN117014727B_ABST
Patent Text Reader

Abstract

The present application provides a data acquisition method and device, which acquires a first frame of TOF data including projection-off data and infrared data, and after determining that there is a data block in the infrared data where the number of data points with a value greater than a first threshold is greater than a second threshold, the TOF data used to generate the first frame of TOF image is acquired based on the difference between the infrared data and the projection-off data. Because the data block where the number of data points with a value greater than the first threshold is greater than the second threshold represents an overexposed data block, and the projection-off data is the TOF data collected by the TOF camera when the TOF light source is turned off, the difference between the infrared data and the projection-off data can correct the overexposure, thereby improving the quality of the first frame of TOF data. The first frame of TOF data with higher quality is used for facial recognition, which not only has higher security, but also has higher accuracy and execution speed.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] This application is a divisional application of the invention patent application with application number 202110925831.X submitted to the China Patent Office, application date August 12, 2021, and invention name “Data Acquisition Method and Device”. Technical Field

[0002] The present application relates to the field of electronic information, and in particular to a data acquisition method and device. Background Art

[0003] Facial recognition is widely used in electronic devices to identify authorized users. For example, for the face unlocking function, whether to unlock the screen depends on whether the facial recognition is passed.

[0004] In view of the above scenarios, in terms of security, we need to consider the problem of malicious attacks. In terms of accuracy, we need to consider the problem of inaccurate recognition results due to low image quality. In terms of user experience, we need to consider the problem of latency.

[0005] It can be seen that the problem that needs to be solved is how to achieve facial recognition safely, accurately, and quickly. Summary of the invention

[0006] The present application provides a data acquisition method and device, the purpose of which is to solve the problem of how to achieve facial recognition safely, accurately, and quickly.

[0007] In order to achieve the above objectives, this application provides the following technical solutions:

[0008] The first aspect of the present application provides a data acquisition method, including: acquiring a first frame of time-of-flight TOF data, the first frame of TOF data including projection-off data and infrared data, the projection-off data being TOF data collected by a TOF camera when the TOF light source is turned off, determining that there is a data block that meets a preset condition in the infrared data, the preset condition including that the number of data points in the data block whose values ​​are greater than a first threshold is greater than a second threshold, and acquiring TOF data for generating a first frame of TOF image based on the difference between the infrared data and the projection-off data. Because the data block that meets the preset condition is an overexposed data block, and the projection-off data is TOF data collected by the TOF camera when the TOF light source is turned off, the difference between the infrared data and the projection-off data can correct the overexposure to improve the quality of the first frame of TOF data. The first frame of TOF data with higher quality is used for facial recognition, which not only has higher security, but also has higher accuracy and execution speed.

[0009] Optionally, acquiring the first frame of TOF data includes: collecting the projection-off data after the TOF camera turns off the TOF light source; and collecting the infrared data after the TOF camera turns on the TOF light source. By controlling the turning off and on of the TOF light source of the TOF camera, collecting the projection-off data in the form of infrared data, respectively, it is easy to implement.

[0010] Optionally, before collecting the projection off data, the method further includes: collecting depth data for generating a depth image when the TOF camera turns on the TOF light source. The infrared image generated by the depth image and the infrared data can be used for anti-counterfeiting recognition in facial recognition, thereby improving the security of facial recognition.

[0011] Optionally, the timing of controlling the TOF camera to turn off the TOF light source is determined according to the first exposure time used to collect the depth data. This can not only ensure that the depth data is collected, but also minimize the delay in collecting the first frame of TOF data.

[0012] Optionally, the timing of controlling the TOF camera to turn on the TOF light source is determined according to the exposure time for collecting the projection-off data, which can not only ensure that the projection-off data is collected, but also minimize the delay in collecting the first frame of TOF data.

[0013] Optionally, the exposure duration for collecting the projection-off data is the first exposure duration for collecting the depth data, or the second exposure duration for collecting the infrared data.

[0014] Optionally, the TOF camera turns off the TOF light source, including: controlling the TOF light source to turn off through the TOF sensor of the TOF camera; the TOF camera turns on the TOF light source, including: controlling the TOF light source to turn on through the TOF sensor of the TOF camera. Using the TOF sensor to control the turning off and on of the TOF light source, that is, controlling hardware through hardware, can achieve faster execution speed.

[0015] Optionally, it further includes: determining that the data block does not exist in the infrared data, and processing the first frame of TOF data collected into a first frame of TOF image. If there is no data block that meets the preset conditions in the infrared data, it means that the infrared data is not overexposed and can be directly processed into a TOF image.

[0016] Optionally, the method further includes: using the TOF data for generating the first frame of TOF image to generate a first frame of TOF image; and performing facial recognition using the first frame of TOF image to obtain a recognition result. Using the first frame of TOF image to obtain a facial recognition result can improve the execution speed of facial recognition.

[0017] Optionally, the first frame of TOF data includes: a facial recognition frame.

[0018] Optionally, before acquiring the first frame of TOF data, the method further includes: determining that the human eye is safe when the TOF light source is turned on based on the safety indication frame collected before collecting the first frame of TOF data. The facial recognition frame is used for processing to obtain TOF data for generating the first frame of TOF image, and before processing, the safety of the human eye is determined using the safety indication frame, and by classifying the types of TOF data, the safety of the human eye is ensured under the premise of using TOF data.

[0019] Optionally, it also includes: determining, based on the safety indication frame, that human eyes are unsafe when the TOF light source is turned on, and controlling the TOF camera to turn off, so as to avoid damage to human eyes by the infrared light of the TOF camera.

[0020] Optionally, the specific implementation method of obtaining the first frame of TOF data is: through the kernel layer, the first frame of TOF data collected by the TOF camera is stored in the trusted execution environment TEE. The specific implementation method of determining that there is a data block that meets the preset conditions in the infrared data, and obtaining the TOF data used to generate the first frame of TOF image based on the difference between the infrared data and the projection-off data is: in the TEE, determining that there is a data block that meets the preset conditions in the infrared data, and obtaining the TOF data used to generate the first frame of TOF image based on the difference between the infrared data and the projection-off data. The above specific implementation method realizes the purpose of processing TOF data in TEE to improve the security of TOF data.

[0021] The second aspect of the present application provides a data acquisition method, including: collecting TOF data using exposure parameters at a first moment, and collecting a first frame of TOF data using exposure parameters at a second moment whose interval with the first moment is within a preset range, wherein the first moment is earlier than the second moment. Because the interval between the first moment and the second moment is within the preset range, the preset range can be set so that the interval between the second moment and the first moment is not long, so that the external environment for collecting the first frame of TOF data is similar to that for collecting TOF data at the first moment, so that the first frame of TOF data is collected at the second moment using the exposure parameters of the first moment to improve the possibility of adapting the exposure parameters to the environment, thereby obtaining a first frame of TOF data with higher quality, which is beneficial to improving the accuracy and speed of facial recognition under the premise of using TOF data for facial recognition to obtain higher security.

[0022] Optionally, before using the exposure parameters to collect the second TOF data, the method further includes: obtaining the moment when the TOF data was last collected before the second moment as the first moment; and determining that the interval between the second moment and the first moment is within the preset range. Taking the moment when the TOF data was last collected before the second moment as the first moment can ensure the minimum calculation cost, so it is beneficial to save computing resources.

[0023] Optionally, the exposure parameter is an exposure parameter adjusted according to an automatic exposure AE result. The exposure parameter adjusted according to the AE result has a higher degree of adaptability to the ambient light, so it is conducive to obtaining higher quality TOF data.

[0024] The third aspect of the present application provides a data acquisition method, including: obtaining the light intensity of the environment in which the electronic device is located, and using the exposure time corresponding to the light intensity to collect the first frame of TOF data. Based on the light intensity of the environment in which the electronic device is located, the exposure time for collecting the first frame of TOF data is obtained, so it is beneficial to collect TOF data with brightness that meets the requirements of facial recognition in the first frame, thereby improving the accuracy and speed of facial recognition.

[0025] Optionally, the process of acquiring the exposure time corresponding to the light intensity includes: confirming a target interval in each preset light intensity interval, where the target interval is the interval to which the light intensity of the environment in which the electronic device is located belongs; and acquiring the exposure time corresponding to the target interval.

[0026] Optionally, each light intensity interval includes an indoor light intensity interval and an outdoor light intensity interval. The division of the indoor light intensity interval and the outdoor light intensity interval lays a foundation for the refinement of the distinction granularity.

[0027] Optionally, the larger the value in each light intensity interval is, the shorter the corresponding exposure time is, so as to follow the principle that the longer the exposure time is, the higher the brightness of the image is, thereby ensuring the rationality of controlling the image quality by the exposure time.

[0028] The fourth aspect of the present application provides an electronic device, comprising: a TOF camera, a memory, and a processor. The TOF camera is used to collect a first frame of TOF data, wherein the first frame of TOF data includes projection-off data and infrared data, and the projection-off data is TOF data collected by the TOF camera when the TOF light source is turned off. The memory is used to store program code. The processor is used to run the program code to implement the data acquisition method provided in the first aspect, the second aspect, and the third aspect.

[0029] The fifth aspect of the present application provides a chip system, comprising: at least one processor and an interface, the interface being used to receive code instructions and transmit them to the at least one processor; the at least one processor runs the code instructions to implement the data acquisition method provided in the first aspect, the second aspect or the third aspect.

[0030] A sixth aspect of the present application provides a readable storage medium having a program stored thereon, and when the program is read and executed by a computing device, the data acquisition method provided by the first aspect, the second aspect or the third aspect is implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1a An example diagram of a scenario in which facial recognition is performed using an electronic device;

[0032] Figure 1b This is an example diagram of a scene in which an electronic device is used for facial recognition under strong light;

[0033] Figure 2 A schematic diagram of the structure of an electronic device disclosed in an embodiment of the present application;

[0034] Figure 3 A schematic diagram of a software framework for implementing facial recognition in an electronic device disclosed in an embodiment of the present application;

[0035] Figure 4 A flowchart of a facial recognition method disclosed in an embodiment of the present application;

[0036] Figure 5 A flowchart of another facial recognition method disclosed in an embodiment of the present application;

[0037] Figure 6 A flowchart of another facial recognition method disclosed in an embodiment of the present application;

[0038] Figure 7 A flowchart of a data processing method disclosed in an embodiment of the present application;

[0039] Figure 8 A flowchart of another data processing method disclosed in an embodiment of the present application;

[0040] Fig. 9 for Figure 8 The specific implementation flow chart of S819;

[0041] Fig.10 A flowchart of a data acquisition method disclosed in an embodiment of the present application;

[0042] Fig.11 A flowchart of another data acquisition method disclosed in an embodiment of the present application;

[0043] Fig.12 A flowchart of another data acquisition method disclosed in an embodiment of the present application;

[0044] Fig.13 A diagram of another software framework for implementing facial recognition in electronic devices. DETAILED DESCRIPTION

[0045] Figure 1a In the scenario of facial recognition being applied in electronic devices, when a user uses an electronic device such as a mobile phone to implement a function that requires user authorization, the user voluntarily or based on a prompt message from the electronic device, points the camera of the electronic device at the face. Electronic devices are currently usually equipped with RGB (Red Green Blue) cameras. After the electronic device captures the image through the RGB camera, it compares the image with the stored facial template and performs anti-counterfeiting identification. After obtaining the recognition result, the task is performed based on the recognition result.

[0046] For example, if a user intends to use a face to unlock the screen, the user will turn the phone screen toward the face so that the front camera of the phone can capture the facial image. After capturing the image, the phone will perform facial recognition on the image. If the recognition is successful, the screen will be unlocked. If the recognition is not successful, the screen will remain locked.

[0047] During the research process, the inventors found that the security of electronic devices when using recognition results to identify authorized users needs to be improved: currently malicious attacks on the face can be mainly divided into plane attacks (such as using photos to impersonate real faces), head mask attacks (such as using 3D models to impersonate real faces) and similar facial imitation.

[0048] During the research, the inventors also found that using time-of-flight (TOF) data for facial recognition has the following characteristics: 1. TOF data can generate depth images and infrared images, so it has good anti-counterfeiting performance (i.e., preventing plane attacks and head model mask attacks). 2. TOF data is less affected by light, so the quality of TOF data is higher, so a higher threshold value can be used for face comparison, thereby reducing the possibility of counterfeiting of similar faces. It can be seen that using TOF data for facial recognition can improve security.

[0049] However, the current mainstream hardware platforms only support the processing of RGB data, but not TOF data. For example, the image signal processor (ISP) in the current mainstream processors only supports the processing of RGB camera raw data (Camera Raw Data) collected by RGB cameras into RGB raw data (Raw Data), but cannot process TOF data (such as Camera Raw Data) collected by TOF cameras. The cost of redesigning the hardware platform to process TOF data collected by TOF cameras is too high. Therefore, it is necessary to use a software framework to process TOF data collected by TOF cameras. Compared with hardware, software is more vulnerable to attacks. In this case, even if TOF data is used for facial recognition, there are still security vulnerabilities.

[0050] The following embodiments of the present application provide a facial recognition method, which is applied to an electronic device, with the purpose of improving the security of facial recognition.

[0051] Figure 2 This is an example diagram of an electronic device, including: a TOF camera 1, a processor 2, a memory 3, and an I / O subsystem 4.

[0052] The TOF camera 1 is used to collect TOF data. In some implementations, the TOF camera 1 is configured as a front camera of an electronic device, and is used to collect the display screen ( Figure 2 For example, the TOF data of a face located in front of the display screen of the electronic device is collected.

[0053] The TOF camera 1 includes a TOF sensor 11 , a TOF sensor controller 12 , a TOF light source 13 , and a TOF light source controller 14 .

[0054] In some embodiments, the TOF light source controller 14 is controlled by the TOF sensor controller 12 to control the TOF light source 13. The TOF light source 13 emits infrared (IR) light under the control of the TOF light source controller 14. The TOF sensor 11 is used to sense the infrared (IR) light reflected from an object such as a face to collect TOF data.

[0055] The TOF sensor controller 12 and the TOF light source controller 14 are disposed in the I / O subsystem 4 , and communicate with the processor 2 through the I / O subsystem 4 .

[0056] The memory 3 can be used to store computer executable program codes. Specifically, the memory 3 may include a program storage area and a data storage area. The program storage area may store program codes required to implement an operating system, a software system, or at least one function. The data storage area may store data acquired, generated, and used during the use of the electronic device.

[0057] In some implementations, all or part of the memory 3 may be integrated into the processor 2 as the internal memory of the processor 2. In some implementations, the memory 3 is an external memory relative to the processor 2 and communicates with the processor 2 through the external memory interface of the processor 2.

[0058] In some implementations, the memory 3 may include a high-speed random access memory and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0059] The processor 2 may include one or more processing units. For example, the processor 2 may include an application processor (AP), a graphics processing unit (GPU), an image signal processor (ISP), etc.

[0060] In some implementations, the processor 2 may include a secure area processor 21, and the data involved in facial recognition may be stored in the secure area processor 21, and the secure area processor 21 executes the processing flow of these data by calling the program code in the memory 3 to improve the security of facial recognition.

[0061] In some implementations, the secure area processor 21 is used to store the TOF data collected by the TOF camera 1, generate a TOF image using the TOF data, and perform facial recognition processing using the TOF image to obtain a recognition result. The processor 2 can use the recognition result to perform tasks such as face unlocking or face payment.

[0062] Optionally, the electronic device may further include an ambient light sensor ( Figure 2 The ambient light sensor is used to sense the light intensity of the environment in which the electronic device is located. It can be understood that the ambient light sensor is provided in the I / O subsystem 4 through the ambient light sensor controller ( Figure 2 (not shown) communicates with processor 2.

[0063] It is to be understood that the structure illustrated in this embodiment does not constitute a specific limitation on the electronic device. In other embodiments, the electronic device may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0064] It is understandable that the operating system implemented by the processor 2 by running the code stored in the memory 3 may be an iOS operating system, an Android open source operating system, a Windows operating system, a Hongmeng operating system, etc. In the following embodiments, the Android open source operating system will be taken as an example for description.

[0065] The processor 2 of the electronic device runs the program code stored in the memory 3 to achieve Figure 3 The software framework based on the Android open source operating system is shown:

[0066] The software framework includes a Trusted Execution Environment (TEE) and a Rich Execution Environment (REE) that can execute rich instructions. TEE can be implemented by the secure area processor 21 in the processor 2 by running program code.

[0067] REE includes various layers of the Android open source operating system, including but not limited to: application layer, application framework layer, hardware abstraction layer and kernel layer.

[0068] The application layer can include a series of application packages. Figure 3 As shown, the application package may include a payment application and a lock screen application. Each application may initiate a task request in different scenarios to trigger facial recognition.

[0069] The application framework layer provides application programming interface (API) and programming framework for the applications in the application layer. The application framework layer includes some predefined functions. Figure 3 As shown, the application framework layer may include a camera service and a face service. The camera service is used to implement the camera function. The face service is used to implement the face recognition function.

[0070] The hardware abstraction layer is used to abstract the hardware. It hides the hardware interface details of a specific platform and provides a virtual hardware platform for the operating system, making it hardware-independent and portable on multiple platforms. The camera hardware abstraction layer (Camera HAL3) in the hardware abstraction layer is used to respond to the instructions of the Camera service and control the camera. The face client application (Face Client application, Face CA, also known as the face client application) accesses the trusted application of TEE by calling the API (Application Programming Interface) of the TEE client located in REE, thereby using the security functions provided by TEE and trusted applications.

[0071] The kernel layer is the layer between hardware and software. The hardware driver software is usually set in the kernel layer. Figure 3 In the embodiment, one of the functions of ISP-Lite in the kernel layer is to drive ISP-Lite (part of ISP) in the processor to convert TOF data (such as TOF camera Raw Data) collected by the TOF sensor into a data format commonly used for image processing, such as TOF raw data (Raw Data). In the following embodiments, the data processed by ISP-Lite and the TOF data collected by the TOF camera are collectively referred to as TOF data. The TOF camera driver in the kernel layer is used to drive the TOF camera. TEE includes various applications, modules and units for facial recognition based on TOF data. Among them, the role of the Face Trusted Application (Face TA) is to schedule various modules and units in TEE and communicate with REE. The data processing module is used to process data such as Raw Data. The facial recognition module is used to implement functions such as facial recognition using facial templates and facial template management.

[0072] based on Figure 3 In the framework shown, the electronic device can process TOF data into TOF images in TEE, and use the TOF images for facial recognition in TEE to obtain recognition results. That is, on the basis of using TOF data for facial recognition to improve security, data processing and recognition are both implemented in TEE, which can further improve security.

[0073] The following will be combined Figure 3 , the facial recognition method disclosed in the embodiment of the present application is described in detail, such as Figure 4 As shown, the facial recognition method performed by the electronic device disclosed in the embodiment of the present application includes the following steps:

[0074] S401. Transmitting a task request to a face service (Face service) of an application framework layer through an application of the application layer.

[0075] It is understandable that the application can initiate a task and send a task request to the face service in response to at least one instruction, at least one user operation, or at least one sensor signal. Taking the lock screen application as an example, the user presses the power button to trigger the unlock task, and the unlock application sends an unlock request to the Face service.

[0076] S402: Transmitting a task request to the face application client (Face Client application, Face CA) of the hardware abstraction layer through the Face service.

[0077] S403: Transmit an image request to the camera service of the application framework layer through Face CA to respond to the task request.

[0078] S404: Transmit an image request to the camera hardware abstraction layer (Camera HAL3) of the hardware abstraction layer through the Camera service.

[0079] S405, responding to the image request through Camera HAL3, transmitting an image output instruction to the TOF camera driver of the kernel layer to drive the TOF camera to collect TOF data.

[0080] S406, the ISP-Lite of the kernel layer stores the received and processed TOF data in a first secure buffer storage unit (Secure Buffer) of the TEE with first storage information. The first storage information indicates a storage address.

[0081] ISP-Lite stores the received and processed TOF data into TEE, which is equivalent to the TOF data being directly stored from the hardware (ISP of the processor) into the secure area, thus reducing the possibility of being attacked.

[0082] S407 . Transmit the first storage information to Camera HAL3 of the hardware abstraction layer through ISP-Lite of the kernel layer.

[0083] Because the first storage information is transmitted in REE, in order to ensure the security of the first storage information, the first storage information can be encrypted information, that is, ciphertext. In this step, an example of encrypted information is a file descriptor (FD), which is used to describe the storage location and reading method.

[0084] For the convenience of description, the ciphertext of the first storage information is referred to as FD1.

[0085] S408 . Transmit FD1 and calibration data to the Camera service of the application framework layer through Camera HAL3.

[0086] Because calibration data is needed for the subsequent generation of TOF images, Camera HAL3 obtains the calibration data pre-configured in REE and transmits it to TEE together with FD1.

[0087] Optionally, Camera HAL3 may obtain a portion of calibration data from a storage unit of the REE and may obtain another portion of calibration data from a TOF camera.

[0088] S409, transmitting FD1 and calibration data to Face CA of the hardware abstraction layer through the Camera service.

[0089] S410 , transmitting FD1 and calibration data to the Face Trusted Application (Face TA) of TEE through Face CA.

[0090] It can be understood that the information interaction between the modules of each layer in the above REE can comply with the communication protocol between the layers.

[0091] Based on the existing operating system framework, the storage information of TOF data is transmitted to TEE, laying the foundation for processing TOF data in TEE. In addition, ISP-Lite stores the received and processed TOF data in TEE, ensuring the security of TOF data.

[0092] S411. Transmit FD1 to the data processing module of TEE through the Face TA of TEE, and store the calibration data in the calibration data storage unit of TEE.

[0093] S412 , through the data processing module, read the TOF data from the first safety buffer storage unit according to FD1 , and read the calibration data from the calibration data storage unit.

[0094] S413 . Generate a depth image and an infrared image by using the TOF data and the calibration data of the TOF camera through a data processing module.

[0095] S414. Store the TOF image in a second secure buffer storage unit of the TEE using second storage information through the data processing module.

[0096] Because all are implemented in TEE, the second storage information does not need to be encrypted.

[0097] S415. Transmit the second storage information to Face TA through the data processing module.

[0098] S416: Transmit the second stored information to the facial recognition module via Face TA.

[0099] S417 . Read the TOF image from the second security buffer storage unit according to the second storage information through the facial recognition module.

[0100] S418, performing facial recognition on the read TOF image through a facial recognition module to obtain a recognition result.

[0101] Specifically, the facial recognition module performs facial comparison on the TOF image by calling a pre-configured facial template to obtain a comparison result, and uses the TOF image for anti-counterfeiting recognition to obtain an anti-counterfeiting result. The recognition result can be determined based on the comparison result and the anti-counterfeiting result. For example, if the comparison result is a passed comparison and the anti-counterfeiting result is a passed anti-counterfeiting result, then the recognition result is a passed one.

[0102] As mentioned above, because the TOF image includes a depth image and an infrared image, a higher comparison threshold can be used when comparing the infrared image with the face module, so the accuracy is higher, thereby reducing the possibility of similar facial impersonation. And because the depth image and the infrared image can be used for anti-counterfeiting identification at the same time, it has a higher anti-counterfeiting performance (i.e., it has a higher performance in preventing plane attacks and head model mask attacks).

[0103] S419: Transmit the recognition result to Face TA through the facial recognition module.

[0104] S420: Transmit the recognition result to Face CA of REE through Face TA of TEE.

[0105] S421. Transmit the recognition result to the Face service of the application framework layer through Face CA.

[0106] S422. Transmit the recognition result to the application that initiates the task request in the application layer through the Face service.

[0107] Taking the lock screen application at the application layer as an example, after the lock screen application initiates an unlock request, it receives the recognition result, and based on the recognition result indicating that the recognition is successful, it executes the unlocking task. Based on the recognition result indicating that the recognition is unsuccessful, it executes the task of maintaining the locked screen state.

[0108] It is understandable that the process of other applications in the application layer initiating task requests and obtaining facial recognition results is similar and will not be repeated here.

[0109] The facial recognition method described in this embodiment uses TOF data for facial recognition, so it has high security. Furthermore, facial recognition based on TOF data is implemented in TEE, so the security of TOF data, data processing process and recognition process can be guaranteed, thereby further improving the security of facial recognition.

[0110] Moreover, based on the various layers of the Android operating system, the transmission of data storage information, recognition results, and calibration data between REE and TEE is realized, with good compatibility.

[0111] It is understandable that, because the calibration data belongs to camera domain data rather than face domain data, in some implementations, the calibration data can also be transferred from the Camera service to the Face CA via the Face service.

[0112] It is understandable that after the calibration data is stored in the calibration data storage unit of the TEE, as long as the processor is not powered off, the calibration data in the calibration data storage unit of the TEE will not be lost, so there is no need to reload the calibration data. Of course, the calibration data can also be reloaded, which is not limited here. Transmitting the calibration data together with FD1 is only one implementation method, and it can also be transmitted separately, and the transmission path is as described above.

[0113] It can be understood that the TOF camera collects a set of TOF data by exposing once, and after exposing multiple times continuously and collecting multiple sets of TOF data, it stops for the first time length, exposes multiple times continuously again, and then stops for the first time length, and so on... The multiple sets of TOF data collected continuously are called a frame of TOF data, and the multiple sets of TOF data collected again after the first time length interval are another frame of TOF data. Without considering the processing delay and transmission delay of other modules themselves, ISP-Lite also stores each frame of TOF data in the first security buffer storage unit in sequence at the first time length interval, and also transmits the storage information of each frame of TOF data to Face CA in sequence at the first time length interval. Face CA also transmits the storage information of each frame of TOF data to Face TA in sequence at the first time length interval. Therefore, Face TA also receives the storage information of each frame of TOF data in sequence at the first time length interval. Figure 4 S406 in the figure can be used to store any frame of TOF data, S407-S411 can be used to transmit any frame of TOF data, S413-S419 can be used to process the TOF data, and S420-S422 can be used to process the recognition result of the TOF data, or can be used to process the recognition result of multiple frames of TOF data.

[0114] Because TOF cameras collect TOF data by projecting infrared light, it is necessary to pay attention to the safety of human eyes during the collection process. Because the quality of TOF images is related to the accuracy of recognition results, and the exposure parameters of TOF cameras are directly related to the quality of TOF images, it is necessary to adjust the exposure parameters of TOF cameras.

[0115] In the embodiments of the present application, Figure 4 The process shown is further improved, and the data processing module can generate different parameters for adjusting the TOF camera according to the type of TOF data received.

[0116] Figure 5 The following is a flow chart of generating and feeding back the automatic exposure (AE) result and controlling the camera according to the AE result. Figure 5 and Figure 4 The difference:

[0117] S413a, through the data processing module of TEE, in response to the TOF data being a facial recognition frame, generating a depth image, an infrared image and an AE result.

[0118] In an embodiment of the present application, the TOF camera divides the collected TOF data into a safety indication frame and a facial recognition frame. The safety indication frame carries a human eye safety flag to indicate whether the infrared light emitted by the TOF camera is safe for the human eye. The facial recognition frame is a TOF data frame used for facial recognition. In some implementations, the type of the TOF data frame is indicated by at least one of a certain value, character, or string in the TOF data frame.

[0119] In some implementations, in the TOF data collected by the TOF data collection, the first to fourth frames of TOF data are safety indication frames, and the subsequent frames of TOF data are face recognition frames. The safety indication frame includes a human eye safety flag.

[0120] The AE result is used to adjust the exposure parameters of the TOF camera. Optionally, the AE result includes but is not limited to the exposure time and the physical gain of the TOF sensor. One way to generate the AE result is to extract the TOF raw data (Raw Data), that is, the facial area in the original image, calculate the brightness value of the facial area, and compare it with the pre-configured target brightness value to obtain the exposure time and the physical gain of the TOF sensor. One AE result can be generated for each frame of TOF data, or one AE result can be generated for multiple frames of TOF data.

[0121] S415a, transmitting the second storage information and the AE result to the Face TA of the TEE through the data processing module of the TEE.

[0122] It is understandable that, when the AE result of the previous frame of TOF data is the same as the latest AE result, the latest AE result may not be transmitted to save TEE resources.

[0123] S420a, through the Face TA of TEE, transmit the recognition result and AE result to the FaceCA of the hardware abstraction layer of REE.

[0124] S423a, through Face CA, transmit the AE result to the Camera service of the application framework layer of REE.

[0125] S424a, transmit the AE result to Camera HAL3 of the hardware abstraction layer through the Camera service.

[0126] S425a, through Camera HAL3, according to the AE result, adjust the TOF camera through the TOF camera driver.

[0127] Camera HAL3 can send AE results to the TOF sensor controller through the TOF camera driver, and the TOF sensor controller controls the TOF sensor to collect TOF data according to the AE results.

[0128] Figure 6 The following is a process of generating and feeding back a safety mark and controlling the camera according to the safety mark. Figure 6 and Figure 4 The difference:

[0129] S413b: Generate a security identifier through the data processing module of the TEE in response to the TOF data being a security indication frame.

[0130] The safety mark is used to indicate whether the TOF light source is safe for human eyes.

[0131] Optionally, the safety mark is generated in the following manner: after receiving a TOF data frame carrying an eye safety mark (i.e., a safety indication frame), the eye safety mark is extracted, and whether the eye is safe is determined based on the extracted eye mark, and a safety mark is generated. If the eye safety mark indicates safety, a safety mark indicating safety is obtained. If the eye safety mark indicates unsafe, a safety mark indicating unsafe is obtained.

[0132] S415b, transmit the second storage information and the security identifier to FaceTA of TEE through the data processing module of TEE.

[0133] Generally, during the process of starting and ending the acquisition of the TOF camera, the eye safety flag bits carried by each safety indication frame are the same. For example, in the first to fourth safety indication frames, the eye safety flag bits all indicate that the eye is safe, or all indicate that the eye is unsafe. In this case, the multiple safety identifiers obtained are the same, so only one safety identifier can be transmitted to save TEE resources.

[0134] However, it is possible that in some cases, the eye flags in multiple security indication frames are different. Therefore, only the last security flag, that is, the security flag determined based on the last received security indication frame, can be transmitted to save TEE resources.

[0135] S420b, through the Face TA of TEE, transmit the recognition result and security identification to the Face CA of the hardware abstraction layer of REE.

[0136] S423b, transmit the security identifier to the Camera service of the application framework layer of REE through Face CA.

[0137] S424b, transmit the security identifier to Camera HAL3 of the hardware abstraction layer through the Camera service.

[0138] S425b, through Camera HAL3, according to the security identifier, adjust the TOF camera through the TOF camera driver.

[0139] Adjusting the TOF camera includes but is not limited to: if the safety mark indicates that it is not safe for human eyes, then turning off the TOF camera, or reducing the emission intensity of the TOF light source. If the safety mark indicates that it is safe for human eyes, then collecting the TOF data frame and marking it as a TOF data frame for facial recognition (i.e., facial recognition frame).

[0140] based on Figure 2 , Optionally, Camera HAL3 can send a shutdown command to the TOF sensor controller through the TOF camera driver to turn off the TOF camera.

[0141] To summarize, after the electronic device obtains TOF Raw Data through the processor, it processes the TOF Raw Data into a TOF image in the TEE, and uses the TOF image for facial recognition, making facial recognition more secure.

[0142] Moreover, by combining various layers of the Android system, the storage information of TOF data is transmitted to TEE, which not only lays the foundation for processing in TEE, but also achieves compatibility with the Android system.

[0143] It can be seen that even if the processor of the electronic device does not support the generation of TOF images, the above framework and process described in this embodiment can also achieve facial recognition using TOF data, thereby improving security.

[0144] Furthermore, it can also achieve human eye safety protection and adjustment of exposure parameters, thereby improving the performance of electronic devices while improving the security of facial recognition.

[0145] It is understandable that the above Figure 4-Figure 6 Although the Android operating system is used as an example for explanation, the facial recognition method described in the embodiment of the present application is not limited to the Android operating system. For example, the operation of TEE is not limited to the Android operating system, so the functions of each application, module and unit in TEE can also be implemented in other operating systems. The operation of REE is also not limited to the Android operating system, so the transmission path for transmitting TOF data and calibration data to TEE, as well as the transmission path for task requests, the feedback path for recognition results, etc., can all be adapted to different operating systems and adjusted, as long as TOF data and calibration data can be transmitted from REE to TEE, and task requests can trigger the collection of TOF data and the recognition results for executing tasks.

[0146] The inventors found that in addition to security, the accuracy and execution speed of facial recognition also have room for improvement, that is, facial recognition still has the following problems:

[0147] Inaccurate recognition results due to poor image quality: Due to the influence of ambient light, the data collected by the camera may have quality defects such as overexposure, blur, and insufficient brightness. Facial recognition is based on the data collected by the camera, so there may be problems with incorrect comparison results. For example, it is originally the same person, but the face recognition result fails.

[0148] Excessive latency leads to poor user experience: Figure 1a In the scenario shown, if the facial recognition speed is too slow, the user needs to wait too long, which will reduce the user's comfort. Poor image quality is usually the cause of excessive latency.

[0149] Improving image quality and improving execution speed are contradictory: to improve image quality, the image must be iterated, denoised, and processed to make the image converge. The more processing methods are used, the slower the processing speed.

[0150] The inventors also found during the research that TOF imaging has the following characteristics:

[0151] (1) The TOF data collected by the TOF camera is less affected by ambient light, so in most scenarios, the quality of the first frame of TOF data (such as the first frame of facial recognition) collected by the TOF camera can meet the requirements of recognition accuracy. Therefore, in most scenarios, using the first frame of TOF data can obtain a facial recognition result with higher accuracy.

[0152] Let's take the face unlocking service as an example: suppose the face trying to unlock is the face that has been recorded in the electronic device, that is, the face trying to unlock can unlock the screen. In this case, the first frame of TOF data can achieve unlocking.

[0153] (2) When the feedback adjustment mechanism takes effect (the feedback adjustment mechanism refers to using the previously collected data and the camera's calibration data to adjust the exposure parameters of the subsequently collected data to improve the image quality), the TOF data converges (convergence means no longer changing) quickly, and converges in about 3-4 frames of TOF data.

[0154] Continuing with the previous example, assuming that the first frame of TOF data collected is not unlocked, then when the feedback adjustment mechanism is used to adjust the exposure parameters, the collected 3rd or 4th frame of TOF data converges (that is, even if the exposure parameters are adjusted later, the TOF data will no longer change), then the collected 3rd or 4th frame of TOF data can be unlocked.

[0155] Based on the above characteristics of TOF imaging, it is possible to balance accuracy and speed while using TOF data for facial recognition to improve security.

[0156] In the following embodiments, based on the aforementioned method of storing each frame of TOF data in TEE in sequence at a first time interval and transmitting the storage information of each frame of TOF data to TEE, combined with the above-mentioned characteristics of TOF imaging, the data processing module and the facial recognition module are scheduled by Face TA to achieve the purpose of accurately and quickly obtaining recognition results.

[0157] Figure 7 A data processing method disclosed in an embodiment of the present application, and Figure 4 , Figure 5 or Figure 6 Compared with REE, the improvement lies in the applications and modules in TEE. The functions and data transmission methods of each module in REE are the same as those in REE. Figure 4 , Figure 5 or Figure 6 The same, no further elaboration. Figure 7 The process shown in the figure starts after Face TA receives FD1 and calibration data, and includes the following steps:

[0158] S701 , after Face TA receives the FD (abbreviated as FD1 ) of the first frame of TOF data, it transmits FD1 to the data processing module.

[0159] In some implementations, the first frame of TOF data is the first frame of TOF data collected by the TOF camera by the Camera HAL3 in response to an image request and controlling the TOF camera driver.

[0160] In some other implementations, the TOF data frames collected by the TOF camera are marked as safety indication frames and facial recognition frames. The safety indication frame carries a human eye safety flag, which is used to indicate whether the infrared light emitted by the TOF camera is safe for the human eye. The facial recognition frame is a TOF data frame used for facial recognition. In this case, the first frame of TOF data in this step is the first frame of facial recognition frame.

[0161] S702 , the data processing module uses FD1 to read the first frame of TOF data from the first safety buffer storage unit, and reads calibration data from the calibration data storage unit.

[0162] In this embodiment, it is assumed that the calibration data has been stored in the calibration data storage unit.

[0163] S703 , the data processing module generates a TOF image using the first frame of TOF data and the calibration data, and generates an automatic exposure (AE) result.

[0164] As mentioned above, the TOF image includes a depth image and an infrared image.

[0165] The AE result is used by the TOF camera to adjust the exposure parameters for collecting TOF data. The AE result includes but is not limited to the exposure time and the physical gain of the TOF sensor. One way to generate the AE result is to extract the facial area in the TOF raw image (Raw Data), calculate the brightness value of the facial area, and compare it with the pre-configured target brightness value to obtain the exposure time and the physical gain of the TOF sensor.

[0166] For the convenience of description, the TOF image obtained by processing the first frame of TOF data is referred to as the first frame of TOF image, and the AE result obtained by processing the first frame of TOF data is referred to as the first AE result.

[0167] S704: The data processing module stores the first frame of TOF image in the second security buffer storage unit.

[0168] S705: The data processing module transmits the storage information of the first frame of TOF image and the first AE result to Face TA.

[0169] It can be understood that, because the storage information of the first frame of TOF image is only transmitted in TEE, it can be transmitted in plain text.

[0170] S706. Face TA transmits the first AE result to the TOF camera.

[0171] The first AE result is transmitted to the TOF camera, combined with Figure 4 , Figure 5 or Figure 6 The examples shown are: Face TA, Face CA, Camera service, Camera HAL3, and a TOF sensor controller of a TOF camera. The TOF sensor controller then uses the first AE result to adjust parameters of the TOF camera, including but not limited to exposure duration.

[0172] It is understandable that the TOF data frame that the first AE result can act on is related to the time length of the processor processing the TOF data. Taking the 30ms acquisition interval of the TOF camera as an example, assuming that the total time taken by the processor to generate the first TOF image and the first AE result is also 30ms, the first AE result can act on the third frame of TOF data collected by the TOF camera at the earliest.

[0173] Generating and transmitting AE results is an optional step, and its purpose is to improve the quality of the subsequently collected TOF data, thereby obtaining better quality TOF images and more accurate recognition results.

[0174] The first frame of TOF data can be acquired using pre-configured fixed exposure parameters.

[0175] S707. Face TA transmits the storage information of the first frame of TOF image to the facial recognition module to trigger the facial recognition module to perform facial recognition.

[0176] It is understandable that the execution order of S706 and S707 is not limited.

[0177] S708. The facial recognition module uses the storage information of the first frame of TOF image to read the first frame of TOF image from the second security buffer storage unit.

[0178] S709: The facial recognition module uses the first frame of TOF image to perform facial recognition to obtain a first recognition result.

[0179] Combination Figure 3 , the facial recognition module calls the facial template and performs facial comparison on the first frame of TOF image. In addition to facial comparison, depth map and infrared map can also be used for anti-counterfeiting recognition. Therefore, facial recognition includes facial comparison and anti-counterfeiting recognition. If both facial comparison and anti-counterfeiting recognition pass, the recognition result of facial recognition indicates pass.

[0180] S710. The facial recognition module transmits the first recognition result to Face TA.

[0181] S711. Face TA determines whether the first recognition result indicates that the recognition is successful. If yes, execute S712; if no, execute S713.

[0182] S712. Face TA transmits the first recognition result to Face CA.

[0183] As mentioned above, Face CA transmits the first recognition result to the initiator of the task, such as unlocking the application.

[0184] S713. Face TA determines whether the end condition is met. If yes, execute S712; if not, execute S714.

[0185] The termination condition may be preset. In this step, the termination condition includes: the duration of executing the task reaches the second duration threshold. An example of the second duration threshold is 5 seconds. The termination condition is optional, and it is also possible not to determine whether the termination condition is met. If the first recognition result indicates that the recognition fails, S714 is executed.

[0186] It can be seen that the purpose of setting the end condition is to avoid unnecessary time consumption during task execution: if facial recognition still fails after a certain period of time, it can be basically determined that the facial recognition result is determined to be failed, for example, the face of the person trying to unlock is not pre-stored in the electronic device. So there is no need to continue facial recognition. Instead, the facial recognition result should be fed back to the task as soon as possible to reduce the delay of task execution and ensure a better user experience.

[0187] S714. Face TA receives a frame of TOF data from the TOF camera again.

[0188] Taking the TOF camera's acquisition interval of 30ms, the time taken to generate the first frame of TOF image and the first AE result is also 30ms as an example. In this case, the next frame of TOF data received from the TOF camera is the seventh frame of TOF data acquired by the TOF camera.

[0189] It is understandable that, because the first AE can act on the third frame of TOF data at the earliest, the seventh frame of TOF data is a TOF data frame collected after adjusting the exposure parameters using the first AE result.

[0190] After receiving the TOF data frame again, the subsequent process to be executed by Face TA can be referred to S701-S714, that is, the first frame of TOF data as the processing object in S701-S714 is replaced with a frame of TOF data received again, such as the seventh frame of TOF data, and the processing result of the first frame of TOF data is adaptively replaced with the processing result of the frame of TOF data received again (for example, the first frame of TOF image is replaced with the second frame of TOF image), which will not be repeated here.

[0191] Figure 7 The process shown has at least the following differences compared to the existing facial recognition process: the image data processing process in the prior art includes the step of iteratively processing the image data to obtain converged image data, with the aim of obtaining an image whose quality meets the recognition accuracy requirements.

[0192] Taking RGB data as an example, because RGB data is easily affected by ambient light, the quality of the first frame of RGB data collected usually cannot meet the requirements of recognition accuracy. Therefore, it is necessary to iteratively process the RGB data to obtain converged RGB data, and then use the converged RGB data to perform facial recognition to ensure the accuracy of the recognition results. Usually, based on different iterative algorithms, RGB data can converge in about 10 frames at the fastest and 30-40 frames at the slowest. Therefore, the consensus of technicians in this field is that the first frame of RGB data is likely to be of poor quality due to the influence of ambient light, and it is meaningless to use it directly for facial recognition.

[0193] In this embodiment, based on the above TOF imaging principle (1), in most scenarios, the first frame of TOF data converges, so it is known that the first frame of TOF data can most likely obtain a relatively accurate recognition result. Therefore, after the electronic device collects the first frame of TOF data, the facial recognition result is obtained by using the first frame of TOF image generated by the first frame of TOF data. Therefore, a smaller processing delay can be obtained while ensuring that the recognition result has a high accuracy. In other words, based on the above principle of TOF imaging, the step of iterating image data in the hope of obtaining converged image data is omitted in exchange for an increase in the image data processing speed.

[0194] Furthermore, the inventors found in their research that the interval for a TOF camera to collect TOF data and the time it takes for a processor to generate a TOF image are usually 30ms, and the time it takes for face recognition is usually 150ms, so the time it takes to get the recognition result using TOF data is: 30ms+150ms=180ms. Under the same conditions, this value can show that the method described in this embodiment can make the user feel the unlocking speed significantly faster, so it can improve the user experience.

[0195] Because the second frame of TOF data is collected using the exposure parameters determined by the first AE result, the probability of convergence is higher. Therefore, even if the first frame of TOF data fails to be recognized due to quality issues, the second frame of TOF data received can also be recognized successfully. The time taken is: (30ms+150ms)*2=360ms, which is still advantageous compared to the time it takes for RGB data to converge.

[0196] A more specific example of a scenario in which this embodiment is applied is:

[0197] Limited by the hardware conditions of the mobile phone, there is only one processor (such as Figure 2 The secure area processor 21) shown can be used to implement facial recognition.

[0198] Assuming that the face of the person trying to unlock is the face that has been recorded in the electronic device, after the user points the face to the mobile phone screen, the TOF camera collects TOF data at intervals of 30ms and transmits it to the processor. After receiving the first frame of TOF data, the processor processes the first frame of TOF data into a first frame of depth image and a first frame of infrared image according to the process described in this embodiment, and uses the first frame of depth image and the first frame of infrared image to obtain a first recognition result. In the process of the processor generating the first frame of TOF image and calculating the recognition result, because the processor is occupied, although the TOF camera is still transmitting TOF data frames to the processor, the processor can no longer receive subsequent TOF data frames. That is to say, except for the first frame of TOF data received, other TOF data frames are discarded.

[0199] It can be seen that this embodiment does not consider the process of the TOF data iteration process, which just fits the scenario of the above-mentioned one processor. That is to say, in a scenario where only one processor is used for TOF data processing and facial recognition, if both recognition processing and TOF data iteration are to be performed to wait for the convergence of TOF data, the processor must process them serially, so it takes too long. The method described in this embodiment can shorten the time consumption because it does not perform the iteration step of TOF data. Because TOF data converges in the first frame in most cases, it can shorten the time consumption of facial recognition and ensure that the recognition result is relatively accurate.

[0200] To sum up, the data method described in this embodiment, based on the characteristic that TOF data converges with the first frame with a high probability, ensures that facial recognition can be performed accurately and quickly in most cases. Because TOF data is used for facial recognition with high security, it can achieve the purpose of facial recognition safely, accurately, and quickly.

[0201] During the research, the inventors further discovered that in outdoor scenes, because natural light also contains light with a wavelength close to that of infrared light emitted by the TOF light source, the first frame of TOF data collected may be overexposed. In this case, the quality of the first frame of TOF data is not enough to obtain accurate recognition results. As in the above example, when the first frame cannot be unlocked, the required unlocking delay is at least 360ms, and this delay can be further reduced.

[0202] Another data processing method disclosed in the embodiments of the present application is as follows: Figure 8 As shown:

[0203] S801-S806 are the same as S701-S706. For details, please refer to Figure 8 , I will not go into details here.

[0204] S807-S812 are the processing flow of the second frame of TOF data after receiving the storage information of the second frame of TOF data. The processing flow is the same as that of the first frame of TOF data. For details, please refer to Figure 8 , I will not go into details here.

[0205] It is understandable that the second frame of TOF data may be a TOF data frame collected by the TOF camera using the exposure parameters adjusted based on the first AE result, or it may have the same exposure parameters as the first frame of TOF data. Which of the above situations is specific depends on the interval at which the TOF camera collects the TOF data frame (such as the aforementioned first time interval) and the time from the processor receiving the TOF data frame to feeding back the AE result.

[0206] Assuming that the interval between the TOF camera collecting and transmitting data is 30ms, and the time taken by the processor to process the TOF data into a TOF image and obtain the AE result is also 30ms, the first AE result cannot be used for the collection of the second frame of TOF data.

[0207] S813-S818 are the processing flow of the third frame of TOF data after receiving the storage information of the third frame of TOF data. The processing flow is the same as that of the first frame of TOF data. For details, please refer to Figure 8 , I will not go into details here.

[0208] It can be seen from the above process that the difference between this embodiment and the above embodiment is that the first three frames of TOF data are received and processed.

[0209] S819. Face TA calls the facial recognition module to perform facial recognition.

[0210] An optional implementation of S819 is as follows Fig. 9 As shown:

[0211] S901. Face TA transmits the storage information of the third frame TOF image to the facial recognition module.

[0212] S902: After the facial recognition module uses the storage information of the third frame of TOF image to read the third frame of TOF image, the facial recognition module uses the third frame of TOF image to perform facial recognition to obtain a recognition result of the third frame of TOF image.

[0213] It can be understood that the facial recognition module reads the third frame of TOF image from the second security buffer storage unit. Fig. 9 I will not go into details here.

[0214] For the sake of distinction, the recognition result of the third frame image is referred to as the third recognition result.

[0215] S903. The facial recognition module transmits the third recognition result to Face TA.

[0216] S904. Face TA determines whether the third recognition result indicates that the recognition is passed. If yes, execute S905; if no, execute S906.

[0217] S905. Face TA transmits the third recognition result to Face CA.

[0218] Because the third frame of TOF data is most likely collected after adjusting the exposure parameters using the AE results, the quality is most likely to be optimal. Therefore, using the third frame of TOF image for facial recognition first can further reduce the time consumption.

[0219] S906. Face TA transmits the storage information of the first frame TOF image and the storage information of the second frame TOF image to the facial recognition module.

[0220] S907: The facial recognition module obtains a first recognition result and a second recognition result.

[0221] The first recognition result is the result of performing facial recognition on the first frame of TOF image. The second recognition result is the result of performing facial recognition on the second frame of TOF image.

[0222] It can be understood that the order of using the first frame of TOF image and the second frame of TOF image for facial recognition is not limited.

[0223] S908. The facial recognition module transmits the first recognition result and the second recognition result to Face TA.

[0224] S909: If at least one of the first recognition result and the second recognition result indicates a pass, the Face TA transmits the recognition result indicating a pass to the TOF CA.

[0225] S910: If both the first recognition result and the second recognition result indicate that the recognition fails, Face TA determines whether the termination condition is met. If yes, execute S911; if no, execute again. Figure 8 The process shown.

[0226] The definition of the end condition can refer to the above embodiment. In this step, the duration threshold in the end condition can be shorter than the duration threshold of the end condition in the above embodiment, because in this embodiment, multiple frames of TOF data have been collected and processed. In this step, an example of the duration threshold is 3 seconds.

[0227] It should be noted that executing Figure 8 In the process shown in the figure, "Nth frame TOF data", N is no longer the sequence of data frames collected by the TOF camera, N = 1, 2, 3, which means that the processor executes Figure 8 For example, when S801 is executed again, the TOF camera may collect the 9th frame of TOF data, but in this execution Figure 8 In the process shown, the processor receives the first frame of TOF data. Therefore, the "first frame of TOF data" in S801 refers to the data received during this execution. Figure 8 The first frame of TOF data in the process shown is not the first frame of TOF data actually collected by the TOF camera.

[0228] S911. Face TA transmits the recognition result that fails the recognition to TOF CA.

[0229] It is understandable that, in addition to Fig. 9 In addition to the above, another method can be used to implement S819, such as using the first frame TOF image, the second frame TOF image and the third frame TOF image in sequence for facial recognition, or only using the third frame TOF image for recognition, and if the recognition result indicates that it fails, re-execute Figure 8 The process shown in the figure can save memory and other resources. Other implementation methods are not listed here one by one. The method described in this embodiment is based on the principle of TOF imaging mentioned above (2) that TOF data converges after 2-3 frames, continuously processes 3 frames of TOF data, and obtains recognition results. For outdoor scenes, a smaller processing delay can be obtained than that of the above embodiment.

[0230] Figure 8 Examples of application scenarios for the process shown are:

[0231] Assuming that the face of the person trying to unlock is the face that has been recorded in the electronic device, the user points the face to the screen in an outdoor environment. The TOF camera collects TOF data at intervals of 30ms and transmits it to the processor. After receiving the first frame of TOF data, the processor processes the received TOF data to generate the first TOF image and the first AE result.

[0232] Because the electronic device is in an outdoor environment, it is highly likely that it will not be unlocked based on the first frame of TOF data. Therefore, after the processor generates the first TOF image and transmits the first AE result, it will not perform facial recognition, but continue to receive and process the TOF data frame. Because the time to generate the TOF image and AE result is 30ms, which is equivalent to the time it takes for the TOF camera to collect and transmit TOF data, after processing the first frame of TOF data, the second frame of TOF data can be received and processed, and after processing the second frame of TOF data, the third frame of TOF data can be received and processed.

[0233] Based on the above duration, the first AE result generated by the first frame of TOF data can be applied to the third frame of TOF data at the earliest. Therefore, the third frame of TOF data will most likely be unlocked successfully.

[0234] The shortest time consumed to obtain the recognition result through the third frame TOF data is: 30ms*3+150ms. Compared with the above embodiment, when the unlocking of the first frame TOF data is unsuccessful, the unlocking is changed from the seventh frame to the third frame, so the speed can be increased and the delay can be reduced.

[0235] It can be seen that if the first frame of TOF data can be unlocked, compared with the above embodiment, the processing time of two TOF data (generating TOF image and AE result) is increased by 60ms. Therefore, the data processing method described in this embodiment improves the unlocking speed in outdoor strong light scenes by sacrificing the unlocking speed of the first frame of TOF data.

[0236] It is understandable that, because the first AE result acts on the third frame TOF data, optionally, in Figure 8 In the process, the second frame TOF data and the AE result of the second frame TOF data may not be processed, that is, S807-S812 are not executed to save resources. Accordingly, the second frame TOF image is no longer involved in obtaining the recognition result.

[0237] In summary, the second frame of TOF data may be simply received without being processed, or the second frame of TOF data may be discarded without being received.

[0238] In this embodiment, taking the continuous processing of three frames of TOF data in one process as an example, in fact, the number of TOF data frames processed continuously is related to the time consumed in generating and feeding back the AE results, and the first time interval for the TOF camera to collect the data frames.

[0239] Based on the scenarios in which the above embodiments are respectively applicable, it can be understood that the above Figure 7 or Figure 8 The method described can use judgment conditions, select the best one, based on Figure 3 The software framework shown in the figure, another data processing method disclosed in the embodiment of the present application includes the following steps:

[0240] 1. Face CA obtains the intensity value of ambient light.

[0241] based on Figure 3 In the software framework shown, after Face CA receives the task request, it can also send a light intensity request to obtain the intensity value of the ambient light. It can be understood that the light intensity request can be sent to the corresponding driver of the kernel layer through the application framework layer and the corresponding module of the hardware abstraction layer, and the corresponding driver of the kernel layer drives the light sensor to sense the intensity value of the ambient light and feed it back to Face CA.

[0242] 2. Face CA transmits the intensity value of ambient light to Face TA.

[0243] 3. Face TA determines whether the intensity of the ambient light is greater than the preset first intensity threshold. If so, execute Figure 7 The data processing flow shown, if not, execute Figure 8 The data processing flow is shown.

[0244] In this embodiment, the intensity value of light is used to determine the environment in which the electronic device is located, and a process that is more suitable for the environment is used to obtain facial recognition results. Therefore, on the basis of improving security, the accuracy and speed of obtaining facial recognition results can be maximized.

[0245] During the research process, the inventors further discovered that, whether in outdoor or indoor scenes, when the ambient light is extremely strong and the sensor of the TOF camera faces a strong light source, the quality of the first frame of TOF data cannot support accurate facial recognition results. Therefore, the TOF data needs to be converged to increase the duration of facial recognition.

[0246] For example Figure 1bIn the case of a user who is outdoors in extremely strong sunlight and whose back is to the sun, the user points the front camera of the phone at the face in anticipation of face unlocking (assuming that the user's face has been stored in the phone as a face module), and it takes more time to unlock compared to an indoor environment.

[0247] In response to the above problems, an embodiment of the present application provides a data acquisition method, the purpose of which is to obtain higher quality first-frame TOF data, so as to improve the accuracy of the recognition results of the first-frame TOF data while using TOF data for facial recognition to obtain higher security, thereby further achieving the purpose of quickly completing facial recognition.

[0248] Fig.10 A data acquisition method disclosed in an embodiment of the present application is executed by the above electronic device, combined with Figure 2-Figure 6 As shown, Fig.10 The process starts from Camera HAL3 driving the TOF camera through the TOF camera driver to collect data, and includes the following steps:

[0249] S1001. Camera HAL3 transmits an image output instruction to the TOF camera in response to the image request.

[0250] Combination Figure 2 as well as Figure 3 It can be understood that Camera HAL3 can transmit image output instructions to the TOF camera through the TOF camera driver at the kernel layer.

[0251] S1002 : The TOF camera collects a first frame of TOF data including depth data, projection-off data, and infrared data in response to a picture output instruction.

[0252] Depth data is data used to generate depth images. Projection-off data is TOF data collected by the TOF camera when the TOF light source is turned off. Infrared data is data used to generate infrared images.

[0253] The depth data and infrared data are both TOF data collected by the TOF camera when the TOF light source is turned on.

[0254] When the TOF camera is powered on, the TOF light source is turned on by default. In some implementations, the TOF sensor controller of the TOF camera transmits an output command to the TOF sensor, and the TOF sensor responds to the output command, collects depth data, and transmits a shutdown command to the TOF light source controller. The TOF light source controller responds to the shutdown command and turns off the TOF light source, and the TOF sensor collects projection shutdown data. The TOF sensor transmits an on command to the TOF light source controller, and the TOF light source controller responds to the on command and turns on the TOF light source, and the TOF sensor collects infrared data.

[0255] In some implementations, an example of the first frame of TOF data is: 4 sets of depth data, a set of projection-off data, and a set of infrared data. "A set of data" can be understood as a two-dimensional array.

[0256] The TOF sensor first collects each set of depth data in sequence with the first exposure time, collects infrared data with the second exposure time, and collects projection off data with the first exposure time or the second exposure time. The first exposure time and the second exposure time can be carried by the Camera HAL3 in the output instruction or transmitted to the TOF camera separately, or can be transmitted to the TOF camera by other modules, or can be obtained by the TOF camera in advance from the storage module.

[0257] It is understandable that the timing when the TOF sensor sends a closing instruction to the TOF light source controller is related to the first exposure duration, and the timing when the TOF sensor sends an opening instruction to the TOF light source controller is related to the exposure duration of the projected closing data.

[0258] In some implementations, the TOF sensor may mark the TOF data collected within the first time range as projection-off data. The first time range may be determined based on the interval between the time when the TOF sensor issues a closing instruction and the time when the TOF sensor issues an opening instruction. The TOF sensor may mark the TOF data collected before the first time range as depth data, and mark the TOF data collected after the first time range as infrared data.

[0259] It is understandable that the order in which the TOF camera collects depth data, projection off data, and infrared data is not limited. For example, the data may be collected in the order of projection off data, infrared data, and depth data, or in the order of infrared data, depth data, and projection off data. The timing of the TOF sensor transmitting the closing instruction and the opening instruction to the TOF light source controller is adjusted according to the order.

[0260] In this step, a TOF sensor is used to control the turning on or off of the TOF light source, which has a higher execution speed.

[0261] S1003, after the data processing module obtains the first frame of TOF data, it determines whether the infrared data in the first frame of TOF data contains the target data block, if yes, execute S1004, if not, execute S1005.

[0262] The specific method of the data processing module to obtain the first frame of TOF data is as follows: Figure 4 , Figure 5 or Figure 6 As shown, no further details are given.

[0263] The infrared data is a two-dimensional array. It can be understood that the two-dimensional array includes some values ​​arranged in rows and columns. Each value can be regarded as a data point. The target data block is a data block that meets the following preset conditions: the number of data points with values ​​greater than the first threshold is greater than the second threshold.

[0264] As mentioned above, the infrared data collected by the TOF sensor is a two-dimensional array, which is processed by ISP-Lite into infrared raw data, that is, infrared raw images. Therefore, it can be understood that the target data block is the target area in the infrared raw image, and each value in the target data block is the brightness value of the corresponding pixel point in the target area. Therefore, for the infrared raw image, the target area is: the area where the number of pixels with brightness values ​​greater than the first threshold is greater than the second threshold.

[0265] When the ambient light is extremely strong and the TOF camera faces a strong light source, the TOF data collected by the TOF camera will usually be overexposed due to the strong light. Therefore, the brightness of most pixel values ​​in the overexposed area of ​​the infrared original image generated by the TOF data is too high, which affects recognition. The conditions of the target area (data block) in this step are set based on this principle.

[0266] S1004 , using the infrared data in the first frame of TOF data, subtracting the projection-off data in the first frame of TOF data to obtain a processed first frame of TOF data, and processing the processed first frame of TOF data into a TOF image.

[0267] S1005: Process the first frame of TOF data into a TOF image.

[0268] It can be understood that processing the first frame of TOF data into a TOF image means processing the depth data in the first frame of TOF data into a depth image, and processing the infrared data in the first frame of TOF data into an infrared image.

[0269] The method described in this embodiment, when the infrared data in the first frame TOF data is overexposed, removes the influence of ambient light on the infrared data by subtracting the projection-off data from the infrared data in the first frame TOF data, thereby improving the quality of the first frame TOF data, and further obtaining a higher quality TOF image, which is beneficial to improving the accuracy and speed of facial recognition.

[0270] Fig.11 Another data acquisition method disclosed in the embodiment of the present application is as follows: Figure 3 The Camera HAL3 implementation shown in , includes the following steps:

[0271] S1101. In response to an image request, determine whether the interval duration is within a preset range. If yes, execute S1102; if not, execute S1103.

[0272] The interval duration is the interval duration between the moment when TOF data is collected for the last time (referred to as the first moment) and the moment when the first frame of TOF data is to be collected (referred to as the second moment).

[0273] It is understandable that, because Camera HAL3 drives the TOF camera to collect TOF data through the TOF camera driver of the kernel layer, Camera HAL3 can select the second moment according to the time when the image output instruction is sent to the TOF camera driver. For example, the moment after a certain delay is added to the moment when the image output instruction is sent to the TOF camera driver as the second moment. For another example, in order to simplify the process, Camera HAL3 can directly use the current moment of the system as the second moment.

[0274] In this embodiment, the first frame of TOF data refers to, Figure 4 , Figure 5 ,or Figure 6 As shown, the application sends a task request once, which triggers the first frame of TOF data collected by the TOF camera. The last time can be the last frame of TOF data collected by the TOF camera triggered by the task request sent by the application last time.

[0275] S1102 , instructing the TOF camera to use the exposure parameters used for the last TOF data acquisition to acquire the first frame of TOF data at the second moment.

[0276] As mentioned earlier, because the AE results can be used to adjust the parameters of the TOF camera, the exposure parameters used for the last TOF data collection are very likely to be the adjusted exposure parameters. Because the interval between the first moment and the second moment is within the preset range, it is very likely that the environment is the same as the last collection. Therefore, the exposure parameters used for the last TOF data collection are very likely to be applicable to the current environment, which is conducive to obtaining higher quality TOF data, and thus higher quality TOF images.

[0277] S1103 : Instruct the TOF camera to collect the first frame of TOF data using pre-configured exposure parameters.

[0278] Because if the interval between the first moment and the second moment is long, it means that a long time has passed since the last TOF data was collected. Therefore, it is very likely that the environment in which the electronic device is located has changed. Therefore, the exposure parameters used for the last TOF data collection are no longer suitable for the current environment. Therefore, it is meaningless to use the exposure parameters used for the last TOF data collection. Therefore, the pre-configured exposure parameters are used.

[0279] The data acquisition method described in this embodiment makes full use of the exposure parameters adjusted by the AE adjustment mechanism to improve the quality of the first frame TOF data, and further obtains a higher quality TOF image, which is beneficial to improving the accuracy and speed of facial recognition.

[0280] It is understandable that in the above steps, the exposure parameters used for the last TOF data collection are only one implementation method, because the moment of the last TOF data collection is the latest moment from the second moment, so the purpose of comparing with the last TOF data collection is to save computing resources.

[0281] However, in this embodiment, the comparison is not limited to the last TOF data acquisition, and any acquisition before the second moment can be compared with the second moment. Therefore, the conditions satisfied by the first moment can be summarized as: earlier than the second moment and the interval time with the second moment is within a preset range.

[0282] Fig.12 Another data acquisition method disclosed in the embodiment of the present application is as follows: Figure 3 The Camera HAL3 implementation shown in , includes the following steps:

[0283] S1201. In response to an image request, obtain light intensity.

[0284] In this step, an ambient light sensor on the electronic device may be used to obtain the light intensity of the environment in which the electronic device is located.

[0285] S1202. Instruct the TOF camera to collect the first frame of TOF data using the exposure duration corresponding to the light intensity.

[0286] In this embodiment, the corresponding relationships between multiple light intensity intervals and exposure durations are preconfigured, and the corresponding relationships satisfy the following principles:

[0287] 1. The light intensity intervals include an indoor light intensity interval and an outdoor light intensity interval.

[0288] The preconfigured fixed exposure duration may not be applicable to the outdoor environment. Therefore, the exposure durations used in the indoor environment and the outdoor environment are different. Therefore, it is necessary to use light intensity intervals to reflect this difference.

[0289] Furthermore, because in the outdoor environment, the change range of the light intensity is very large, it is necessary to further distinguish the exposure parameters used in different light intensity intervals in the outdoor environment.

[0290] 2. The larger the value in each light intensity interval, the shorter the corresponding exposure duration.

[0291] Because the longer the exposure duration, the higher the brightness of the image. Therefore, in the outdoor environment, it is necessary to shorten the exposure duration to avoid the problem that the brightness of the image is too high and the clarity is reduced.

[0292] An example of the corresponding relationship configured based on the above principles is:

[0293] L <= 500 lux, t = 1 ms (default fixed value);

[0294] 500 lux < L <= 3000 lux, t = 0.7 ms;

[0295] 3000 lux < L <= 30000 lux, t = 0.5 ms;

[0296] L > 30000 lux, t = 0.3 ms.

[0297] In the above example, L represents the light intensity value, t represents the exposure duration, L <= 500 lux is the indoor light intensity interval, that is, when less than 500 lux, it can be considered that the electronic device is in the indoor environment. In this case, the default fixed exposure duration can be used.

[0298] 500 lux < L <= 3000 lux, 3000 lux < L <= 30000 lux, and L > 30000 lux are the intervals further divided for the outdoor environment. It can be seen that among the three intervals, the larger the value of the interval, the shorter the corresponding exposure duration.

[0299] It can be understood that the granularity of the intensity range of the outdoor environment in the above example can be adjusted. The smaller the granularity, the finer the control of the exposure time, which is more conducive to improving the image quality and thus the speed of image processing.

[0300] It can be understood that Camera HAL3 can be driven by the TOF camera to transmit the exposure time corresponding to the light intensity to the TOF sensor controller of the TOF camera.

[0301] It can be seen that in this embodiment, the exposure time for collecting TOF data is obtained based on the light intensity of the environment in which the electronic device is located. Therefore, it is beneficial to collect TOF data with brightness that meets the facial recognition requirements in the first frame, thereby helping to improve the accuracy and speed of facial recognition.

[0302] It should be noted that Figure 7-Figure 12 The process shown is not limited to Figure 3 The software framework shown can also be applied in Fig.13 In the software framework shown.

[0303] Fig.13 In the , TEE is not set, that is, the processing of TOF data and facial recognition are performed in REE, specifically, in Face CA. Therefore, TOF data can be directly transmitted between modules instead of the storage information of TOF data. Fig.13 and Figure 3 The difference is that after ISP-Lite receives the TOF data collected by the TOF camera, it transmits the TOF data to Face CA via Camera HAL3 and Camera service. Face CA processes the TOF data into TOF images and uses the TOF images for facial recognition.

[0304] based on Fig.13 , Figure 7 In the process shown, each step is executed by Face CA: after Face CA receives the first frame of TOF data, it processes the first frame of TOF data into the first frame of TOF image, obtains the first AE result, transmits the first AE result to Camera HAL3, and uses the first frame of TOF image to perform facial recognition to obtain a first recognition result. If the first recognition result indicates that the recognition is successful, the first recognition result is transmitted to Face service. If the first recognition result indicates that the recognition is unsuccessful, the seventh frame of TOF data is received again.

[0305] based on Fig.13 , Figure 8 In the process shown, each step is performed by Face CA and will not be repeated here.

[0306] based on Fig.13 , Fig.10 The process shown in the figure can be replaced by Face CA for the data processing module, which will not be described in detail.

[0307] Figure 11-Figure 12 The process shown is applicable to Fig.13 .

[0308] Understandably, Figure 7-Figure 12 The process shown is not limited to the Android operating system. In other operating systems, a module with the same function as the data module can implement the steps performed by the above data module, and a module with the same function as the Camera HAL3 can implement the steps performed by the above Camera HAL3.

[0309] An embodiment of the present application also discloses a chip system, including: at least one processor and an interface, wherein the interface is used to receive code instructions and transmit them to at least one processor, and the at least one processor executes the code instructions to implement at least one of the above-mentioned facial recognition method, data acquisition method, and data processing method.

[0310] The specific process of the processor implementing the above functions can be found in the above embodiments and will not be repeated here.

[0311] The embodiment of the present application also discloses a computer-readable storage medium having program code stored thereon. When the program code is executed by a computer device, at least one of the facial recognition method, data acquisition method, and data processing method described in the above embodiment is implemented.

Claims

1. A data acquisition method, characterized in that: Applied to an electronic device, the electronic device includes a time-of-flight TOF camera, the TOF camera is used to collect TOF data, the TOF data includes a safety indication frame and multiple facial recognition frames, the multiple facial recognition frames include a first facial recognition frame, the first facial recognition frame is the first frame of the multiple facial recognition frames, and the method includes: Acquire the TOF data, wherein the TOF data is data collected by the TOF camera triggered by a task request issued by an application program; Acquire the safety indication frame, wherein the safety indication frame includes a human eye safety flag; If the eye safety flag indicates safety, acquiring the first facial recognition frame, wherein the first facial recognition frame includes projection-off data and infrared data, wherein the projection-off data is TOF data collected by the TOF camera when the TOF light source is turned off, and the infrared data is data used to generate an infrared image; Determining that there is a data block that meets a preset condition in the infrared data, wherein the preset condition includes that the number of data points in the data block whose brightness values ​​are greater than a first threshold is greater than a second threshold; According to the difference between the infrared data and the projection-off data, acquiring TOF data for generating a first frame of TOF image; Generate a first frame of TOF image using the TOF data for generating the first frame of TOF image; The first frame of TOF image is used to perform facial recognition to obtain a recognition result.

2. The method according to claim 1, characterized in that The acquiring of the first facial recognition frame comprises: After the TOF camera turns off the TOF light source, collecting the projection-off data; and After the TOF camera turns on the TOF light source, the infrared data is collected.

3. The method according to claim 2, characterized in that Before collecting the projection closing data, the method further includes: When the TOF camera turns on the TOF light source, depth data for generating a depth image is collected.

4. The method according to claim 3, characterized in that: The timing of controlling the TOF camera to turn off the TOF light source is determined according to the first exposure time used to collect the depth data.

5. The method according to claim 3 or 4, characterized in that: The timing of controlling the TOF camera to turn on the TOF light source is determined according to the exposure time for collecting the projection-off data.

6. The method according to any one of claims 3 to 5, characterized in that: The exposure time for collecting the projection-off data is the first exposure time for collecting the depth data, or the second exposure time for collecting the infrared data.

7. The method according to any one of claims 2 to 6, characterized in that: The TOF camera turns off the TOF light source, comprising: Controlling the TOF light source to be turned off through the TOF sensor of the TOF camera; The TOF camera turns on the TOF light source, including: The TOF light source is controlled to turn on through the TOF sensor of the TOF camera.

8. The method according to any one of claims 1 to 6, characterized in that: Also includes: It is determined that the data block does not exist in the infrared data, and the collected first face recognition frame is processed into a first TOF image frame.

9. The method according to claim 8, characterized in that Also includes: If the eye safety flag indicates unsafe, the TOF camera is controlled to be turned off.

10. The method according to any one of claims 1 to 4, characterized in that: The step of obtaining a first facial recognition frame includes: The first facial recognition frame captured by the TOF camera is stored in the trusted execution environment TEE through the kernel layer; The determining that there is a data block satisfying a preset condition in the infrared data, and acquiring TOF data for generating a first frame of TOF image according to a difference between the infrared data and the projection-off data, comprises: In the TEE, it is determined that there is a data block satisfying the preset condition in the infrared data, and TOF data for generating a first frame of TOF image is acquired according to a difference between the infrared data and the projection-off data.

11. The method according to any one of claims 1 to 4, characterized in that: The TOF camera is a front TOF camera.

12. An electronic device, characterized in that: include: A TOF camera is used to collect TOF data, wherein the TOF data includes a safety indication frame and multiple facial recognition frames, wherein the multiple facial recognition frames include a first facial recognition frame, wherein the first facial recognition frame is the first frame of the multiple facial recognition frames, and wherein the TOF data is the TOF data collected by the TOF camera triggered by an application sending a task request, wherein the first facial recognition frame includes projection-off data and infrared data, and the projection-off data is the TOF data collected by the TOF camera when the TOF light source is turned off; A memory for storing program codes; A processor is used to run the program code to implement the data acquisition method described in any one of claims 1-11.

13. The method according to claim 12, characterized in that The TOF camera is a front TOF camera.

14. A chip system, characterized in that: include: at least one processor and an interface, the interface being used to receive code instructions and transmit them to the at least one processor; The at least one processor runs the code instructions to implement the data acquisition method according to any one of claims 1 to 11.

15. A readable storage medium having a program stored thereon, characterized in that: When the program is read and executed by a computing device, the data acquisition method according to any one of claims 1 to 11 is implemented.

Citation Information

Patent Citations

  • System security processing method and device

    CN109522722A

  • Terminal and control method

    CN109981902A

  • Image acquisition method, device and equipment and computer readable storage medium

    CN111524088A

  • Laser safety circuit and laser safety equipment

    CN113126067A

  • Distance measurement method, terminal and storage medium

    CN113219476A