A power consumption mode switching method and device, electronic equipment and storage medium

Abstract

This application relates to the field of computer technology, and more particularly to the field of artificial intelligence technology. It provides a method, apparatus, electronic device, and storage medium for switching power consumption modes. The method includes: in a first power consumption mode, capturing a set of original environmental images of the surrounding environment and performing optimization processing to obtain a corresponding optimized environmental image set; based on the optimized environmental image set, determining that the surrounding environment meets preset power consumption switching conditions, extracting an image optimization parameter set from the optimized environmental image set, and adjusting the image capture parameter set based on the second power consumption mode to be switched; initializing all other parameters except the image capture parameter set, and reloading the image optimization parameter set; executing the second power consumption mode based on the image optimization parameter set and the adjusted image capture parameter set, and performing biometric information capture and recognition. Since this application only requires parameter adjustment to switch power consumption modes, it reduces the time consumed in exiting a low-power mode.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method, apparatus, electronic device, and storage medium for switching power consumption modes. Background Technology

[0002] Currently, biometric identification technology is being used more and more widely in various fields, such as palm scanning and facial recognition. These methods rely on collecting and analyzing the unique biometric characteristics of the human body to authenticate identities, thus providing a more secure and convenient authentication method than traditional password authentication.

[0003] Biometric authentication typically relies on specialized biometric identification devices. To reduce resource consumption and extend device lifespan, these devices usually incorporate a low-power mode. When not in use for a certain period, the device automatically enters a low-power mode by adjusting its hardware operating status, such as disabling unnecessary functions or degrading some hardware features. When needed again, it exits the low-power mode and resumes normal operation.

[0004] In related technologies, exiting low-power mode typically involves a reboot. First, a reboot command is issued from the application, which reaches the kernel layer via an interface. The kernel layer loads and initializes the relevant drivers, which then initialize the relevant hardware. After all components and hardware are initialized, the disabled functions are re-enabled, and degraded hardware returns to normal operation, completing the reboot process. As can be seen, the entire reboot process involves numerous steps, resulting in excessively long exit times for low-power mode, low efficiency in biometric identification, and a prolonged waiting period for successful biometric identification, further compromising the user experience.

[0005] In summary, reducing the time consumed in exiting low-power mode is an urgent problem to be solved. Summary of the Invention

[0006] This application provides a method, apparatus, electronic device, and storage medium for switching power consumption modes to reduce the time consumed in exiting a low-power mode.

[0007] On the one hand, a method for switching power consumption modes is provided, applicable to biometric identification devices, including:

[0008] In the first power consumption mode, a set of original environmental images of the surrounding environment is captured, and the original environmental image set is optimized to obtain a corresponding optimized environmental image set.

[0009] Based on the optimized environment image set, when it is determined that the surrounding environment meets the preset power consumption switching conditions, an image optimization parameter set is extracted from the optimized environment image set, and the image capture parameter set is adjusted based on the second power consumption mode to be switched; the power consumption corresponding to the second power consumption mode is greater than that of the first power consumption mode.

[0010] Initialize all parameters other than the image capture parameter set, and reload the image optimization parameter set;

[0011] Based on the image optimization parameter set and the adjusted image capture parameter set, the second power consumption mode is executed, and under the second power consumption mode, biometric information is captured and identified on the target object.

[0012] On the one hand, a power consumption mode switching device is provided, comprising:

[0013] The shooting unit is used to capture a set of original environmental images of the surrounding environment in a first power consumption mode, and to optimize the set of original environmental images to obtain a corresponding optimized environmental image set.

[0014] An adjustment unit is configured to, based on the optimized environment image set, determine when the surrounding environment meets preset power consumption switching conditions, extract an image optimization parameter set from the optimized environment image set, and adjust the image capture parameter set based on the second power consumption mode to be switched; the power consumption corresponding to the second power consumption mode is greater than that of the first power consumption mode.

[0015] The loading unit is used to initialize all parameters other than the image capture parameter set and reload the image optimization parameter set.

[0016] The execution unit is configured to execute the second power consumption mode based on the image optimization parameter set and the adjusted image acquisition parameter set, and to perform biometric information acquisition and recognition on the target object under the second power consumption mode.

[0017] Optionally, the adjustment unit is specifically used to determine whether the surrounding environment meets the preset power switching conditions using any of the following methods:

[0018] The optimized environment image set is located, and the objects to be identified contained in each optimized environment image are located. The actual distance between each object to be identified and the biometric identification device is measured. When it is identified that there is at least one object to be identified whose object type is the target type and the corresponding actual distance reaches the preset distance threshold, it is determined that the surrounding environment meets the power consumption switching condition.

[0019] The light intensity information in each optimized environment image is detected. After determining that the light intensity change in the surrounding environment reaches a preset change threshold, it is determined that the surrounding environment meets the power consumption switching condition.

[0020] Optionally, the adjustment unit is specifically used to determine whether the object type of each object to be identified is the target type in the following manner:

[0021] Input the optimized environment image containing the object to be identified into the target recognition model;

[0022] Feature extraction is performed on the optimized environment image to obtain an initial image feature set; the initial image feature set contains the initial image features of the optimized environment image in each feature dimension;

[0023] Based on the attention mechanism, the information richness of the optimized environment image represented by each initial image feature is combined to obtain the feature weights corresponding to each initial image feature; and based on each feature weight, the corresponding initial image features are adjusted to obtain the target image features corresponding to each initial image feature.

[0024] Based on the target image features and the corresponding initial image features, the object type of the object to be identified is predicted to obtain the corresponding type prediction value.

[0025] Based on the type prediction value, determine whether the object type of the object to be identified is the target type.

[0026] Optionally, the adjustment unit is specifically used for:

[0027] Select an optimized environment image from the set of optimized environment images;

[0028] Iterate through each pixel of the optimized environment image and obtain the pixel value distribution of each pixel in different color spaces;

[0029] The set of image optimization parameters is determined based on the distribution of pixel values.

[0030] Optionally, the loading unit is specifically used for:

[0031] A restart operation is performed on the registers corresponding to each of the other parameters in the image sensor, and on the image signal processor, to restore each of the other parameters to its default value;

[0032] The image optimization parameter set is reloaded into the image signal processor.

[0033] Optionally, the device further includes:

[0034] The training unit is used to obtain the target recognition model in the following manner:

[0035] Obtain a training sample set; wherein each training sample contains a sample image and a sample label, the sample label representing whether the object type of the sample object in the corresponding sample image is the target type;

[0036] The initial recognition model is trained iteratively based on the training sample set to obtain the target recognition model.

[0037] Optionally, the training unit is specifically configured to perform the following operations during each iteration:

[0038] Feature extraction is performed on multiple sample images to obtain multiple initial image feature sets;

[0039] For each initial image feature in each initial image feature set, based on the attention mechanism, the feature weights corresponding to each initial image feature are obtained by combining the information richness of the corresponding sample image represented by each initial image feature; and based on each feature weight, the corresponding initial image features are adjusted to obtain the target image features corresponding to each initial image feature.

[0040] For each initial image feature set, based on the target image features and the corresponding initial image features, the object type of the corresponding sample object is predicted to obtain the corresponding type prediction value;

[0041] The parameters of the initial recognition model are adjusted based on the differences between the predicted values ​​of each type and the corresponding sample labels.

[0042] Optionally, the execution unit is specifically used for:

[0043] Perform bio-information imaging on the target object to acquire at least one bio-information image;

[0044] The image quality of the at least one bio-information image is evaluated based on a preset image quality index. After determining that the image quality of the at least one bio-information image meets the preset recognition conditions, the resources occupied by the image sensor and the image signal processor are released and the released resources are allocated to the bio-information recognition task until the next shooting.

[0045] Biometric identification is performed on the at least one biometric image.

[0046] Optionally, the device further includes:

[0047] The determining unit is used to obtain the capture time of multiple historical biological information captured within a preset time period;

[0048] Based on each capture time, the capture pattern of historical biological information in the time dimension was determined;

[0049] Based on the capture pattern, the power consumption switching cycle of the biometric identification device is determined;

[0050] At the transition point between power consumption switching cycles, when the power consumption mode of the biometric identification device is determined to be the first power consumption mode, a power consumption mode switching operation is performed.

[0051] On one hand, an electronic device is provided, including a processor and a memory, wherein the memory stores a computer program that, when executed by the processor, causes the processor to perform the steps of any of the above-described power consumption mode switching methods.

[0052] On the one hand, a computer-readable storage medium is provided, comprising a computer program that, when run on an electronic device, causes the electronic device to perform the steps of any of the above-described power consumption mode switching methods.

[0053] On one hand, a computer program product is provided, the computer program product including a computer program stored in a computer-readable storage medium; when a processor of an electronic device reads the computer program from the computer-readable storage medium, the processor executes the computer program, causing the electronic device to perform the steps of any of the above-described power consumption mode switching methods.

[0054] The beneficial effects of this application are as follows:

[0055] This application provides a method, apparatus, electronic device, and storage medium for switching power consumption modes. In a first power consumption mode, the biometric identification device can periodically capture images of the surrounding environment, obtaining one or more original environmental images each time. The original environmental images are then optimized to obtain one or more optimized environmental images. Based on the optimized environmental images, the surrounding environment of the biometric identification device can be determined, and further, based on the surrounding environmental conditions, it can be determined whether to exit the first power consumption mode and switch to a second power consumption mode with higher power.

[0056] As can be seen, compared with the existing technology that can only adjust the power consumption mode through manual triggering, this application can detect changes in the surrounding environment based on the optimized environmental image, and then automatically determine whether to switch to the second power consumption mode according to the specific situation of the surrounding environment. This allows the first power consumption mode to be exited in advance before the biometric information recognition operation is actually performed, ensuring that the response is not slow due to the first power consumption mode when performing biometric information recognition, thus improving the response speed of power consumption mode switching and realizing adaptive adjustment of power consumption mode.

[0057] Furthermore, this application considers that, in biometric identification devices, the most important thing is to enable the camera device to exit the first power consumption mode as soon as possible in order to quickly and accurately collect biometric information. Therefore, after determining that it is necessary to switch to the second power consumption mode, the biometric identification device can adjust the parameters of the image capturing parameter set based on the second power consumption mode to be switched to, and extract and save the image optimization parameter set from the optimized environment image set. Then, it uses an initialization method to quickly adjust all other parameters except the image capturing parameter set, and reloads the previously saved image optimization parameter set so that the captured image can be optimized using the image optimization parameter set later.

[0058] Subsequently, the biometric identification device can execute the second power mode based on the image optimization parameter set and the adjusted image capture parameter set, and perform biometric information capture and identification on the target object in the second power mode. The power mode switching method proposed in this application focuses on the recovery of the camera device in the biometric identification device. The mode switching is achieved by adjusting and initializing some parameters, as well as reloading some parameters, without restarting the entire biometric identification device, which greatly shortens the time to exit the first power mode.

[0059] Other features and advantages of this application will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings. Attached Figure Description

[0060] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0061] Figure 1 A schematic diagram of a prior art method for exiting low-power mode, provided as an embodiment of this application;

[0062] Figure 2 An overall module architecture diagram provided for an embodiment of this application;

[0063] Figure 3 A schematic diagram illustrating an application scenario of a power consumption mode switching method provided in an embodiment of this application;

[0064] Figure 4 A flowchart illustrating a power consumption mode switching method provided in an embodiment of this application;

[0065] Figure 5 This is a schematic diagram of an optimization process provided in an embodiment of this application;

[0066] Figure 6 A schematic diagram illustrating the acquisition of surrounding environment images by a biometric identification device provided in an embodiment of this application;

[0067] Figure 7 A schematic diagram illustrating how to determine if the surrounding environment meets the power consumption switching conditions, as provided in an embodiment of this application;

[0068] Figure 8 This is a schematic diagram of the structure of a residual network provided in an embodiment of this application;

[0069] Figure 9 A schematic diagram illustrating a method for obtaining target image features provided in an embodiment of this application;

[0070] Figure 10 A schematic diagram illustrating a method for adjusting various parameters provided in an embodiment of this application;

[0071] Figure 11 A schematic diagram illustrating resource release as provided in an embodiment of this application;

[0072] Figure 12 A flowchart illustrating another power consumption mode switching method provided in this application embodiment;

[0073] Figure 13 A schematic diagram of the composition structure of a power consumption mode switching device provided in an embodiment of this application;

[0074] Figure 14 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application;

[0075] Figure 15 A schematic diagram of the hardware structure of another electronic device provided in the embodiments of this application. Detailed Implementation

[0076] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings of the embodiments of this application. Obviously, the described embodiments are only some embodiments of the technical solutions of this application, and not all embodiments. Based on the embodiments recorded in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the technical solutions of this application.

[0077] The following describes some of the concepts involved in the embodiments of this application.

[0078] Power consumption mode: In different usage scenarios, the device can change the device's power consumption and performance by adjusting the hardware operation mode; in this application, it mainly includes a first power consumption mode and a second power consumption mode. The device consumes relatively less power in the first power consumption mode, which is also known as the low power consumption mode. The second power consumption mode is the power consumption mode under normal operating conditions.

[0079] Image capture parameters: Parameters related to the shooting method of the camera device in the biometric identification device, including at least parameters related to the shooting frame rate; the image capture parameters are stored in the corresponding registers of the image sensor of the camera device.

[0080] Image optimization parameters: These are parameters related to the image signal processor (ISP) of the camera device. They are responsible for optimizing the captured image by performing noise reduction, color correction, color enhancement, and other optimization processes to improve image quality.

[0081] The design concept of the embodiments of this application is briefly introduced below:

[0082] Currently, identity authentication using biometric identification technology typically relies on specialized biometric identification devices. To reduce resource consumption and extend device lifespan, these devices usually include a low-power mode. When not in use for a certain period, they automatically enter a low-power mode by adjusting the hardware's operating status. When needed again, they exit the low-power mode and resume normal operation.

[0083] Under relevant technologies, the operation of exiting low-power mode for biometric identification devices can usually only be triggered manually. For example, a palm-swipe access control device is associated with a corresponding access control application (APP). Personnel can trigger an exit command from low-power mode through the relevant interface of the access control APP. After the command is issued, the device exits low-power mode by restarting itself. However, the entire process is too lengthy. Figure 1 The diagram shown is a logical schematic of exiting low-power mode according to an embodiment of this application. First, a restart command needs to be issued from the APP. The APP communicates with the ISPServer through IspClient, transmits the command using the dbus protocol, and transmits the command to RK AIQ and Algo Libs through the ISPServer. RK AIQ and Algo Libs are responsible for providing image processing capabilities.

[0084] The reboot command further reaches the kernel layer via the v4l2 framework and media framework. Upon receiving the command, the v4l2 framework and media framework call the image signal processor driver and image sensor driver to execute specific hardware operations. The RGB and infrared sensors in the hardware layer, upon receiving the command, exit low-power mode and begin data acquisition based on normal operating mode.

[0085] It is evident that the entire restart process involves numerous steps, resulting in an excessively long time required to exit low-power mode.

[0086] Based on this, embodiments of this application provide a method, apparatus, electronic device, and storage medium for switching power consumption modes, such as... Figure 2 The diagram shown is an overall module architecture diagram improved in this application embodiment. It mainly includes an Artificial Intelligence (AI) detection module and a parameter adjustment module. The AI ​​detection module is responsible for identifying whether there are pedestrians around and, in conjunction with a distance sensor, determining whether a pedestrian is approaching, thereby determining whether it is necessary to exit the first power consumption mode. There are also other methods for determining whether it is necessary to exit the first power consumption mode, which will be described in detail in S401 to S404, and will not be repeated here. The parameter adjustment module can quickly switch power consumption modes by quickly replying to relevant parameters.

[0087] For example, in the first power consumption mode, the biometric identification device can periodically take pictures of the surrounding environment, obtain one or more original environmental images each time, and optimize the original environmental images to obtain one or more optimized environmental images. Based on the optimized environmental images, the situation of the surrounding environment where the biometric identification device is located can be determined, and further, based on the surrounding environmental situation, it can be determined whether it is necessary to exit the first power consumption mode and switch to the second power consumption mode with higher power.

[0088] As can be seen, compared with the existing technology that can only adjust the power consumption mode through manual triggering, this application can detect changes in the surrounding environment based on the optimized environmental image, and then automatically determine whether to switch to the second power consumption mode according to the specific situation of the surrounding environment. This allows the first power consumption mode to be exited in advance before the biometric information recognition operation is actually performed, ensuring that the response is not slow due to the first power consumption mode when performing biometric information recognition, thus improving the response speed of power consumption mode switching and realizing adaptive adjustment of power consumption mode.

[0089] Furthermore, this application considers that, in biometric identification devices, the most important thing is to enable the camera device to exit the first power consumption mode as soon as possible in order to quickly and accurately collect biometric information. Therefore, after determining that it is necessary to switch to the second power consumption mode, the biometric identification device can adjust the parameters of the image capturing parameter set based on the second power consumption mode to be switched to, and extract and save the image optimization parameter set from the optimized environment image set. Then, it uses an initialization method to quickly adjust all other parameters except the image capturing parameter set, and reloads the previously saved image optimization parameter set so that the captured image can be optimized using the image optimization parameter set later.

[0090] Subsequently, the biometric identification device can execute the second power mode based on the image optimization parameter set and the adjusted image capture parameter set, and perform biometric information capture and identification on the target object in the second power mode. The power mode switching method proposed in this application focuses on the recovery of the camera device in the biometric identification device. The mode switching is achieved by adjusting and initializing some parameters, as well as reloading some parameters, without restarting the entire biometric identification device, which greatly shortens the time to exit the first power mode.

[0091] The preferred embodiments of this application are described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit this application. Furthermore, the embodiments and features in the embodiments of this application can be combined with each other without conflict.

[0092] like Figure 3 The diagram shown is an application scenario illustration of an embodiment of this application. The application scenario diagram includes a biometric identification device 310.

[0093] In this application embodiment, the biometric identification device 310 includes, but is not limited to, a palm-swiping device, a face recognition device, an iris recognition device, etc.; the main hardware of the biometric identification device 310 is a camera device, which includes an image signal processor, an image sensor, etc., wherein the image sensor is responsible for capturing images (such as the original environmental image in this application), and the image signal processor is responsible for optimizing the captured original image.

[0094] The power consumption mode switching method in the various embodiments of this application can be executed by the biometric identification device 310. For example, in the first power consumption mode, the biometric identification device 310 captures a set of original environmental images of the surrounding environment and optimizes the original environmental image set to obtain a corresponding optimized environmental image set; based on the optimized environmental image set, when the biometric identification device 310 determines that the surrounding environment meets the preset power consumption switching conditions, it extracts an image optimization parameter set from the optimized environmental image set and adjusts the image capture parameter set based on the second power consumption mode to be switched.

[0095] Afterwards, the biometric identification device 310 initializes all other parameters except the image capture parameter set, restores all other parameters to their default values, and reloads the image signal processor with the image optimization parameter set value. Subsequently, the biometric identification device 310 can execute the second power mode based on the image optimization parameter set and the adjusted image capture parameter set, and perform biometric identification on the target object in the second power mode.

[0096] It should be noted that, Figure 3 The examples shown are merely illustrative; in reality, the number of biometric identification devices is unlimited and is not specifically limited in the embodiments of this application.

[0097] Furthermore, the solution proposed in this application can also be executed jointly by a biometric identification device and a server. For example, the process of determining whether the surrounding environment meets the preset power consumption switching conditions based on the optimized environmental image set can be deployed to the server for execution, or the process of performing biometric identification on the target object can be deployed to the server for execution. This application does not make specific limitations, but for the sake of convenience, unless otherwise specified, this application will assume that the power consumption mode switching solution proposed in this application is executed by a biometric identification device.

[0098] The biometric identification device and the server can communicate via a communication network. In one alternative implementation, the communication network can be a wired network or a wireless network.

[0099] The following describes the power consumption mode switching method provided by the exemplary embodiments of this application in conjunction with the application scenarios described above and with reference to the accompanying drawings. It should be noted that the above application scenarios are only shown to facilitate understanding of the spirit and principles of this application, and the embodiments of this application are not limited in any way in this respect.

[0100] See Figure 4 The diagram shown is a flowchart illustrating an implementation of a power consumption mode switching method provided in this application. The specific implementation process of this method is as follows:

[0101] S401: In the first power consumption mode, the biometric identification device captures a set of original environmental images of the surrounding environment and optimizes the original environmental image set to obtain a corresponding optimized environmental image set.

[0102] In this application, the biometric identification device has two power consumption modes, namely a first power consumption mode and a second power consumption mode. The power consumption of the second power consumption mode is greater than that of the first power consumption mode. That is, in the same amount of time, the biometric identification device consumes less power and has lower power in the first power consumption mode, while it consumes more power and has higher power in the second power consumption mode. The first power consumption mode can also be called a low power consumption mode, and the second power consumption mode can also be called a normal power consumption mode. In other words, the second power consumption mode is the power consumption mode of the biometric identification device under normal working conditions.

[0103] Biometric identification devices can be palm recognition devices, face recognition devices, iris recognition devices, etc. In this application, the biometric identification device is usually in an idle state (the biometric identification device is not operated for a long time) or enters a first power consumption mode under human setting. In the first power consumption mode, some unnecessary functions of the biometric identification device will be turned off, and some functions will be downgraded; for example, the screen brightness will be reduced, and the shooting frame rate and resolution of the camera device will be reduced, etc. However, the camera device will not be turned off in the first power consumption mode and still needs to work.

[0104] Even in the first power consumption mode, the biometric identification device will periodically photograph the surrounding environment. The image sensor in the camera device captures a raw image set of the surrounding environment, and then the image signal processor optimizes each raw image based on its characteristics. For example, ... Figure 5 The diagram shown is an example of an optimization process provided in this application. Taking an original environmental image as an example, the image signal processor can reduce noise in the original environmental image, adjust the white balance according to the color cast of the original environmental image, adjust the brightness according to the overall light intensity of the original environmental image, and so on.

[0105] The above adjustment process is based on ISP parameters. The image signal processor determines the corresponding ISP parameters, such as the values ​​of chromatic aberration compensation, saturation, contrast, color temperature, color cast, and sharpening intensity, according to the characteristics of an original environmental image. Then, it optimizes the original environmental image based on these parameter values ​​to obtain the corresponding optimized environmental image.

[0106] S402: When the biometric identification device determines that the surrounding environment meets the preset power consumption switching conditions based on the optimized environment image set, it extracts the image optimization parameter set from the optimized environment image set and adjusts the image capture parameter set based on the second power consumption mode to be switched.

[0107] In this application, the biometric identification device includes an AI detection module and a distance sensor. The former is used to identify the object type of the object to be identified from the optimized environmental image set, and the latter is used to detect the actual distance between the object to be identified and the biometric identification device.

[0108] For example, biometric identification devices mainly determine whether the surrounding environment meets the power consumption switching conditions through the following two methods.

[0109] Method 1: Locate the optimized environment image set, each optimized environment image contains the object to be identified, and measure the actual distance between each object to be identified and the biometric identification device; when the object type of at least one object to be identified is identified as the target type, and the corresponding actual distance reaches the preset distance threshold, it is determined that the surrounding environment meets the power consumption switching condition.

[0110] Specifically, the optimized environment image set contains one or more optimized environment images. Taking one optimized environment image as an example, the AI ​​detection module first determines whether there is an object to be identified in the optimized environment image. If there is one or more objects to be identified, the object type of one or more objects to be identified is further identified.

[0111] In this application, the object to be identified can be a person, cat, dog, etc. The AI ​​detection module can determine whether the object to be identified exists in the optimized environment image by visual features or by whether there is a moving object in multiple consecutive optimized environment images. The two methods of determining the object to be identified can also be used in combination. Furthermore, the target type in this application can be "person". Other object types besides the target object can include "cat", "dog", etc. The AI ​​detection module can identify whether the object type of the object to be identified is the target type, or it can directly identify the specific object type of the object to be identified.

[0112] The specific details of the objects to be identified, object types, and target types mentioned above are merely illustrative examples. In specific applications, adjustments can be made based on actual scenarios, and this application does not impose any specific limitations.

[0113] While identifying the object type, the distance sensor can acquire the distance information of all objects to be identified within a preset range (that is, the actual distance from the object to be identified to the biometric identification device). The AI ​​detection module can determine the actual distance between each object to be identified and the biometric identification device based on the positional relationship of each object in the optimized environment image and the magnitude of the multiple distance information acquired by the distance sensor.

[0114] like Figure 6The diagram illustrates a biometric identification device acquiring images of its surrounding environment, as provided in an embodiment of this application. Two objects to be identified are positioned in front of the device, distinguished by a thicker solid line (referred to as object A) and a thinner dashed line (referred to as object B). Object A is closer to the biometric identification device; therefore, in the optimized environmental image captured (and optimized), object A appears further forward. Assuming the distance sensor obtains two distance readings of 1.1m and 1.5m respectively, the actual distance between object A and the biometric identification device can be determined to be 1.1m, and the actual distance between object B and the biometric identification device can be determined to be 1.5m.

[0115] Then, the object type is directly identified for all objects to be identified in the optimized environment image to determine whether there are any objects of the target type and whether the actual distance between the target type object and the biometric identification device reaches the distance threshold.

[0116] Furthermore, in this application, the distance sensor can also acquire only the distance information of the nearest object to be identified (i.e., the actual distance from the nearest object to the biometric identification device); when multiple objects to be identified exist in the vicinity, the AI ​​detection module can detect only the object type of the object to be identified that is in the foreground as shown in the optimized environment image, such as... Figure 6 The process involves either identifying object A in the image or detecting the object type of all objects to be identified. Once it is determined that one or more objects of the target type are of the target type, it is then determined whether the target type of the object to be identified is located at the foreground in the optimized environment image.

[0117] Finally, the following methods can be used to determine whether the surrounding environment meets the power consumption switching conditions.

[0118] 1. If there is an optimized environment image in which the object to be identified is of the target type, and the actual distance between the object to be identified and the biometric identification device reaches the distance threshold, then the surrounding environment is determined to meet the power consumption switching conditions.

[0119] 2. If an object to be identified is present in multiple consecutive optimized environment images, and the object to be identified in each optimized environment image can be the same object or different objects, and the object to be identified in each optimized environment image is of the target type, and the actual distance between the object to be identified and the biometric identification device reaches the distance threshold, then the surrounding environment is determined to meet the power consumption switching condition.

[0120] 3. If an object to be identified exists in multiple consecutive optimized environment images, and the object to be identified in each optimized environment image is the same object to be identified, and the object to be identified is of the target type, and the actual distance between the object and the biometric identification device reaches the distance threshold, then the surrounding environment is determined to meet the power consumption switching conditions.

[0121] Method 2: Detect the light intensity information in each optimized environment image. After determining that the light intensity change in the surrounding environment reaches the preset change threshold, determine that the surrounding environment meets the power consumption switching conditions.

[0122] For example, such as Figure 7 The diagram shown is a schematic representation of an embodiment of this application for determining whether the surrounding environment meets the power consumption switching conditions. It can determine whether the degree of light intensity change in the surrounding environment reaches a change threshold by using the light intensity information represented in two adjacent optimized environment images. Figure 7 After α), it is determined that the surrounding environment meets the power consumption switching condition; or, in multiple consecutive optimized environment images, the light intensity difference between each two adjacent optimized environment images reaches the change threshold, thereby determining that the surrounding environment meets the power consumption switching condition.

[0123] Furthermore, a judgment can also be made based on an optimized environment image. Specifically, if the ambient light intensity represented by the optimized environment image is too low, below the preset light intensity threshold, then it is determined that the surrounding environment meets the power consumption switching conditions.

[0124] This method takes into account that when someone approaches the biometric identification device, they will block the ambient light around the device. Therefore, when the light suddenly dims or the light intensity is too low, it can be assumed that someone is approaching the biometric identification device and needs to perform biometric identification.

[0125] By employing the two methods described above, it's possible to predict the need for biometric identification based on the target's approaching behavior before the actual biometric identification (such as palm swiping) is performed. This allows for early exit from the first power consumption mode, eliminating the need to wait until the target has begun the palm swiping operation before gradually switching power consumption modes. This shortens the time for biometric information collection and identification, thus improving the palm swiping experience for the target.

[0126] Next, we will elaborate on the specific process of determining whether the object type of the object to be identified is the target type in Method 1.

[0127] The AI ​​detection module contains a pre-trained target recognition model. The biometric identification device can input an optimized environment image containing the object to be identified into the target recognition model. Then, feature extraction is performed on the optimized environment image to obtain an initial image feature set. The initial image feature set contains the initial image features of the optimized environment image in each feature dimension.

[0128] Furthermore, based on the attention mechanism, the information richness of the optimized environment image represented by each initial image feature is combined to obtain the feature weights corresponding to each initial image feature. Based on each feature weight, the corresponding initial image features are adjusted to obtain the target image features corresponding to each initial image feature. Finally, based on each target image feature and the corresponding initial image features, the object type of the object to be identified is predicted to obtain the corresponding type prediction value. Based on the type prediction value, it is determined whether the object type of the object to be identified is the target type.

[0129] The target recognition model in this application can employ a ResNet residual network; furthermore, to enhance the model's ability to focus on important features, a squeeze-and-excitation (SE) module, i.e., a channel attention mechanism, can be embedded in each residual block of the residual network. Figure 8 The diagram shown is a structural schematic of a residual network provided in an embodiment of this application. The optimized environmental image is input into the target recognition model, and the initial convolutional layer is responsible for feature extraction.

[0130] Furthermore, the residual layer following the initial convolutional layer contains one or more stacked residual blocks, and a residual block can further contain multiple convolutional layers, such as... Figure 8 The residual block shown contains convolutional layer 1 and convolutional layer 2. Convolutional layer 1 and convolutional layer 2 can further extract features based on the output of the initial convolutional layer, so that each optimized environment image can obtain a corresponding initial image feature set. After convolutional layer 1 and convolutional layer 2, there is a normalization layer 1 and a normalization layer 2, which are responsible for performing batch normalization on the output of the corresponding convolutional layer.

[0131] The SE module, based on an attention mechanism, obtains the feature weights corresponding to each initial image feature in an initial image feature set. The SE module determines the feature weights based on the information richness of the optimized environment image represented by each initial image feature. For initial image features with obvious features (such as edges, textures, and color variations), their feature weights are larger, thus enabling the model to focus more on these key features. Then, based on each feature weight, the corresponding initial image features are adjusted to obtain the target image features corresponding to each initial image feature. Specifically, this adjustment can be achieved by multiplying the feature weights by the corresponding initial image features.

[0132] The SE module mainly consists of global average pooling, two fully connected layers, and recalibration. Global average pooling can compress spatial information by pooling the input feature map into a 1x1 output. The fully connected layers learn the importance of each channel (feature weights). Recalibration is used to scale the initial image features based on the feature weights to highlight the more important features.

[0133] After obtaining the target image features, the object type of the object to be identified can be predicted based on each target image feature and its corresponding initial image features. Specifically, the residual connection adds each target image feature to its corresponding initial image feature. If the number of target image features and initial image features (i.e., the number of channels) is inconsistent during this process, a 1x1 convolution can be used to adjust the number of channels. Then, the spatial dimensions (height and width) are reduced by the pooling layer, and finally, the fully connected layer performs classification and outputs the predicted type value of the object to be identified.

[0134] In this application, the residual layers included in the target recognition model can be one or more, such as... Figure 8 The model structure shown contains only one residual layer, but it can also be set to multiple residual layers, such as 4; similarly, the number of residual blocks contained in each residual layer can also be one or more, and this application does not make a specific limitation.

[0135] In addition, the target recognition model can also be a Faster Region-based Convolutional Neural Network (Faster R-CNN), a real-time target detection algorithm (such as You Only LookOnce, YOLO), or a Single Shot MultiBox Detector (SSD), etc. In order to improve the accuracy and robustness of recognition, the prediction results of multiple models can also be combined. This application does not make specific limitations.

[0136] The target recognition model described above is obtained by iteratively training the initial recognition model based on the training sample set. The training sample set contains multiple training samples, each of which contains a sample image and a sample label. The sample label can be the object type of the sample object in the corresponding sample image, or it can be whether the object type of the sample object in the corresponding sample image is a target type. The latter will be used as an example for detailed explanation below.

[0137] When constructing the training sample set, in addition to collecting complete images of people, images containing only a part of the human body, such as the face, hands, arms, or torso, can be collected as positive samples. Furthermore, images of other non-human creatures such as cats and dogs can be collected as negative samples. Then, the positive and negative samples are labeled based on whether the object in the image is human, thus obtaining sample labels.

[0138] Data cleaning is performed on the acquired images, including but not limited to removing duplicate images, images with missing data, and images with abnormal acquisition.

[0139] Furthermore, median filtering can be applied to the image to remove noise. Details are as follows.

[0140] Create a two-dimensional array with a size equal to the width and height of the image, and a one-dimensional array with a length equal to the width × height of the filtering window. The two-dimensional array is used to store the filtered image result, and the one-dimensional array is used to store the neighboring pixels around the current pixel.

[0141] The process iterates through all pixels in the image. For each pixel, it extracts its neighboring pixels and stores them in a one-dimensional array. Then, it sorts all pixel values ​​in the array, takes the median value of each sorted pixel, and assigns it to the current pixel. Once all pixels have undergone these steps, the filtering and cleaning process is complete.

[0142] Furthermore, this application can also standardize the acquired images, such as adjusting the image size, normalizing pixel values ​​to the range of 0-1, and applying some data augmentation techniques such as rotation, scaling, and cropping, to enhance the generalization ability of the model.

[0143] Next, training samples are constructed based on the above images, and an initial recognition model is trained based on the training samples to finally obtain the target recognition model. The specific process is as follows.

[0144] Feature extraction is performed on multiple sample images to obtain multiple initial image feature sets. For each initial image feature in each initial image feature set, based on an attention mechanism, the information richness of the corresponding sample image represented by each initial image feature is combined to obtain the feature weight corresponding to each initial image feature. Based on each feature weight, the corresponding initial image features are adjusted to obtain the target image features corresponding to each initial image feature. For each initial image feature set, based on each target image feature and the corresponding initial image feature, the object type of the corresponding sample object is predicted to obtain the corresponding type prediction value. Based on the difference between each type prediction value and the corresponding sample label, the parameters of the initial recognition model are adjusted.

[0145] like Figure 9 The diagram illustrates a method for obtaining target image features according to an embodiment of this application. Taking a sample image as an example, after feature extraction of the sample image, a corresponding initial image feature set is obtained. Assuming that the initial image feature set contains three initial image features, feature weight 1, feature weight 2, and feature weight 3 are determined according to the information richness of the sample image represented by each of the three initial image features, wherein feature weight 2 > feature weight 1 > feature weight 3. The feature weights are multiplied by the corresponding initial image features to obtain target image feature 1, target image feature 2, and target image feature 3.

[0146] In this application, the training process of the model is the same as that of the aforementioned model when actually identifying the object type of the object to be identified, and will not be repeated here.

[0147] In addition, the Histogram of Oriented Gradients (HOG) can be used to extract image features from sample images. Specifically, the HOG function is imported to extract features from the image, resulting in HOG feature vectors and a visualized HOG image. The HOG image can display the gradient direction of each pixel. For the feature extraction process, the number of directions in the gradient histogram (e.g., 6, 8, etc.), the cell size to which the image is divided (e.g., 16x16 pixels), the number of cells contained in a local region, and so on can be set. If the image is a color image, the color channels used when calculating the gradient can be further specified.

[0148] Finally, the features extracted by HOG are combined into a large one-dimensional vector, which is then used as the overall feature of the sample image and input into the model to perform the model training process.

[0149] After training, the target recognition model is deployed to the AI ​​detection module to perform object type detection.

[0150] Next, returning to the aforementioned step of the biometric identification device determining whether the surrounding environment meets the power consumption switching conditions, once the biometric identification device determines that the surrounding environment meets the power consumption switching conditions through method one or method two, that is, after someone approaches the biometric identification device, it can prepare to exit the first power consumption mode.

[0151] This application takes into account that one of the most important parts of a biometric identification device is the camera device. In order for the biometric identification device to quickly capture biometric images and ensure that the biometric images captured by the camera device are clear enough, the camera device first needs to be restored to the normal power consumption mode (i.e., the second power consumption mode) as soon as possible. That is, the image sensor and image signal processor in the camera device need to be restored to the normal power consumption mode as soon as possible.

[0152] Furthermore, while the camera is not shut down in the first power consumption mode, its functionality is degraded, such as a reduced frame rate, meaning fewer photos are taken per unit time. Essentially, this process adjusts relevant parameters of the camera, specifically those related to the frame rate, causing a decrease in the frame rate. Besides the frame rate, other parameters of the camera can also be adjusted according to actual needs after entering the first power consumption mode.

[0153] Therefore, in order to restore the camera equipment to normal power consumption mode as soon as possible, this application mainly adjusts the relevant parameters of the camera equipment.

[0154] First, select an optimization environment image from the optimization environment image set; traverse each image pixel of the optimization environment image and obtain the pixel value distribution of each image pixel in different color spaces; based on the pixel value distribution, determine the image optimization parameter set.

[0155] For example, a histogram can be used to determine the set of image optimization parameters. A histogram can include one or more of the following: a grayscale histogram, a red channel histogram, a green channel histogram, and a blue channel histogram, used to represent different color spaces. In the grayscale histogram, pixel values ​​are grayscale values; in the red channel histogram, pixel values ​​are red component values; in the green channel histogram, pixel values ​​are green component values; and in the blue channel histogram, pixel values ​​are blue component values.

[0156] First, for an optimized environment image, iterate through each image pixel, calling the `compute_histogram` method to calculate the frequency of each grayscale value and generate a grayscale histogram. In the grayscale image, determine the maximum and minimum grayscale values ​​based on the specific grayscale values ​​of each pixel, and determine the average grayscale value based on the weighted average of the grayscale values. Then, further extract parameters related to image signal processing, such as contrast, sharpening intensity, color temperature / color cast, and white balance, using the `extract_isp_parameters` method.

[0157] After obtaining the image optimization parameter set, the image optimization parameter set can be saved to the specified storage area; and while obtaining the image optimization parameter set, the biometric identification device adjusts the image capturing parameters according to the requirements of the second power consumption mode.

[0158] Considering that the purpose of this application is to complete the collection and recognition of biological information as quickly as possible, the image capture parameters can be parameters related to the capture frame rate, such as the vertical total size (VTS). Assuming that the normal capture frame rate of the biological information recognition device in the second power mode is set to 60 frames, and after entering the first power mode, the capture frame rate is reduced to 10 frames in order to reduce power consumption, the parameters related to the capture frame rate need to be adjusted every time the power mode is switched. Therefore, the image capture parameters are first adjusted according to the capture frame rate corresponding to the second power mode to increase the capture frame rate, thereby improving the speed of biological information collection.

[0159] S403: The biometric identification device initializes all parameters other than the image acquisition parameter set and reloads the image optimization parameter set.

[0160] In biometric identification devices, the camera also involves other parameters, such as clock signal, line length, frame length, capture window width, capture window height, number of data transmission channels, etc. These other parameters may also be adjusted after entering the first power mode, such as reducing the number of data transmission channels, reducing the width and height of the capture window, etc. Therefore, if it is necessary to switch from the first power mode to the second power mode, other parameters also need to be restored to normal. However, there are too many other parameters. Whether they are adjusted one by one, or all parameters are pre-stored in a certain storage area before entering the first power mode and then read into the corresponding registers when switching power modes, writing a large number of parameters will consume a lot of time, resulting in a slow mode switching speed. Furthermore, the parameters in the registers need to wait for the processing of the current frame image to be completed before they can be updated, which further prolongs the exit time of the low power mode.

[0161] Therefore, this application achieves rapid recovery of parameters other than the image capture parameter set through rapid stop-start streaming. Rapid stop streaming refers to stopping the streaming media output of the camera device; during this process, the image sensor remains powered on, and the image signal processor stops processing the image. Rapid start streaming refers to starting the streaming media output of the camera device and reloading the image optimization parameter set; during this process, the image signal processor resumes image processing.

[0162] For example, a restart operation is performed on the registers corresponding to each of the other parameters in the image sensor, and on the image signal processor, so that each of the other parameters is directly restored to its default value, thereby achieving a rapid "adjustment" of the other parameters; and the image optimization parameter set is reloaded into the image signal processor.

[0163] In the above process, restarting the image signal processor can interrupt the image processing flow, stop the streaming media output of the camera device, and stop the image signal processor from outputting images. This allows the registers to update parameters without waiting for the current frame to end. Furthermore, compared to the method of reading or adjusting all other parameters, restarting the register array to directly initialize (i.e. restore to default values) the parameters in each register is faster, greatly shortening the time for power mode switching.

[0164] Furthermore, since the image signal processor also performs a restart operation, the corresponding image optimization parameter set is also cleared or restored to its default value. However, in order to ensure that the final image output by the camera device is clear enough, the image optimization parameters need to be adjusted as needed according to the image output by the image processor. The image output by the image processor is directly affected by the ambient light. In a very short time, the ambient light can be considered to remain unchanged. Therefore, this application pre-saves the image optimization parameter set in S402, which can be directly reloaded after restarting, avoiding the need to readjust the image optimization parameters and further shortening the time required for mode switching.

[0165] like Figure 10 The diagram shown is a schematic of a method for adjusting various parameters provided in an embodiment of this application. Taking the image capture parameter set as an example that only contains the VTS value, the VTS value can be adjusted automatically according to the requirements of the shooting frame rate under the second power consumption mode. Other parameters 1, 2, ..., n in the image sensor are restored to their default values ​​after a quick start-stop. After the quick start-stop, the image signal processor reloads the pre-stored image optimization parameters into the image signal processor.

[0166] Quickly stopping and starting, as well as reloading the image optimization parameter set, can be achieved through the stop_stream and start_stream methods. Then, the apply_isp_parameters method can be used to apply the image optimization parameter set to the image processing pipeline.

[0167] Image capture parameters (such as VTS values) are written from the corresponding register (such as the VTS register) to the sensor register and take effect.

[0168] S404: The biometric identification device executes a second power consumption mode based on the image optimization parameter set and the adjusted image capture parameter set, and performs biometric information capture and identification on the target object in the second power consumption mode.

[0169] After adjusting the image acquisition parameter set, reloading the image optimization parameter set, and restoring other parameters by relying on quick stop and start, the second power mode can be executed. In the second power mode, biological information images are acquired and biological information is identified.

[0170] It is understood that in the specific implementation of this application, if data related to bioinformatics is involved, and when this application applies the above data to specific products or technologies, permission or consent from the target object is required, and the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0171] Furthermore, immediately after switching to the second power mode, the Central Processing Unit (CPU) is still being scheduled, and many tasks await resource allocation. To improve CPU throughput, such as... Figure 11 The diagram illustrates a resource release mechanism provided in this application embodiment. Once the image quality of the bio-information image captured and optimized by the camera device meets the requirements, the infrared image data stream and color image data stream of the camera device can be temporarily shut down to release resources. This allows the threads related to the bio-information recognition algorithm to obtain more resources for better bio-information recognition. For example, bio-information is captured on the target object to obtain at least one bio-information image. The quality of the at least one bio-information image is evaluated based on preset image quality indicators. After determining that the image quality of at least one bio-information image meets preset recognition conditions, the resources occupied by the image sensor and image signal processor are released, and the released resources are allocated to the bio-information recognition task until the next capture. Then, the bio-information recognition algorithm performs bio-information recognition on the at least one bio-information image.

[0172] Meanwhile, considering that the activities of the target object are regular within a fixed period, the power consumption mode can be switched automatically and periodically based on this regularity. This allows the low-power mode to exit in advance before the actual collection and identification of biometric information is required, and to automatically enter the low-power mode more quickly and intelligently after the biometric identification task is no longer needed.

[0173] For example, the biometric identification device acquires the capture time of multiple historical biometric information captured within a preset time period; based on each capture time, it determines the capture pattern of the historical biometric information in the time dimension; based on the capture pattern, it determines the power consumption switching cycle of the biometric identification device; at the cycle transition point of each power consumption switching cycle, when it is determined that the power consumption mode of the biometric identification device is the first power consumption mode, it performs a power consumption mode switching operation.

[0174] For example, for a company's swipe access control device, there are usually a lot of biometric identification tasks during working hours on weekdays (assuming 8-9 am), while there are fewer biometric identification tasks at other times. The biometric identification device can capture this pattern and then exit the first power mode before 8 am and enter the first power mode after 9 am. Between 9 am on the same day and 8 am the next day, it can also periodically exit the first power mode, such as once an hour, etc.

[0175] It should be noted that the above-described specific implementation of automatically switching power consumption modes based on capture patterns is merely an example and is not intended to limit the scope of this application.

[0176] One alternative implementation is to proceed as follows: Figure 12 The flowchart shown illustrates steps S401 to S404, including the following steps:

[0177] S1201: The biometric identification device captures a set of original environmental images of the surrounding environment and optimizes the original environmental image set to obtain a corresponding optimized environmental image set.

[0178] The above process is mainly accomplished by the image sensor and image signal processor in the camera equipment.

[0179] S1202: The biometric identification device locates the optimized environment image set, each optimized environment image contains the object to be identified, and obtains the actual distance between each object to be identified and the biometric identification device, as well as the object type of each object to be identified.

[0180] S1203: Biometric identification equipment detects light intensity information in various optimized environment images to determine the degree of light intensity change in the surrounding environment.

[0181] S1202 and S1203 can be executed individually or in parallel.

[0182] S1204: The biometric identification device determines whether it needs to exit the first power consumption mode. If yes, it executes S1205; otherwise, it returns to S1201.

[0183] If the object type of the object to be identified is the target type, and the actual distance between it and the biometric identification device meets the distance threshold, or the degree of change in the light intensity of the surrounding environment meets the change threshold, then it is determined to exit the first power consumption mode.

[0184] S1205: The biometric identification device adjusts the image capture parameter set based on the second power consumption mode to be switched.

[0185] S1206: Biometric identification device extracts image optimization parameter set from optimized environment image set.

[0186] Specifically, an image optimization parameter set can be obtained by optimizing the histogram of an optimized environment image in the optimized environment image set, and the image optimization parameter set can be saved to a specified storage location.

[0187] S1205 and S1206 can be executed in parallel or sequentially, and the execution order is not limited.

[0188] S1207: Rapid shutdown of biometric identification equipment.

[0189] Perform a restart operation on the registers corresponding to each of the other parameters in the image sensor, as well as on the image signal processor.

[0190] S1208: Rapid power-on for biometric identification devices.

[0191] After quickly opening the flow, all other parameters are directly restored to their default values.

[0192] S1209: Biometric identification device reloads image optimization parameter set.

[0193] S1210: The biometric identification device is executing the second power consumption mode.

[0194] This application adds an AI detection module, which can automatically determine whether it is necessary to exit low power mode. By adjusting the parameter adjustment module's fast start-stop flow, the power mode can be switched quickly, greatly reducing the time required to exit low power mode.

[0195] Based on the same inventive concept, embodiments of this application also provide a power consumption mode switching device. For example... Figure 13 As shown, this is a schematic diagram of the power consumption mode switching device 130, which may include:

[0196] The imaging unit 1301 is used to capture a set of original environmental images of the surrounding environment in the first power consumption mode, and to optimize the original environmental image set to obtain a corresponding optimized environmental image set.

[0197] The adjustment unit 1302 is used to extract an image optimization parameter set from the optimized environment image set when the surrounding environment meets the preset power consumption switching conditions based on the optimized environment image set, and to adjust the image capture parameter set based on the second power consumption mode to be switched; the power consumption corresponding to the second power consumption mode is greater than that of the first power consumption mode.

[0198] The loading unit 1303 is used to initialize all parameters other than the image capture parameter set and reload the image optimization parameter set;

[0199] The execution unit 1304 is used to execute a second power mode based on the image optimization parameter set and the adjusted image acquisition parameter set, and to perform biometric information acquisition and recognition on the target object in the second power mode.

[0200] Optionally, the adjustment unit 1302 is specifically used to determine whether the surrounding environment meets the preset power switching conditions using any of the following methods:

[0201] The system locates and optimizes the environment image set, identifies the objects to be identified in each optimized environment image, and measures the actual distance between each object to be identified and the biometric identification device. When it is identified that there is at least one object to be identified whose type is the target type and the corresponding actual distance reaches the preset distance threshold, it is determined that the surrounding environment meets the power consumption switching condition.

[0202] The light intensity information in each optimized environment image is detected. After determining that the light intensity change in the surrounding environment reaches the preset change threshold, the surrounding environment is determined to meet the power consumption switching condition.

[0203] Optionally, the adjustment unit 1302 is specifically used to determine whether the object type of each object to be identified is the target type in the following manner:

[0204] Input the optimized environment image containing the object to be identified into the target recognition model;

[0205] Feature extraction is performed on the optimized environment image to obtain an initial image feature set; the initial image feature set contains the initial image features of the optimized environment image under each feature dimension.

[0206] Based on the attention mechanism, the information richness of the optimized environment image represented by each initial image feature is combined to obtain the feature weights corresponding to each initial image feature; and based on each feature weight, the corresponding initial image features are adjusted to obtain the target image features corresponding to each initial image feature.

[0207] Based on the features of each target image and the corresponding initial image features, the object type of the object to be identified is predicted, and the corresponding type prediction value is obtained;

[0208] Based on the type prediction value, determine whether the object type of the object to be identified is the target type.

[0209] Optionally, the adjustment unit 1302 is specifically used for:

[0210] Select an optimized environment image from the optimized environment image set;

[0211] Iterate through each pixel of an optimized environment image and obtain the pixel value distribution of each pixel in different color spaces;

[0212] Based on the distribution of pixel values, a set of image optimization parameters is determined.

[0213] Optionally, loading unit 1303 is specifically used for:

[0214] Reset the registers corresponding to each of the other parameters in the image sensor, and perform a restart operation on the image signal processor to restore each of the other parameters to its default value;

[0215] Reload the image optimization parameter set into the image signal processor.

[0216] Optionally, the device also includes:

[0217] Training unit 1305 is used to obtain the target recognition model in the following manner:

[0218] Obtain the training sample set; where each training sample contains a sample image and a sample label, and the sample label indicates whether the object type of the sample object in the corresponding sample image is the target type;

[0219] The initial recognition model is trained iteratively based on the training sample set to obtain the target recognition model.

[0220] Optionally, training unit 1305 is specifically used to perform the following operations during each loop iteration:

[0221] Feature extraction is performed on multiple sample images to obtain multiple initial image feature sets;

[0222] For each initial image feature in each initial image feature set, based on the attention mechanism, the information richness of the corresponding sample image represented by each initial image feature is combined to obtain the feature weight corresponding to each initial image feature; and based on each feature weight, the corresponding initial image feature is adjusted to obtain the target image feature corresponding to each initial image feature.

[0223] For each initial image feature set, based on the features of each target image and the corresponding initial image features, the object type of the corresponding sample object is predicted to obtain the corresponding type prediction value;

[0224] The parameters of the initial recognition model are adjusted based on the differences between the predicted values ​​of each type and the corresponding sample labels.

[0225] Optionally, execution unit 1304 is specifically used for:

[0226] Perform bio-information imaging on the target object to acquire at least one bio-information image;

[0227] Based on preset image quality indicators, at least one bio-information image is evaluated for quality. After determining that the image quality of at least one bio-information image meets the preset recognition conditions, the resources occupied by the image sensor and image signal processor are released and the released resources are allocated to the bio-information recognition task until the next shot.

[0228] Perform biometric identification on at least one biometric image.

[0229] Optionally, the device also includes:

[0230] The determining unit 1306 is used to obtain the capture time of multiple historical biological information captured within a preset time period;

[0231] Based on each capture time, the capture pattern of historical biological information in the time dimension was determined;

[0232] Based on the capture pattern, the power consumption switching cycle of the biometric identification device is determined;

[0233] At the transition point between power consumption switching cycles, when the power consumption mode of the biometric identification device is determined to be the first power consumption mode, a power consumption mode switching operation is performed.

[0234] For ease of description, the above sections are divided into modules (or units) according to their functions and described separately. Of course, in implementing this application, the functions of each module (or unit) can be implemented in one or more software or hardware components.

[0235] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0236] Having introduced the power consumption mode switching method and apparatus according to exemplary embodiments of this application, we will now introduce an electronic device according to another exemplary embodiment of this application.

[0237] Those skilled in the art will understand that various aspects of this application can be implemented as a system, method, or program product. Therefore, various aspects of this application can be specifically implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, collectively referred to herein as a "circuit," "module," or "system."

[0238] Based on the same inventive concept as the above-described method embodiments, this application also provides an electronic device. In one embodiment, the electronic device may be a server. In this embodiment, the structure of the electronic device may be as follows: Figure 14 As shown, it includes a memory 1401, a communication module 1403, and one or more processors 1402.

[0239] The memory 1401 is used to store computer programs executed by the processor 1402. The memory 1401 may mainly include a program storage area and a data storage area. The program storage area may store the operating system and programs required to run instant messaging functions, etc.; the data storage area may store various instant messaging information and operation instruction sets, etc.

[0240] Memory 1401 may be volatile memory, such as random-access memory (RAM); memory 1401 may also be non-volatile memory, such as read-only memory, flash memory, hard disk drive (HDD), or solid-state drive (SSD); or memory 1401 may be any other medium capable of carrying or storing a desired computer program having the form of instructions or data structures and accessible by a computer, but is not limited thereto. Memory 1401 may be a combination of the above-described memories.

[0241] Processor 1402 may include one or more central processing units (CPUs) or digital processing units, etc. Processor 1402 is used to implement methods such as object type recognition and biometric information recognition when calling computer programs stored in memory 1401.

[0242] The communication module 1403 is used to communicate with terminal devices and other servers.

[0243] This application embodiment does not limit the specific connection medium between the memory 1401, communication module 1403, and processor 1402. This application embodiment... Figure 14 The memory 1401 and the processor 1402 are connected via a bus 1404, and the bus 1404 is in Figure 14 The diagram uses thick lines to describe the connections between other components; these are for illustrative purposes only and should not be considered limiting. The 1404 bus can be divided into address bus, data bus, control bus, etc. For ease of description, Figure 14It is described using only a thick line, but does not indicate that there is only one bus or one type of bus.

[0244] The memory 1401 stores a computer storage medium containing computer-executable instructions. These instructions are used to implement the object type recognition, biometric information recognition, and other methods described in this application. The processor 1402 is used to execute the object type recognition, biometric information recognition, and other methods.

[0245] In another embodiment, the electronic device may also be other electronic devices, such as... Figure 3 The biometric identification device 310 shown is illustrated. In this embodiment, the electronic device can be structured as follows: Figure 15 As shown, it includes components such as: communication component 1510, memory 1520, display unit 1530, camera device 1540, sensor 1550, audio circuit 1560, Bluetooth module 1570, processor 1580, etc.

[0246] The communication component 1510 is used to communicate with the server. In some embodiments, it may include a Circuit-Wireless Fidelity (WiFi) module, which is a short-range wireless transmission technology. Electronic devices can use the WiFi module to help objects (such as users) send and receive information.

[0247] The memory 1520 can be used to store software programs and data. The processor 1580 executes various functions and data processing of the biometric identification device 310 by running the software programs or data stored in the memory 1520. The memory 1520 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. The memory 1520 stores an operating system that enables the biometric identification device 310 to run. In this application, the memory 1520 may store the operating system and various application programs, and may also store a computer program that executes the power mode switching method of the embodiments of this application.

[0248] The display unit 1530 can also be used to display information input by the object or information provided to the object, as well as a graphical user interface (GUI) of various menus of the biometric identification device 310. Specifically, the display unit 1530 may include a display screen 1532 disposed on the front of the biometric identification device 310. The display screen 1532 may be configured as a liquid crystal display, a light-emitting diode, or the like.

[0249] The display unit 1530 can also be used to receive input digital or character information and generate signal inputs related to object setting and function control of the biometric identification device 310. Specifically, the display unit 1530 may include a touch screen 1531 disposed on the front of the biometric identification device 310, which can collect touch operations on or near the object, such as clicking a button, dragging a scroll bar, etc.

[0250] The touchscreen 1531 can be placed on top of the display screen 1532, or the touchscreen 1531 and the display screen 1532 can be integrated to realize the input and output functions of the biometric identification device 310. After integration, it can be referred to as a touch display screen. In this application, the display unit 1530 can display the application program and the corresponding operation steps.

[0251] The camera device 1540 can be used to capture still images, and it mainly includes an image sensor, an image signal processor, and a register array. The image sensor is responsible for capturing images, the image signal processor is responsible for optimizing the images captured by the image sensor, and the register array stores various parameters related to image capture and image optimization; there can be one or more camera devices 1540.

[0252] The biometric identification device may also include at least one sensor 1550, which includes at least a distance sensor 1552 for detecting whether the actual distance between the object to be identified and the biometric identification device has reached a preset distance threshold, so that the processor can further determine whether it is necessary to exit the low-power mode. In addition, other sensors may be equipped according to actual needs, such as a pressure sensor 1551, a fingerprint sensor 1553, etc. The biometric identification device may also be configured with other sensors such as a gyroscope, barometer, hygrometer, thermometer, infrared sensor, light sensor, motion sensor, etc., which are not specifically limited in this application.

[0253] Audio circuitry 1560, speaker 1561, and microphone 1562 provide an audio interface between the object and the biometric identification device 310. Audio circuitry 1560 converts received audio data into electrical signals, which are then transmitted to speaker 1561, where they are converted into sound signals for output. Biometric identification device 310 may also be equipped with volume buttons for adjusting the volume of the sound signal. Conversely, microphone 1562 converts collected sound signals into electrical signals, which are then received by audio circuitry 1560, converted into audio data, and output to communication component 1510 for transmission to, for example, another biometric identification device 310, or to memory 1520 for further processing.

[0254] The Bluetooth module 1570 is used to interact with other Bluetooth devices that also have a Bluetooth module via the Bluetooth protocol. For example, a biometric identification device can establish a Bluetooth connection with a wearable electronic device (such as a smartwatch) that also has a Bluetooth module through the Bluetooth module 1570, thereby exchanging data.

[0255] The processor 1580 is the control center of the biometric identification device. It connects various parts of the terminal via various interfaces and lines. By running or executing software programs stored in the memory 1520 and calling data stored in the memory 1520, it performs various functions of the biometric identification device and processes data, such as object type recognition and biometric information recognition. In some embodiments, the processor 1580 may include one or more processing units; the processor 1580 may also integrate an application processor and a baseband processor, wherein the application processor mainly handles the operating system, object interface, and applications, and the baseband processor mainly handles wireless communication. It is understood that the baseband processor may not be integrated into the processor 1580. In this application, the processor 1580 can run the operating system, applications, object interface display and touch response, and the power consumption mode switching method of this embodiment. Furthermore, the processor 1580 is coupled to the display unit 1530.

[0256] In some possible implementations, various aspects of the power mode switching method provided in this application can also be implemented as a program product, which includes a computer program. When the program product is run on an electronic device, the computer program causes the electronic device to perform the steps in the power mode switching method according to the various exemplary embodiments of this application described above. For example, the electronic device can perform actions such as... Figure 4 or Figure 12 The steps are shown in the figure.

[0257] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0258] The program product of the embodiments of this application may employ a portable compact disc read-only memory (CD-ROM) and include a computer program, and may run on an electronic device. However, the program product of this application is not limited thereto. In this document, the readable storage medium may be any tangible medium that contains or stores a program that may be used by or in conjunction with a command execution system, apparatus, or device.

[0259] A readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying a readable computer program. This propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting a program for use by or in conjunction with a command execution system, apparatus, or device.

[0260] Computer programs contained on readable media may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0261] Computer programs for performing the operations of this application can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The computer program can execute entirely on the target electronic device, partially on the target electronic device, as a standalone software package, partially on the target electronic device and partially on a remote electronic device, or entirely on a remote electronic device or server. In cases involving remote electronic devices, the remote electronic device can be connected to the target electronic device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external electronic device (e.g., via the Internet using an Internet service provider).

[0262] It should be noted that although several units or sub-units of the device have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of this application, the features and functions of two or more units described above can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided and embodied by multiple units.

[0263] Furthermore, although the operations of the method of this application are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all the operations shown must be performed to achieve the desired result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0264] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing a computer-usable computer program.

[0265] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, produce a machine for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0266] These computer program commands may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the commands stored in the computer-readable storage medium produce an article of manufacture including command means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0267] These computer program commands can also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing the commands executed on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0268] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0269] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for switching power consumption modes, characterized in that, The method, applied to biometric identification devices, includes: In the first power consumption mode, a set of original environmental images of the surrounding environment is captured, and the original environmental image set is optimized to obtain a corresponding optimized environmental image set. Based on the optimized environment image set, when it is determined that the surrounding environment meets the preset power consumption switching conditions, an image optimization parameter set is extracted from the optimized environment image set, and the image capture parameter set is adjusted based on the second power consumption mode to be switched; the power consumption corresponding to the second power consumption mode is greater than that of the first power consumption mode. Initialize all parameters other than the image capture parameter set, and reload the image optimization parameter set; Based on the image optimization parameter set and the adjusted image capture parameter set, the second power consumption mode is executed, and under the second power consumption mode, biometric information is captured and identified on the target object.

2. The method as described in claim 1, characterized in that, Determining that the surrounding environment meets the preset power switching conditions can be done using any of the following methods: The optimized environment image set is located, and the objects to be identified contained in each optimized environment image are located. The actual distance between each object to be identified and the biometric identification device is measured. When it is identified that there is at least one object to be identified whose object type is the target type and the corresponding actual distance reaches the preset distance threshold, it is determined that the surrounding environment meets the power consumption switching condition. The light intensity information in each optimized environment image is detected. After determining that the light intensity change in the surrounding environment reaches a preset change threshold, it is determined that the surrounding environment meets the power consumption switching condition.

3. The method as described in claim 2, characterized in that, Whether the object type of each object to be identified is the target type is determined in the following way: Input the optimized environment image containing the object to be identified into the target recognition model; Feature extraction is performed on the optimized environment image to obtain an initial image feature set; the initial image feature set contains the initial image features of the optimized environment image in each feature dimension; Based on the attention mechanism, the information richness of the optimized environment image represented by each initial image feature is combined to obtain the feature weights corresponding to each initial image feature; and based on each feature weight, the corresponding initial image features are adjusted to obtain the target image features corresponding to each initial image feature. Based on the target image features and the corresponding initial image features, the object type of the object to be identified is predicted to obtain the corresponding type prediction value. Based on the type prediction value, determine whether the object type of the object to be identified is the target type.

4. The method according to any one of claims 1 to 3, characterized in that, The step of extracting the image optimization parameter set from the optimized environment image set includes: Select an optimized environment image from the set of optimized environment images; Iterate through each pixel of the optimized environment image and obtain the pixel value distribution of each pixel in different color spaces; The set of image optimization parameters is determined based on the distribution of pixel values.

5. The method according to any one of claims 1 to 3, characterized in that, The initialization of all parameters other than the image capture parameter set, and the reloading of the image optimization parameter set, includes: A restart operation is performed on the registers corresponding to each of the other parameters in the image sensor, and on the image signal processor, to restore each of the other parameters to its default value; The image optimization parameter set is reloaded into the image signal processor.

6. The method as described in claim 3, characterized in that, The target recognition model was obtained in the following way: Obtain a training sample set; wherein each training sample contains a sample image and a sample label, the sample label representing whether the object type of the sample object in the corresponding sample image is the target type; The initial recognition model is trained iteratively based on the training sample set to obtain the target recognition model.

7. The method as described in claim 6, characterized in that, Perform the following operations during each loop iteration: Feature extraction is performed on multiple sample images to obtain multiple initial image feature sets; For each initial image feature in each initial image feature set, based on the attention mechanism, the feature weights corresponding to each initial image feature are obtained by combining the information richness of the corresponding sample image represented by each initial image feature; and based on each feature weight, the corresponding initial image features are adjusted to obtain the target image features corresponding to each initial image feature. For each initial image feature set, based on the target image features and the corresponding initial image features, the object type of the corresponding sample object is predicted to obtain the corresponding type prediction value; The parameters of the initial recognition model are adjusted based on the differences between the predicted values ​​of each type and the corresponding sample labels.

8. The method as described in claim 6, characterized in that, The process of capturing and identifying biometric information of the target object includes: Perform bio-information imaging on the target object to acquire at least one bio-information image; The image quality of the at least one bio-information image is evaluated based on a preset image quality index. After determining that the image quality of the at least one bio-information image meets the preset recognition conditions, the resources occupied by the image sensor and the image signal processor are released and the released resources are allocated to the bio-information recognition task until the next shooting. Biometric identification is performed on the at least one biometric image.

9. The method according to any one of claims 1 to 3, 6 to 8, characterized in that, The method further includes: Acquire the capture time of multiple historical biological information captured within a preset time period; Based on each capture time, the capture pattern of historical biological information in the time dimension was determined; Based on the capture pattern, the power consumption switching cycle of the biometric identification device is determined; At the transition point between power consumption switching cycles, when the power consumption mode of the biometric identification device is determined to be the first power consumption mode, a power consumption mode switching operation is performed.

10. A power consumption mode switching device, characterized in that, include: The shooting unit is used to capture a set of original environmental images of the surrounding environment in a first power consumption mode, and to optimize the set of original environmental images to obtain a corresponding optimized environmental image set. An adjustment unit is configured to, based on the optimized environment image set, determine when the surrounding environment meets preset power consumption switching conditions, extract an image optimization parameter set from the optimized environment image set, and adjust the image capture parameter set based on the second power consumption mode to be switched; the power consumption corresponding to the second power consumption mode is greater than that of the first power consumption mode. The loading unit is used to initialize all parameters other than the image capture parameter set and reload the image optimization parameter set. The execution unit is configured to execute the second power consumption mode based on the image optimization parameter set and the adjusted image acquisition parameter set, and to perform biometric information acquisition and recognition on the target object under the second power consumption mode.

11. An electronic device, characterized in that, It includes a processor and a memory, wherein the memory stores a computer program that, when executed by the processor, causes the processor to perform the steps of any one of the methods described in claims 1 to 9.

12. A computer-readable storage medium, characterized in that, It includes a computer program that, when run on an electronic device, causes the electronic device to perform the steps of any of the methods described in claims 1 to 9.

13. A computer program product, characterized in that, The method includes a computer program stored in a computer-readable storage medium; when a processor of an electronic device reads the computer program from the computer-readable storage medium, the processor executes the computer program, causing the electronic device to perform the steps of any one of claims 1 to 9.

Images