Electronic device and processing method

Abstract

This application discloses an electronic device and a processing method, including: an environmental information sensing module, an image acquisition module, and a detection module. The environmental information sensing module is configured to be continuously active, used to detect environmental information in the environment where the electronic device is located, and when the environmental information meets a first condition, triggers the image acquisition module to start and sends the environmental information to the image acquisition module; the image acquisition module is used to adjust the image acquisition parameters to target acquisition parameters according to the environmental information, and acquire a target image based on the target acquisition parameters; after the quality of the acquired target image meets a preset image quality condition, triggers the detection module to start and sends the target image to the detection module; the detection module is used to perform target object detection on the target image, and controls the working state of the electronic device based on the detection result of the target object.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more particularly to an electronic device and a processing method thereof. Background Technology

[0002] AON (Always-On) low-power sensing systems are widely used in scenarios such as face wake-up and gesture recognition.

[0003] However, when recognizing images to achieve face wake-up or gesture recognition, the modules in the AON low-power sensing system work independently, resulting in no information transmission between them. This causes each module to consume high power to complete face wake-up or gesture recognition. For example, the image acquisition module in the system usually needs to acquire multiple frames to obtain a clear image for recognition, which increases power consumption and fails to demonstrate the advantages of the AON low-power sensing system. Summary of the Invention

[0004] In view of the above, this application provides an electronic device and a processing method, the specific solution of which is as follows:

[0005] An electronic device includes: an environmental information sensing module, an image acquisition module, and a detection module;

[0006] The environmental information sensing module is configured to be continuously turned on, used to detect environmental sensing information in the environment where the electronic device is located, and when the environmental sensing information meets a first condition, to trigger the image acquisition module to start and send the environmental sensing information to the image acquisition module.

[0007] The image acquisition module is used to adjust the image acquisition parameters to the target acquisition parameters according to the environmental perception information, and acquire the target image based on the target acquisition parameters. After the quality of the acquired target image meets the preset image quality conditions, the detection module is triggered to start and the target image is sent to the detection module.

[0008] The detection module is used to detect target objects in the target image and control the working state of the electronic device based on the detection results of the target objects.

[0009] Furthermore, the image acquisition module is also used for:

[0010] Based on the target image, the target model parameters of the model used to perform target object detection in the detection module are determined, and the target model parameters are sent to the detection module;

[0011] The detection module is used for:

[0012] The target image is used to detect the target object using a detection model set as the parameters of the target model, and the detection result is obtained so as to control the working state of the electronic device based on the detection result.

[0013] Furthermore, the detection module is used for:

[0014] The target image is used to detect the target object using a detection model set as the parameters of the target model, and the detection result is obtained. When the detection result meets the second condition, the main system is triggered to start so that the main system can control the switching of the working state of the electronic device.

[0015] Furthermore, the detection module is also used for:

[0016] When it is determined that the detection result does not meet the second condition, the detection result is sent to the image acquisition module so that the image acquisition module can re-acquire the target image and use the detection module to detect the target object in the re-acquired target image.

[0017] Furthermore, the environmental information sensing module is also used for:

[0018] After the image acquisition module is started, it switches to a first low-power state where the power consumption is lower than that of the continuously on state;

[0019] The image acquisition module is further configured to: after the detection module is started, switch to a second low-power state with power consumption lower than the start-up state, wherein the start-up state is when the image acquisition module is in a state in which it can adjust the image acquisition parameters to the target acquisition parameters and acquire the target image based on the target acquisition parameters.

[0020] Furthermore, the environmental information sensing module includes: a distance sensor, a brightness sensor, and a motion sensor;

[0021] The distance sensor is used to collect object movement information, and when the object movement information indicates that there is a moving object in the environment where the electronic device is located, the brightness sensor is triggered to start.

[0022] The brightness sensor is used to collect ambient brightness information in the environment where the electronic device is located, and to trigger the motion sensor to start when the ambient brightness information meets the brightness threshold.

[0023] The motion sensor is used to detect the motion information of the moving object. When the motion information meets the motion conditions, it is determined that the environmental perception information meets the first condition, and the image acquisition module is triggered to start.

[0024] Furthermore, the environmental information sensing module also includes:

[0025] The feedback control unit is used to obtain the detection results transmitted by the detection module, adjust the acquisition frequency of the distance sensor when collecting object movement information based on the detection results, and adjust the detection frequency of the motion sensor when detecting the motion information.

[0026] Furthermore, the image acquisition module adjusts the image acquisition parameters to the target acquisition parameters based on the environmental perception information, including:

[0027] The image acquisition module adjusts the focus parameter in the image acquisition parameters according to the object movement information, and adjusts the exposure parameter in the image acquisition parameters according to the ambient brightness information, motion information, and preset image brightness data.

[0028] A processing method includes:

[0029] The environmental information sensing module, which is configured to be continuously on, detects environmental sensing information in the environment in which the electronic device is located.

[0030] In response to the environmental perception information meeting the first condition, the image acquisition module is triggered to start, and the environmental information perception module is controlled to send the environmental perception information to the image acquisition module;

[0031] The image acquisition module is controlled to acquire a target image based on target acquisition parameters, wherein the target acquisition parameters are obtained by adjusting the image acquisition parameters based on the environmental perception information.

[0032] Once the target image is acquired by the image acquisition module and the target image quality meets the preset image quality conditions, the detection module is triggered to start and the target image is sent to the detection module.

[0033] The detection module performs target object detection on the target image, and controls the working state of the electronic device based on the detection results of the target object.

[0034] Furthermore, it also includes:

[0035] In response to the activation of the image acquisition module, the environmental information sensing module is controlled to switch to a first low-power state where the power consumption is lower than that of the continuously on state.

[0036] In response to the activation of the detection module, the image acquisition module is controlled to switch to a second low-power state with power consumption lower than that of the activation state. The activation state is when the image acquisition module is in a state where it can adjust the image acquisition parameters to the target acquisition parameters and acquire the target image based on the target acquisition parameters. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0038] Figure 1 This is a schematic diagram of the structure of an electronic device disclosed in an embodiment of this application;

[0039] Figure 2 This is a schematic diagram of the structure of an electronic device disclosed in an embodiment of this application;

[0040] Figure 3 This is a schematic diagram of the structure of an environmental information sensing module disclosed in an embodiment of this application;

[0041] Figure 4 This is a schematic diagram of the structure of an electronic device disclosed in an embodiment of this application;

[0042] Figure 5 This is a schematic diagram of an electronic device and its implementation process disclosed in an embodiment of this application;

[0043] Figure 6 This is a structural block diagram of the modules included in an electronic device disclosed in an embodiment of this application;

[0044] Figure 7 This is a schematic diagram of the structure of an electronic device disclosed in an embodiment of this application. Detailed Implementation

[0045] The embodiments of this application are described below with reference to the accompanying drawings. The terminology used in the implementation section of this application is for explaining specific embodiments only and is not intended to limit the scope of this application.

[0046] The embodiments of this application will now be described with reference to the accompanying drawings. Those skilled in the art will recognize that, with technological advancements and the emergence of new scenarios, the technical solutions provided in the embodiments of this application are equally applicable to similar technical problems.

[0047] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of elements is not necessarily limited to those elements, but may include other elements not explicitly listed or inherent to those processes, methods, products, or apparatuses.

[0048] This application discloses an electronic device, the schematic diagram of which is shown below. Figure 1 As shown, it includes:

[0049] The system includes an environmental information sensing module 11, an image acquisition module 12, and a detection module 13.

[0050] The environmental information sensing module 11 is configured to be continuously turned on, used to detect environmental sensing information in the environment where the electronic device is located, and when the environmental sensing information meets the first condition, it triggers the image acquisition module to start and sends the environmental sensing information to the image acquisition module.

[0051] The image acquisition module 12 is used to adjust the image acquisition parameters to the target acquisition parameters according to the environmental perception information, and acquire the target image based on the target acquisition parameters. After the quality of the acquired target image meets the preset image quality conditions, the detection module is triggered to start and the target image is sent to the detection module.

[0052] The detection module 13 is used to detect target objects in the target image and control the working state of the electronic device based on the detection results of the target objects.

[0053] AON (Always-On) low-power sensing systems are widely used in scenarios such as face wake-up and gesture recognition.

[0054] However, when recognizing images to achieve face wake-up or gesture recognition, the modules in the AON low-power sensing system work independently, resulting in no information transmission between them. This causes each module to consume high power to complete face wake-up or gesture recognition. For example, the image acquisition module in the system usually needs to acquire multiple frames to obtain a clear image for recognition, which increases power consumption and fails to demonstrate the advantages of the AON low-power sensing system.

[0055] Based on this, in the electronic device disclosed in this solution, only the environmental information sensing module 11 is always in the on state. It is used to detect the environmental sensing information in the environment where the electronic device is located. When it is determined that the environmental sensing information meets the first condition, it triggers the image acquisition module 12 to start. Then, the image acquisition module 12 is used to acquire images and triggers the detection module 13 to start so that the detection module 13 can detect the acquired images, thereby realizing the control of the working state of the electronic device. This realizes the phased wake-up of different modules in the electronic device and effectively reduces the power consumption of the electronic device. In addition, when the environmental information sensing module 11 determines that the environmental sensing information meets the first condition, it triggers the image acquisition module 12 to start and sends the environmental sensing information to the image acquisition module 12. This allows the image acquisition module 12 to first adjust the image acquisition parameters based on the environmental information sensing module 11 and then acquire the target image based on the adjusted parameters. This ensures that the acquired target image conforms to the environmental sensing information and that the acquired image is a valid and high-quality image. The acquired image can be used to adjust the working state of the electronic device without repeated acquisition.

[0056] The electronic device disclosed in this embodiment can be a desktop computer, a laptop computer, a tablet computer, or a smart mobile terminal (such as a mobile phone). An AON low-power sensing system can be applied to the electronic device to achieve functions such as waking up, unlocking, or gesture control. Furthermore, when the AON low-power sensing system is applied to the electronic device, the display screen can be in a screen-off state, meaning that unlocking, waking up, or gesture control can be performed while the screen is off.

[0057] The AON low-power sensing system in the electronic device can be implemented by an environmental information sensing module 11, an image acquisition module 12, and a detection module 13. The environmental information sensing module 11, the image acquisition module 12, and the detection module 13 can also be applied to other functions in the electronic device, which are not specifically limited here.

[0058] Taking the application of the AON low-power sensing system in the screen-off unlocking scenario of electronic devices as an example: When the electronic device meets the low-power sensing conditions, the AON low-power sensing system is activated. At this time, the environmental information sensing module 11 in the electronic device is continuously turned on, while the image acquisition module 12 and the detection module 13 are not turned on to ensure the overall power consumption of the electronic device is reduced. Specifically, meeting the low-power sensing conditions means that the display screen of the electronic device is in a screen-off state. In addition, the AON low-power sensing system can also be applied to the automatic screen-locking scenario of electronic devices. In this scenario, the AON low-power sensing system is always on, meaning that no low-power sensing conditions need to be set, and the AON low-power sensing system is always running. Alternatively, in either scenario, the AON low-power sensing system is always on. When the automatic screen-locking conditions are met, the AON low-power sensing system is used to automatically lock the screen; when the screen-off unlocking conditions are met, the AON low-power sensing system is used to unlock the screen, and so on.

[0059] If the environmental information sensing module 11 is continuously turned on, it continuously detects the environmental sensing information in the environment where the electronic device is located and determines whether the detected environmental sensing information meets the first condition. If it is determined that the detected environmental sensing information meets the first condition at a certain moment, it triggers the image acquisition module 12 to start and sends the environmental sensing information that meets the first condition to the image acquisition module 12 so that the image acquisition module 12 can receive the environmental sensing information after it is started.

[0060] After the image acquisition module 12 is started, it receives environmental perception information sent by the environmental information perception module 11. Based on this environmental perception information, the image acquisition module 12 can adjust the image acquisition parameters to the target acquisition parameters and acquire the target image based on the target acquisition parameters.

[0061] In other words, the target acquisition parameters are matched with the currently received environmental perception information. The target image acquired based on the target acquisition parameters is more consistent with the current environmental perception information, which ensures the quality of the target image. This enables the image acquisition module 12 to obtain a high-quality image with only one acquisition, without the need for multiple acquisitions and adjustments.

[0062] For example, the environmental perception information includes the ambient brightness. Based on the current ambient brightness, the exposure value in the image acquisition parameters of the image acquisition module 11 is adjusted so that the adjusted exposure value can match the current ambient brightness. Based on this, the acquired image matches the current ambient brightness, avoiding situations where the acquired image is too bright or too dark, thereby ensuring the quality of the acquired image.

[0063] In existing solutions, an image is first acquired through an image acquisition module (e.g., image 1). Then, the image acquisition parameters are adjusted based on the acquired image, and another image is acquired according to the adjusted parameters (e.g., image 2). The quality of image 2 is higher than that of image 1. However, this method requires multiple image acquisitions. In this solution, the environmental information sensing module 11 directly obtains the environmental sensing information and adjusts the image acquisition parameters based on the environmental sensing information. The image is then acquired directly based on the adjusted parameters. In this case, the quality of the acquired image is directly superior to that of image 1, reducing the number of image acquisitions and improving the efficiency of acquiring high-quality images.

[0064] After the image acquisition module 12 acquires the target image, it is necessary to determine the quality of the acquired target image to see if the target image quality meets the preset image quality conditions. Only when the target image quality meets the preset image quality conditions can the detection module 13 be triggered to start and send the target image to the detection module 13 after it has started, so that the detection module 13 can receive the target image.

[0065] If the target image quality does not meet the preset image quality conditions, the image acquisition module 12 needs to be controlled to re-acquire the target image; or, the image acquisition module 12 readjusts the image acquisition parameters based on the target image that does not meet the preset image quality conditions, and re-acquires the target image using the image acquisition module 12 with the readjusted image acquisition parameters; or, the image acquisition module 12 sends the result that the acquired target image quality does not meet the preset image quality conditions to the environmental information perception module 11, so that the environmental information perception module 11 re-detects the environmental perception information and sends the environmental perception information to the image acquisition module 12, so that the image acquisition module 12 readjusts the image acquisition parameters based on the re-acquired environmental perception information, and re-acquires the target image using the image acquisition module 12 with the readjusted image acquisition parameters.

[0066] After the detection module 13 is started, the detection module 13 performs target object detection on the received target image. That is, the detection module 13 analyzes the target image to determine whether the target image contains a target object, such as whether the target image contains a face, or whether the target image contains gestures, etc.

[0067] The detection module 13 controls the working state of the electronic device based on the detection result of the target object in the target image. Specifically, if it is determined that the detection result of the target object in the target image meets the target conditions, the working state of the electronic device can be switched; if it is determined that the detection result of the target object in the target image does not meet the target conditions, the working state of the electronic device will not be switched.

[0068] Specifically, for cases where the target conditions are met, the following can be included: if it is determined that the detection result of the target object in the target image meets the first target condition, the electronic device can be controlled to switch from the first working state to the second working state; if it is determined that the detection result of the target object in the target image meets the second target condition, the electronic device can be controlled to switch from the first working state to the third working state.

[0069] For example, if a face is detected in the target image, the electronic device can be controlled to switch from the current standby state to the system running state; if no face is detected, the electronic device can be controlled to maintain the current standby state.

[0070] The image acquisition module 12 can be located in the microcontroller unit (MCU) of the electronic device, while the detection module 13 can be located in the dedicated processor of the electronic device. This dedicated processor can be: an embedded neural network processor (eNPU) used for model inference on the device side, optimized for executing AI models (such as convolution and matrix calculations), with low power consumption, typically hardware-accelerated for fixed model architectures, and highly efficient, but with low flexibility; or a programmable neural network processor (pNPU) used for flexible and efficient model calculation, which can be configured by software to adapt to different AI models and algorithms, and can handle more complex or newer models while maintaining high energy efficiency; or a digital signal processor (DSP) used for streaming media signal processing, which is good at fixed-point or floating-point operations on continuous data, with low latency. The core task of the DSP is to process digital signals after analog signals such as audio, video, and sensor signals are converted.

[0071] In addition, the environmental information sensing module 11 can be set in the microcontroller unit (MCU) or the digital signal processor (DSP).

[0072] It should be noted that if the detection module 13 in this embodiment is an NPU, then the detection module 13 can be a low-power NPU rather than the main NPU. Alternatively, the detection module 13 can be used in a low-power state under the main NPU. Whether it is a low-power NPU or a low-power state under the main NPU, the power consumption of the detection module 13 during operation is low, so as to meet the low-power requirements of the AON low-power sensing system.

[0073] The electronic device disclosed in this embodiment includes an environmental information sensing module 11, an image acquisition module 12, and a detection module 13. Only the environmental information sensing module 11 is configured to be continuously turned on. The environmental information sensing module is used to trigger the image acquisition module 12 to start when the environmental sensing information meets a first condition, and to send the environmental sensing information to the image acquisition module 12. After the image acquisition module 12 starts, it can first adjust the image acquisition parameters based on the environmental sensing information, so that the image acquisition module 12 can acquire the target image based on the adjusted image acquisition parameters. After the target image meets the preset image quality conditions, the detection module 13 is triggered to start and detect the target image in order to control the working state of the electronic device. This solution ensures that the environmental information sensing module 11 remains continuously active. Only when the environmental information meets the first condition will the image acquisition module 12 and the detection module 13, which consume relatively high power, be activated. This strategy results in lower power consumption for the electronic device. In addition, when the image acquisition module 12 is activated, it directly receives the current environmental sensing information sent by the environmental information sensing module 11. This allows the image acquisition module 12 to adjust its image acquisition parameters before acquiring an image, ensuring that the adjusted parameters conform to the current environment. This results in images that match the current environment, avoiding the acquisition of invalid or low-quality images and improving the detection success rate.

[0074] This embodiment discloses an electronic device, the structural schematic diagram of which is shown below. Figure 1 As shown, it includes:

[0075] The system includes an environmental information sensing module 11, an image acquisition module 12, and a detection module 13.

[0076] In addition to the same structure as the previous embodiment, in the electronic device disclosed in this embodiment, the image acquisition module 12 is also used to: determine the target model parameters of the model used by the detection module 13 to perform target object detection based on the target image, and send the target model parameters to the detection module 13; the detection module 13 is also used to: perform target object detection on the target image using the detection model set as the target model parameters, and obtain the detection result, so as to control the working state of the electronic device based on the detection result.

[0077] After acquiring the target image, the image acquisition module 12 can analyze the characteristics of the target image. The characteristics of the target image can be: the scene corresponding to the target image, such as: indoor or outdoor; the brightness of the target image; and the distance of objects in the target image, etc.

[0078] Different target images have different characteristics, which leads to different model parameters for the models used to detect and analyze the target objects within them. In other words, models with different parameters excel at detecting different types of objects. For example, a model with the first set of parameters may have a higher accuracy rate in detecting the first type of object than models with other parameters; a model with the second set of parameters may have a higher accuracy rate in detecting the first scene than models with other parameters. Therefore, the detection types that models with different parameters excel at can be predetermined. Then, after determining the characteristics of the target image, the corresponding model parameters can be determined based on these characteristics.

[0079] After the target model parameters are determined, the image acquisition module 12 triggers the start of the detection module 13 and sends the target image and target model parameters to the detection module 13. After receiving the target model parameters, the detection module 13 can configure the detection model according to the target model parameters. After the configuration is completed, the detection module 13 performs target object detection on the target image based on the configured detection model to ensure the accuracy of the detection model in detecting the target object.

[0080] After receiving the target model parameters, the detection module 13 can determine the detection model in the detection module 13. If there is only one detection model in the detection module 13, the model parameters of the detection model are configured directly according to the target model parameters to obtain the detection model configured as the target model parameters. Then, the detection model configured as the target model parameters is used to detect the target object in the target image, so that the configured detection model has high accuracy in detecting the target object in the target image.

[0081] In addition, if there are multiple detection models in the detection module, a model can be selected from the multiple detection models according to the target model parameters. The selected model is used as the target model, and the target model is used to detect the target object in the target image. Selecting a model from multiple detection models involves determining the model parameters of each detection model. Then, the model parameters of each detection model are compared with the target model parameters, and the detection model with the closest target model parameters is selected as the target model to ensure the accuracy of the detection of the target object in the target image.

[0082] Alternatively, one could select one of multiple detection models as the target model based on the target model parameters. If the model parameters of the target model do not perfectly match the target model parameters, the model parameters of the target model can be adjusted to ensure that they perfectly match the target model parameters. This further guarantees the accuracy of the detection of the target object in the target image. Furthermore, adjusting the model parameters of the selected model involves smaller adjustments, which improves the efficiency of the adjustment and avoids large changes in the model parameters.

[0083] It should be noted that in the electronic device disclosed in this embodiment, the process of determining the target model parameters based on the target image is implemented by the image acquisition module 12. After acquiring the target image, the image acquisition module 12 can simultaneously execute the process of determining the target model parameters and triggering the detection module 13 to start. By using parallel processing, the data processing process is effectively reduced, ensuring that the target model parameters can be received after the detection module 13 starts, and the detection model can be configured based on the received target model parameters. During this process, the detection module 13 does not need to perform additional calculations, reducing the delay caused by calculations and speeding up the overall speed from triggering to the final control response.

[0084] In addition, in the electronic device disclosed in this embodiment, the process of determining the target model parameters based on the target image can also be performed by other modules, such as: a model management module. After the image acquisition module 12 acquires the target image, it sends the target image to the model management module. The model management module records the model information of the detection model in the detection module 13. The model management module analyzes the target image it receives, determines the target model parameters, and adjusts the model parameters in the model information so that the model parameters of the detection model are the target model parameters, that is, the parameter configuration of the detection model is completed. Then, the detection module 13 is triggered to start. After the detection module 13 starts, it loads the detection model after the parameter configuration is completed, obtains the target image, runs the detection model, and uses the detection model to detect the target object in the target image.

[0085] Alternatively, the model management module may analyze only the target image to determine the target model parameters, and then send the target model parameters to the detection module 13 after startup, so that the detection model in the detection module 13 can be configured according to the target model parameters, and the configured detection model can be used to detect the target object in the target image.

[0086] The model management module can be set together with the image acquisition module 12 in the microcontroller unit (MCU).

[0087] The electronic device disclosed in this embodiment includes an environmental information sensing module 11, an image acquisition module 12, and a detection module 13. The environmental information sensing module 11 detects environmental information in the environment where the electronic device is located. When the environmental information meets a first condition, it triggers the image acquisition module 12 to start. The image acquisition module 12 adjusts the image acquisition parameters based on the environmental information, acquires a target image, and judges the quality of the target image. After acquiring a target image that meets the preset image quality conditions, it determines the target model parameters based on the target image. Then, it triggers the detection module 13 to start. The detection module 13 sets a detection model based on the target model parameters and detects the target image based on the detection model set with the target model parameters, so as to control the working state of the electronic device. In this solution, after the image acquisition module 12 acquires an image that meets the preset image quality conditions, it can determine the model parameters that best match the target based on the analysis of the acquired image. This allows the detection model in the detection module 13 to detect the image according to the model parameters determined by the image acquisition module 12, thereby ensuring the accuracy of detection when the parameters of the detection model match the characteristics of the image.

[0088] This embodiment discloses an electronic device, the structural schematic diagram of which is shown below. Figure 1 As shown, it includes:

[0089] The system includes an environmental information sensing module 11, an image acquisition module 12, and a detection module 13.

[0090] In addition to the same structure as the previous embodiment, in the electronic device disclosed in this embodiment, the detection module 13 can be specifically used to: use the detection model set as the target model parameter to perform target object detection on the target image, obtain the detection result, and trigger the main system to start when the detection result meets the second condition, so that the main system can control the switching of the working state of the electronic device.

[0091] When the detection result meets the second condition, the main system is triggered to start, and the main system executes the process of switching the working state of the electronic device. That is, the detection module 13 only detects the target image, determines whether the detection result meets the second condition, and triggers the main system to start when the second condition is met.

[0092] Among them, determining whether the detection result meets the second condition, that is, determining whether there is a target object in the target image, the target object can be a gesture or a face. Accordingly, determining whether the detection result meets the second condition can be specifically: determining whether the user's gesture was detected, or determining whether a face was detected.

[0093] Alternatively, determining whether the detection result meets the second condition can also be: whether a face with a number of feature points greater than a specific threshold is detected. A face with a number of feature points greater than a specific threshold indicates a higher degree of clarity of the face, and it can be accurately determined whether the working state of the electronic device can be switched based on the current target image.

[0094] When the detection module 13 determines that the detection result meets the second condition, it triggers the main system to start, and the main system further determines whether the working state of the electronic device can be switched.

[0095] Specifically, the structural schematic diagram of the electronic device disclosed in this embodiment can be shown as follows: Figure 2 As shown, it includes: an environmental information perception module 21, an image acquisition module 22, a detection module 23, and a main system 24.

[0096] The detection module 23 detects the target image only to determine whether a target object exists, while the main system 24 needs to further determine whether the target object meets the corresponding conditions. For example, the main system 24 determines whether the target object meets the matching conditions. If the main system 24 determines that the target object matches the pre-stored object features, it can be determined that the matching conditions are met; if the main system 24 determines that the target object does not match the pre-stored object features, it can be determined that the matching conditions are not met.

[0097] When the main system 24 determines that the target object meets the matching conditions, it controls the switching of the working state of the electronic device; when the main system 24 determines that the target object does not meet the matching conditions, it controls the switch of the working state of the electronic device.

[0098] Furthermore, regardless of whether the main system 24 determines whether the target object meets the matching conditions, the final result needs to be fed back to the image acquisition module 22.

[0099] If the main system 24 determines that the target object meets the matching conditions, it controls the switching of the working state of the electronic device and feeds back the result of the target object meeting the matching conditions in the target image to the detection module 23, and then feeds it back to the image acquisition module 22 through the detection module 23, and then feeds it back to the environmental information perception module 21 through the image acquisition module 22, so that each module can clearly understand that the switching of the working state of the electronic device has been realized based on the target image acquired under the environmental perception information. Each module can record the environmental perception information, the target acquisition parameters determined based on the environmental perception information, and the completion of the switching of the working state of the electronic device, so that the same environmental perception information can be detected again in the future and the target acquisition parameters can be determined by referring to the record, so as to improve the processing efficiency.

[0100] Alternatively, when the main system 24 determines that the target object meets the matching conditions, while controlling the switching of the working state of the electronic device, it only feeds back the result that the target object in the target image meets the matching conditions to the detection module 23, and feeds it back to the image acquisition module 22 through the detection module 23. The corresponding information is stored in the detection module 23 and the image acquisition module 22 respectively, so that when the image acquisition module 22 receives the same environmental perception information again, it can refer to the record to determine the target acquisition parameters. This process can bypass the environmental information perception module 21, thereby reducing the data recorded in the environmental information perception module 21 and ensuring data processing efficiency.

[0101] Alternatively, the main system 24 can directly send the result of the target object in the target image meeting the matching conditions to the image acquisition module 22 without going through the detection module 23. As long as there is a data transmission path between the image acquisition module 22 and the main system 24, this method can also ensure data processing efficiency.

[0102] Furthermore, if the detection result does not meet the second condition, the detection result is sent to the image acquisition module 22 so that the image acquisition module 22 can re-acquire the target image and use the detection module 23 to detect the target object in the re-acquired target image.

[0103] If the main system 24 determines that the target object does not meet the matching conditions, then there is no need to control the working state of the electronic device to switch. Instead, the result is directly fed back to the image acquisition module 22. Alternatively, the main system 24 can send the result to the detection module 23, and then the detection module 23 can send the result to the image acquisition module 22.

[0104] After the image acquisition module 22 receives the result that the target object in the target image does not meet the matching conditions, the image acquisition module 22 needs to acquire the image again based on the result that the target object in the target image does not meet the matching conditions, so that the re-acquired image can be used as a new target image and sent to the detection module 23. The detection module 23 detects the new target image and obtains a new detection result. After the new detection result meets the second condition, the main system 24 performs object matching on the target object in the new target image to determine whether the matching conditions are met. If they are still not met, the result that the matching conditions are not met needs to be sent to the image acquisition module 22 again until the target object in the re-acquired image meets the matching conditions, and then the working state of the electronic device is switched.

[0105] In this scenario, after the detection module 23 determines that a new detection result meets the second condition, the main system 24 can be restarted. This requires the main system 24 to exit the current startup state after sending a result that does not meet the matching condition to the image acquisition module 22, until the detection module 23 determines that a new detection result meets the second condition, and then restart the main system 24. Alternatively, the main system 24 can remain in the startup state after sending a result that does not meet the matching condition to the image acquisition module 22, until the main system 24 controls the switching of the working state of the electronic device, and then determines whether the main system 24 needs to exit the startup state based on the switched working state.

[0106] In addition, after the image acquisition module 22 obtains the result that the target object in the target image does not meet the matching conditions, since the result of not meeting the matching conditions is due to the mismatch of the target object in the target image, rather than due to the low pixel count or poor brightness of the target image, there is no need to readjust the image acquisition parameters. The image can be acquired again directly according to the current target acquisition parameters.

[0107] It should be noted that when the main system 24 determines that the target object does not meet the matching conditions, it can collect the target image multiple times. When the number of collections reaches the target threshold, if it is still determined that the matching conditions are not met, the current process is terminated and the target image is no longer collected again. Instead, all modules except the environmental information perception module 21 are controlled to exit the startup state, and only the environmental information perception module 21 is kept in the open state to continue to detect environmental information, so as to avoid the increase in power consumption caused by repeated detection.

[0108] This embodiment discloses an electronic device, the structural schematic diagram of which is shown below. Figure 1 As shown, it includes:

[0109] The system includes an environmental information sensing module 11, an image acquisition module 12, and a detection module 13.

[0110] In addition to the same structure as the previous embodiment, in the electronic device disclosed in this embodiment, the environmental information sensing module 11 is also used to: switch to a first low-power state where the power consumption is lower than that of the continuously on state after the image acquisition module 12 is started; the image acquisition module 12 is also used to: switch to a second low-power state where the power consumption is lower than that of the start state after the detection module 13 is started, the start state being that the image acquisition module 12 is in a state where it can adjust the image acquisition parameters to the target acquisition parameters and acquire the target image based on the target acquisition parameters.

[0111] In the electronic device disclosed in this embodiment, only the environmental information sensing module 11 is configured to be continuously turned on, while the image acquisition module 12 and the detection module 13 will only switch to the start-up state after being triggered, so as to ensure that the power consumption of the electronic device is low and avoid the increase in power consumption caused by the environmental information sensing module 11, the image acquisition module 12 and the detection module 13 all being in the start-up state.

[0112] When the environmental information sensing module 11 detects that the environmental sensing information in the environment where the electronic device is located meets the first condition, it triggers the image acquisition module 12 to start and sends the environmental sensing information to the image acquisition module 12. Then, the image acquisition module 12 adjusts the image acquisition parameters and acquires the image. During this process, the environmental information sensing module 11 does not need to continue to detect the environmental sensing information. Therefore, after the image acquisition module 12 starts and receives the environmental sensing information, it can control the environmental information sensing module 11 to switch to the first low-power state, that is, control the environmental information sensing module 11 to exit the current continuously on state.

[0113] In the first low-power state, the power consumption of the environmental information sensing module 11 is lower than that of the environmental information sensing module 11 when it is continuously turned on. That is, after the image acquisition module 12 is started, the environmental information sensing module 11 can be controlled to switch to a lower power consumption state. So after the image acquisition module 12 is started, only the image acquisition module 12 is in the started state, while other modules, such as the environmental information sensing module 11 and the detection module 13, are in a relatively low power consumption state to avoid high power consumption of electronic devices.

[0114] The first low-power state of the environmental information sensing module 11 can directly control each sensor in the environmental information sensing module 11 to be in standby mode, or reduce the frequency of each sensor in the environmental information sensing module 11, so as to reduce the power consumption of the environmental information sensing module 11.

[0115] Correspondingly, after the image acquisition module 12 triggers the detection module 13 to start, the image acquisition module 12 sends the target image to the detection module 13. After that, the image acquisition module 12 switches to the second low-power state, that is, controls the image acquisition module 12 to exit the current start state. The start state is the state in which the image acquisition module 12 is running normally, such as: the image acquisition module 12 is in a state in which it can adjust the image acquisition parameters to the target acquisition parameters and acquire the target image based on the target acquisition parameters.

[0116] In the second low-power state, the power consumption of the image acquisition module 12 is lower than that of the image acquisition module 12 when it is in the start state. That is, after the detection module 13 is started, the image acquisition module 12 can be controlled to switch to a lower power consumption state. So after the detection module 13 is started, only the detection module 13 is in the start state, while other modules are in a relatively low power consumption state, so as to avoid the problem of high power consumption of electronic devices.

[0117] The second low-power state of the image acquisition module 12 can be: all devices in the image acquisition module 12 are in standby mode; or, the voltage of each device in the image acquisition module 12 is reduced to reduce the overall power consumption of the image acquisition module 12; or, different devices in the image acquisition module 12 are in different low-power states, such as: the focusing motor in the image acquisition module 12 is directly powered off, the image acquisition sensor in the image acquisition module 12 is in standby mode, that is, only the core power supply is maintained, and the pixel array and readout circuit are turned off; or, the image acquisition sensor is in deep sleep mode, at which time the image acquisition sensor can only be woken up by a specific hardware signal, and the power consumption is very low. In addition, the image signal processor and related circuits in the image acquisition module 12 can be directly powered off. Even if the image signal processor is powered on again and runs, it will not consume much time, but direct power off can ensure the lowest power consumption. Alternatively, the image signal processor is not directly powered off, but the operating parameters of the image signal processor are reduced, such as: reducing the operating voltage of the image signal processor.

[0118] Specifically, the structural schematic diagram of the environmental information sensing module 11 in the electronic device disclosed in this embodiment can be shown as follows: Figure 3 As shown, it includes: a distance sensor 31, a brightness sensor 32, and a motion sensor 33.

[0119] Among them, the distance sensor 31 is used to collect object movement information. When the object movement information indicates that there is a moving object in the environment where the electronic device is located, the brightness sensor 32 is triggered to start.

[0120] The brightness sensor 32 is used to collect ambient brightness information in the environment where the electronic device is located. When the ambient brightness information meets the brightness threshold, the motion sensor 33 is triggered to start.

[0121] The motion sensor 33 is used to detect the motion information of a moving object. When the motion information meets the motion conditions, it determines that the environmental perception information meets the first condition and triggers the image acquisition module 12 to start.

[0122] The environmental information sensing module 11 includes multiple different sensors. In this embodiment, not all of the multiple different sensors included in the environmental information sensing module 11, which is in a continuously active state, are in an active state.

[0123] Specifically, the environmental information sensing module 11 may include a distance sensor 31, a brightness sensor 32, and a motion sensor 33. The distance sensor 31 may be continuously turned on in the environmental information sensing module 11, while the brightness sensor 32 and the motion sensor 33 may be in a low power consumption state, such as being in a standby state. When triggered to start, they can directly switch from the standby state to the start state, so as to reduce the start-up delay while ensuring low power consumption.

[0124] The distance sensor 31 is continuously on and is used to detect object movement information. That is, the distance sensor 31 uses information about the environment in which the electronic device is located. When there is a moving object in the environment, the distance sensor 31 can detect that the object has moved, that is, it detects the object movement information. Specifically, the distance sensor 31 can be a TOF (Time-of-Flight) sensor.

[0125] When the distance sensor 31 detects that there is a moving object in the environment, it triggers the start of the brightness sensor AIS Sensor 32. After the brightness sensor 32 is started, it collects the ambient brightness information of the environment in which the electronic device is currently located to determine whether the current ambient brightness information is within a certain threshold range, that is, whether the current ambient brightness information meets the brightness threshold.

[0126] Specifically, the brightness threshold can be a brightness threshold range. If the ambient brightness value in the current ambient brightness information is less than the minimum value in the brightness threshold range, it indicates that the current ambient brightness value is low. If an image is captured at this time, the ambient brightness in the captured image will be dark, affecting the image quality. Therefore, when the current ambient brightness value is less than the minimum value in the brightness threshold range, subsequent detection can be omitted, that is, the motion sensor 33 is not triggered to start. Instead, the brightness sensor 32 is controlled to exit the current start state, and the distance sensor 31 continues to detect the object movement information in the environment. For example, if the current environment is completely dark, even if an image is captured, the image acquisition module 12 cannot detect a face in the image it captures. Therefore, when the ambient brightness value is less than the minimum value in the brightness threshold range, no further detection is required.

[0127] If the ambient brightness value in the current ambient brightness information is greater than the maximum value in the brightness threshold range, it indicates that the current ambient brightness value is high. If an image is captured at this time, the ambient brightness in the captured image will be bright, which will also affect the image quality. Therefore, when the current ambient brightness value is greater than the maximum value in the brightness threshold range, subsequent detection can be omitted, that is, the motion sensor 33 is not triggered to start, but the brightness sensor 32 is controlled to exit the current start state, and the distance sensor 31 continues to detect the object movement information in the environment.

[0128] If the ambient brightness value in the current ambient brightness information is within the brightness threshold range, it indicates that the brightness in the current environment is relatively moderate. At this time, even if the ambient brightness is not at the optimal value, the quality of the acquired image can be adjusted by adjusting the image acquisition parameters in the image acquisition module 12. Therefore, when it is determined that the ambient brightness value is within the brightness threshold range, the brightness sensor 32 directly triggers the motion sensor 33 to start.

[0129] After the motion sensor 33 is activated, it detects the motion information of the moving object. If the motion information meets the motion conditions, it can be determined that the environmental perception information meets the first condition. At this time, the image acquisition module 12 is activated.

[0130] The motion sensor 33 is used to detect whether the moving object is undergoing violent motion. If it is determined that the moving object is undergoing violent motion, and an image is acquired at this time, the image acquired by the image acquisition module 12 may be blurred due to the violent motion of the moving object. Therefore, when it is determined that the moving object is undergoing violent motion, it can be determined that the motion information of the moving object does not meet the motion conditions. If it is determined that the moving object is not undergoing violent motion, it can be determined that the motion conditions are met, and the image acquisition module 12 can be triggered to acquire an image.

[0131] In determining whether a moving object has undergone violent movement, the motion sensor 33 can detect whether the value detected within a short period of time changes by an amplitude greater than a certain threshold. If the value detected within a short period of time changes by an amplitude greater than the certain threshold, it can be determined that the moving object has undergone violent movement. If the amplitude changes of the values ​​detected within a short period of time are all less than the threshold, it can be determined that the moving object has not undergone violent movement.

[0132] In addition, the motion sensor 33 can also be used to detect whether the electronic device is moving violently. When the electronic device is moving violently, the image captured by the image acquisition module 12 will also appear blurry. Therefore, the motion sensor 33 can also be used to detect whether the electronic device is moving violently. When it is determined that the electronic device is moving violently, the image acquisition module 12 is not triggered. At this time, the motion sensor 33 and the brightness sensor 32 are both kept in a low power consumption state, and only the distance sensor 31 is kept in a continuously turned-on state. When it is determined that the electronic device is not moving violently, the image acquisition module 12 can be triggered.

[0133] Alternatively, the motion sensor 33 can be a gyroscope, which is installed on the electronic device to detect the motion state of the electronic device, i.e., to detect whether the electronic device is undergoing violent motion; the motion sensor 33 can also be an accelerometer, which is installed on the electronic device to detect the motion state of a moving object, i.e., to detect whether the moving object is undergoing violent motion; the motion sensor 33 can also be other sensors, which are not specifically limited here.

[0134] When the distance sensor 31 triggers the brightness sensor 32 to start, the distance sensor 31 can switch to a low power consumption state. Similarly, when the brightness sensor 32 triggers the motion sensor 33 to start, the brightness sensor 32 can also switch to a low power consumption state to ensure low power consumption operation of the electronic device.

[0135] Specifically, the distance sensor 31 can switch to a lower power consumption state by: reducing the sampling frequency of the distance sensor 31, such as switching from continuous real-time detection to intermittent detection; reducing the transmission power, such as reducing the transmission power of the light pulse or radio wave used for ranging; or directly shutting down some circuits in the distance sensor 31, such as shutting down non-core circuit units such as data processing or communication interfaces while maintaining basic sensing capabilities, so that the distance sensor 31 as a whole is in a lower power consumption state.

[0136] The brightness sensor 32 can switch to a low-power state in the following ways: after the brightness sensor 32 has finished working, it can be directly controlled to turn off or power down until it is triggered again, so as to reduce the power consumption of the brightness sensor 32; or, it can be controlled to enter a low-power standby mode. In this mode, the brightness sensor 32 only maintains the minimum operation of the listening circuit so that it can receive the wake-up signal and respond to the wake-up signal.

[0137] The motion sensor 33 can switch to a lower power consumption state, which can be achieved by directly controlling the motion sensor 33 to turn off or power down after it has finished working, until it is triggered again, so as to reduce the power consumption of the motion sensor 33; or, the motion sensor 33 can be controlled to enter a low power standby mode, in which the motion sensor 33 only maintains a minimum of the listening circuit operation so that it can receive the wake-up signal and respond to the wake-up signal.

[0138] Of course, the environmental information sensing module 11 in the electronic device disclosed in this embodiment may also include only a portion of the distance sensor 31, the brightness sensor 32, and the motion sensor 33. For example, the environmental information sensing module 11 may only include the distance sensor 31 and the brightness sensor 32. When the object movement information detected by the distance sensor 31 indicates that there is a moving object in the environment where the electronic device is located, the brightness sensor 32 is triggered to start. After the brightness sensor 32 is activated, it detects the ambient brightness information in the environment where the electronic device is located. When the ambient brightness information meets the brightness threshold, it indicates that the environmental sensing information meets the first condition. At this time, the image acquisition module 12 is triggered to start. Another example is that the environmental information sensing module 11 may only include the distance sensor 31. When the object movement information detected by the distance sensor 31 indicates that there is a moving object in the environment where the electronic device is located, the image acquisition module 12 is triggered to start, etc.

[0139] The electronic device disclosed in this embodiment includes: an environmental information sensing module 11, an image acquisition module 12, and a detection module 13. The environmental information sensing module 11 is continuously powered on. When the environmental sensing information detected by the environmental information sensing module 11 meets a first condition, it triggers the image acquisition module 12 to start and controls the environmental information sensing module 11 to switch to a first low-power state. After the image acquisition module 12 acquires an image, it triggers the detection module 13 to start. After the detection module 13 starts, it controls the image acquisition module 12 to switch to a second low-power state. This embodiment, through relay-style hierarchical state management, ensures the continuity of the detection function while achieving further refinement and maximum reduction of the overall power consumption of the electronic device. When the next-level module starts up, the previous-level module switches to a low-power state, maintaining the necessary state with extremely low power consumption. This ensures that only one module is working at any given time, while other modules are in a low-power state, resulting in a significant reduction in the power consumption of the electronic device. In addition, controlling the previous-level module to be in a low-power state, rather than a shut-down state, allows the module to maintain a certain level of responsiveness and state integrity. This ensures that the previous-level module can quickly resume operation when needed, avoiding delays and additional startup power consumption.

[0140] Furthermore, the environmental information sensing module 11 in the electronic device disclosed in this embodiment may also include a feedback control unit, and the structural schematic diagram of the electronic device can be shown as follows. Figure 4 As shown, it includes:

[0141] The system includes an environmental information sensing module 41, an image acquisition module 42, and a detection module 43.

[0142] The environmental information sensing module 41 includes: a distance sensor 411, a brightness sensor 412, a motion sensor 413, and a feedback control unit 414.

[0143] In addition to the same structure as the previous embodiment, the environmental information sensing module 41 in the electronic device disclosed in this embodiment adds a feedback control unit 414. The feedback control unit 414 is used to obtain the detection results transmitted by the detection module 43, adjust the acquisition frequency of the distance sensor 411 when collecting object movement information based on the detection results, and adjust the detection frequency of the motion sensor 413 when detecting motion information.

[0144] The detection module 43 obtains the detection result, which may include: the detection result of the target object in the target image obtained after adjusting the image acquisition parameters based on the environmental perception information, indicating that the target object has passed the detection, or the detection result of the target object has failed the detection.

[0145] In addition, the detection results may also include: the target object's moving speed (based on the moving speed, it can be determined whether the target object is moving slowly or rapidly approaching or moving away), the target object's acceleration (used to determine whether the target object is moving at a constant speed, accelerating, or decelerating), the target object's moving direction information (used to determine whether the target object's trajectory is a straight line, a curve, or an uncertain direction), and the target object's motion state (used to characterize whether the target object is in continuous motion, intermittent motion, or has changed from motion to a stationary state), etc.

[0146] The detection module 43 sends the detection results to the environmental information sensing module 41 so that the feedback control unit 414 in the environmental information sensing module 41 can adjust the modules included in the environmental information sensing module 41 based on the detection results.

[0147] The feedback control unit 414 obtains the detection results. If the detection results determine that the target object is rapidly approaching the electronic device, then, in order to ensure the accuracy of image acquisition and the accuracy of detection, the acquisition frequency of the distance sensor 411 can be increased to acquire distance change data more densely. Correspondingly, the detection frequency of the motion sensor 413 can also be increased. If the detection results determine that the target object is stationary or moving slowly, then the acquisition frequency of the distance sensor 411 and the detection frequency of the motion sensor 413 can be reduced to save power consumption.

[0148] In addition, the detection results may also include: motion state information of electronic devices determined by the detection and analysis of target images, such as: the motion amplitude of electronic devices (based on the motion amplitude of electronic devices, it can be determined whether the electronic devices are slightly shaking or moving significantly), the intensity of the motion of electronic devices (used to characterize whether the motion of electronic devices is gentle, rapid or violent), the stability of the motion of electronic devices (used to characterize whether electronic devices are in a stable state or in a state of continuous shaking), and the motion mode of electronic devices (based on the scene, it can be determined whether the electronic devices are currently in handheld walking mode, running mode, or vehicle bump mode, etc.).

[0149] The feedback control unit 414 obtains the detection results. If the detection results determine that the electronic device is shaking violently or is in a bumpy environment, the detection frequency of the motion sensor 413 can be increased to obtain more detailed motion data. If the detection results determine that the electronic device is in a completely stationary and stable state, the detection frequency of the motion sensor 413 can be reduced to reduce power consumption.

[0150] Secondly, the detection result may also include: the first time consumed by the detection module 43 from acquiring the target image to obtaining the detection result, or the time when the detection result is obtained, so that the feedback control unit 414 can determine the second time corresponding to the time between triggering the image acquisition module 42 to start and obtaining the detection result. Taking the first time as an example, if the first time exceeds the first time threshold, it indicates that the accuracy of the acquired target image is low. Therefore, the acquisition frequency of the distance sensor 411 and the detection frequency of the motion sensor 413 can be increased to shorten the detection time by improving the image acquisition accuracy. Specifically, the frequency range to be adjusted can be as follows: if the first time exceeds the first time threshold t1, the acquisition frequency of the distance sensor 411 is reduced to half of its original value, and the detection frequency of the motion sensor 413 is also reduced to half of its original value; if the first time exceeds the first time threshold t2, the acquisition frequency of the distance sensor 411 is reduced to one-third of its original value, and the detection frequency of the motion sensor 413 is also reduced to one-third of its original value, and so on, until the frequency reaches 1 fps.

[0151] The feedback control unit 414 obtains the detection result output by the detection module 43 in the following ways: the detection module 43 is directly connected to the feedback control unit 414 in the environmental information perception module 41, and the feedback control unit 414 can directly obtain the detection result of the detection module 43 without going through any other module or unit; or the detection module 43 outputs the detection result to the image acquisition module 42, and the image acquisition module 42 outputs the detection result to the feedback control unit 414 in the environmental information perception module 41, so that the image acquisition module 42 can also obtain the detection result, and can also adjust the image acquisition parameters based on the detection result to ensure the quality of the acquired image.

[0152] In addition, the environmental information sensing module 41 in the electronic device disclosed in this embodiment may include, in addition to the aforementioned sensors and feedback control unit 414, an interactive control unit.

[0153] The interactive control unit transmits the environmental perception information detected by the environmental information perception module 41 to the image acquisition module 42, and is also used to obtain the information transmitted from the image acquisition module 42 to the environmental information perception module 41.

[0154] Correspondingly, the image acquisition module 42 may also include an interactive control unit to enable information exchange between the two modules (environmental information perception module 41 and image acquisition module 42) with the interactive control unit in the environmental information perception module 41, so as to ensure the transmission of information between the modules.

[0155] The electronic device disclosed in this embodiment includes: an environmental information sensing module 41, an image acquisition module 42, and a detection module 43. The environmental information sensing module 41 includes: a distance sensor 411, a brightness sensor 412, a motion sensor 413, and a feedback control unit 414. The feedback control unit 414 can obtain the detection results transmitted by the detection module 43 and adjust the acquisition frequency of the distance sensor 414 when acquiring object movement information, and adjust the detection frequency of the motion sensor 411 when detecting motion information, based on the detection results. This solution achieves the adjustment from fixed-frequency sensing to dynamic adaptive sensing through the feedback control unit 414, further optimizing power consumption and improving detection efficiency while ensuring sensing response capability.

[0156] Furthermore, the image acquisition module 42 in the electronic device disclosed in this embodiment adjusts the image acquisition parameters to the target acquisition parameters, which can be specifically as follows:

[0157] The image acquisition module 42 adjusts the focus parameters in the image acquisition parameters according to the object movement information, and adjusts the exposure parameters in the image acquisition parameters according to the ambient brightness information, motion information and preset image brightness data.

[0158] The image acquisition module 42 obtains environmental perception information, which may include object movement information, ambient brightness information, and motion information. Before acquiring an image, the image acquisition module 42 first adjusts the image acquisition parameters based on the obtained environmental perception information. Only after the adjustment is completed can the adjusted image acquisition module 42 be used to acquire the image, so as to ensure that a high-quality image can be acquired in one go.

[0159] During the process of adjusting the image acquisition parameters, the image acquisition module 42 can adjust the focus parameters in the image acquisition parameters based on object movement information, and adjust the exposure parameters in the image acquisition parameters based on ambient brightness information and motion information.

[0160] The object movement information detected by the TOF411 distance sensor provides precise distance data of the detected object. The image acquisition module 42 can directly convert this distance value into the focus position or focus motor steps required by the lens in the image acquisition module 42. Specifically, a distance-position mapping table can be preset. This mapping table records the correspondence between the distance between the image acquisition module 42 and the measured object and the precise position that the focus motor needs to move to. The precise position that the focus motor needs to move to is the position of the motor required to acquire the sharpest image at the distance value corresponding to the distance between the image acquisition module 42 and the measured object.

[0161] Adjusting exposure parameters based on ambient brightness and motion information can specifically involve adjusting the exposure amount based on ambient brightness information and adjusting the exposure time based on motion information.

[0162] The ambient brightness information collected by the brightness sensor 412 represents the overall light intensity of the scene in which the electronic device is located. A preset image brightness value can be set, which represents the desired brightness level of the captured image. At this brightness level, the image quality is high. The ambient brightness information is compared with the preset image brightness value to determine the exposure required to achieve this brightness level. This exposure can be achieved through aperture, shutter speed, and gain. If the aperture is adjustable, the exposure is adjusted primarily through aperture control. By adjusting the aperture, shutter speed, and gain, overexposure or underexposure of the image captured by the image acquisition module 42 is avoided.

[0163] The motion sensor 413 detects motion information. Taking a gyroscope as an example, when the motion sensor 413 is a gyroscope, the motion information detected by the gyroscope is the motion information of the electronic device, such as the angular velocity and / or jitter amplitude data of the electronic device. These data are converted into a quantitative indicator that reflects the intensity of the motion of the electronic device.

[0164] Based on the motion information of the electronic device, the system dynamically calculates a maximum allowable exposure time, i.e., a safe shutter speed. The basic logic of the safe shutter speed is as follows: the more violent the movement of the electronic device, the shorter the safe shutter speed, i.e., the shorter the exposure time, so as to capture the instantaneous action and ensure the clarity of the captured image; the more slight the movement of the electronic device or the stationary state, the longer the allowable safe shutter speed, i.e., the longer the exposure time, so as to absorb more light.

[0165] The calculated safe shutter speed can be used as a constraint to limit and guide the exposure parameters. For example, the exposure time of the image acquisition module 42 can be directly controlled to not exceed the safe shutter speed. Alternatively, since shortening the exposure time will reduce the amount of light entering the camera, which may cause the image to darken, the brightness lost due to shortening the exposure time can be compensated by increasing the gain, thereby ensuring that the overall exposure of the acquired image can reach the corresponding brightness level.

[0166] In addition, after adjusting the image acquisition parameters based on environmental perception information, the image acquisition module 42 acquires the target image based on the adjusted image acquisition parameters. After the image acquisition module 42 acquires the target image, it can directly trigger the detection module 43 to start and send the target image to the detection module 43. Alternatively, after the image acquisition module 42 acquires the target image, the image acquisition module 42 itself can first analyze the target image to determine whether the brightness and sharpness of the target image acquired by the image acquisition module 42 meet the requirements. The brightness requirement is the preset image brightness threshold, and the sharpness requirement can be a preset sharpness value. The adjustment of the exposure time and the adjustment of the focus parameters are related to the preset sharpness value. If the brightness and clarity of the acquired target image meet the requirements, the detection module 43 is triggered to start and the target image is transmitted to the detection module 43. If the requirements are not met, the current detection can be exited and a detection failure result can be generated for recording and feedback. Alternatively, the image acquisition module 42 can be controlled to re-acquire the image. When re-acquiring the image, the image acquisition parameters can be readjusted based on the brightness or clarity that does not meet the requirements so that the re-acquired image can meet the brightness and clarity requirements.

[0167] In the electronic device disclosed in this embodiment, the image acquisition module adjusts the focus parameters in the image acquisition parameters based on object movement information and pre-set image sharpness data, and adjusts the exposure parameters based on ambient brightness information, motion information, and pre-set image brightness data. This achieves feedforward pre-adjustment of the image acquisition parameters. By fusing multi-sensor data, high-quality images are directly obtained, thereby significantly improving the accuracy of target detection and the overall efficiency of detection. After the image acquisition module starts, this solution directly adjusts the image acquisition parameters based on object movement information, ambient brightness information, and motion information, ensuring the quality of the images acquired by the image acquisition module and providing a high-quality input source for subsequent detection. Furthermore, adjusting the focus parameters based on object movement information and adjusting the exposure parameters based on ambient brightness information and motion information enables the image acquisition module to quickly output stable and reliable images even in complex scenarios such as target movement, sudden changes in lighting, and device instability, thus improving detection efficiency.

[0168] Specifically, the electronic device and its implementation process disclosed in this embodiment can be described as follows: Figure 5 As shown, the electronic device includes an environmental information sensing module, an image acquisition module, and a detection module.

[0169] The environmental information sensing module may include: a time-of-flight (TOF) distance sensor, an ambient light sensor (AIS) brightness sensor, a motion sensor (such as a gyroscope), and a feedback control unit.

[0170] Among them, the Time-of-Flight (TOF) distance sensor is used for distance detection. TOF is used to detect moving objects, determine the distance between the moving object and the electronic device, and can trigger the AIS brightness sensor to start. In addition, the information detected by TOF can also assist in adjusting the focus parameters when the image acquisition module adjusts the image acquisition parameters.

[0171] The AIS brightness sensor is used to detect ambient brightness information, and when the ambient brightness information meets the brightness threshold, it triggers the Gyro motion sensor to start.

[0172] The motion sensor Gyro detects the motion information of moving objects, and triggers the image acquisition module to start when the motion information meets the motion conditions;

[0173] The feedback control unit is used to obtain the detection results from the detection module and adjust the acquisition frequency of the distance sensor and the detection frequency of the motion sensor based on the detection results.

[0174] The image acquisition module obtains the environmental perception information output by the environmental information perception module. The environmental perception information includes: ambient brightness information, motion information, and object movement information (distance information).

[0175] For brightness information, the brightness information can be compared with a preset image brightness threshold to determine the exposure. Based on the determined exposure, the image acquisition parameters are adjusted. After the adjustment is completed, the image acquisition module acquires an image based on the adjusted image. At this time, the brightness of the acquired image meets the preset image brightness threshold. If the determined exposure cannot be achieved by adjusting the image acquisition parameters, the working mode of each image acquisition sensor in the image acquisition module can be switched to maximize the brightness of the image acquired by the image acquisition module after switching the working mode.

[0176] For motion information, it can be determined whether the motion speed in the motion information exceeds a threshold, that is, whether the movement of the electronic device is intense (in the example, the motion sensor is a gyroscope, so the motion information detected by the gyroscope is the motion information of the electronic device). If the motion speed exceeds the threshold, there is a risk of image blurring due to intense movement. Therefore, the exposure time can be reduced to eliminate the motion blur problem, so that the adjusted image acquisition module can output a clear image. If the motion speed does not exceed the threshold, the existing exposure time can be maintained without adjustment.

[0177] For distance information, if the image acquisition module is equipped with a motor, the distance information needs to be acquired and the motor movement distance needs to be calculated based on the distance information to ensure that the adjusted image acquisition module can focus clearly and output a clear image.

[0178] In addition, the image acquisition module can also be used for decision control. The image acquisition module can determine the target model parameters based on ambient brightness information, motion information, and distance information in order to configure the detection model in the detection module. It can also check whether the currently acquired target image meets the requirements for clarity and brightness. If it does, it can trigger the detection module to start and control the image acquisition module to enter a low-power state. If it does not meet the requirements, it can exit the detection. Secondly, the image acquisition module can also perform face recognition (i.e., target object recognition) on the acquired image to determine whether the acquired image contains a face. If it contains a face, it triggers the detection module to start. If it does not contain a face, it needs to re-acquire the image (of course, the process of target object recognition can also be performed by the detection module).

[0179] After the detection module is started, it may include a power-on / off model loading module, which is used to control the start of the detection model. After the model is started, the detection model is run so that the detection model can detect the target image and control the working state of the electronic device based on the detection results of the target object.

[0180] like Figure 6The diagram shown is a structural block diagram of the various modules included in the electronic device. The electronic device disclosed in this embodiment may include three parts, of which one part is an environmental information sensing module, which may be set in an MCU or a DSP; another part is an image acquisition module, which may be set in an MCU; and the third part is a detection module, which may be set in an NPU or a DSP. The NPU may be an eNPU or a pNPU.

[0181] The image acquisition module can include three sub-parts: a perception information preprocessing sub-part, an image data acquisition sub-part, and a model management sub-part. The perception information preprocessing sub-part can include: a sub-module capable of adjusting exposure based on ambient brightness information; a sub-module capable of adjusting exposure time based on motion information; a sub-module capable of adjusting focus parameters based on distance information; and further includes: a sub-module capable of adjusting the operating mode of each image acquisition unit (Sensor) in the image acquisition module; a sub-module capable of adjusting the model parameters of the detection model; and a sub-module capable of determining whether to use single-frame detection or continuous multi-frame detection.

[0182] After preprocessing, the preprocessed image acquisition module is used for image acquisition, which is the image data acquisition sub-section. This sub-section is divided into single-frame detection and continuous multi-frame detection. For single-frame detection, the image acquisition unit (Sensor) is used to acquire the target image based on the adjusted focus parameters, exposure amount, and exposure time. For continuous multi-frame detection, the first frame is the same as single-frame detection, while the acquisition of the second and subsequent image frames requires adjustment of the exposure parameters based on the feedback from the previous frame. In addition, real-time fine-tuning is also required based on the data from the image acquisition unit (Sensor) to ensure that higher quality frame images can be acquired. The acquired images are then transmitted to the image signal processor (ISP) for processing. Image processing may include: de-mosaic processing, noise reduction processing, automatic white balance processing, color correction and enhancement processing, gamma correction processing, and sharpening processing.

[0183] In addition, the image data acquisition sub-section also includes brightness and sharpness judgment, which is mainly used to evaluate the acquired image. First, the 2A / 3A statistical value can be extracted and the sharpness judgment can be performed. Then, the statistical value and the result of the sharpness judgment are used as feedback signals to feed back to the front-end control unit so that the acquisition parameters for the next frame image can be adjusted based on the information of the current frame image.

[0184] Among them, the sharpness judgment mainly targets the evaluation of autofocus. By analyzing the high-frequency details, edge sharpness, or contrast of the image, a sharpness score representing the current focus state is calculated. The higher the score, the more accurate the focus. In the 2A / 3A statistical value, 3A mainly refers to the quantitative indicators of automatic exposure AE (adjusting exposure parameters according to the brightness of the image), automatic focus AF (adjusting the focus position according to the sharpness of the image), and automatic white balance AWB (adjusting the color according to the light source so that white objects appear white), while 2A refers to the quantitative indicators of AE and AF.

[0185] In addition, the image acquisition module may also include a model management sub-section, which includes: a model initialization and loading module, a multi-model data sharing module, a multi-model feedback module, and a model data transformation module. The model initialization and loading module is used to initialize the model and load it into the detection module. The multi-model data sharing module is used to realize the data transmission and sharing between different models. The multi-model feedback module is used to collect and analyze the actual operating performance of each model and provide feedback on the analysis results in order to dynamically optimize the entire recognition process. The model data transformation module is used to transform the data so that different models can recognize and analyze the transformed data, ensuring that data from different sources and in different formats can be processed correctly and efficiently by different models.

[0186] The specific implementation details of the environmental information sensing module and the detection module will not be elaborated here.

[0187] This embodiment discloses a processing method, the flowchart of which is as follows: Figure 7 As shown, it includes:

[0188] Step S71: Detect environmental perception information in the environment where the electronic device is located by an environmental information perception module that is configured to be continuously turned on.

[0189] Step S72: In response to the environmental perception information meeting the first condition, the image acquisition module is triggered to start, and the environmental information perception module is controlled to send the environmental perception information to the image acquisition module;

[0190] Step S73: Control the image acquisition module to acquire the target image based on the target acquisition parameters, which are obtained by adjusting the image acquisition parameters based on environmental perception information;

[0191] Step S74: The target image is acquired based on the image acquisition module, and the quality of the target image meets the preset image quality conditions. The detection module is then triggered to start and the target image is sent to the detection module.

[0192] Step S75: Detect target objects in the target image using the detection module, and control the working state of the electronic device based on the detection results of the target objects.

[0193] Furthermore, the processing method disclosed in this embodiment may also include:

[0194] In response to the start of the image acquisition module, the control environment information perception module switches to a first low-power state where the power consumption is lower than that of the continuously on state; in response to the start of the detection module, the control image acquisition module switches to a second low-power state where the power consumption is lower than that of the start state. The start state is when the image acquisition module is in a state where it can adjust the image acquisition parameters to the target acquisition parameters and acquire the target image based on the target acquisition parameters.

[0195] The processing method disclosed in this embodiment is implemented based on the electronic device disclosed in the above embodiments, and will not be described again here.

[0196] The processing method disclosed in this embodiment detects environmental perception information in the environment where the electronic device is located through an environmental information perception module configured to be continuously on; in response to the environmental perception information meeting a first condition, the image acquisition module is triggered to start and the environmental information perception module is controlled to send the environmental perception information to the image acquisition module; the image acquisition module is controlled to acquire a target image based on target acquisition parameters, which are obtained by adjusting the image acquisition parameters based on the environmental perception information; based on the completion of the target image acquisition by the image acquisition module and the target image quality meeting the preset image quality conditions, the detection module is triggered to start and the target image is sent to the detection module; the detection module performs target object detection on the target image, and the working state of the electronic device is controlled based on the detection result of the target object. This solution keeps the environmental information sensing module continuously active. The image acquisition and detection modules, which consume significant power, are only activated when the environmental information meets the first condition. This strategy results in lower power consumption for the electronic device. Furthermore, when the image acquisition module is activated, it directly receives the current environmental information from the environmental information sensing module. This allows the image acquisition module to adjust its parameters before acquiring an image, ensuring that the adjusted parameters match the current environment. This results in images that are appropriate for the environment, avoiding the acquisition of invalid or low-quality images and improving the detection success rate.

[0197] In the above embodiments, the facial image data involved is collected with user authorization or is facial feature data registered locally on the device, and does not involve the illegal acquisition or use of personal biometric information.

[0198] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the device embodiment drawings provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0199] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, training equipment, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0200] In the above embodiments, the implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, in the form of a computer program product.

[0201] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, training device, or data center to another website, computer, training device, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).

Claims

1. An electronic device, comprising: The environmental information sensing module, the image acquisition module and the detection module; The environmental information sensing module is configured to be in a continuous start state, detects environmental sensing information in an environment where the electronic device is located, and triggers the image acquisition module to start and sends the environmental sensing information to the image acquisition module when the environmental sensing information meets a first condition; The image acquisition module is configured to adjust image acquisition parameters to target acquisition parameters according to the environmental sensing information, acquire a target image based on the target acquisition parameters, and trigger the detection module to start and send the target image to the detection module when the quality of the acquired target image meets a preset image quality condition; The detection module is configured to perform target object detection on the target image and control the working state of the electronic device based on the detection result of the target object.

2. The electronic device of claim 1, wherein the image acquisition module is further configured to: determine target model parameters of a model used for performing target object detection in the detection module based on the target image, and send the target model parameters to the detection module; and the detection module is configured to: perform target object detection on the target image by using a detection model set as the target model parameters to obtain a detection result, so as to control the working state of the electronic device based on the detection result.

3. The electronic device of claim 2, wherein the detection module is configured to: perform target object detection on the target image by using a detection model set as the target model parameters to obtain a detection result, and trigger a main system to start when the detection result meets a second condition, so that the main system controls the working state switching of the electronic device.

4. The electronic device of claim 3, wherein the detection module is further configured to: when it is determined that the detection result does not meet the second condition, send the detection result to the image acquisition module, so that the image acquisition module re-acquires a target image and performs target object detection on the re-acquired target image by using the detection module.

5. The electronic device of claim 1, wherein the environmental information sensing module is further configured to: switch to a first low-power consumption state with lower power consumption than the continuous start state after the image acquisition module starts; and the image acquisition module is further configured to switch to a second low-power consumption state with lower power consumption than the start state after the detection module starts, wherein the start state is a state in which the image acquisition module can adjust image acquisition parameters to target acquisition parameters and acquire a target image based on the target acquisition parameters.

6. The electronic device of claim 1, the environmental information perception module comprising: The distance sensor, the brightness sensor and the motion sensor; The distance sensor is configured to acquire object movement information and trigger the brightness sensor to start when the object movement information indicates that there is a moving object in an environment where the electronic device is located; The brightness sensor is configured to acquire environmental brightness information in an environment where the electronic device is located and trigger the motion sensor to start when the environmental brightness information meets a brightness threshold value; and the motion sensor is configured to acquire motion information in the environment where the electronic device is located and trigger the distance sensor to start when the motion information meets a motion threshold value. The motion sensor is used to detect the motion information of the moving object. When the motion information meets the motion conditions, it is determined that the environmental perception information meets the first condition, and the image acquisition module is triggered to start.

7. The electronic device according to claim 6, wherein the environmental information sensing module further comprises: The feedback control unit is used to obtain the detection results transmitted by the detection module, adjust the acquisition frequency of the distance sensor when collecting object movement information based on the detection results, and adjust the detection frequency of the motion sensor when detecting the motion information.

8. The electronic device according to claim 6, wherein the image acquisition module adjusts the image acquisition parameters to the target acquisition parameters based on the environmental perception information, comprising: The image acquisition module adjusts the focus parameter in the image acquisition parameters according to the object movement information, and adjusts the exposure parameter in the image acquisition parameters according to the ambient brightness information, motion information, and preset image brightness data.

9. A processing method, comprising: The environmental information sensing module, which is configured to be continuously on, detects environmental sensing information in the environment where the electronic device is located. In response to the environmental perception information meeting the first condition, the image acquisition module is triggered to start, and the environmental information perception module is controlled to send the environmental perception information to the image acquisition module; The image acquisition module is controlled to acquire a target image based on target acquisition parameters, wherein the target acquisition parameters are obtained by adjusting the image acquisition parameters based on the environmental perception information. Once the target image is acquired by the image acquisition module and the target image quality meets the preset image quality conditions, the detection module is triggered to start and the target image is sent to the detection module. The detection module performs target object detection on the target image, and controls the working state of the electronic device based on the detection results of the target object.

10. The method of claim 9, further comprising: In response to the activation of the image acquisition module, the environmental information sensing module is controlled to switch to a first low-power state where the power consumption is lower than that of the continuously on state. In response to the activation of the detection module, the image acquisition module is controlled to switch to a second low-power state with power consumption lower than that of the activation state. The activation state is when the image acquisition module is in a state where it can adjust the image acquisition parameters to the target acquisition parameters and acquire the target image based on the target acquisition parameters.

Images