Ambient light reporting method, terminal equipment and storage medium

Abstract

The embodiment of the invention relates to an ambient light reporting method, terminal equipment and a storage medium. And the SCP processor of the terminal equipment determines to start a gain adjustment function according to the acquired ambient light value, and sets an end flag bit of the gain adjustment function when the execution of the gain adjustment function is completed. And the application processor of the terminal equipment performs de-noising processing on the ambient light value generated by the SCP processor to obtain a noiseless ambient light value, and sends the noiseless ambient light value to the SCP processor. And the SCP processor calculates an ambient light lux value according to the noiseless ambient light value, and reports the ambient light lux value to the application processor in response to an end flag bit of the gain adjustment function. According to the invention, the SCP processor can immediately report the calculated ambient light lux value according to the end flag bit of the adjustment gain function, and does not need to wait for the ambient light sensor to carry out integration on the ambient light data collected in the next preset period and then report the ambient light data, thereby reducing the time consumed by the SCP processor to report the ambient light lux value.

Description

Technical Field

[0001] The present application relates to the technical field of terminal equipment, and in particular to an ambient light reporting method, a terminal equipment and a storage medium. Background Art

[0002] Current terminal devices (such as smart phones) are equipped with ambient light sensors to measure the light intensity of the surrounding environment in order to automatically adjust the brightness of the terminal device. When the ambient light changes, the ambient light reported by the ambient light sensor will fluctuate greatly, and the brightness adjustment will be turned on. For example, when a user carries a terminal device from a bright environment to a dark environment, the ambient light sensor of the terminal device can detect that the light intensity in the dark environment is insufficient, and automatically turns on the brightness adjustment to increase the screen brightness so that the user can see the content on the display screen. However, when the ambient light changes, the terminal device needs to go through a startup dimming stage before the brightness adjustment process can begin. The startup dimming stage refers to the time period from when the ambient light changes to when the dimming is turned on. The startup dimming stage of existing terminal devices is too long, resulting in the terminal device taking a long time to achieve brightness adjustment, affecting the user experience. Summary of the invention

[0003] In view of the above, it is necessary to provide an ambient light reporting method, a terminal device and a storage medium to solve the technical problem that the terminal device takes too long to start the dimming stage.

[0004] In a first aspect, an embodiment of the present application provides an ambient light reporting method, which is applied to a terminal device, and the method includes: a sensor of the terminal device cooperates with an SCP processor to obtain ambient light data according to a preset period, and obtains an ambient light value based on the ambient light data; if the SCP processor determines to turn on the gain adjustment function according to the ambient light value, when it detects that the gain adjustment function is completed, the end flag of the gain adjustment function is set; the SCP processor reports the ambient light value to an application processor of the terminal device; the application processor denoises the ambient light value and obtains a noise-free ambient light value, and sends the noise-free ambient light value to the SCP processor; the SCP processor calculates the ambient light illuminance lux value based on the noise-free ambient light value; the SCP processor responds to the end flag of the gain adjustment function and reports the ambient light lux value to the application processor. When the above technical solution detects that the ambient light sensor turns on the gain adjustment function, the SCP processor monitors the action of the gain adjustment function, and when the gain adjustment function ends, the SCP processor can immediately report the noise-free ambient light value returned by the AP processor according to the end flag of the gain adjustment function, without waiting for the ambient light sensor to integrate the ambient light data collected in the next preset period before reporting, thereby further reducing the time consumed by the SCP processor to report the ambient light lux value.

[0005] In one embodiment of the present application, after determining to turn on the gain adjustment function, the method further includes: the SCP processor sends a notification message to the application processor, the notification message is used to instruct the SCP processor to turn on the gain adjustment function; the application processor cancels the median filtering process according to the notification message, and the median filtering process refers to sorting all the ambient light lux values ​​input into the noise memory of the terminal device, and selecting the ambient light reported value in the middle position for output. In the above technology, when it is detected that the ambient light sensor turns on the gain adjustment function, the SCP processor notifies the AP processor to cancel the median filtering process, so as to ensure that the ambient light lux value of the first frame of ambient light data reported by the SCP processor received by the AP processor will not be filtered by the median filtering process, thereby further reducing the time consumed by the SCP processor to report the ambient light lux value.

[0006] In one embodiment of the present application, canceling the median filtering process includes: clearing the value in the noise memory. The above technical solution can cancel the median filtering of the ambient light lux value by clearing the value in the noise memory.

[0007] In one embodiment of the present application, the application processor performs denoising on the ambient light value to obtain a noise-free ambient light value, including: the application processor calls a noise algorithm library to perform denoising on the ambient light value and removes image noise and backlight noise in the ambient light value to obtain a noise-free ambient light value. In the above technical solution, the application processor calls a noise algorithm library to perform denoising on the ambient light value and removes image noise and backlight noise in the ambient light value, which can remove the impact of the screen of the terminal device on the ambient light.

[0008] In one embodiment of the present application, the SCP processor determines to start the adjustment gain function according to the ambient light value, including: if the ambient light value is greater than or equal to the first preset ambient light threshold, determining to start the adjustment gain function; if the ambient light value is less than the first preset ambient light threshold, determining not to start the adjustment gain function.

[0009] In one embodiment of the present application, after determining to turn on the gain adjustment function, the method further includes: the SCP processor turns on the gain adjustment function to adjust the ambient light value. In the above technical solution, the SCP processor turns on the gain adjustment function to adjust the ambient light value, and can quickly adjust the ambient light value output by the ambient light sensor when the ambient light changes.

[0010] In one embodiment of the present application, the SCP processor activates the gain adjustment function to adjust the ambient light value, including: the SCP processor determines the gain value according to the ambient light value, adjusts the ambient light value according to the gain value, and integrates the adjusted ambient light value. In the above technical solution, the SCP processor activates the gain adjustment function to adjust the ambient light value, and can quickly adjust the ambient light integral value output by the ambient light sensor when the ambient light changes.

[0011] In one embodiment of the present application, after reporting the ambient light lux value to the application processor, the method further includes: the SCP processor clears the end flag of the gain adjustment function. In the above technical solution, the SCP processor clears the end flag of the gain adjustment function so as to report the lux value of the next ambient light data.

[0012] In an embodiment of the present application, after the ambient light lux value is reported to the application processor, the method further includes: the application processor adjusting the brightness according to the ambient light lux value.

[0013] In a second aspect, an embodiment of the present application provides a terminal device, which includes a memory and a processor: the memory is used to store program instructions; the processor is used to read and execute the program instructions stored in the memory, and when the program instructions are executed by the processor, the terminal device executes the above-mentioned ambient light reporting method.

[0014] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, which stores program instructions. When the program instructions are executed on a terminal device, the terminal device executes the above-mentioned ambient light reporting method.

[0015] In addition, the technical effects brought about by the second to third aspects can be found in the descriptions related to the methods of each design in the above method part, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0017] Figure 1 Schematic diagram of the time consumed by existing smartphones of different brands during the startup dimming stage.

[0018] Figure 2 A schematic diagram of a process for an ambient light sensor of a terminal device to report an ambient light value provided in an embodiment of the present application.

[0019] Figure 3 A schematic diagram of a median filtering process provided in one embodiment of the present application.

[0020] Figure 4 A schematic diagram of reporting ambient light data provided in an embodiment of the present application.

[0021] Figure 5 A technical architecture corresponding to a method for obtaining target ambient light through initial ambient light and content displayed on a display screen provided in an embodiment of the present application.

[0022] Figure 6 This is a flow chart of an ambient light reporting method provided in one embodiment of the present application.

[0023] Figure 7 This is a flow chart of an ambient light reporting method provided in another embodiment of the present application.

[0024] Figure 8 A schematic diagram of a terminal device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0025] In the following, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a concrete way.

[0026] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those generally understood by technicians in the technical field in this application. The terms used in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application. It should be understood that, unless otherwise specified in this application, " / " means or. For example, A / B can represent A or B. "And / or" in this application is only a description of the association relationship of associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. "At least one" means one or more. "Multiple" means two or more than two. For example, at least one of a, b or c can represent: a, b, c, a and b, a and c, b and c, a, b and c seven situations.

[0027] Current terminal devices (such as smart phones) are equipped with ambient light sensors to measure the light intensity of the surrounding environment in order to automatically adjust the brightness of the terminal device. When the ambient light changes, the ambient light reported by the ambient light sensor will fluctuate greatly, turning on the brightness adjustment. For example, when a user carries a terminal device from a bright environment to a dark environment, the ambient light sensor of the terminal device can detect that the light intensity in the dark environment is insufficient, and automatically turns on the brightness adjustment to increase the screen brightness so that the user can see the content on the display screen. However, when the ambient light changes, the terminal device needs to go through a startup dimming stage before the brightness adjustment process can begin. The startup dimming stage refers to the time period from when the ambient light changes to when the dimming is turned on. The startup dimming stage of existing terminal devices is too long, resulting in the terminal device taking a long time to achieve brightness adjustment, affecting the user experience.

[0028] Figure 1 The time consumed by smartphones of different brands in the startup dimming stage (i.e., startup dimming time) is shown. Among them, the time consumed by brand A smartphone in the startup dimming stage is 1.36s, the time consumed by brand B smartphone in the startup dimming stage is 1.18s, the time consumed by brand C smartphone in the startup dimming stage is 1.15s, and the time consumed by brand D smartphone in the startup dimming stage is 2.51s. Thus, according to the experimental data, it can be seen that the time consumed by smartphones of different brands in the startup dimming stage has reached the second level, which is a user-perceivable impact, resulting in a poor user experience.

[0029] refer to Figure 2 As shown, it is a schematic diagram of the process of the ambient light sensor of the terminal device provided in one embodiment of the present application reporting the ambient light value. The processor in the terminal device is a multi-core processor, and the multi-core processor at least includes: an application processor (application processor, referred to as AP processor) and a sensor coprocessor (sensor coprocessor, referred to as SCP processor). Among them, the AP processor is the application processor in the terminal device, and the operating system, user interface and application program all run on the AP processor. The SCP processor is a coprocessor that can assist the AP processor in matters related to images, sensors (for example, ambient light sensors), etc.

[0030] The following is an example of the process of the ambient light sensor reporting the ambient light value, taking the terminal device as a smartphone. The ambient light sensor obtains ambient light data and integrates the ambient light data according to a preset period. If the ambient light changes, the ambient light sensor integrates the obtained ambient light data and outputs an integral value greater than the first preset ambient light threshold. Since the integral value is greater than the preset ambient light range, the gain adjustment (GAIN) function of the ambient light sensor is turned on, where the gain adjustment function refers to the ambient light sensor performing gain adjustment on the integral value and integrating the adjusted integral value again. During the period of gain adjustment, the integral value output by the ambient light sensor is unstable. Therefore, during the period of gain adjustment, the integral value will not be reported to the SCP processor as the ambient light value. When the gain adjustment function ends or the gain adjustment function is not needed, the ambient light sensor will not immediately report the ambient light value after the ambient light change to the application according to the ambient light reporting cycle. There are mainly the following two problems.

[0031] The first problem is that when the gain adjustment function ends or does not need to be adjusted, the SCP processor calculates the ambient light illuminance (lux) value and reports the ambient light lux value to the AP processor. The AP processor stores the ambient light lux value in the noise memory and performs median filtering on the ambient light lux value. Median filtering refers to sorting all the ambient light lux values ​​input into the noise memory, and then selecting the ambient light report value in the middle position for output. Because the ambient light lux value corresponding to the first frame of ambient light data after the ambient light changes has a larger change than the ambient light data before the ambient light changes, it cannot occupy the middle position and will be filtered out, resulting in the inability to report the ambient light lux value of the first frame of ambient light data to the application. For example, refer to Figure 3 , the three ambient light lux values ​​stored in the noise memory are 1000, 20, and 50, of which 1000 is the ambient light lux value of the first frame of ambient light data. The AP processor sorts the three ambient light lux values ​​stored in the noise memory to get 20, 50, and 1000, and selects the ambient light lux value 50 in the middle position for reporting, resulting in the ambient light lux value 1000 of the first frame of ambient light data not being reported. The ambient light lux value of the second frame of ambient light data after an ambient light reporting cycle will be reported after median filtering (reference Figure 4 ).

[0032] The second problem is that after the ambient light lux value is median filtered, the AP processor performs cutout processing on the ambient light lux value after median filtering to remove the image noise and backlight noise in the ambient light lux value, so as to remove the impact of the screen on the ambient light, and report the noise-removed ambient light lux value to the SCP processor for storage. The ambient light lux value after removing the image noise and backlight noise will not be calculated until the next ambient light data integration, and the calculation result will be reported. Therefore, after the ambient light changes, it is necessary to wait until the third frame of ambient light data in the next ambient light reporting cycle to match the current ambient light, and report the ambient light lux value of the third frame of ambient light data to the application, which causes the smartphone to take too long to start the dimming stage.

[0033] In order to solve the technical problem that the terminal device takes too long to start the dimming stage, the present application provides an ambient light reporting method. The method is applied in the terminal device. Figure 5 The technical architecture corresponding to the method for obtaining target ambient light through initial ambient light and content displayed on a display screen provided in an embodiment of the present application is described.

[0034] like Figure 5 As shown, the processor in the terminal device includes an AP processor and an SCP processor. Figure 5 Only the AP processor and the SCP processor are shown. In practical applications, the multi-core processor may also include other processors. For example, when the terminal device is a smart phone, the multi-core processor may also include a baseband (BP) processor that runs the smart phone radio frequency communication control software and is responsible for sending and receiving data.

[0035] Figure 5 The AP processor only shows the content related to the embodiments of the present application. The implementation of the embodiments of the present application needs to rely on the following layers in the AP processor: application layer (Application), java framework layer (Framework Java), native framework layer (Framework native), hardware abstraction layer (HAL), kernel layer (kernel) and hardware layer (hardware).

[0036] Figure 5 The SCP processor in the SCP processor can be understood as a sensor control center (sensor hub), which can control the sensor and process the relevant data of the sensor. The implementation of the embodiment of the present application needs to rely on the SCP processor: the cooperative application layer (Hub APK), the cooperative framework layer (Hub FWK), the cooperative driver layer (Hub DRV), and the cooperative hardware layer (Hubhardware).

[0037] There are various applications in the application layer of the AP processor. Figure 5 , application A and application B are shown. Taking application A as an example, after the user starts application A, the display screen will display the interface of application A. Specifically, application A sends the display parameters of the interface to be displayed by application A (for example, the memory address and color of the interface to be displayed) to the display engine service.

[0038] The display engine service in the AP processor sends the received display parameters of the interface to be displayed to the SurfaceFlinger of the native framework layer (Framework native) of the AP processor.

[0039] SurfaceFlinger in the native framework layer (Framework native) of the AP processor is responsible for controlling the fusion of the interface (surface). As an example, the overlapping area of ​​at least two overlapping interfaces is calculated. The interface here may be an interface presented by the status bar, system bar, application itself (the interface to be displayed by application A), wallpaper, background, etc. Therefore, SurfaceFlinger can not only obtain the display parameters of the interface to be displayed by application A, but also obtain the display parameters of other interfaces.

[0040] The hardware abstraction layer of the AP processor exists in HWC (Hardware composer HAL). HWC is the module for interface synthesis and display in the system, providing hardware support for the surfaceFlinger service. Step A1 is that SurfaceFlinger sends the display parameters (e.g., memory address, color, etc.) of each interface to HWC through the interface (e.g., setLayerBuffer, setLayerColor,) for interface fusion.

[0041] Usually, in the synthesis of images (for example, when the terminal device displays an image, the status bar, system bar, application itself and wallpaper background need to be synthesized), the HWC obtains the synthesized image according to the display parameters of each interface through the underlying hardware of the HWC (for example, the hardware synthesizer). The HWC in the hardware abstraction layer of the AP processor sends the image synthesized by the underlying hardware to the OLED driver, see step A2.

[0042] The OLED driver of the core layer of the AP processor sends the synthesized image to the display subsystem (DSS) of the hardware layer of the AP processor, see step A3. The display subsystem (DSS) in the hardware layer of the AP processor can perform secondary processing on the synthesized image (for example, HDR10 processing for enhancing image quality, etc.), and the secondary processed image can be sent for display after the secondary processing. In actual applications, secondary processing may not be performed. Taking the example of not performing secondary processing, the display subsystem of the hardware layer of the AP processor sends the synthesized image to the OLED screen for display.

[0043] Taking the startup of application A as an example, the synthesized image displayed on the OLED screen is an interface synthesized from the interface to be displayed by application A and the interface corresponding to the status bar.

[0044] In the above manner, the OLED screen can complete one image refresh and display.

[0045] The embodiment of the present application can control the display subsystem (DSS) to store the entire frame image (or an image larger than the target area in the entire frame image, or an image corresponding to the target area in the entire frame image) in the memory of the kernel layer of the AP processor before sending the secondary processed image (or synthesized image) for display. Since this process belongs to concurrent write-back of image frame data, the memory can be recorded as Concurrent Write Back (CWB) memory, see step A4.

[0046] In the embodiment of the present application, the display subsystem stores the entire frame of the image in the CWB memory of the AP processor as an example. After the display subsystem successfully stores the entire frame of the image in the CWB memory, the display subsystem can send a storage success signal to the HWC. The entire frame of the image corresponding to the image stored in the CWB memory by the display subsystem can be recorded as the image to be refreshed (the image to be refreshed can also be understood as the image after this refresh), and the entire frame of the image corresponding to the image stored in the CWB memory by the display subsystem can also be recorded as the first image.

[0047] The AP processor may also be configured to allow the HWC to access the CWB memory. The HWC may obtain the target image from the CWB memory after receiving a storage success signal sent by the display subsystem, as shown in step A5.

[0048] It should be noted that no matter whether the CWB memory stores a whole frame image or an image of a partial region in the whole frame image, the HWC can obtain the target image from the CWB memory. The process of the HWC obtaining the target image from the CWB memory can be recorded as the HWC cutting out the image from the CWB memory.

[0049] For the convenience of description, the image stored in the CWB memory of the display subsystem may also be recorded as the second image. As mentioned above, the second image may be the first image, the target image, or an image between the range of the target image and the range of the first image. The range of the target image may be understood as the range size defined by the length and width of the target image, and the range of the first image may be the range of the entire frame image, and the range size may also be defined by the length and width.

[0050] Continuing with application A as an example, when application A needs to adjust brightness due to interface switching, application A sends the brightness to be adjusted to the display engine service.

[0051] The display engine service in the AP processor sends the brightness to be adjusted to the kernel node in the kernel layer of the AP processor, so that the relevant hardware can adjust the brightness of the OLED screen according to the brightness to be adjusted stored in the kernel node.

[0052] According to the above method, the OLED screen can complete one brightness adjustment.

[0053] In the embodiment of the present application, the HWC may also be configured to obtain the brightness to be adjusted from the core node, and the brightness to be adjusted may also be recorded as the brightness after this adjustment, see step A5' for details.

[0054] In the specific implementation, HWC can monitor whether the data stored in the kernel node changes based on the uevent mechanism. After monitoring the data changes in the kernel node, the currently stored data is obtained from the kernel node, that is, the brightness value to be adjusted (the brightness value to be adjusted is used to adjust the brightness of the display screen, so it can also be recorded as the brightness value of the display screen). After HWC obtains the target image or the brightness information to be adjusted, it can send the target image or the brightness information to be adjusted to the noise algorithm library of the hardware abstraction layer of the AP processor. See step A6. The noise algorithm library can calculate the fusion noise at the refresh moment of the target image each time the target image is obtained. After each brightness is obtained, the fusion noise at the brightness adjustment moment is calculated. The noise algorithm library stores the calculated fusion noise in the noise memory of the noise algorithm library.

[0055] In actual applications, after the HWC obtains the target image, the HWC can store the target image, and the HWC can send the storage address of the target image to the noise algorithm library, and the noise algorithm library can cache a frame of the target image at the latest moment by recording the address. After the HWC obtains the brightness to be adjusted, the HWC can send the brightness to be adjusted to the noise algorithm library, and the noise algorithm library can cache a brightness at the latest moment. For ease of description, the subsequent embodiments of this application are described by HWC sending the target image to the noise algorithm library. In actual applications, it can be said that the HWC stores the target image after obtaining it, and sends the storage address of the target image to the noise algorithm library.

[0056] As an example, after the noise algorithm library receives the storage address of the first frame of the target image, it caches the storage address of the first frame of the target image. Each subsequent time a new storage address of a target image is received, the storage address of the new target image is used as the storage address of the cached latest target image. Correspondingly, after the noise algorithm library receives the first brightness, it caches the first brightness, and each subsequent time a new brightness is received, the new brightness is used as the cached latest brightness. In the embodiment of the present application, the noise algorithm library caches the acquired target image and brightness value to the data repository. The target image and brightness value stored in the data repository can be recorded as screen data, that is, the screen data stored in the data repository includes: target image and brightness value.

[0057] In addition, in order to describe the transmission relationship between the target image, the brightness to be adjusted and other parameters, the embodiment of the present application is described by taking the HWC sending the target image, the brightness to be adjusted and other parameters to the noise algorithm library as an example. In practical applications, the relationship between the HWC and the noise algorithm library is that the HWC calls the noise algorithm library. When the HWC calls the noise algorithm library, the HWC inputs the target image (storage address of the target image), the brightness to be adjusted and other parameters as independent variables of the calculation model in the noise algorithm library to the noise algorithm library. Other parameters will not be given one by one.

[0058] Since brightness adjustment and image refresh are two completely independent processes, the image may be refreshed at a certain moment while the brightness remains unchanged. In this case, the target image corresponding to the refreshed image and the current brightness (the brightness value stored in the noise algorithm library before the moment indicated by the timestamp of the target image) are used to calculate the fusion noise at this moment. For the sake of description, the fusion noise at the moment of image refresh calculated due to image refresh can be recorded as the image noise at the moment of image refresh. Similarly, if the image is not refreshed at a certain moment but the brightness is adjusted, the adjusted brightness and the current target image (the target image stored in the noise algorithm library before the moment indicated by the timestamp of the brightness value) are used to calculate the fusion noise at this moment. For the sake of description, the fusion noise at the moment of brightness adjustment calculated due to brightness adjustment can be recorded as the backlight noise at the moment of brightness adjustment.

[0059] The target image and brightness sent by HWC to the noise algorithm library are both timestamped, and accordingly, the image noise and backlight noise calculated by the noise algorithm library are also timestamped. The timestamp of the image noise is the same as the timestamp of the target image, and the timestamp of the backlight noise is the same as the timestamp of the brightness to be adjusted. The timestamp of the image noise should strictly be the image refresh time. In practical applications, other time nodes close to the image refresh time can also be used as the image refresh time, for example, the start time (or end time, or any time between the start time and the end time) of the target image obtained by HWC from the CWB memory is used as the image refresh time. The timestamp of the backlight noise should strictly be the backlight adjustment time. In practical applications, other time nodes close to the backlight adjustment time can also be used as the backlight adjustment time, for example, the start time (or end time, or any time between the start time and the end time) of the brightness to be adjusted obtained from the kernel node by HWC is used as the brightness adjustment time. The timestamp of the image noise and the timestamp of the backlight noise facilitate the subsequent denoising of the initial ambient light collected by the ambient light sensor over the time span to obtain the target ambient light. The noise algorithm library stores the image noise and backlight noise in the noise memory. When the noise algorithm library stores the image noise, it also stores the timestamp of the image noise. When the noise algorithm library stores the backlight noise, it also stores the timestamp of the backlight noise.

[0060] After the ambient light sensor in the hardware layer of the SCP processor is started (usually, the ambient light sensor is started after the terminal device is turned on), it collects the initial ambient light at a certain preset period. For example, the ambient light sensor in the hardware layer of the SCP processor enters an integration time period, from t0 (t 01) moment. The ambient light sensor of the SCP processor transmits the initial ambient light information to the ambient light sensor driver (ALSDRV) of the hub driver layer (Hub DRV) of the SCP processor, see step E2. In step A1, the image is refreshed at time t0 (t01), and the SurfaceFlinger in the native framework layer of the AP processor sends the display parameters of the interface to the HWC in the hardware abstraction layer of the AP processor. The HWC can send the display parameters of each layer of the interface sent by SurfaceFlinger to the hardware at the bottom of the HWC, and the hardware at the bottom of the HWC obtains the composite image of each layer of the interface according to the display parameters of each layer of the interface. The hardware at the bottom of the HWC returns the composite image to the HWC.

[0061] In step A2, the HWC in the hardware abstraction layer of the AP processor sends the synthesized image to the OLED driver in the core layer of the AP processor.

[0062] In step A3, the OLED driver in the kernel layer of the AP processor sends the synthesized image to the display subsystem in the hardware layer of the AP processor.

[0063] In step A4, the display subsystem in the hardware layer of the AP processor stores the image before display in the CWB memory in the core layer of the AP processor.

[0064] In the embodiment of the present application, after the HWC sends the synthesized image to the OLED driver, the HWC will wait for a storage success signal sent by the display subsystem.

[0065] After the display subsystem successfully stores the image before display in the CWB memory, it will send a storage success signal to the HWC. After receiving the storage success signal from the display subsystem, the HWC cuts out the image before display stored in the CWB memory in the kernel layer to obtain the target image.

[0066] In step A5, the HWC in the hardware abstraction layer of the AP processor obtains the target image by clipping the image before display stored in the CWB memory in the kernel layer.

[0067] In step A6, after the HWC in the hardware abstraction layer of the AP processor obtains the target image, it sends the target image to the noise algorithm library of this layer. After receiving the target image, the noise algorithm library calculates the image noise or backlight noise at time t01 according to the target image and the cached current brightness information. During the execution of steps A1 to A6, the ambient light sensor in the hardware layer of the SCP processor is always in an integration process within a preset period.

[0068] In step A7, the noise algorithm library in the hardware abstraction layer in the AP processor sends the image noise or backlight noise at time t01 to the HWC of the same layer.

[0069] In step A8, after the AP processor calculates and obtains the image noise or backlight noise at time t01, the image noise or backlight noise at time t01 is sent to the ambient light sensor application of the cooperative application layer of the SCP processor.

[0070] In step E5 , the ambient light sensor application of the co-application layer of the SCP processor sends the image noise at time t01 to the noise memory of the co-driver layer of the SCP processor.

[0071] In step E6, the denoising module in the SCP processor co-driving layer retrieves the fused noise from the noise memory of the layer.

[0072] In step E3, the SCP processor co-driver layer obtains the raw values ​​of the four channels of the initial ambient light from the ambient light sensor of this layer.

[0073] In step E4, the SCP processor cooperative driving layer calculates the target ambient light according to the raw values ​​on the four channels of the initial ambient light and the image noise and backlight noise that interfere with the initial ambient light. In step E7, the calculation module in the cooperative driving layer of the SCP processor obtains the target ambient light lux value according to the raw values ​​on the four channels of the target ambient light. In step E8, the calculation module in the SCP processor sends the calculated target ambient light lux value to the ambient light sensor application of the cooperative application layer through the interface of the cooperative framework layer.

[0074] In step E9, the ambient light sensor application of the cooperative application layer in the SCP processor transmits the target ambient light lux value to the light service (lightservice) of the native framework layer in the AP processor through the second inter-core communication (communication from the SCP processor to the light service of the AP processor).

[0075] The light service can send the target ambient light lux value to the display engine service. The display engine service can send the target ambient light lux value to the upper layer so that the application in the application layer can determine whether to adjust the brightness. The display engine service can also send the target ambient light lux value to the kernel node so that the relevant hardware can adjust the brightness of the display screen according to the target ambient light lux value stored in the kernel node.

[0076] The following will be combined Figure 6 The following specifically introduces an ambient light reporting method provided by an embodiment of the present application. Figure 6The method of the example includes one or more steps, but does not constitute a limitation of the present application. In addition, the order of the steps of the method is only an example, and the order of the steps can be changed. Without departing from the content disclosed in the present application, additional steps can be added or steps can be reduced. The method specifically includes the following steps.

[0077] In step S601, the SCP processor acquires ambient light data according to a preset period, and obtains an ambient light value based on the ambient light data.

[0078] In one embodiment of the present application, the ambient light sensor of the SCP processor collects ambient light data according to a preset period, and obtains an ambient light value by integrating the ambient light data. The ambient light sensor transmits the ambient light value to the ambient light sensor driver (ALSDRV) of the hub driver layer (Hub DRV) of the SCP processor.

[0079] In step S602, the SCP processor determines whether to enable the gain adjustment function according to the ambient light value. If the gain adjustment function is enabled, step S603 is executed; if the gain adjustment function is not enabled, step S607 is executed.

[0080] In one embodiment of the present application, the SCP processor determines whether to turn on the gain adjustment function according to the ambient light value, including: determining whether the ambient light value is greater than or equal to a first preset ambient light threshold, and if the ambient light value is greater than or equal to the first preset ambient light threshold, determining to turn on the gain adjustment function. If the ambient light value is less than the first preset ambient light threshold, determining not to turn on the gain adjustment function. In one embodiment of the present application, the ambient light value being greater than or equal to the first preset ambient light threshold indicates that the ambient light around the terminal device has undergone a large fluctuation. For example, when the terminal device enters a dark ambient light environment from a bright ambient light environment, the ambient light value fluctuates greatly, resulting in the ambient light value being greater than or equal to the first preset ambient light threshold, and determining that the gain adjustment function needs to be turned on. The first preset ambient light threshold can be set according to actual needs, and the present application does not impose any restrictions on this.

[0081] Step S603: The SCP processor sends a notification message to the AP processor, where the notification message is used to instruct the SCP processor to enable a gain adjustment function.

[0082] In one embodiment of the present application, the notification message can be agreed upon by the SCP processor and the AP processor, for example, the notification message can be agreed upon as a preset error code. The SCP processor sends the error code as a notification message to the AP processor, which is used to notify the AP processor of the information that the SCP processor turns on the gain adjustment function.

[0083] Step S604: the SCP processor starts a gain adjustment function to adjust the ambient light value.

[0084] In one embodiment of the present application, the SCP processor starts the gain adjustment function to adjust the ambient light value, including: the SCP processor determines the gain value according to the ambient light value, adjusts the ambient light value according to the gain value, and integrates the adjusted ambient light value to obtain the adjusted ambient light value. In one embodiment of the present application, the SCP processor determines the gain value according to the intensity of the ambient light value.

[0085] In step S605, the SCP processor determines whether the gain adjustment function is completed according to the adjusted ambient light value. If the gain adjustment function is completed, step S606 is executed. If the gain adjustment function is not completed, the process returns to step S604 and calls the gain adjustment function to adjust the ambient light value.

[0086] In one embodiment of the present application, the SCP processor determines whether the adjustment gain function is executed based on the adjusted ambient light value, including: if the adjusted ambient light value is less than the second preset ambient light threshold, determining that the adjustment gain function is executed; if the adjusted ambient light value is greater than or equal to the second preset ambient light threshold, determining that the adjustment gain function is not executed.

[0087] Step S606: The SCP processor sets an end flag of the gain adjustment function.

[0088] In one embodiment of the present application, the SCP processor indicates the end of the gain adjustment function of the ambient light sensor by setting an end flag of the gain adjustment function. The end flag may be a character string, and the character string may include numbers, letters, or a combination of the two.

[0089] Step S607: The SCP processor reports the ambient light value to the AP processor.

[0090] In one embodiment of the present application, the SCP processor reports the ambient light value to the AP processor when the gain adjustment function is not turned on or when the gain adjustment function is completed. In one embodiment of the present application, reporting the ambient light value to the AP processor includes: calculating the ambient light lux value (i.e., the illuminance value of the ambient light) according to the ambient light value, and reporting the ambient light lux value to the AP processor. In one embodiment of the present application, the denoising module in the SCP processor cooperative driving layer reads the fused noise from the noise memory of this layer, obtains the initial (raw) value of the ambient light value on four channels from the ambient light sensor of this layer, wherein the fused noise includes image noise and backlight noise; calculates the ambient light lux value according to the raw value of the four channels of the ambient light value and the image noise and backlight noise that interfere with the ambient light value; the SCP processor sends the calculated ambient light lux value to the HWC in the AP processor through the interface of the cooperative framework layer.

[0091] Step S608: The AP processor cancels the median filtering process according to the notification message, performs denoising on the ambient light value to obtain a noise-free ambient light value, and sends the noise-free ambient light value to the SCP processor.

[0092] In one embodiment of the present application, the AP processor cancels the median filtering process according to the notification message, including: clearing the value in the noise memory in the AP processor. In one embodiment of the present application, the noise memory is a FIFO (First Input First Output) memory. The FIFO memory is a first-in-first-out dual-port buffer, and one of the two ports of the memory is an input port of the memory and the other is an output port of the memory.

[0093] In one embodiment of the present application, the HWC in the AP processor calls the noise algorithm library to perform denoising on the ambient light value and remove the image noise and backlight noise in the ambient light value to obtain a noise-free ambient light value. In one embodiment of the present application, the noise algorithm library can calculate the image noise and backlight noise at the time of refreshing the target image each time the target image currently displayed by the terminal device is obtained. The HWC in the AP processor calls the noise algorithm library to obtain the image noise and backlight noise at the time of refreshing the target image, and performs denoising on the ambient light value based on the obtained image noise and backlight noise to remove the image noise and backlight noise in the ambient light value, and obtain a noise-free ambient light value.

[0094] Step S609: The SCP processor calculates the corresponding ambient light lux value according to the noise-free ambient light value.

[0095] In one embodiment of the present application, the SCP processor co-driving layer calculates the corresponding ambient light lux value based on the noise-free ambient light value.

[0096] Step S610: The SCP processor responds to the end flag of the gain adjustment function and reports the ambient light lux value of the noise-free ambient light value to the AP processor.

[0097] In one embodiment of the present application, the SCP processor sends the calculated noise-free ambient light value of the ambient light lux value to the ambient light sensor application of the cooperative application layer through the interface of the cooperative framework layer, and the ambient light sensor application of the SCP processor transmits the ambient light lux value to the light service (lightservice) of the native framework layer in the AP processor.

[0098] Step S611: The SCP processor clears the end flag of the gain adjustment function.

[0099] In one embodiment of the present application, the SCP processor clears the end flag of the gain adjustment function so as to report the lux value of the next ambient light data.

[0100] In an embodiment of the present application, the method further includes: the AP processor adjusts the brightness according to the ambient light lux value.

[0101] In one embodiment of the present application, the light service in the AP processor sends the ambient light lux value to the display engine service. The display engine service can send the ambient light lux value to the application in the application layer, and the application in the application layer performs brightness adjustment.

[0102] In the embodiment of the present application, when it is detected that the ambient light sensor turns on the gain adjustment function, the SCP processor notifies the AP processor to cancel the median filtering process, so that the ambient light lux value of the first frame of ambient light data reported by the SCP processor received by the AP processor will not be filtered by the median filtering process, thereby reducing the time consumed by the SCP processor to report the ambient light lux value. At the same time, the SCP processor monitors the action of the gain adjustment function, and when the gain adjustment function is completed, the SCP processor can immediately report the noise-free ambient light value returned by the AP processor according to the end flag of the gain adjustment function when receiving the ambient light value, without waiting for the ambient light sensor to integrate the ambient light data collected in the next preset period before reporting, thereby further reducing the time consumed by the SCP processor to report the ambient light lux value.

[0103] refer to Figure 7 FIG. 2 is a flow chart of an ambient light reporting method provided by another embodiment of the present application. The method comprises the following steps.

[0104] In step S701, the SCP processor acquires ambient light data according to a preset period, and obtains an ambient light value based on the ambient light data.

[0105] In step S702, the SCP processor determines whether to enable the gain adjustment function according to the ambient light value. If the gain adjustment function is enabled, step S703 is executed. If the gain adjustment function is not enabled, step S704 is executed.

[0106] Steps S701-S702 and Figure 6 Steps S601-S602 are the same, and the specific implementation contents of steps S701-S702 can be referred to Figure 6 The implementation contents of steps S601-S602 will not be described again here.

[0107] Step S703, when it is detected that the gain adjustment function is completed, the end flag of the gain adjustment function is set. In one embodiment of the present application, after determining to start the gain adjustment function, the SCP processor starts the gain adjustment function to adjust the ambient light value and detects whether the gain adjustment function is completed. If it is detected that the gain adjustment function is completed, the end flag of the gain adjustment function is set.

[0108] Step S704: The SCP processor transmits the ambient light value to the AP processor.

[0109] In one embodiment of the present application, the SCP processor reports the ambient light value to the AP processor when the gain adjustment function is not enabled or the gain adjustment function is terminated. Figure 6 The same as step S607, the specific implementation content can refer to Figure 6 Implementation content of step S607.

[0110] Step S705: The AP processor performs denoising on the ambient light value to obtain a noise-free ambient light value, and sends the noise-free ambient light value to the SCP processor.

[0111] In one embodiment of the present application, the HWC in the AP processor calls the noise algorithm library to perform denoising on the ambient light value to remove the image noise and backlight noise in the ambient light value. In one embodiment of the present application, the noise algorithm library can calculate the image noise and backlight noise at the time of refreshing the target image each time after obtaining the target image currently displayed by the terminal device. The HWC in the AP processor calls the noise algorithm library to obtain the image noise and backlight noise at the time of refreshing the target image, and performs denoising on the ambient light value based on the obtained image noise and backlight noise to remove the image noise and backlight noise in the ambient light value, and obtain a noise-free ambient light value.

[0112] Step S706: The SCP processor calculates the ambient light lux value according to the noise-free ambient light value.

[0113] Step S707: The SCP processor responds to the end flag of the gain adjustment function and reports the ambient light lux value to the AP processor.

[0114] Step S708: The SCP processor clears the end flag of the gain adjustment function.

[0115] Steps S706-S708 and Figure 6 Steps S609-S611 are the same, and the specific implementation content of steps S706-S708 can be referred to Figure 6 The implementation contents of steps S609-S611 will not be described again here.

[0116] In the embodiment of the present application, when it is detected that the ambient light sensor turns on the gain adjustment function, the SCP processor monitors the action of the gain adjustment function, and when the gain adjustment function ends, the SCP processor can immediately report the noise-free ambient light value returned by the AP processor according to the end flag of the gain adjustment function, without waiting for the ambient light sensor to integrate the ambient light data collected in the next preset period before reporting, thereby further reducing the time consumed by the SCP processor to report the ambient light lux value.

[0117] The following is an introduction to the terminal devices involved in the embodiments of the present application.

[0118] refer to Figure 8 In this embodiment, the terminal device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0119] It is understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the terminal device 100. In other embodiments of the present application, the terminal device 100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0120] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0121] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0122] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0123] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0124] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple groups of I2C buses. The processor 110 may be coupled to the touch sensor 180K, the charger, the flash, the camera 193, etc. through different I2C bus interfaces. For example: the processor 110 may be coupled to the touch sensor 180K through the I2C interface, so that the processor 110 communicates with the touch sensor 180K through the I2C bus interface, thereby realizing the touch function of the terminal device 100.

[0125] The I2S interface can be used for audio communication. In some embodiments, the processor 110 can include multiple I2S buses. The processor 110 can be coupled to the audio module 170 via the I2S bus to achieve communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 can transmit an audio signal to the wireless communication module 160 via the I2S interface to achieve the function of answering a call through a Bluetooth headset.

[0126] The PCM interface can also be used for audio communication, sampling, quantizing and encoding analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 can be coupled via a PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 via the PCM interface to realize the function of answering calls via a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

[0127] The UART interface is a universal serial data bus for asynchronous communication. The bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, the UART interface is generally used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 through the UART interface to implement the Bluetooth function. In some embodiments, the audio module 170 can transmit an audio signal to the wireless communication module 160 through the UART interface to implement the function of playing music through a Bluetooth headset.

[0128] The MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a camera serial interface (CSI), a display serial interface (DSI), etc. In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to implement the shooting function of the terminal device 100. The processor 110 and the display screen 194 communicate via the DSI interface to implement the display function of the terminal device 100.

[0129] The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 with the camera 193, the display 194, the wireless communication module 160, the audio module 170, the sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0130] The USB interface 130 is an interface that complies with the USB standard specification, and specifically can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the terminal device 100, and can also be used to transmit data between the terminal device 100 and peripheral devices. It can also be used to connect headphones to play audio through the headphones. The interface can also be used to connect other terminal devices 100, such as AR devices, etc.

[0131] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present application is only a schematic illustration and does not constitute a structural limitation on the terminal device 100. In other embodiments of the present application, the terminal device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0132] The charging management module 140 is used to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 may receive wireless charging input through a wireless charging coil of the terminal device 100. While the charging management module 140 is charging the battery 142, it may also power the terminal device 100 through the power management module 141.

[0133] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the display screen 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc. In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0134] The wireless communication function of the terminal device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0135] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in terminal device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0136] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to the terminal device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0137] The modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 170A, a receiver 170B, etc.), or displays an image or video through a display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

[0138] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the terminal device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0139] In some embodiments, the antenna 1 of the terminal device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the terminal device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0140] The terminal device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or change display information.

[0141] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light emitting diode or an active-matrix organic light emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the terminal device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0142] The terminal device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0143] ISP is used to process the data fed back by camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, ISP can be set in camera 193.

[0144] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the terminal device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0145] The digital signal processor is used to process digital signals, and can process not only digital image signals but also other digital signals. For example, when the terminal device 100 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0146] Video codecs are used to compress or decompress digital videos. The terminal device 100 may support one or more video codecs. Thus, the terminal device 100 may play or record videos in various coding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0147] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. Through NPU, applications such as intelligent cognition of the terminal device 100 can be realized, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0148] The internal memory 121 may include one or more random access memories (RAM) and one or more non-volatile memories (NVM).

[0149] Random access memory may include static random-access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM, for example, the fifth generation DDR SDRAM is generally referred to as DDR5 SDRAM), etc.; non-volatile memory may include disk storage devices and flash memory.

[0150] Flash memory can be divided into NOR FLASH, NAND FLASH, 3D NAND FLASH, etc. according to the operating principle; single-level cell (SLC), multi-level cell (MLC), triple-level cell (TLC), quad-level cell (QLC), etc. according to the storage unit potential level; universal flash storage (UFS), embedded multi media card (eMMC), etc. according to the storage specification.

[0151] The random access memory can be directly read and written by the processor 110, and can be used to store executable programs (such as machine instructions) of the operating system or other running programs, and can also be used to store user and application data.

[0152] The non-volatile memory may also store executable programs and user and application data, etc., and may be loaded into the random access memory in advance for direct reading and writing by the processor 110 .

[0153] The external memory interface 120 can be used to connect to an external non-volatile memory to expand the storage capacity of the terminal device 100. The external non-volatile memory communicates with the processor 110 through the external memory interface 120 to implement a data storage function. For example, files such as music and videos are stored in the external non-volatile memory.

[0154] The internal memory 121 or the external memory interface 120 is used to store one or more computer programs. The one or more computer programs are configured to be executed by the processor 110. The one or more computer programs include multiple instructions. When the multiple instructions are executed by the processor 110, the ambient light reporting method in the above embodiment can be executed on the terminal device 100 to implement the ambient light reporting function of the terminal device 100.

[0155] The terminal device 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0156] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be arranged in the processor 110, or some functional modules of the audio module 170 can be arranged in the processor 110.

[0157] The speaker 170A, also called a "speaker", is used to convert an audio electrical signal into a sound signal. The terminal device 100 can listen to music or listen to a hands-free call through the speaker 170A.

[0158] The receiver 170B, also called a "handset", is used to convert audio electrical signals into sound signals. When the terminal device 100 receives a call or voice message, the voice can be received by placing the receiver 170B close to the ear.

[0159] Microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak by putting their mouth close to the microphone 170C to input the sound signal into the microphone 170C. The terminal device 100 can be provided with at least one microphone 170C. In other embodiments, the terminal device 100 can be provided with two microphones 170C, which can not only collect sound signals but also realize noise reduction function. In other embodiments, the terminal device 100 can also be provided with three, four or more microphones 170C to realize the collection of sound signals, noise reduction, identification of sound sources, realization of directional recording function, etc.

[0160] The earphone interface 170D is used to connect a wired earphone and can be a USB interface 130, or a 3.5 mm open mobile terminal platform (OMTP) standard interface or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0161] The pressure sensor 180A is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A can be set on the display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. The capacitive pressure sensor can be a parallel plate including at least two conductive materials. When a force acts on the pressure sensor 180A, the capacitance between the electrodes changes. The terminal device 100 determines the intensity of the pressure according to the change in capacitance. When a touch operation acts on the display screen 194, the terminal device 100 detects the touch operation intensity according to the pressure sensor 180A. The terminal device 100 can also calculate the touch position according to the detection signal of the pressure sensor 180A. In some embodiments, touch operations acting on the same touch position but with different touch operation intensities can correspond to different operation instructions. For example: when a touch operation with a touch operation intensity less than the first pressure threshold acts on the short message application icon, an instruction to view the short message is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure threshold acts on the short message application icon, an instruction to create a new short message is executed.

[0162] The gyroscope sensor 180B can be used to determine the motion posture of the terminal device 100. In some embodiments, the angular velocity of the terminal device 100 around three axes (i.e., x, y, and z axes) can be determined by the gyroscope sensor 180B. The gyroscope sensor 180B can be used for anti-shake shooting. For example, when the shutter is pressed, the gyroscope sensor 180B detects the angle of the terminal device 100 shaking, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to offset the shaking of the terminal device 100 through reverse movement to achieve anti-shake. The gyroscope sensor 180B can also be used for navigation and somatosensory game scenes.

[0163] The air pressure sensor 180C is used to measure air pressure. In some embodiments, the terminal device 100 calculates the altitude through the air pressure value measured by the air pressure sensor 180C to assist positioning and navigation.

[0164] The magnetic sensor 180D includes a Hall sensor. The terminal device 100 can use the magnetic sensor 180D to detect the opening and closing of the flip leather case. In some embodiments, when the terminal device 100 is a flip phone, the terminal device 100 can detect the opening and closing of the flip cover according to the magnetic sensor 180D. Then, according to the detected opening and closing state of the leather case or the opening and closing state of the flip cover, the flip cover automatic unlocking and other features are set.

[0165] The acceleration sensor 180E can detect the magnitude of the acceleration of the terminal device 100 in all directions (generally three axes). When the terminal device 100 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of the terminal device 100 and applied to applications such as horizontal and vertical screen switching and pedometers.

[0166] The distance sensor 180F is used to measure the distance. The terminal device 100 can measure the distance by infrared or laser. In some embodiments, when shooting a scene, the terminal device 100 can use the distance sensor 180F to measure the distance to achieve fast focusing.

[0167] The proximity light sensor 180G may include, for example, a light emitting diode (LED) and a light detector, such as a photodiode. The light emitting diode may be an infrared light emitting diode. The terminal device 100 emits infrared light outward through the light emitting diode. The terminal device 100 uses a photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the terminal device 100. When insufficient reflected light is detected, the terminal device 100 can determine that there is no object near the terminal device 100. The terminal device 100 can use the proximity light sensor 180G to detect that the user holds the terminal device 100 close to the ear to talk, so as to automatically turn off the screen to save power. The proximity light sensor 180G can also be used in leather case mode, and pocket mode automatically unlocks and locks the screen.

[0168] The ambient light sensor 180L is used to sense the ambient light brightness. The terminal device 100 can adaptively adjust the brightness of the display screen 194 according to the perceived ambient light brightness. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the terminal device 100 is in a pocket to prevent accidental touch.

[0169] The fingerprint sensor 180H is used to collect fingerprints. The terminal device 100 can use the collected fingerprint characteristics to achieve fingerprint unlocking, access application locks, fingerprint photography, fingerprint call answering, etc.

[0170] The temperature sensor 180J is used to detect temperature. In some embodiments, the terminal device 100 uses the temperature detected by the temperature sensor 180J to execute a temperature processing strategy. For example, when the temperature reported by the temperature sensor 180J exceeds a threshold, the terminal device 100 reduces the performance of the processor located near the temperature sensor 180J to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is lower than another threshold, the terminal device 100 heats the battery 142 to avoid abnormal shutdown of the terminal device 100 due to low temperature. In other embodiments, when the temperature is lower than another threshold, the terminal device 100 performs a boost on the output voltage of the battery 142 to avoid abnormal shutdown caused by low temperature.

[0171] The touch sensor 180K is also called a "touch control device". The touch sensor 180K can be set on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, also called a "touch control screen". The touch sensor 180K is used to detect touch operations acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In other embodiments, the touch sensor 180K can also be set on the surface of the terminal device 100, which is different from the position of the display screen 194.

[0172] The bone conduction sensor 180M can obtain a vibration signal. In some embodiments, the bone conduction sensor 180M can obtain a vibration signal of a vibrating bone block of the vocal part of the human body. The bone conduction sensor 180M can also contact the human pulse to receive a blood pressure beat signal. In some embodiments, the bone conduction sensor 180M can also be set in an earphone and combined into a bone conduction earphone. The audio module 170 can parse out a voice signal based on the vibration signal of the vibrating bone block of the vocal part obtained by the bone conduction sensor 180M to realize a voice function. The application processor can parse the heart rate information based on the blood pressure beat signal obtained by the bone conduction sensor 180M to realize a heart rate detection function.

[0173] The key 190 includes a power key, a volume key, etc. The key 190 may be a mechanical key or a touch key. The terminal device 100 may receive key input and generate key signal input related to user settings and function control of the terminal device 100.

[0174] Motor 191 can generate vibration prompts. Motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, audio playback, etc.) can correspond to different vibration feedback effects. For touch operations acting on different areas of the display screen 194, motor 191 can also correspond to different vibration feedback effects. Different application scenarios (for example: time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

[0175] Indicator 192 may be an indicator light, which may be used to indicate charging status, power changes, messages, missed calls, notifications, etc.

[0176] The SIM card interface 195 is used to connect a SIM card. The SIM card can be connected to and separated from the terminal device 100 by inserting the SIM card interface 195 or pulling it out from the SIM card interface 195. The terminal device 100 can support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, and the like. Multiple cards can be inserted into the same SIM card interface 195 at the same time. The types of the multiple cards can be the same or different. The SIM card interface 195 can also be compatible with different types of SIM cards. The SIM card interface 195 can also be compatible with external memory cards. The terminal device 100 interacts with the network through the SIM card to implement functions such as calls and data communications. In some embodiments, the terminal device 100 uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the terminal device 100 and cannot be separated from the terminal device 100.

[0177] This embodiment further provides a computer storage medium, in which computer instructions are stored. When the computer instructions are executed on the terminal device 100, the terminal device 100 executes the above-mentioned related method steps to implement the ambient light reporting method in the above-mentioned embodiment.

[0178] This embodiment further provides a computer program product. When the computer program product is run on a computer, the computer is enabled to execute the above-mentioned related steps to implement the ambient light reporting method in the above-mentioned embodiment.

[0179] In addition, an embodiment of the present application also provides a device, which may specifically be a chip, a component or a module, and the device may include a connected processor and a memory; wherein the memory is used to store computer-executable instructions, and when the device is running, the processor may execute the computer-executable instructions stored in the memory so that the chip executes the ambient light reporting method in the above-mentioned method embodiments.

[0180] Among them, the terminal device 100, computer storage medium, computer program product or chip provided in this embodiment are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0181] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0182] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0183] The unit described as a separate component may or may not be physically separated, and the component shown as a unit may be one physical unit or multiple physical units, that is, it may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiment.

[0184] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0185] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of each embodiment method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store program code.

[0186] The above implementation modes are only used to illustrate the technical solutions of the present application and are not intended to limit the present application. Although the present application has been described in detail with reference to the above preferred implementation modes, a person skilled in the art should understand that the technical solutions of the present application may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present application.

Claims

1. A method for reporting ambient light, applied to a terminal device, characterized in that: The method comprises: The sensor co-processor of the terminal device acquires ambient light data according to a preset period, and obtains an ambient light value based on the ambient light data; If the sensor co-processor determines to start the gain adjustment function according to the ambient light value, when detecting that the gain adjustment function is completed, setting an end flag of the gain adjustment function; The sensor co-processor reports the ambient light value to the application processor of the terminal device; The application processor performs denoising on the ambient light value to obtain a noise-free ambient light value, and sends the noise-free ambient light value to the sensor co-processor; The sensor co-processor calculates the ambient light illumination value according to the noise-free ambient light value; The sensor co-processor responds to the end flag of the gain adjustment function and reports the ambient light illumination value to the application processor.

2. The ambient light reporting method according to claim 1, characterized in that: After determining to start the gain adjustment function, the method further includes: The sensor co-processor sends a notification message to the application processor, where the notification message is used to instruct the sensor co-processor to enable the gain adjustment function; The application processor cancels the median filtering process according to the notification message. The median filtering process refers to sorting all the ambient light illumination values ​​input into the noise memory of the terminal device and selecting the ambient light report value in the middle position for output.

3. The ambient light reporting method according to claim 2, characterized in that: The canceling of the median filtering process comprises: Clear the value in the noise memory.

4. The ambient light reporting method according to claim 1, characterized in that: The application processor performs denoising on the ambient light value to obtain a noise-free ambient light value, including: The application processor calls a noise algorithm library to perform denoising on the ambient light value and remove image noise and backlight noise in the ambient light value to obtain the noise-free ambient light value.

5. The ambient light reporting method according to claim 1, characterized in that: The sensor co-processor determines to start a gain adjustment function according to the ambient light value, including: If the ambient light value is greater than or equal to a first preset ambient light threshold, determining to enable the gain adjustment function; If the ambient light value is less than the first preset ambient light threshold, it is determined not to enable the gain adjustment function.

6. The ambient light reporting method according to claim 1, characterized in that: After determining to start the gain adjustment function, the method further includes: The sensor co-processor enables the gain adjustment function to adjust the ambient light value.

7. The ambient light reporting method according to claim 6, characterized in that: The sensor co-processor starts the gain adjustment function to adjust the ambient light value, including: The sensor co-processor determines a gain value according to the ambient light value, adjusts the ambient light value according to the gain value, and integrates the adjusted ambient light value.

8. The ambient light reporting method according to claim 1, characterized in that: After reporting the ambient light illumination value to the application processor, the method further includes: The sensor co-processor clears an end flag of the gain adjustment function.

9. The ambient light reporting method according to claim 1, characterized in that: After reporting the ambient light illumination value to the application processor, the method further includes: The application processor performs brightness adjustment according to the ambient light illumination value.

10. A terminal device, characterized in that: The terminal device includes a memory and a processor: The memory is used to store program instructions; The processor is used to read and execute the program instructions stored in the memory. When the program instructions are executed by the processor, the terminal device executes the ambient light reporting method as described in any one of claims 1 to 9.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program instructions, and when the program instructions are executed on a terminal device, the terminal device executes the ambient light reporting method according to any one of claims 1 to 9.

Images