Image processing method and related device

Abstract

The present application provides an image processing method and related devices. In this method, after the electronic device starts the camera application, in the selfie mode, during the process of switching from the unfolded state to the folded state and then switching to the unfolded state, the electronic device can use valid Stats statistical parameters to calculate the exposure parameters after the camera for collecting images is switched from the external screen camera to the internal screen camera, and then use the internal screen camera 203 to collect images based on the exposure parameters. Valid Stats statistical parameters refer to: Stats statistical parameters obtained based on non-completely black images collected by the internal screen camera 203. In this way, the user can observe the normally exposed image during the process of switching cameras.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method and related devices. Background Art

[0002] An electronic device with a foldable screen may include multiple cameras, such as an inner screen camera, an outer screen camera, and a rear camera. The multiple cameras may be distributed in different locations.

[0003] When the folding screen electronic device starts the camera, the camera for collecting images will be switched while the folding screen of the electronic device is switched. How to make the electronic device display a better preview image during the switching of the folding screen is an urgent problem to be solved in the art. Summary of the invention

[0004] The present application provides an image processing method and related devices, so that after the electronic device starts the camera application, in the selfie mode, when switching from the unfolded state to the folded state and then switching back to the unfolded state, the displayed preview image will not be overexposed.

[0005] In a first aspect, the present application provides an image processing method, which is applied to an electronic device, wherein the electronic device comprises: a first display screen, a second display screen, and a third display screen, wherein the first display screen is connected to the second display screen via a folding component, the second display screen and the display surface of the third display screen are opposite to each other, an inner screen camera is arranged on the first display screen, and an outer screen camera is arranged on the third display screen, and when the electronic device is in a folded state, the display surfaces of the first display screen and the second display screen are opposite to each other;

[0006] The method includes:

[0007] The electronic device is in an unfolded state, and the inner screen camera is activated;

[0008] When the electronic device switches from the unfolded state to the folded state, the inner screen camera is turned off and the outer screen camera is turned on;

[0009] When the electronic device switches from the folded state to the unfolded state, the outer screen camera is turned off and the inner screen camera is turned on;

[0010] The electronic device displays a first preview image captured by the internal screen camera on the first display screen and the second display screen;

[0011] Among them, the first preview image is the first frame image displayed after the electronic device turns off the external screen camera and starts the internal screen camera, and the first preview image is exposed normally.

[0012] By implementing the above method, after the electronic device starts the camera application, in the selfie mode, when switching from the unfolded state to the folded state and then to the unfolded state, the preview image displayed will not be overexposed. In this way, the preview image observed by the user during the process of switching between the folded state and the unfolded state using the electronic device's camera is always normally exposed, improving the user experience.

[0013] In combination with the first aspect, in some implementations, after the electronic device turns off the external screen camera and starts the internal screen camera, the method further includes:

[0014] The electronic device determines a first exposure parameter according to a first Stats statistical parameter;

[0015] The electronic device collects the first preview image through the internal screen camera according to the first exposure parameter;

[0016] Among them, the first Stats statistical parameter includes: a parameter used to indicate the brightness of the second preview image, and the second preview image is a non-completely black image captured by the electronic device through the internal screen camera.

[0017] In some implementations, the first Stats statistical parameter can be obtained based on the non-total black image captured by the internal screen camera 203 last time, that is, the first Stats statistical parameter is valid. The first exposure parameter obtained by the electronic device based on the valid first Stats statistical parameter can be used to obtain a preview image with normal exposure.

[0018] In combination with the first aspect, in some implementations, the first Stats statistical parameter also includes a configuration parameter of the internal screen camera, and the value of the first exposure parameter is within an adjustment range of the internal screen camera.

[0019] In some implementations, the Stats statistical parameters may also include a series of camera configuration parameters, such as ISPStatsROIParams, ISPHDRStatsConfig, ISPHDRBHistStatsConfig, ISPBFStatsConfig, etc. This series of parameters may limit the adjustment range of the exposure parameters, so that the internal screen camera may adopt the first exposure parameters obtained based on the first Stats statistical parameters within a normal applicable range.

[0020] In combination with the first aspect, in some implementations, the first exposure parameter includes one or more of the following: exposure time, ISO, and aperture.

[0021] In a possible implementation, the electronic device may use the value in the first exposure parameter to adjust the exposure time, ISO, and aperture of the internal screen camera. Optionally, the aperture of the internal screen camera in the electronic device may be adjusted. Alternatively, the aperture of the internal screen camera in the electronic device may not be adjusted.

[0022] In combination with the first aspect, in some implementations, the electronic device is in an unfolded state, and after starting the inner-screen camera, the method further includes:

[0023] The electronic device calculates the Stats statistical parameters of each frame of image captured by the internal screen camera;

[0024] The electronic device stores the Stats statistical parameters of multiple frames of images captured by the internal screen camera in a first queue in chronological order. The multiple frames of images include: non-completely black images captured by the internal screen camera. The first Stats statistical parameter is the earliest parameter stored in the first queue.

[0025] In some implementations, the above-mentioned multiple frames can be based on the speed at which the user folds the electronic device. For example, the multiple frames can be set to 6 frames. In this way, the Stats statistical parameter with the earliest storage time read from the first queue can be obtained based on the non-completely black image obtained based on the image captured by the internal screen camera 203 in the fully unfolded state. In this way, the first Stats statistical parameter is more effective.

[0026] In conjunction with the first aspect, in some implementations, after the electronic device captures the first preview image through the internal screen camera, the method further includes:

[0027] The electronic device clears the Stats statistical parameters stored in the first queue, and stores the Stats statistical parameters of the first preview image in the first queue.

[0028] In this way, after the first queue is used, the Stats statistical parameters in the first queue are cleared and the latest acquired Stats statistical parameters are stored in chronological order. In this way, it can be ensured that the subsequent electronic device reads the Stats statistical parameters with the earliest storage time from the first queue with stronger real-time performance, making the acquired exposure parameters more accurate.

[0029] In combination with the first aspect, in some implementations, a frameluma parameter of a frame grayscale component in each Stats statistical parameter in the first queue is greater than or equal to 1.

[0030] In this way, the electronic device can determine whether the current Stats statistical parameters are valid according to the frameluma parameters, and the electronic device does not need to perform more complicated calculations to determine whether the Stats statistical parameters are valid, thus saving time.

[0031] In combination with the first aspect, in some implementations, when the electronic device switches from the unfolded state to the folded state, before turning off the inner screen camera and starting the outer screen camera, the method further includes:

[0032] When the electronic device switches from the unfolded state to the folded state, it is detected that the angle between the first display screen and the second display screen decreases to a first threshold value.

[0033] In some implementations, when the electronic device switches from the unfolded state to the folded state, after detecting that the angle between the first display screen and the second display screen decreases to a first threshold, the electronic device generates a first signal. Based on the first signal, the electronic device can turn off the inner screen camera and start the outer screen camera.

[0034] In combination with the first aspect, in some implementations, when the electronic device switches from the folded state to the unfolded state, before turning off the external screen camera and starting the internal screen camera, the method further includes:

[0035] When the electronic device switches from the folded state to the unfolded state, it is detected that the angle between the first display screen and the second display screen increases to the first threshold value.

[0036] In some implementations, when the electronic device switches from the folded state to the unfolded state, after detecting that the angle between the first display screen and the second display screen increases to the first threshold, the electronic device generates a second signal. Based on the second signal, the electronic device can turn off the external screen camera and start the internal screen camera.

[0037] In combination with the first aspect, in some implementations, the electronic device is in an unfolded state, and after starting the inner-screen camera, the method further includes:

[0038] The electronic device displays the preview image captured by the internal screen camera on the first display screen and the second display screen.

[0039] In combination with the first aspect, in some implementations, when the electronic device switches from the unfolded state to the folded state, after turning off the inner screen camera and starting the outer screen camera, the method further includes:

[0040] The electronic device displays the preview image captured by the external screen camera on the third display screen.

[0041] In a second aspect, the present application provides an electronic device, the electronic device comprising a host and a solid state drive SSD; the host comprising one or more processors and one or more memories. The SSD is coupled to the one or more processors, the SSD is used to store computer program code, the computer program code comprises computer instructions, when the one or more processors execute the computer instructions, the electronic device executes the method of the first aspect or any embodiment of the first aspect.

[0042] In a third aspect, an embodiment of the present application provides a computer storage medium, including computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes the method of the first aspect or any implementation method of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 A schematic diagram of an AE algorithm framework provided in an embodiment of the present application;

[0044] Figure 2A-2C Schematic diagrams of the structures of several electronic devices provided in the embodiments of the present application;

[0045] Figure 3 A schematic diagram of the camera layout of an electronic device provided in an embodiment of the present application;

[0046] Figure 4 A schematic diagram of an application scenario of switching cameras of an electronic device provided in an embodiment of the present application;

[0047] Figure 5 A flowchart of an image processing method provided in an embodiment of the present application;

[0048] Figure 6 A schematic diagram of the hardware architecture of an electronic device 100 provided in an embodiment of the present application;

[0049] Figure 7 A schematic diagram of the software architecture of the electronic device 100 provided in an embodiment of the present application;

[0050] Fig. 8A A schematic diagram of a Stats statistical parameter determination module provided in an embodiment of the present application;

[0051] Figure 8B A schematic diagram of a Stats statistical parameter storage module provided in an embodiment of the present application;

[0052] Figure 8C A schematic diagram of a camera switching determination module provided in an embodiment of the present application. DETAILED DESCRIPTION

[0053] The technical solutions in the embodiments of the present application will be described clearly and in detail below in conjunction with the accompanying drawings. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in the text is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0054] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features, and in the description of the embodiments of the present application, unless otherwise specified, "plurality" means two or more.

[0055] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments.

[0056] The term "user interface (UI)" in the following embodiments of the present application refers to a medium interface for interaction and information exchange between an application or operating system and a user, which realizes the conversion between the internal form of information and the form acceptable to the user. The user interface is a source code written in a specific computer language such as Java and extensible markup language (XML). The interface source code is parsed and rendered on an electronic device, and finally presented as content that can be recognized by the user. The commonly used form of user interface is a graphical user interface (GUI), which refers to a user interface related to computer operation displayed in a graphical manner. It can be a visual interface element such as text, icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets, etc. displayed on the display screen of an electronic device.

[0057] The electronic device can start the camera function and display a preview image on the display screen. The following are some related terms about the process of the electronic device presenting the preview image:

[0058] Exposure refers to the process in which the light emitted or reflected by the photographic object is projected onto the photosensitive film in the electronic device through the camera lens, causing the photosensitive film to undergo chemical changes and produce a latent image. From a physical point of view, the exposure value can be referred to as the product of light intensity and the duration of light action, that is, exposure value = illumination * exposure time. Among them, light intensity refers to the luminous flux of light received per unit area, that is, illumination. The duration of light action refers to the time the photoreceptor receives light irradiation, that is, exposure time. If an electronic device wants to obtain a certain amount of exposure, the greater the light intensity, the shorter the exposure time; the smaller the light intensity, the longer the exposure time. Among them, the shutter can be used to control the time that light is on the photosensitive film, and the aperture can be used to control the amount of light reaching the photosensitive film. As an item in the ISO standard of the International Organization for Standardization, sensitivity can be quantitatively specified by ISO, that is, ISO can be used to control the sensitivity of the camera to light.

[0059] Normal exposure can mean that the light and dark distribution of the image is uniform, or that an object with a certain reflectivity is restored to the corresponding gray level in the image displayed on the electronic device. This is because the brightness and color of an object are determined by the reflectivity of the object to light. For example, the reflectivity of pure black is 0, the reflectivity of pure white is 100%, and the reflectivity of the gray level in the middle is 18%.

[0060] Overexposure refers to the phenomenon that the image brightness displayed on the electronic device is too high and the photo is too white. The causes of overexposure may include but are not limited to: too large aperture, too slow shutter speed, too high exposure parameters, etc.

[0061] Stats statistical parameters: refers to the parameters obtained by the electronic device based on the current image frame and the configuration parameters of the camera. Stats statistical parameters are used to determine new exposure parameters.

[0062] The parameters obtained based on the current image frame in the Stats statistical parameters can be used to indicate the brightness of the current image. The parameters obtained based on the current image frame may include, but are not limited to: a brightness histogram and brightness area statistics. For example, a brightness histogram may include: a 256-segment histogram of the entire frame image; a brightness area statistics may include: R / Gr / Gb / B four-component average value statistics of the entire frame image, or R / Gr / Gb / B four-component average value statistics of each block of the M*N blocks of the entire frame image.

[0063] Specifically, the camera configuration parameters in the Stats statistical parameters can refer to the chiispstatsdefs.h header file definition, which may include ISPStatsROIParams, ISPHDRStatsConfig, ISPHDRBHistStatsConfig, ISPBFStatsConfig and other parameters.

[0064] Among them, ISPStatsROIParams is used to indicate the region of interest (ROI) and ROI mode set by the camera of the electronic device for the current image. ISPHDRStatsConfig is used to indicate the high dynamic range imaging (HDR) configuration set by the camera of the electronic device for the current image. ISPHDRBHistStatsConfig is used to indicate the historical information of the HDR configuration set by the camera of the electronic device.

[0065] The exposure parameters may include, but are not limited to, exposure time and ISO gain value. The exposure time generally refers to the time for light to act, and the ISO gain value indicates the image brightness gain. The larger the exposure value, the brighter the image; the larger the ISO gain value, the brighter the image. Optionally, the exposure parameters may also include aperture, and the larger the aperture, the brighter the image.

[0066] In some implementations, the external screen camera and the internal screen camera of the electronic device do not share a set of exposure parameters due to the different types of cameras and different internal components of the cameras. The exposure parameters of the external screen camera and the internal screen camera are calculated separately.

[0067] The target brightness may refer to the brightness of the preview image finally presented by the electronic device adjusting the brightness of the captured image. The target brightness may be set by the electronic device or by the user.

[0068] The automatic exposure algorithm (AE) refers to an algorithm that continuously adjusts the exposure parameters according to the image obtained by the electronic device so that the electronic device can subsequently obtain a normally exposed image. It can also be understood as a mechanism for the electronic device to automatically adjust the exposure parameters according to the strength of the external light.

[0069] Combine the following Figure 1 The AE algorithm provided in the embodiment of the present application is introduced.

[0070] Figure 1 An AE algorithm framework diagram provided in an embodiment of the present application is exemplified.

[0071] refer to Figure 1 , Figure 1 The AE algorithm shown is actually a cyclic process, thereby achieving the goal of infinitely approaching the ideal exposure parameters. For the first time, the electronic device collects an image through a camera and generates an image based on default exposure parameters.

[0072] The electronic device then processes the image to obtain Stats statistical parameters.

[0073] Next, the electronic device generates new exposure parameters based on the Stats statistical parameters and the target brightness.

[0074] Finally, the electronic device can adjust the brightness of the acquired new frame image based on the new exposure parameters, thereby obtaining an image displaying the new frame. Among them, exposure intensity = exposure time * aperture size * ISO. Exposure intensity is used to characterize image brightness. The aperture size in electronic devices is generally fixed. Therefore, the electronic device can adjust the brightness of the image by adjusting the exposure time and ISO size. Specifically, the above parameters can be sent to the camera in the request process of the application, and the electronic device usually gives priority to adjusting the exposure time. When the adjustment degree of the exposure time reaches the maximum degree and still cannot satisfy the brightness of the image to reach or approach the target brightness, adjust the ISO. Optionally, when the aperture in the electronic device can be adjusted, the electronic device can also adjust the image brightness by changing the aperture.

[0075] In some implementations, the AE algorithm may be composed of two modules, namely, an AE statistics module and an AE strategy module. Among them, the AE statistics module may be used to obtain Stats statistical parameters. Specifically, the AE statistics module may be used to obtain Stats statistical parameters by obtaining brightness histograms and brightness area statistics, etc. Specifically, the AE statistics module may generate a 256-segment histogram and R / Gr / Gb / B four-component average statistical information for the entire frame image, and may also generate R / Gr / Gb / B four-component average statistical information for each block of the M*N blocks of the entire frame image. The AE strategy module may be used to perform real-time calculations based on the Stats statistical parameters generated by the AE statistics module to generate new exposure parameters.

[0076] In a possible implementation, the AE strategy module may generate a value of the exposure parameter within the camera adjustment range according to the Stats statistical parameters generated by the AE statistical module. In other words, the above camera configuration parameters may affect the specific value of the exposure parameter obtained.

[0077] The electronic device is set with a target brightness. When the target brightness differs greatly from the brightness of the current image, the electronic device cannot adjust the exposure parameters to the target brightness in a single pass through the AE algorithm, and needs to make multiple adjustments so that the brightness of the image obtained by the electronic device is closer and closer to the target brightness.

[0078] The following is a schematic diagram of the structural morphology of a group of electronic devices provided in an embodiment of the present application.

[0079] Figure 2A-2C The folding screen of an electronic device including screen A, screen B and screen C is only taken as an example, which should not constitute a limitation to the present application.

[0080] Among them, screen A and screen B belong to the inner folding screen, screen C belongs to the outer folding screen, and screen A and screen B can be connected by a folding component (such as a hinge). Since information is mainly displayed through screen A during the unfolding process of the folding screen, screen A can be used as the first display screen, and screens B and C can be used as secondary screens. In addition, since screen B in the inner folding screen and screen C in the outer folding screen are parallel to each other, the folding screen angle θ in this application can represent both the angle between screen A and screen B, and the angle between screen A and screen C.

[0081] Figure 2A The structural form of an electronic device when the folding screen is in a folded state is shown by way of example.

[0082] like Figure 2A As shown, when the folding screen is in a fully folded state, the folding screen angle is 0 degrees, denoted as θ=0°. At this time, the orientations of screens A and B are opposite, and the orientations of screens A and C are the same. In some implementations, the electronic device is in a Figure 2A In the structural form shown, the electronic device can be in a screen-off state, for example, screen A, screen B, and screen C are all in a screen-off state. The electronic device can also display interface content through screen C. For example, screen C can display the screen-off interface, the lock screen interface, and any user interface. It is worth noting that when the electronic device is in a folded state, neither screen A nor screen B will display any content.

[0083] Figure 2B The structural form of the electronic device is exemplified in the dynamic process of the folding screen switching from a folded state to an unfolded state.

[0084] like Figure 2B As shown, during the process of switching the folding screen of the electronic device from a folded state to an unfolded state, the angle of the folding screen gradually increases, and the electronic device can switch from displaying the interface content on screen C to displaying the interface content on screen A, and then switch to displaying the interface content on both screens A and B.

[0085] Specifically, in the dynamic process of the electronic device switching from the folded state to the unfolded state, when the folding screen angle of the electronic device increases and exceeds the first threshold, the electronic device can display information through screen A, and screens B and C are in the off state. Figure 2B As shown in (a) in FIG. 1 , when θ=50°, the electronic device displays information on screen A;

[0086] When the folding screen angle of the electronic device continues to increase and exceeds the second threshold, the electronic device can display information through screen A and screen B together, and screen C is in a screen-off state. Figure 2B As shown in (b) in FIG. 1 , when θ=150°, the electronic device displays information jointly by screen A and screen B. The first threshold is less than the second threshold, the first threshold may be, for example, θ=0°, and the second threshold may be, for example, θ=90°. The first threshold and the second threshold are not specifically limited in the embodiment of the present application.

[0087] In another possible implementation, when the folding screen of the electronic device switches from the folded state to the unfolded state, the folding screen angle gradually increases, and the electronic device can directly switch from displaying the interface content on the C screen to displaying the interface content on the A screen and the B screen. Specifically, when the folding screen angle of the mat device increases and exceeds the first threshold, the electronic device can display the interface content on the A screen and the B screen. Exemplarily, the first threshold can be θ=20°.

[0088] From the above content, it can be seen that when the folding screen angle is less than 20°, the electronic device is considered to be in a folded state.

[0089] In the embodiment of the present application, when the folding screen of the electronic device switches from the unfolded state to the folded state, the relationship between the angle change of the folding screen and the screen of the display interface of the electronic device can refer to the above Figure 2B The relevant instructions are not repeated here.

[0090] Figure 2C The structural form of the electronic device when the folding screen is in the unfolded state is shown by way of example.

[0091] like Figure 2C As shown in FIG. 1 , when the folding screen is in a fully unfolded state, the folding screen angle is 180 degrees. Note that θ=180°. At this time, the orientations of screens A and B are the same, and the orientations of screens A and C are opposite. In some implementations, the electronic device is in a Figure 2C In the structural form shown, the electronic device can be in a screen-off state, for example, screen A, screen B, and screen C are all in a screen-off state. The electronic device can also display interface content through screen A and screen B together. For example, screen A and screen B can jointly display the screen-off interface, the lock screen interface, and any user interface.

[0092] In some implementations, screen B of the electronic device faces screen C. When the electronic device is in the folded state, screen A faces screen B.

[0093] In some implementations, screen A may be referred to as a first display screen, screen B may be referred to as a second display screen, and screen C may be referred to as a third display screen.

[0094] Optionally, when the angle of the folding screen changes, the electronic device is not limited to changing the display screen used, but can also change the display content on the display screen. Figure 2A-2C The example of changing the display screen used by an electronic device is merely used for illustration and should not be construed as limiting the present application.

[0095] In the embodiments of the present application, the above-mentioned folding screen angle and the corresponding display screen used are only examples and should not constitute a limitation to the present application.

[0096] In combination with the above structural forms of electronic devices, the camera layout of electronic devices in different structural forms is introduced below.

[0097] Figure 3 The layout of various cameras of the electronic device in the folded state and in the unfolded state is exemplified.

[0098] like Figure 3 As shown, Figure 3 The electronic device is shown in a front view and a rear view in a folded state, and in a front view and a rear view in an unfolded state.

[0099] When the electronic device is in the folded state, it can be seen from the front view of the electronic device that the electronic device includes an external screen camera 201, and the external screen camera 201 is located on the C screen. It can be seen from the rear view of the electronic device that the electronic device also includes a rear camera 202, and the shooting direction of the rear camera 202 is opposite to the direction of the C screen.

[0100] When the electronic device is in the unfolded state, it can be seen from the front view of the electronic device that the electronic device includes an inner screen camera 203, and the inner screen camera 203 is located on screen A. Optionally, the inner screen camera 203 can also be located on screen B. It can be seen from the rear view of the electronic device that the electronic device also includes a rear camera 205 and an outer screen camera 204, and the shooting directions of the rear camera 205 and the outer screen camera 204 are consistent.

[0101] Among them, the above-mentioned rear cameras may refer to a group of cameras, which may include but are not limited to: a wide-angle camera, an ultra-wide-angle camera, a telephoto camera, a macro camera, a depth of field camera, a black and white camera, etc.

[0102] It is worth noting that the external screen camera 201 is the same camera as the external screen camera 204. The rear camera 202 is the same camera as the rear camera 205.

[0103] When using the camera function, the electronic device can switch the photo taking mode, such as the selfie mode and the other-photo mode. When the electronic device uses the selfie mode, if the electronic device is in the folded state, the electronic device can take pictures through the external screen camera 201; if the electronic device is in the unfolded state, the electronic device can take pictures through the internal screen camera 203. When the electronic device uses the other-photo mode, if the electronic device is in the folded state, the electronic device can take pictures through the rear camera 202; if the electronic device is in the unfolded state, the electronic device can take pictures through the rear camera 205, and not through the external screen camera 204.

[0104] After the electronic device launches the camera application, in selfie mode, when switching from the unfolded state to the folded state and then switching back to the unfolded state, the displayed preview image will appear overexposed.

[0105] Combine the following Figure 4 To illustrate the application scenario of switching cameras of the above electronic device.

[0106] like Figure 4 As shown in (a), the electronic device is in the unfolded state and collects images through the internal screen camera 203. The A screen and the B screen jointly display the user interface 300 including the preview image. And the preview image is normally exposed.

[0107] like Figure 4 As shown in (b), the electronic device switches to the folded state and collects images through the external screen camera 201. The C screen displays a user interface 301 including a preview image.

[0108] like Figure 4 As shown in (c), the electronic device switches back to the unfolded state and collects images through the internal screen camera 203. Screen A and screen B jointly display a user interface 302 containing a preview image. The preview image displayed in the user interface 302 may be overexposed.

[0109] The following explains why the preview image displayed in the user interface 302 may be overexposed.

[0110] During the process of switching the electronic device from the unfolded state to the folded state, first, when the folding screen angle of the electronic device reaches the first threshold (for example, θ=20°), the electronic device sends a signal to the camera service, triggering the electronic device to turn off the internal screen camera 203; thereafter, the folding screen angle of the electronic device is further reduced to 0, and the electronic device is completely folded. In this process, if the time duration for the folding screen angle to decrease from the first threshold to 0 is less than the effective time duration of the above signal, the internal screen camera 203 will not be turned off and is still collecting images when the folding screen angle is extremely small or even 0°. In this case, since the internal screen camera 203 receives extremely little light, the last frame of image captured by the internal screen camera 203 before being turned off is almost a completely black image.

[0111] Afterwards, when the electronic device switches from the folded state to the unfolded state, the electronic device starts the internal screen camera 203 again. At this time, the internal screen camera 203 is initialized first, and then the Stats statistical parameters of the last frame of the image captured before the internal screen camera 203 was turned off are obtained, and the exposure parameters used when the internal screen camera 203 captures the image this time are determined based on the statistical parameters. Since the last frame of the image captured before the internal screen camera 203 was turned off last time was almost a completely black image, the exposure parameters determined by the electronic device this time are too high, resulting in overexposure of the preview image.

[0112] Exemplarily, the electronic device mistakenly believes that if the brightness of the image generated based on the current ambient light is darker, this will cause the brightness of the image generated based on the current ambient light to differ greatly from the target brightness through the Stats statistical parameters of the last frame of the image collected before the last internal screen camera 203 was turned off. The electronic device will consider adjusting the exposure parameters to increase the brightness of the image. For example, the brightness of the last frame of the image collected before the last internal screen camera 203 was turned off was 1 nit, and the target brightness set by the electronic device was 50 nit. The electronic device mistakenly believes that the exposure parameters need to be adjusted to increase the brightness by 49 nit through the Stats statistical parameters obtained based on the last frame of the image collected before the last internal screen camera 203 was turned off. However, the electronic device mistakenly believes that the brightness needs to be increased by 49 nit, and the current ambient light will increase the brightness of the image by 40 nit, which will cause the electronic device to display an image with a brightness of 89 nit. In this way, the electronic device may have the problem of overexposure of the preview image displayed.

[0113] In order to solve the above problems, an embodiment of the present application provides an image processing method, in which, after the electronic device starts the camera application, in the selfie mode, during the process of switching from the unfolded state to the folded state and then switching to the unfolded state, the electronic device can use valid Stats statistical parameters to calculate exposure parameters after the camera for collecting images switches from the external screen camera to the internal screen camera, and then use the internal screen camera 203 to collect images based on the exposure parameters. Valid Stats statistical parameters refer to: Stats statistical parameters obtained based on non-completely black images collected by the internal screen camera 203.

[0114] In this way, it is possible to avoid overexposure of the preview image displayed after the folding screen of an electronic device with a camera function activated is switched from a folded state to an unfolded state, thereby allowing the user to observe a normally exposed image during the process of switching the electronic device from a folded state to an unfolded state, thereby improving the user experience.

[0115] The following describes the image processing method provided by the embodiments of the present application.

[0116] Figure 5 The process of the image processing method provided by the embodiment of the present application is exemplified. The method comprises:

[0117] S501. When the electronic device is in the unfolded state, the camera function is started and the selfie mode is entered, the internal screen camera 203 is enabled to capture images, and the preview images from the internal screen camera 203 are displayed on the A screen and the B screen.

[0118] The unfolded state may refer to a state in which the folding screen of the electronic device has an angle of 180° or approximately 180°. Figure 2A Related instructions.

[0119] In some implementations, the electronic device may start a camera function in response to a user operation. The user operation may include, but is not limited to, a click operation, a voice command, and a tidying up command. Exemplarily, the electronic device may start a camera function in response to a user clicking an icon of a camera application located on a desktop interface. The camera function may be a function provided by a system application in the electronic device, or a function provided by a third-party application.

[0120] In some implementations, after the electronic device starts the camera function, when the electronic device is in the unfolded state, the rear camera 205 is generally used by default to capture images, and the electronic device can switch the camera in response to a user operation. Exemplarily, the electronic device can switch the camera for capturing images from the rear camera 205 to the internal screen camera 203 in response to a user's click operation on the switching control, and a preview image from the internal screen camera 203 can be displayed on the A screen and the B screen.

[0121] S502. During the startup of the internal screen camera 203, obtain the Stats statistical parameters of each frame preview image from the internal screen camera 203, and determine whether it is valid, and store the valid Stats statistical parameters in the queue.

[0122] In some implementations, after the camera function is activated, the electronic device in the unfolded state may display a preview image. The preview image may be acquired by the internal screen camera 203. Figure 4 .

[0123] In some implementations, the electronic device may use the above AE algorithm to obtain the Stats statistical parameters corresponding to each frame of the preview image. The Stats statistical parameters obtained based on one frame may be referred to as one-frame Stats statistical parameters. For a description of the Stats statistical parameters, please refer to the above Figure 1 The detailed introduction of is not given here.

[0124] Specifically, the electronic device can determine whether the Stats statistical parameters obtained by the electronic device are valid through the frame grayscale component (frameluma) parameter output by the AE algorithm. Exemplarily, when the electronic device detects that the value of the frameluma parameter is greater than or equal to 1.0, it means that the Stats statistical parameters are obtained based on a non-total black image, so it can be determined that the Stats statistical parameters currently obtained by the electronic device are valid.

[0125] In the embodiment of the present application, there may be other ways to determine whether the Stats statistical parameters are valid, such as determining that the current preview image has a certain brightness by analyzing the brightness histogram and the brightness area statistics, which is not limited to this.

[0126] In some implementations, not all valid Stats statistical parameters need to be stored in the queue. Specifically, the electronic device may store the most recently obtained multiple Stats statistical parameters in the queue.

[0127] Preferably, the electronic device sets the maximum storage capacity of the queue based on the consideration of the user's folding speed. In this way, when the electronic device reads the Stats statistical data from the head of the queue, it may be based on the image captured by the internal screen camera when the electronic device is fully unfolded, so that the Stats statistical data read from the head of the queue is more effective.

[0128] For example, the electronic device can store the latest acquired 6 frames of Stats statistical parameters in the queue. This is based on the folding speed of the electronic device. Based on empirical values, the electronic device is composed of Figure 2CThe expanded state shown is switched to Figure 2A The time for the folded state shown is as fast as 180ms and as slow as 360ms. In this way, even if the electronic device determines that the Stats statistical parameters obtained based on the images collected during the process of switching from the unfolded state to the folded state are all valid, the Stats statistical parameters stored in the queue also include the Stats statistical parameters obtained based on the images collected when the electronic device is in the unfolded state. In this way, the availability of the Stats statistical parameters is improved.

[0129] Exemplarily, the electronic device is provided with a dummy queue for storing Stats statistical parameters, and the maximum storage capacity of the dummy queue is 6 frames of Stats statistical parameters. The electronic device collects a total of 200 frames of images through the internal screen camera 203. Among them, the first 197 frames of images are collected when the electronic device is in the unfolded state, and the 198th frame image to the 200th frame image are collected during the process of the electronic device switching from the unfolded state to the folded state.

[0130] If the 200 frames of Stats statistical parameters obtained by the electronic device based on the 200 frames of images are all valid, the electronic device can store the 195th frame Stats statistical parameters to the 200th frame Stats statistical parameters in the dummy queue. The above 6 frames of Stats statistical parameters are arranged in the dummy queue in the order of acquisition, with the 195th frame Stats statistical parameters at the head of the queue and the 200th frame Stats statistical parameters at the tail of the queue.

[0131] In some implementations, the queue storing valid Stats statistical parameters may be referred to as a first queue.

[0132] S503: When the electronic device switches from the unfolded state to the folded state, it detects that the angle of the folding screen reaches a first threshold, generates a first signal, and based on the first signal, turns off the inner screen camera 203 and starts the outer screen camera 201.

[0133] Before executing S503, the electronic device may respond to a user operation to switch the electronic device from the unfolded state to the folded state.

[0134] In some implementations, when the electronic device switches from the unfolded state to the folded state, the electronic device can determine whether the camera needs to be switched by detecting the folding screen angle. Specifically, the electronic device can generate a first signal when detecting that the folding screen angle reaches a first threshold.

[0135] Before executing S503, the electronic device can respond to the user operation and switch the electronic device from the unfolded state to the folded state. It is worth noting that the camera function of the electronic device is always turned on during the process of switching the folding screen form. In addition, the shooting mode set by the electronic device is always the selfie mode.

[0136] After detecting that the electronic device is in the folded state, the electronic device can collect images through the external screen camera 201. At this time, the electronic device can display a preview image from the external screen camera 201 on the C screen.

[0137] In a possible implementation, when the electronic device is in the folded state, the electronic device can directly respond to user operations and then switch the electronic device from the folded state to the unfolded state.

[0138] In another possible implementation, when the electronic device is in the folded state, the electronic device can turn off the camera function and start any other function. Then the camera function is started again and the shooting mode is set to the selfie mode. Finally, the electronic device is switched from the folded state to the unfolded state.

[0139] In an embodiment of the present application, between executing S503 and S504, that is, when the electronic device is in the folded state, the electronic device starts any function (such as capturing images through the rear camera 202, etc.). It is only necessary to ensure that the electronic device turns on the camera function and the shooting mode is the selfie mode during the process of switching from the folded state to the unfolded state.

[0140] S504: When the electronic device switches from the folded state to the unfolded state, it detects that the angle of the folding screen reaches a first threshold, generates a second signal, and based on the second signal, turns off the external screen camera 201 and starts the internal screen camera 203.

[0141] In some implementations, the electronic device generates a request before generating each frame of preview image, and the request is used to request the electronic device to generate Stats statistical parameters. And the request will carry the camera identifier. The camera identifier is used to indicate which specific camera the Stats statistical parameters are used to generate exposure parameters for the image captured. The electronic device can determine whether the camera that captures the image needs to obtain the Stats statistical parameters stored in the queue based on the camera identifier carried.

[0142] In some implementations, when the electronic device switches from the folded state to the unfolded state, when the folding screen angle of the electronic device reaches a first threshold (e.g., θ=20°), the hinge of the electronic device switches from the closed state to the open state. The electronic device can send a hinge opening signal to the camera service, thereby triggering the electronic device to turn off the external screen camera 201 and start the internal screen camera 203.

[0143] In some implementations, after the external screen camera 201 is turned off and the internal screen camera 203 is started, the Stats statistical parameter at the head of the queue obtained by the electronic device can be referred to as the first Stats statistical parameter. The preview image based on the first Stats statistical parameter can be referred to as the second preview image. In the process of switching from the folded state to the unfolded state, after the external screen camera 201 is turned off and the internal screen camera 203 is started, the first frame of the preview image captured by the electronic device through the internal screen camera 203 can be referred to as the first preview image.

[0144] S505: The electronic device reads the valid Stats statistical parameters of the head of the queue from the queue.

[0145] In some implementations, when the electronic device switches to collecting images by the inner screen camera 203, the angle of the folding screen may be small. Due to the special position of the inner screen camera 203 on the electronic device, the inner screen camera 203 does not receive enough light. This may cause the Stats statistical parameters obtained by the electronic device based on the image currently collected by the inner screen camera 203 to be invalid.

[0146] In another possible implementation, when the electronic device switches to capturing images by the internal screen camera 203, its AE algorithm needs to be initialized, which may also cause the Stats statistical parameters obtained by the electronic device based on the images currently captured by the internal screen camera 203 to be invalid.

[0147] Specifically, when the electronic device switches to collecting images by the internal screen camera 203, it fails to obtain real-time and valid Stats statistical parameters, and the electronic device can read a frame of Stats statistical parameters at the head of the team from the dummy queue. It can be seen from the relevant description in S503 above that the frame of Stats statistical parameters at the head of the team is valid. Among them, the Stats statistical parameters with the longest storage time in the dummy queue are located at the head of the team, and the Stats statistical parameters with the shortest storage time in the dummy queue are located at the end of the team.

[0148] In a possible implementation, after a frame of Stats statistical parameters at the head of the dummy queue is read, the electronic device will update the dummy queue. Then the acquired valid Stats statistical parameters are stored in the dummy queue again. Exemplarily, the electronic device can directly clear the dummy queue after the Stats statistical parameters at the head of the team are read, and then store the new valid Stats statistical parameters in the dummy queue in the order of the time when the Stats statistical parameters were generated. Exemplarily, after the Stats statistical parameters at the head of the team are read, the electronic device can, according to the rules of queue entry and exit, enter the new valid Stats statistical parameters into the dummy queue after the Stats statistical parameters at the head of the team are read, and remove the Stats statistical parameters with the longest storage time in the dummy queue (i.e., the Stats statistical parameters at the head of the team) from the dummy queue.

[0149] S506: The electronic device determines exposure parameters based on valid Stats statistical parameters.

[0150] Specifically, the electronic device can input the valid Stats statistical parameter into the AE algorithm to obtain the exposure parameter. Since the Stats statistical parameter is valid, combined with the above description of the AE algorithm, the exposure parameter without deviation can be obtained.

[0151] S507: The electronic device captures an image through the internal screen camera 203 based on the exposure parameters, and displays a preview image.

[0152] Specifically, the electronic device can input the exposure parameters obtained in S506 into the AE algorithm to obtain a preview image of normal exposure. It is worth noting that the preview image of normal exposure may refer to the first frame image obtained after the electronic device switches to the image captured by the internal screen camera 203. Combined with the above description of the AE algorithm, since the first frame image is normally exposed, the Stats statistical parameters obtained based on the first frame image are also valid, and the new exposure parameters obtained based on the Stats statistical parameters will not deviate, and the second frame image generated based on the new exposure parameters is also normally exposed.

[0153] By implementing the above method, the electronic device can obtain valid Stats statistical parameters and obtain normal exposure parameters before switching to the first frame image captured by the internal screen camera 203. Therefore, the first frame image obtained based on the normal exposure parameters will not be overexposed.

[0154] In this way, it is possible to avoid overexposure after the folding screen of an electronic device with a camera function activated is switched from a folded state to an unfolded state, so that the images observed by the user during the process of switching the electronic device from a folded state to an unfolded state are all normally exposed images, thereby improving the user experience.

[0155] It is worth noting that, in the embodiment of the present application, the electronic device may switch between the inner screen camera and the outer screen camera based on the folding screen angle reaching the first threshold value as a condition for the camera that collects images.

[0156] In a possible implementation, the first threshold may also be a condition for the electronic device to distinguish between the folded state and the unfolded state. For example, when the folding screen angle of the electronic device is less than the first threshold, the electronic device can be considered to be in the folded state; when the folding screen angle of the electronic device is greater than the first threshold, the electronic device can be considered to be in the unfolded state. Exemplarily, the first threshold may be an angle such as 20° or 50°.

[0157] In another possible implementation, the first threshold and the second threshold may be the conditions for the electronic device to distinguish between the folded state and the unfolded state. For example, when the folding screen angle of the electronic device is less than the first threshold, the electronic device can be regarded as a folded state; when the folding screen angle of the electronic device is greater than the second threshold, the electronic device can be regarded as an unfolded state; when the folding screen angle of the electronic device is greater than the first threshold and less than the second threshold, the electronic device can be regarded as a transitional state. Among them, the first threshold is less than the second threshold. Exemplarily, the first threshold can be an angle such as 20° or 50°, and the second threshold can be an angle such as 140° or 160°.

[0158] In another possible implementation, the third threshold and the second threshold may also be conditions for the electronic device to distinguish between the folded state and the unfolded state. The third threshold is less than the first threshold. That is, when the electronic device switches the camera, the electronic device may not be limited to being in the unfolded state or the folded state, but may also be in the process of transitioning from the unfolded state to the folded state (i.e., in a transition state).

[0159] In a possible implementation, the electronic device can obtain the Stats statistical parameters of each frame preview image from the inner screen camera 203 during the startup of the inner screen camera 203, and directly store the Stats statistical parameters in the queue. The electronic device can set the maximum value of the data stored in the queue, for example, 10 frames. When the number of stored in the queue reaches the maximum value, the electronic device can delete the Stats statistical parameters at the head of the queue, and then store the new Stats statistical parameters in the queue. Then, when the electronic device detects that the folding screen angle reaches the first threshold, it detects that the camera for collecting images is switched from the external screen camera 201 to the inner screen camera 203, reads the Stats statistical parameters from the queue, and judges the validity of the read Stats statistical parameters. If the Stats statistical parameters are valid, execute S506-S507; if the Stats statistical parameters are not valid, read the Stats statistical parameters of the next frame from the queue, and then judge the Stats statistical parameters of the next frame until the acquired Stats statistical parameters are valid.

[0160] In another possible implementation, the electronic device can directly preset a set of Stats statistical parameters, which are valid. And, when the electronic device detects that the folding screen angle reaches the first threshold, it detects that the camera for collecting images is switched from the external screen camera 201 to the internal screen camera 203, reads the valid Stats statistical parameters from the queue, and then executes S506-S507. It can also avoid the overexposure of the displayed preview image when the electronic device starts the camera application and switches from the unfolded state to the folded state and then to the unfolded state in the selfie mode.

[0161] Figure 6 A schematic structural diagram of an electronic device 100 is shown.

[0162] The electronic device 100 may be equipped with Or a portable terminal device of other operating systems, the electronic device 100 can be a mobile phone, a tablet computer, a desktop computer, a laptop computer, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, and a cellular phone, a personal digital assistant (PDA), an augmented reality (AR) device, a virtual reality (VR) device, an artificial intelligence (AI) device, a wearable device, a vehicle, a vehicle-mounted device, a smart home device and / or a smart city device, but not limited thereto, the electronic device 100 can also include a laptop computer with a touch-sensitive surface or a touch panel, a desktop computer with a touch-sensitive surface or a touch panel, and other non-portable terminal devices, etc. The embodiment of the present application does not impose any special restrictions on the specific type of the electronic device.

[0163] The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0164] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or separate some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0165] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0166] In some implementations, the processor 110 may be used to detect that the angle of the folding screen decreases to a first threshold value during the process of switching from the unfolded state to the folded state, generate a first signal, and based on the first signal, start the external screen camera 201 and turn off the internal screen camera 203. The processor 110 may also be used to detect that the angle of the folding screen increases and decreases to a first threshold value during the process of switching from the folded state to the unfolded state, generate a second signal, and based on the second signal, turn off the external screen camera 201 and start the internal screen camera 203. The processor 110 may also be used to determine the validity of the Stats statistical parameters of each frame preview image from the internal screen camera 203. The processor 110 may also obtain the information of the Stats statistical parameters in response to the request sent by the internal screen camera 203, and read out the valid Stats statistical parameters. The processor 110 may also be used to determine the exposure parameters based on the valid Stats statistical parameters.

[0167] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0168] In some implementations, a first queue is stored in a memory.

[0169] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0170] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple groups of I2C buses. The processor 110 may be coupled to the touch sensor 180K, the charger, the flash, the camera 193, etc. through different I2C bus interfaces. For example: the processor 110 may be coupled to the touch sensor 180K through the I2C interface, so that the processor 110 communicates with the touch sensor 180K through the I2C bus interface, thereby realizing the touch function of the electronic device 100.

[0171] The I2S interface can be used for audio communication. In some embodiments, the processor 110 can include multiple I2S buses. The processor 110 can be coupled to the audio module 170 via the I2S bus to achieve communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 can transmit an audio signal to the wireless communication module 160 via the I2S interface to achieve the function of answering a call through a Bluetooth headset.

[0172] The PCM interface can also be used for audio communication, sampling, quantizing and encoding analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 can be coupled via a PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 via the PCM interface to realize the function of answering calls via a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

[0173] The UART interface is a universal serial data bus for asynchronous communication. The bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, the UART interface is generally used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 through the UART interface to implement the Bluetooth function. In some embodiments, the audio module 170 can transmit an audio signal to the wireless communication module 160 through the UART interface to implement the function of playing music through a Bluetooth headset.

[0174] The MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a camera serial interface (CSI), a display serial interface (DSI), etc. In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to implement the shooting function of the electronic device 100. The processor 110 and the display screen 194 communicate via the DSI interface to implement the display function of the electronic device 100.

[0175] The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 with the camera 193, the display 194, the wireless communication module 160, the audio module 170, the sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0176] The USB interface 130 is an interface that complies with the USB standard specification, and specifically can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the electronic device 100, and can also be used to transfer data between the electronic device 100 and a peripheral device. It can also be used to connect headphones to play audio through the headphones. The interface can also be used to connect other electronic devices, such as AR devices, etc.

[0177] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present invention is only a schematic illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0178] The charging management module 140 is used to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 may receive wireless charging input through a wireless charging coil of the electronic device 100. While the charging management module 140 is charging the battery 142, it may also power the electronic device through the power management module 141.

[0179] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the display screen 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc. In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0180] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0181] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0182] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0183] The modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 170A, a receiver 170B, etc.), or displays an image or video through a display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

[0184] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, demodulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0185] In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology, etc. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0186] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0187] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0188] In some implementations, the display screen 194 may include screen A, screen B, and screen C. Screen A and screen B may be used to display preview images captured by the internal screen camera 203. Screen C may be used to display preview images captured by the external screen camera 201.

[0189] The electronic device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0190] The ISP is used to process the data fed back by the camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing and converts it into an image visible to the naked eye. The ISP can also perform algorithm optimization on the noise, brightness, and color of the image. The ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, the ISP can be set in the camera 193.

[0191] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0192] In some implementations, the camera 193 may include an inner screen camera 203 , an outer screen camera 201 , and a rear camera 202 .

[0193] The digital signal processor is used to process digital signals, and can process not only digital image signals but also other digital signals. For example, when the electronic device 100 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0194] Video codecs are used to compress or decompress digital videos. The electronic device 100 may support one or more video codecs. Thus, the electronic device 100 may play or record videos in a variety of coding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0195] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. Through NPU, applications such as intelligent cognition of electronic device 100 can be realized, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0196] The internal memory 121 may include one or more random access memories (RAM) and one or more non-volatile memories (NVM).

[0197] Random access memory may include static random-access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM, for example, the fifth generation DDR SDRAM is generally referred to as DDR5 SDRAM), etc.; non-volatile memory may include disk storage devices and flash memory.

[0198] Flash memory can be divided into NOR FLASH, NAND FLASH, 3D NAND FLASH, etc. according to the operating principle; single-level cell (SLC), multi-level cell (MLC), triple-level cell (TLC), quad-level cell (QLC), etc. according to the storage unit potential level; universal flash storage (UFS), embedded multi media card (eMMC), etc. according to the storage specification.

[0199] The random access memory can be directly read and written by the processor 110, and can be used to store executable programs (such as machine instructions) of the operating system or other running programs, and can also be used to store user and application data.

[0200] The non-volatile memory may also store executable programs and user and application data, etc., and may be loaded into the random access memory in advance for direct reading and writing by the processor 110 .

[0201] The external memory interface 120 can be used to connect to an external non-volatile memory to expand the storage capacity of the electronic device 100. The external non-volatile memory communicates with the processor 110 through the external memory interface 120 to implement a data storage function. For example, files such as music and videos are stored in the external non-volatile memory.

[0202] The electronic device 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0203] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be arranged in the processor 110, or some functional modules of the audio module 170 can be arranged in the processor 110.

[0204] The speaker 170A, also called a "speaker", is used to convert an audio electrical signal into a sound signal. The electronic device 100 can listen to music or listen to a hands-free call through the speaker 170A.

[0205] The receiver 170B, also called a "earpiece", is used to convert audio electrical signals into sound signals. When the electronic device 100 receives a call or voice message, the voice can be received by placing the receiver 170B close to the human ear.

[0206] Microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak by putting their mouth close to microphone 170C to input the sound signal into microphone 170C. The electronic device 100 can be provided with at least one microphone 170C. In other embodiments, the electronic device 100 can be provided with two microphones 170C, which can not only collect sound signals but also realize noise reduction function. In other embodiments, the electronic device 100 can also be provided with three, four or more microphones 170C to collect sound signals, reduce noise, identify the sound source, realize directional recording function, etc.

[0207] The earphone interface 170D is used to connect a wired earphone and can be a USB interface 130 or a 3.5 mm open mobile terminal platform (OMTP) standard interface or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0208] The pressure sensor 180A is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A can be set on the display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. The capacitive pressure sensor can be a parallel plate including at least two conductive materials. When a force acts on the pressure sensor 180A, the capacitance between the electrodes changes. The electronic device 100 determines the intensity of the pressure according to the change in capacitance. When a touch operation acts on the display screen 194, the electronic device 100 detects the touch operation intensity according to the pressure sensor 180A. The electronic device 100 can also calculate the touch position according to the detection signal of the pressure sensor 180A. In some embodiments, touch operations acting on the same touch position but with different touch operation intensities can correspond to different operation instructions. For example: when a touch operation with a touch operation intensity less than the first pressure threshold acts on the short message application icon, an instruction to view the short message is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure threshold acts on the short message application icon, an instruction to create a new short message is executed.

[0209] The gyro sensor 180B can be used to determine the motion posture of the electronic device 100. In some embodiments, the angular velocity of the electronic device 100 around three axes (i.e., x, y, and z axes) can be determined by the gyro sensor 180B. The gyro sensor 180B can be used for anti-shake shooting. For example, when the shutter is pressed, the gyro sensor 180B detects the angle of the electronic device 100 shaking, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to offset the shaking of the electronic device 100 through reverse movement to achieve anti-shake. The gyro sensor 180B can also be used for navigation and somatosensory game scenes.

[0210] The air pressure sensor 180C is used to measure air pressure. In some embodiments, the electronic device 100 calculates the altitude through the air pressure value measured by the air pressure sensor 180C to assist positioning and navigation.

[0211] The magnetic sensor 180D includes a Hall sensor. The electronic device 100 can use the magnetic sensor 180D to detect the opening and closing of the flip leather case. In some embodiments, when the electronic device 100 is a flip phone, the electronic device 100 can detect the opening and closing of the flip cover according to the magnetic sensor 180D. Then, according to the detected opening and closing state of the leather case or the opening and closing state of the flip cover, the flip cover can be automatically unlocked.

[0212] The acceleration sensor 180E can detect the magnitude of the acceleration of the electronic device 100 in all directions (generally three axes). When the electronic device 100 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of the electronic device and is applied to applications such as horizontal and vertical screen switching and pedometers.

[0213] The distance sensor 180F is used to measure the distance. The electronic device 100 can measure the distance by infrared or laser. In some embodiments, when shooting a scene, the electronic device 100 can use the distance sensor 180F to measure the distance to achieve fast focusing.

[0214] The proximity light sensor 180G may include, for example, a light emitting diode (LED) and a light detector, such as a photodiode. The light emitting diode may be an infrared light emitting diode. The electronic device 100 emits infrared light outward through the light emitting diode. The electronic device 100 uses a photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the electronic device 100. When insufficient reflected light is detected, the electronic device 100 can determine that there is no object near the electronic device 100. The electronic device 100 can use the proximity light sensor 180G to detect that the user holds the electronic device 100 close to the ear to talk, so as to automatically turn off the screen to save power. The proximity light sensor 180G can also be used in leather case mode and pocket mode to automatically unlock and lock the screen.

[0215] The ambient light sensor 180L is used to sense the ambient light brightness. The electronic device 100 can adaptively adjust the brightness of the display screen 194 according to the perceived ambient light brightness. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the electronic device 100 is in a pocket to prevent accidental touches.

[0216] The fingerprint sensor 180H is used to collect fingerprints. The electronic device 100 can use the collected fingerprint characteristics to implement fingerprint unlocking, access application locks, fingerprint photography, fingerprint call answering, etc.

[0217] The temperature sensor 180J is used to detect temperature. In some embodiments, the electronic device 100 uses the temperature detected by the temperature sensor 180J to execute a temperature processing strategy. For example, when the temperature reported by the temperature sensor 180J exceeds a threshold, the electronic device 100 reduces the performance of a processor located near the temperature sensor 180J to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is lower than another threshold, the electronic device 100 heats the battery 142 to avoid abnormal shutdown of the electronic device 100 due to low temperature. In other embodiments, when the temperature is lower than another threshold, the electronic device 100 performs a boost on the output voltage of the battery 142 to avoid abnormal shutdown caused by low temperature.

[0218] The touch sensor 180K is also called a "touch control device". The touch sensor 180K can be set on the display screen 194. The touch sensor 180K and the display screen 194 form a touch screen, also called a "touch control screen". The touch sensor 180K is used to detect touch operations acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In other embodiments, the touch sensor 180K can also be set on the surface of the electronic device 100, which is different from the position of the display screen 194.

[0219] The bone conduction sensor 180M can obtain a vibration signal. In some embodiments, the bone conduction sensor 180M can obtain a vibration signal of a vibrating bone block of the vocal part of the human body. The bone conduction sensor 180M can also contact the human pulse to receive a blood pressure beat signal. In some embodiments, the bone conduction sensor 180M can also be set in an earphone and combined into a bone conduction earphone. The audio module 170 can parse out a voice signal based on the vibration signal of the vibrating bone block of the vocal part obtained by the bone conduction sensor 180M to realize a voice function. The application processor can parse the heart rate information based on the blood pressure beat signal obtained by the bone conduction sensor 180M to realize a heart rate detection function.

[0220] The key 190 includes a power key, a volume key, etc. The key 190 may be a mechanical key or a touch key. The electronic device 100 may receive key input and generate key signal input related to user settings and function control of the electronic device 100.

[0221] Motor 191 can generate vibration prompts. Motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, audio playback, etc.) can correspond to different vibration feedback effects. For touch operations acting on different areas of the display screen 194, motor 191 can also correspond to different vibration feedback effects. Different application scenarios (for example: time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

[0222] Indicator 192 may be an indicator light, which may be used to indicate charging status, power changes, messages, missed calls, notifications, etc.

[0223] The software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. In the embodiment of the present invention, the Android system of the layered architecture is taken as an example to exemplify the software structure of the electronic device 100.

[0224] Figure 71 is a software structure block diagram of the electronic device 100 according to an embodiment of the present invention.

[0225] The layered architecture divides the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: the application layer, the application framework layer, the Android runtime and system library, and the kernel layer.

[0226] The application layer can include a series of application packages.

[0227] like Figure 7 As shown, the application package may include camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message and other applications.

[0228] The application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0229] like Figure 7 As shown, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, and the like.

[0230] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0231] Content providers are used to store and retrieve data and make it accessible to applications. This data can include video, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0232] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.

[0233] The phone manager is used to provide communication functions of the electronic device 100, such as management of call status (including connecting, hanging up, etc.).

[0234] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0235] The notification manager enables applications to display notification information in the status bar. It can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as notifications of applications running in the background, or a notification that appears on the screen in the form of a dialog window. For example, a text message is displayed in the status bar, a prompt sound is emitted, an electronic device vibrates, an indicator light flashes, etc.

[0236] Android Runtime includes core libraries and virtual machines. Android runtime is responsible for scheduling and management of the Android system.

[0237] The core library consists of two parts: one part is the function that needs to be called by the Java language, and the other part is the Android core library.

[0238] The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0239] The system library can include multiple functional modules, such as surface manager, media library, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc. Figure 7 Not shown in FIG.

[0240] The surface manager is used to manage the display subsystem and provide the fusion of 2D and 3D layers for multiple applications.

[0241] The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0242] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0243] A 2D graphics engine is a drawing engine for 2D drawings.

[0244] In an embodiment of the present application, the system library may further include: a camera service, a Stats statistical parameter judgment module, a Stats statistical parameter storage module, a camera switching judgment module and an AE algorithm module.

[0245] The camera service is used to manage and use the functions of the camera application. In some examples, the camera service can respond to the start of the camera application, start the image sensor, control the image sensor to collect image frames, obtain the image frames collected by the image sensor, etc. The camera service can also be used to distinguish the specific camera that needs to be started.

[0246] The AE algorithm module can include an AE statistics module and an AE strategy module. For more information about the AE algorithm, please refer to the above Figure 1 The relevant instructions are not repeated here.

[0247] refer to Fig. 8A , Fig. 8A The schematic diagram of the Stats statistical parameter judgment module is exemplified.

[0248] In some implementations, the Stats statistical parameter determination module is used to determine the validity of the acquired Stats statistical parameters.

[0249] Specifically, the electronic device can obtain the Stats statistical parameters based on the preview image obtained by the internal screen camera through the AE algorithm. Among them, the Stats statistical parameters include the frameluma parameter. The electronic device can determine whether the Stats statistical parameters obtained by the electronic device are valid by judging the frameluma parameter. Specifically, the electronic device can determine whether the frameluma parameter is greater than or equal to 1. If so, it means that the Stats statistical parameters are obtained based on a non-completely black image, and it can be determined that the Stats statistical parameters currently obtained by the electronic device are valid, and then the Stats statistical parameters are stored in the dummyqueue; if not, it means that the Stats statistical parameters are obtained based on a completely black image, and the Stats statistical parameters will not be stored in the dummyqueue.

[0250] refer to Figure 8B , Figure 8B The schematic diagram of the Stats statistical parameter storage module is shown as an example.

[0251] In some implementations, the Stats statistical parameter storage module is used to manage valid Stats statistical parameters, for example, storage, queue entry, and dequeue processing.

[0252] Specifically, the electronic device may store the Stats statistical parameters judged to be valid in the dummyqueue, wherein the Stats statistical parameters stored in the dummyqueue for the longest time are located at the head of the queue, and the Stats statistical parameters stored in the dummyqueue for the shortest time are located at the tail of the queue.

[0253] The electronic device can set the maximum storage capacity of the dummyqueue. Preferably, the electronic device will set the maximum storage capacity of the queue based on the consideration of the user's folding speed. In this way, when the electronic device reads the Stats statistical data from the head of the queue, it may be based on the image captured by the internal screen camera when the electronic device is fully unfolded. In this way, the Stats statistical data read from the head of the queue is more effective. For example, Figure 8B As shown, the maximum storage capacity of dummyqueue is 6 frames of Stats statistical parameters.

[0254] When the storage capacity in the dummyqueue reaches 6 frames, after receiving the Stats statistical parameter signal sent by the Stats statistical parameter judgment module to store the Stats statistical parameters of the new frame, the Stats statistical parameters at the head of the queue can be deleted, and then the Stats statistical parameters of the new frame are stored at the end of the dummyqueue. In addition, the above process can also realize the update of the dummyqueue, so that the Stats statistical parameters with higher real-time performance can be stored in the dummyqueue.

[0255] In some implementations, after receiving a signal from an electronic device reading a Stats statistical parameter at the head of the queue, the Stats statistical parameter storage module may clear the dummyqueue and then implement storage of valid Stats statistical parameters.

[0256] refer to Figure 8C , Figure 8C The schematic diagram of the camera switching judgment module is exemplarily shown.

[0257] In some implementations, the camera switching judgment module is used to determine when the acquisition camera switches from the external screen camera to the internal screen camera, and when it is determined to be the above-mentioned camera switching situation, a read signal is sent to the Stats statistical parameter storage module, requesting to read the valid Stats statistical parameters at the head of the dummyqueue.

[0258] It is worth noting that after the electronic device starts the camera function, it will generate a request before generating each frame of preview image, and the request is used to request the electronic device to determine the Stats statistical parameters. And the request will carry the identification of the camera. The identification of the camera is used to indicate which specific camera is used to generate the exposure parameters for the image captured by the Stats statistical parameters. For example, if the electronic device currently captures images through the internal screen camera 203, the request generated by the electronic device for determining the Stats statistical parameters will carry the identification of the internal screen camera 203.

[0259] The camera switching judgment module is provided with store_id. The store_id is composed of prev_id located in the upper 4 bits and curr_id located in the lower 4 bits. Among them, prev_id is used to indicate the camera identifier carried by the previous request, and curr_id is used to indicate the camera identifier carried by the current request. Exemplarily, the identifier of the external screen camera can be set to 7, and the identifier of the internal screen camera can be set to 4. In the embodiment of the present application, the specific identifier of the camera is not limited.

[0260] The electronic device obtains the camera identification cameraid carried in the current request and then updates store_id.

[0261] Next, the electronic device can determine the camera switching status according to the store_id. Specifically, the prev_id and curr_id in the store_id are determined.

[0262] In some implementations, if prev_id is consistent with curr_id, it means that the electronic device has not currently switched cameras, and there is no need to send a read signal to the Stats statistical parameter storage module.

[0263] If prev_id and curr_id are inconsistent, you also need to determine the specific values ​​of prev_id and curr_id.

[0264] Exemplarily, if prev_id represents the identifier of the external screen camera 201, and curr_id represents the identifier of the internal screen camera 203, it means that the electronic device currently implements the camera switching, and it is switched from the external screen camera 201 to the internal screen camera 203. At this time, a read signal needs to be sent to the Stats statistical parameter storage module;

[0265] If prev_id represents the ID of the internal screen camera 203, and curr_id represents the ID of the external screen camera 201, it means that the electronic device has currently realized the camera switching, and it is switched from the internal screen camera 203 to the external screen camera 201. At this time, there is no need to send a read signal to the Stats statistical parameter storage module.

[0266] It is worth noting that in order to ensure the real-time judgment, if prev_id is consistent with curr_id, there is no need to change prev_id; if prev_id is inconsistent with curr_id, the value of prev_id needs to be modified to the current value of curr_id.

[0267] The kernel layer is the layer between hardware and software. The kernel layer contains at least display driver, camera driver, audio driver, and sensor driver.

[0268] The following is an illustrative description of the workflow of the software and hardware of the electronic device 100 in conjunction with capturing a photo scene.

[0269] When the touch sensor 180K receives a touch operation, the corresponding hardware interrupt is sent to the kernel layer. The kernel layer processes the touch operation into a raw input event (including touch coordinates, timestamp of the touch operation, and other information). The raw input event is stored in the kernel layer. The application framework layer obtains the raw input event from the kernel layer and identifies the control corresponding to the input event. For example, if the touch operation is a touch single-click operation and the control corresponding to the single-click operation is the control of the camera application icon, the camera application calls the interface of the application framework layer to start the camera application, and then starts the camera driver by calling the kernel layer to capture static images or videos through the camera 193.

[0270] The various implementation modes of the present application can be combined arbitrarily to achieve different technical effects.

[0271] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integration. The available medium can be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state disk (SSD)), etc.

[0272] Those skilled in the art can understand that to implement all or part of the processes in the above-mentioned embodiments, the processes can be completed by computer programs to instruct related hardware, and the programs can be stored in computer-readable storage media. When the programs are executed, they can include the processes of the above-mentioned method embodiments. The aforementioned storage media include: ROM or random access memory RAM, magnetic disk or optical disk and other media that can store program codes.

[0273] In short, the above are only embodiments of the technical solution of the present invention, and are not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made according to the disclosure of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for image processing, characterized in that: The method is applied to an electronic device, wherein the electronic device comprises: a first display screen, a second display screen, and a third display screen, wherein the first display screen and the second display screen are connected via a folding component, the display surfaces of the second display screen and the third display screen are opposite to each other, an inner screen camera is arranged on the first display screen, and an outer screen camera is arranged on the third display screen, and when the electronic device is in a folded state, the display surfaces of the first display screen and the second display screen are opposite to each other; The method comprises: The electronic device is in an unfolded state, and the internal screen camera is activated; Storing a first Stats statistical parameter, the first Stats statistical parameter comprising: a parameter for indicating the brightness of a non-completely black image captured by the inner-screen camera; When the electronic device switches from the unfolded state to the folded state, the inner screen camera is turned off and the outer screen camera is turned on; When the electronic device switches from the folded state to the unfolded state, the external screen camera is turned off and the internal screen camera is turned on; The electronic device determines a first exposure parameter according to the first Stats statistical parameter; The electronic device collects a first preview image through the internal screen camera according to the first exposure parameter; The electronic device displays a first preview image captured by the internal screen camera on the first display screen and the second display screen; The first preview image is the first frame image displayed after the electronic device turns off the external screen camera and starts the internal screen camera, and the first preview image is exposed normally.

2. The method according to claim 1, characterized in that The first Stats statistical parameter also includes the configuration parameters of the internal screen camera, and the value of the first exposure parameter is within the adjustment range of the internal screen camera.

3. The method according to claim 1, characterized in that The first exposure parameter includes one or more of the following: exposure time, ISO, and aperture.

4. The method according to claim 1, characterized in that: Stores the first Stats statistical parameters, including: The electronic device calculates the Stats statistical parameters of each frame of image captured by the internal screen camera; The electronic device stores the Stats statistical parameters of multiple frames of images captured by the internal screen camera in a first queue in chronological order, and the multiple frames of images include: non-completely black images captured by the internal screen camera, and the first Stats statistical parameter is the parameter with the earliest storage time in the first queue.

5. The method according to claim 4, characterized in that After the electronic device captures the first preview image through the internal screen camera, the method further includes: The electronic device clears the Stats statistical parameters stored in the first queue, and stores the Stats statistical parameters of the first preview image in the first queue.

6. The method according to claim 4, characterized in that The frame grayscale component frameluma parameter in each Stats statistical parameter in the first queue is greater than or equal to 1.

7. The method according to claim 1, characterized in that When the electronic device switches from the unfolded state to the folded state, before turning off the inner screen camera and turning on the outer screen camera, the method further includes: When the electronic device switches from the unfolded state to the folded state, it is detected that the angle between the first display screen and the second display screen decreases to a first threshold.

8. The method according to claim 7, characterized in that When the electronic device switches from the folded state to the unfolded state, before turning off the external screen camera and starting the internal screen camera, the method further includes: When the electronic device switches from the folded state to the unfolded state, it is detected that the angle between the first display screen and the second display screen increases to the first threshold value.

9. The method according to claim 1, characterized in that: The electronic device is in the unfolded state, and after the inner-screen camera is started, the method further includes: The electronic device displays the preview image captured by the internal screen camera on the first display screen and the second display screen.

10. The method according to any one of claims 1 to 9, characterized in that When the electronic device switches from the unfolded state to the folded state, after the inner screen camera is turned off and the outer screen camera is turned on, the method further includes: The electronic device displays the preview image captured by the external screen camera on the third display screen.

11. An electronic device, characterized in that: include: One or more processors, one or more memories; wherein the one or more memories are coupled to the one or more processors, the one or more memories are used to store computer program codes, the computer program codes include computer instructions, and when the one or more processors execute the computer instructions, the electronic device executes the method as described in any one of claims 1-10.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions. When the program instructions are executed on an electronic device, the electronic device executes the method according to any one of claims 1 to 10.

Images