Shooting method and electronic equipment

Abstract

A shooting method and electronic device, relating to the field of terminal technology, can reduce memory consumption by reducing the resident memory during the shooting process, so that shooting can be performed efficiently. The method includes: starting a camera application. In response to a selection operation of a first shooting mode, a preview interface for shooting is displayed. In response to a shooting operation in the preview interface, a first running memory is allocated, and a first image processing method is used to process the captured image in the first running memory, and the processing result is written into a second running memory, and the first running memory is released, and the size of the first running memory is larger than the size of the second running memory. A first shooting result is obtained based on the processing result in the second running memory.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of terminal technology, and in particular to a shooting method and electronic device. Background Art

[0002] Electronic devices such as mobile phones and tablets usually have camera functions. Electronic devices can use the camera function to take pictures and videos. In addition, the camera function usually includes multiple shooting modes, such as photo, video, portrait, night scene, high-pixel and other shooting modes. Among them, some shooting modes (such as target shooting modes) consume a lot of memory (referring to running memory, unless otherwise specified in the following text, memory refers to running memory). Taking the target shooting mode as a high-pixel shooting mode of 200 million pixels, in the high-pixel shooting mode of 200 million pixels, the YUV buffer (buffer, belonging to memory) required by the electronic device for JPEG is as high as 300MB.

[0003] It is understandable that the memory of an electronic device is limited. If the memory usage of the electronic device is high and the above target shooting mode is used for shooting, there may not be enough memory to successfully complete the shooting in the target shooting mode, resulting in shooting jams and failure to efficiently capture images. Summary of the invention

[0004] The present application provides a shooting method and an electronic device, which can reduce the resident memory during the shooting process and reduce the memory consumption, thereby enabling efficient shooting.

[0005] To achieve the above objectives, the embodiments of the present application adopt the following technical solutions:

[0006] In the first aspect, the present application provides a shooting method, which is applied to an electronic device, in which a camera application is installed, and the camera application provides a first shooting mode (which can be any shooting mode, especially a shooting mode with high memory usage, such as the high-pixel shooting mode described below), and the first shooting mode includes a first image processing method (such as the β processing channel (FeatureGraph-β) described below). Specifically, the method includes: turning on the camera application. In response to the selection operation of the first shooting mode, a preview interface for shooting is displayed. In response to the shooting operation in the preview interface (such as a click operation on the shooting button), a first running memory (such as the buffer 2 described below) is allocated, and the captured image is processed in the first running memory using the first image processing method, and the processing result is written into the second running memory (such as the buffer 1 described below), and the first running memory is released, and the size of the first running memory is larger than the size of the second running memory. A first shooting result is obtained based on the processing result in the second running memory, such as an image stored in a gallery or an image displayed on a preview interface.

[0007] In summary, when the electronic device processes images using the first image processing method in the first shooting mode, it can use the large first running memory for a short time and release it immediately after use, and will not occupy the memory for a long time. In this way, compared with the method of using a larger resident memory: when the subsequent electronic device processes images using other image processing methods (especially high-load processing methods, such as the α processing channel (FeatureGraph-α) described below) in the first shooting mode, it can complete the image processing more smoothly while occupying less memory, reducing the possibility of shooting freezes.

[0008] In a possible design of the first aspect, the first image processing method includes a first processing stage (JPEG-β as described below). The first image processing method is used to process the acquired image in the first running memory, and the processing result (such as the encoding result) is written into the second running memory, including: completing the processing of the first processing stage in the first running memory, and writing the processing result of the first processing stage into the second running memory.

[0009] That is, the electronic device can use a large first operating memory for processing in the first processing stage, and release the first operating memory after the processing is completed, so that memory optimization can be performed in a targeted manner.

[0010] Furthermore, the size of the running memory required for the processing process of the above-mentioned first processing stage is greater than the first value, such as 200MB, 300MB, 500MB, etc. That is, the first processing stage is a processing stage with a higher demand for memory. Exemplarily, the first processing stage includes a JPEG encoding stage. For example, in high-pixel shooting mode, the memory demand of the JPEG encoding stage can be as high as 300MB, and the memory demand is relatively high. In this way, the electronic device can optimize the memory for the processing stage with a higher demand for memory. Accordingly, the size of the first running memory should also be greater than the first value. After the first running memory is released, the memory occupancy will be greatly reduced, and the effect of memory optimization will be more obvious.

[0011] In a possible design of the first aspect, before the first image processing method is used to process the captured image in the first running memory, the method further includes: receiving a shooting operation in the preview interface in a first scene (a low-light environment as described below); and selecting a first image processing method that matches the first scene. Accordingly, using the first image processing method to process the captured image in the first running memory includes: using the selected first image processing method to process the captured image in the first running memory.

[0012] In this way, the electronic device can process the image in the first image processing manner that matches the current first scene to obtain a suitable shooting result.

[0013] In a possible design of the first aspect, after selecting a first image processing method that matches the first scene and before writing the processing result into the second running memory, the method further includes: if it is the first time that the first image processing method is selected after the camera application is opened, allocating the second running memory; wherein, if it is not the first time that the first image processing method is selected after the camera application is opened, the second running memory is not allocated.

[0014] In this way, the electronic device can accurately allocate the second operating memory when the first image processing method is selected for the first time after the camera application is started this time, and there is no need to repeatedly allocate it later. In other words, after the camera application is started this time, the second operating memory is only allocated once, and it can be used to store the processing results of the first image processing method each time the first image processing method is selected.

[0015] In a possible design of the first aspect, after obtaining the first shooting result based on the processing result in the second running memory, the method further includes: in response to closing the camera application, releasing the second running memory. In this way, after closing the camera application, the resident memory allocated during the running of the camera application can be released.

[0016] At this point, it should be noted that: opening the camera application may refer to starting the application process of the camera application, and correspondingly, closing the camera application refers to closing the running of the application process. Alternatively, opening the camera application may refer to switching the camera application to the foreground, and correspondingly, closing the camera application refers to switching the camera application to the background. This application does not make specific limitations on this.

[0017] In a possible design of the first aspect, during the process of turning the camera on and off, the electronic device can shoot multiple times in the first shooting mode, and different image processing methods can be used for image processing in at least two shootings. In addition to the first image processing method, the image processing method can also include a second image processing method (such as FeatureGraph-α described below).

[0018] After displaying the preview interface of shooting in response to the selection operation of the first shooting mode (i.e., after entering the first shooting mode), in the first scene, before receiving the shooting operation in the preview interface (i.e., before selecting the first image processing method), or, after obtaining the first shooting result based on the processing result in the second running memory (i.e., after shooting is completed using the first image processing method), before closing the camera application, the above method also includes: in the second scene (the highlight environment described below), in response to the shooting operation in the preview interface, allocating the third running memory (such as the buffer 2A described below), processing the captured image in the third running memory using the second image processing method, writing the processing result into the fourth running memory (such as the buffer 1A described below), releasing the third running memory, and the size of the third running memory is greater than the size of the fourth running memory. The second shooting result is obtained based on the processing result in the fourth running memory.

[0019] That is, when the electronic device processes an image using the second image processing method in the first shooting mode, it can also use the large third operating memory for a short time and release it immediately after use, so that the memory will not be occupied for a long time. In this way, when the electronic device subsequently processes an image using the first image processing method in the first shooting mode, it can also complete the image processing more smoothly while occupying less memory.

[0020] Of course, the electronic device may also process one or more processing stages in the second image processing method (such as JPEG-α described below) in a large third operating memory to improve the targetedness of memory optimization.

[0021] In a possible design of the first aspect, the fourth operating memory is allocated when responding to the shooting operation in the first shooting mode for the first time after the camera application is turned on this time. It can be seen that the allocation timing of the fourth operating memory is different from the allocation timing of the second operating memory in the previous text. Among them, after the camera application is turned on this time, the fourth operating memory is allocated when responding to the shooting operation in the first shooting mode for the first time, ensuring that the fourth operating memory is allocated once the shooting in the first shooting mode starts. In this way, after the camera application is turned on this time, the fourth operating memory can be called for use every time when shooting in the first shooting mode.

[0022] In a possible design of the first aspect, after obtaining the second shooting result based on the processing result in the second running memory, the method further includes: in response to closing the camera application, releasing the fourth running memory. In this way, after closing the camera application, the resident memory allocated during the running of the camera application can be released.

[0023] In a possible design of the first aspect, the load of the first image processing method is relatively small, so when the first image processing method is used to process an image, the requirement for the remaining amount of running memory is not high.

[0024] Therefore, in the present design, after displaying the preview interface of shooting in response to the selection operation of the first shooting mode (i.e., after entering the first shooting mode), in the first scenario, before receiving the shooting operation in the preview interface (i.e., before selecting the first image processing method), or, after obtaining the first shooting result based on the processing result in the second running memory (i.e., after shooting is completed using the first image processing method), before closing the camera application, the above method further includes: in the second scenario, in response to the shooting operation in the preview interface, using the second image processing method to process the captured image in the fifth running memory (such as the resident buffer 13 below), and storing the processing result in the fifth running memory. A third shooting result is obtained based on the processing result in the fifth running memory.

[0025] That is to say, for the second image processing method, the electronic device can continue to use a large resident memory (i.e., the fifth memory) to complete the processing and store the processing results. Then, after the second image processing method is completed, the fifth memory will not be released. However, since the first image processing method does not require a high amount of memory, the remaining fifth memory will not affect the processing of the first image processing method.

[0026] In a possible design of the first aspect, the ambient brightness of the first scene is lower than the ambient brightness of the second scene. That is, the electronic device can use corresponding image processing methods for different brightness environments.

[0027] Of course, the first scene and the second scene may also differ in other aspects. For example, the first scene is a scene where the user selects natural light effects, and the second scene is a scene where the user selects stage light effects.

[0028] In a possible design of the first aspect, the electronic device includes a camera service, a channel selection module and an image processing module. In response to a shooting operation in a preview interface, a first running memory is allocated, and a first image processing method is used to process the collected image in the first running memory, and the processing result is written into a second running memory, and the first running memory is released, including: in response to a shooting operation in a preview interface, the camera application sends a shooting request to the camera service. The camera service sends a shooting request to the channel selection module. The channel selection module selects a first image processing method that matches the current first scene of the electronic device. The channel selection module sends indication information of the first image processing method to the image processing module. The image processing module allocates a first running memory based on the indication information, and uses the first image processing method to process the collected image in the first running memory, and writes the processing result into the second running memory, and releases the first running memory.

[0029] That is to say, the electronic device can accurately select the first image processing method that matches the current first scene through the channel selection module, and subsequently, the image processing module can use the first image processing method to perform image processing based on this.

[0030] In a possible design of the first aspect, after the channel selection module selects the first image processing mode that matches the current first scene of the electronic device, the method further includes: if it is the first time that the first image processing mode is selected after the camera application is turned on, the channel selection module allocates the second running memory. Accordingly, the image processing module writes the processing result into the second running memory, including: the image processing module calls the second running memory and writes the processing result into the second running memory.

[0031] That is to say, the electronic device can allocate the second running memory in time through the channel selection module after the first image processing mode is selected for the first time after the camera application is started this time.

[0032] In a possible design of the first aspect, after the camera application sends a shooting request to the camera service, the method further includes: if it is the first time that a shooting request is received in the first shooting mode after the camera application is turned on, the camera service allocates a fourth running memory or a fifth running memory, the fourth running memory is used to write the processing results of the second image processing method included in the first shooting mode, and the fifth running memory is used to process and store the processing results of the second image processing.

[0033] That is to say, the electronic device can, through the camera service, promptly allocate the fourth running memory or the fifth running memory when shooting in the first shooting mode for the first time after the camera application is opened this time.

[0034] In a second aspect, the present application further provides an electronic device, the electronic device comprising a display screen, a memory and one or more processors. The display screen, the memory and the processor are coupled. The memory is used to store computer program code, the computer program code comprising computer instructions, when the computer instructions are executed by the processor, the electronic device executes the method in the first aspect and any possible design thereof.

[0035] In a third aspect, the present application provides a chip system, which is applied to an electronic device including a display screen and a memory; the chip system includes one or more interface circuits and one or more processors; the interface circuit and the processor are interconnected through lines; the interface circuit is used to receive signals from the memory of the electronic device and send signals to the processor, the signals including computer instructions stored in the memory; when the processor executes the computer instructions, the electronic device executes the method as in the first aspect and any possible design method thereof.

[0036] In a fourth aspect, the present application provides a computer storage medium, which includes computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes the method as in the first aspect and any possible design thereof.

[0037] In a fifth aspect, the present application provides a computer program product, which, when executed on a computer, enables the computer to execute the method of the first aspect and any possible design thereof.

[0038] It can be understood that the beneficial effects that can be achieved by the electronic device of the second aspect, the chip system of the third aspect, the computer storage medium of the fourth aspect, and the computer program product of the fifth aspect provided above can be referred to the beneficial effects in the first aspect and any possible design method thereof, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 A schematic diagram of a camera interface provided in an embodiment of the present application;

[0040] Figure 2 A schematic diagram of a processing framework of a high-pixel shooting mode provided in an embodiment of the present application;

[0041] Figure 3 A schematic diagram of memory consumption by a high-pixel shooting mode provided in an embodiment of the present application;

[0042] Figure 4 A hardware structure diagram of an electronic device provided in an embodiment of the present application;

[0043] Figure 5 A software architecture diagram of an electronic device provided in an embodiment of the present application;

[0044] Figure 6 A schematic diagram of a memory optimization solution for a high-pixel shooting mode provided in an embodiment of the present application;

[0045] Figure 7 To adopt Figure 6 One of the flow charts of the camera usage process after the memory optimization scheme shown;

[0046] Figure 8 To adopt Figure 6 The second flowchart of the camera usage process after the memory optimization solution shown;

[0047] Fig. 9 To adopt Figure 6 The timing interaction diagram of a shooting process after the memory optimization scheme shown;

[0048] Fig.10 To adopt Figure 6Schematic diagram of memory allocation when the memory optimization scheme is shown;

[0049] Fig.11 A schematic diagram of another memory optimization solution for a high-pixel shooting mode provided in an embodiment of the present application;

[0050] Fig.12 To adopt Fig.11 One of the flow charts of the camera usage process after the memory optimization scheme shown;

[0051] Fig.13 To adopt Fig.11 The second flowchart of the camera usage process after the memory optimization solution shown;

[0052] Fig.14 To adopt Fig.11 The timing interaction diagram of a shooting process after the memory optimization scheme shown;

[0053] Fig.15 To adopt Fig.11 Schematic diagram of memory allocation when the memory optimization scheme is shown;

[0054] Fig.16 A schematic diagram of a general processing framework of a target shooting mode applicable to an embodiment of the present application;

[0055] Fig.17 For Fig.16 Diagram of the general memory optimization scheme for the target shooting mode shown. DETAILED DESCRIPTION

[0056] The technical solutions in the embodiments of the present application are described below in conjunction with the drawings in the embodiments of the present application. Among them, in the description of the embodiments of the present application, the terms used in the following embodiments are only for the purpose of describing specific embodiments, and are not intended to be used as limitations on the present application. As used in the specification and the appended claims of the present application, the singular expressions "a", "said", "above", "the" and "this" are intended to also include expressions such as "one or more", unless there is a clear indication to the contrary in the context. It should also be understood that in the following embodiments of the present application, "at least one", "one or more" refer to one or more (including two). The term "and / or" is used to describe the association relationship of associated objects, indicating that three relationships can exist; for example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the associated objects before and after are in a "or" relationship.

[0057] References to "one embodiment" or "some embodiments" etc. described in this specification mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Thus, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways. The term "connection" includes direct connection and indirect connection, unless otherwise specified. "First" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated.

[0058] In the embodiments of the present application, the words "exemplarily" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplarily" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplarily" or "for example" is intended to present related concepts in a specific way.

[0059] The shooting method provided in the embodiment of the present application can be used in the scenes where the electronic device uses various shooting modes in the camera function to shoot, and is particularly suitable for the target shooting mode with high memory consumption. Among them, high memory consumption refers to: a shooting mode that requires a large amount of memory for image processing during the shooting process.

[0060] For example, in a high-pixel (eg, 200 million pixels) shooting mode, JPEG requires about 300 MB of memory, which consumes a large amount of memory, so the high-pixel shooting mode belongs to the target shooting mode.

[0061] As another example, some shooting modes include complex image processing algorithms, which also require a large amount of memory support. These shooting modes with relatively complex image processing algorithms also belong to target shooting modes.

[0062] In the following, the target shooting mode is a high-pixel shooting mode as an example to illustrate the present application scheme. Figure 1 The interface 101 of the camera application (i.e., the camera function) shown in the figure includes various shooting modes provided by the camera application, such as photo, video, portrait, night scene, etc. In addition, the interface 101 also includes "more" 1011. In response to the user's selection operation of "more" 1011, the mobile phone can display Figure 1The interface 102 shown in the figure includes more shooting modes provided by the camera function, such as time-lapse photography, dynamic photos, dual-view video and other shooting modes, and also includes a high-pixel shooting mode 1021. In response to the selection operation of the high-pixel shooting mode 1021, the mobile phone can display Figure 1 The preview interface 103 in the high-pixel shooting mode is shown. In practice, the preview interface 103 includes a preview image. The viewfinder interface 103 includes a start shooting button 1031. In response to a click operation on the start shooting button 1031, the mobile phone can start shooting a high-pixel image, that is, shooting in the high-pixel shooting mode. At this time, the electronic device can use the shooting method provided in the embodiment of the present application to shoot.

[0063] Electronic devices can usually allocate corresponding resident memory, such as resident buffer, according to the memory requirements of each processing stage in high-pixel shooting mode. Among them, the resident memory will not be released until the camera application is closed. This may cause shooting lag. Figure 2 and Figure 3 Further explanation.

[0064] In high-pixel shooting mode, the electronic device provides Figure 2 The α processing channel (FeatureGraph-α) and β processing channel (FeatureGraph-β) shown are respectively used for processing images collected in different brightness environments. Among them, the brightness environment includes low-brightness environment and high-brightness environment. Exemplarily, low-brightness environment includes night scenes and indoor environments without lights, and high-brightness environment includes outdoor daytime and bright light environment. For images collected in low-brightness environment, the electronic device uses FeatureGraph-β to significantly increase the brightness of the image to ensure that the brightness of the captured image (such as preview images, images stored in the album) finally provided to the user is appropriate. For images collected in high-brightness environment, the electronic device uses FeatureGraph-α to moderately increase or not increase the brightness of the image to ensure that the brightness of the image provided to the user is appropriate. In short, FeatureGraph-α is used for processing images collected in high-brightness environment (referred to as high-brightness environment), and FeatureGraph-β is used for processing images collected in low-brightness environment (referred to as low-brightness environment).

[0065] It should be noted that the division of processing channels and their applicable scenes in different shooting modes are not based on Figure 2 For example, in the portrait shooting mode, the processing channels can be divided according to the light effects obtained by processing, and different processing channels are suitable for processing different types of light effects. For example, processing channel A is used to process natural light effects, processing channel B is used to process stage light effects, processing channel C is used to process monochrome light effects, etc.

[0066] In practice, during the capture of each frame of image, the electronic device can perform automatic exposure (AutoExposure, AE). One parameter of AE is AE gain (GAIN). Generally speaking, the larger the AE GAIN is, the darker the environment is, and a greater magnification is required to increase the exposure. Therefore, after starting to shoot, the electronic device can select a processing channel based on the AE GAIN corresponding to the previous frame or frames of image before shooting. For example, if AE GAIN is greater than the first value, indicating that the electronic device is in a low-light environment, FeatureGraph-β can be selected; if AE GAIN is less than or equal to the first value, indicating that the electronic device is in a high-light environment, FeatureGraph-α can be selected.

[0067] It should be noted that when shooting in high-pixel shooting mode once, the electronic device can select the corresponding processing channel to process the image based on the brightness multiple. Then, when shooting in high-pixel shooting mode multiple times, such as when the user clicks the start shooting button multiple times, the electronic device can use different processing channels for processing. For example, when the user clicks the start shooting button once, the electronic device captures an outdoor daytime image, then the electronic device can use FeatureGraph-α to process the image; when the user clicks the start shooting button another time, the electronic device captures an indoor image without lights, then the electronic device can use FeatureGraph-β to process the image.

[0068] And, FeatureGraph-α and FeatureGraph-β both include multiple processing stages. Exemplarily, the multiple processing stages include: the image sensor output control stage (Realtime), referred to as output control; the algorithm processing stage (Feature), referred to as algorithm processing; and the JPEG after algorithm processing, referred to as JPEG.

[0069] For example, FeatureGraph-α includes Figure 2 The output control-α, algorithm processing-α and JPEG-α are shown in the figure. FeatureGraph-β includes Figure 2 There are three processing stages in total, namely, output image control-β, algorithm processing-β and JPEG-β. It should be noted that: there are differences between the above-mentioned algorithm processing-α and algorithm processing-β. Algorithm processing-α uses a processing algorithm for images collected in high-brightness environments, and algorithm processing-β uses a processing algorithm for images collected in low-brightness environments. However, output image control-α and output image control-β, JPEG-α and JPEG-β may be the same or different, and the embodiments of the present application do not specifically limit this.

[0070] When the electronic device uses FeatureGraph-β to process the image based on the brightness multiple being greater than the first multiple, the corresponding resident memory will be allocated to the output control-β, algorithm processing-β and JPEG-β respectively, such as Figure 3 The resident buffer 21, resident buffer 22 and resident buffer 23 are shown. Subsequently, when the brightness environment changes, the electronic device uses FeatureGraph-α to process the image based on the brightness multiple being less than the first multiple, and the corresponding resident memory is allocated to the output control-α, algorithm processing-α and JPEG-α respectively, such as Figure 3 The resident buffer 11, resident buffer 12 and resident buffer 13 are shown. Since the resident buffer is not released until the camera is turned off, when using FeatureGraph-α to process images, the memory consumption of the high-pixel shooting mode is the sum of the memory of resident buffer 11, resident buffer 12, resident buffer 13, resident buffer 21, resident buffer 22 and resident buffer 23.

[0071] Compared with the normal shooting mode: the memory occupied by the JPEG (such as JPEG-α and JPEG-β) processing process in the high-pixel shooting mode will be greatly increased. For example, in the normal shooting mode, the JPEG processing process occupies about 15MB of memory, while in the high-pixel shooting mode, the JPEG processing process occupies about 300MB of memory (in practice, the memory usage of JPEG-α and JPEG-β may also be different), which is about 20 times that of the normal shooting mode. In addition, the memory available for the camera in the electronic device is limited, such as about 2.8GB, and in the high-pixel shooting mode, other processing besides JPEG also requires a high memory usage, and accordingly, the remaining memory for JPEG will be very small. It can be seen from this that in the high-pixel shooting mode, there is a high possibility of insufficient memory, which makes it difficult to complete the shooting smoothly, and the shooting will be stuck.

[0072] Based on the above problems, the present application provides a shooting method. When shooting in a high-pixel shooting mode, the electronic device can use a larger temporary memory (hereinafter referred to as buffer 2) to complete the processing of the target stage, write the processing results to a smaller resident memory (hereinafter referred to as buffer 1) and release the larger temporary memory. The target stage refers to the processing stage in FeatureGraph-α and FeatureGraph-β where the processing process occupies a larger memory, such as the target stage is the above-mentioned JPEG.

[0073] In this way, the buffer 2 only exists briefly during the processing of the target stage, and will not occupy a large amount of memory for a long time, thereby reducing memory consumption and avoiding the phenomenon of shooting freezes.

[0074] Exemplarily, the electronic devices may include smart phones, personal digital assistants (PDA), tablet computers, laptop computers, camcorders, video recorders, cameras, smart watches, smart wristbands, cellular phones, car computers, and other devices with camera applications. The embodiments of the present application do not impose any special restrictions on the specific forms of the electronic devices.

[0075] See also Figure 4 , is a hardware structure diagram of an electronic device provided in an embodiment of the present application. Figure 4 As shown, taking the electronic device being a mobile phone as an example, the electronic device may include: a processor 410, an external memory interface 420, an internal memory 421, a universal serial bus (USB) interface 430, a charging management module 440, a power management module 441, a battery 442, an antenna 1, an antenna 2, a mobile communication module 450, a wireless communication module 460, an audio module 470, a speaker 470A, a receiver 470B, a microphone 470C, an earphone interface 470D, a sensor module 480, a button 490, a motor 491, an indicator 492, a camera 493, a display screen 494, and a subscriber identification module (SIM) card interface 495, etc.

[0076] It is to be understood that the structure illustrated in this embodiment does not constitute a specific limitation on the electronic device. In other embodiments, the electronic device may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0077] The processor 410 may include one or more processing units, for example, the processor 410 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0078] The charging management module 440 is used to receive charging input from a charger. The charger can be a wireless charger or a wired charger. The power management module 441 is used to connect the battery 442, the charging management module 440 and the processor 410. The power management module 441 receives input from the battery 442 and / or the charging management module 440, and supplies power to the processor 410, the internal memory 421, the external memory, the display screen 494, the camera 493, and the wireless communication module 460.

[0079] The wireless communication function of the electronic device can be implemented through antenna 1, antenna 2, mobile communication module 450, wireless communication module 460, modem processor and baseband processor.

[0080] The electronic device implements the display function through a GPU, a display screen 494, and an application processor. The GPU is a microprocessor for image processing, connecting the display screen 494 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 410 may include one or more GPUs that execute program instructions to generate or change display information.

[0081] The electronic device can realize the shooting function through ISP, camera 493, video codec, GPU, display screen 494 and application processor. ISP is used to process the data fed back by camera 493. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element (hereinafter referred to as image sensor) through the lens, and the light signal is converted into an electrical signal. The image sensor transmits the electrical signal to ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise and brightness of the image. ISP can also optimize parameters such as exposure and color temperature.

[0082] Camera 493 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the image sensor. The image sensor can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The image sensor converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format.

[0083] In some embodiments, the above-mentioned Realtime, Feature and JPEG are all completed by the ISP, and finally after being processed by JPEG, an image signal in YUV format can be output.

[0084] Of course, ISP can also be understood as a special DSP. Therefore, it can also be understood that the above-mentioned Realtime, Feature and JPEG are all completed by DSP.

[0085] The external memory interface 420 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device.

[0086] The internal memory 421 may be used to store computer executable program codes, which include instructions. The processor 410 executes the instructions stored in the internal memory 421 to perform various functional applications and data processing of the electronic device.

[0087] The electronic device can implement audio functions such as music playing and recording through the audio module 470, the speaker 470A, the receiver 470B, the microphone 470C, the headphone jack 470D, and the application processor.

[0088] The software system of the electronic device can adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. The embodiment of the present application takes the Android system of the layered architecture as an example to illustrate the software structure of the electronic device.

[0089] Reference Figure 5 , is a software structure block diagram of an electronic device provided in an embodiment of the present application. Figure 5As shown, the layered architecture can divide the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom, namely, the application layer (applications), the application framework layer (framework), the hardware abstract layer (HAL) and the kernel layer (kernel, also known as the driver layer).

[0090] The application layer may include multiple application packages, which may include applications such as gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, and short message.

[0091] Furthermore, the application layer may also include a camera application, ie, an application that provides the aforementioned camera functions.

[0092] The application framework layer provides an application programming interface (API) and a programming framework for the application layer. The framework layer includes some predefined functions.

[0093] Among them, the application framework layer may include a window manager, a content provider, a view system, a resource manager, a notification manager, and the like.

[0094] In addition, the application framework layer may also include a camera service. The camera service is mainly used for interaction between the camera application and the bottom layer. Exemplarily, the camera service may be used to transmit requests from the camera application to the HAL layer, such as a request to start shooting, a request to switch shooting modes, etc., and the camera service may also be used to upload images returned by the HAL layer to the camera application.

[0095] The HAL layer is an abstract structure between the kernel layer and the application framework layer. The HAL layer can be a package of hardware drivers, providing a unified interface for the calls of upper-layer applications. Exemplarily, the HAL layer may include Bluetooth HAL, WLAN HAL, display HAL, audio HAL, camera HAL, etc.

[0096] The camera HAL is the core software framework of the camera, and the camera HAL further includes an interface module, a camera abstract device, and an image processing module. The interface module, the camera abstract device, and the image processing module are components in the image data and control instruction transmission pipeline in the camera HAL.

[0097] Of course, different components also have different corresponding functions. Exemplarily, the interface module can be a software interface for the application framework layer, used to interact with the application framework layer, or the interface module can also interact with other modules in the HAL layer (such as camera abstract device, image processing module). The camera abstract device can be a software interface for the driver layer, used to interact with the driver layer, such as calling the camera device driver in the driver layer (hereinafter referred to as the camera driver).

[0098] The image processing module can process the raw image data sent back by the camera module. For example, the image processing module may include an image front end (IFE) node (IFE node) and a Bayer processing segment (BPS) node (BPS node). Exemplarily, the Realtime is completed by the IFE node, and the Feature is completed by the BPS node.

[0099] The camera HAL also includes a channel selection management module and a FeatureGraph (channel selection module). The channel selection management module is used to control the operation of FeatureGraph. FeatureGraph is used to select processing channels according to the brightness environment, such as selecting FeatureGraph-α in a high-brightness environment and FeatureGraph-β in a low-brightness environment.

[0100] Accordingly, the above-mentioned image processing module is used to perform corresponding image processing according to the selected processing channel. For example, if the currently selected processing channel is FeatureGraph-α, the image processing module will perform processing such as output control-α, algorithm processing-α and JPEG-α; or, if the currently selected processing channel is FeatureGraph-β, the image processing module will perform processing such as output control-β, algorithm processing-β and JPEG-β.

[0101] The kernel layer is located below the HAL and is a layer between hardware and software. The modules in the kernel layer are used to drive the operation of various hardware modules. For example, the kernel layer includes Bluetooth driver, WLAN driver, display driver, audio driver, etc., only some of which are shown in the figure.

[0102] Furthermore, the kernel layer also includes a camera driver for driving camera devices to work, wherein the camera devices include cameras, image sensors, and the like.

[0103] The shooting method provided in the present application can be executed in an electronic device having the above-mentioned software and hardware structure. Specifically, the electronic device can complete the processing of the target stage in FeatureGraph-β in buffer 2 (such as 300MB of temporary memory, the same below), and write the processing results into buffer 1 (such as 60MB of resident memory, the same below), while for the target stage in FeatureGraph-α, the processing is still completed in a larger resident memory and the processing results are stored. For details, please refer to the description of method one below. Alternatively, the electronic device can complete the processing of the target stage in FeatureGraph-β in buffer 2 and write the processing results into buffer 1; and, for the target stage in FeatureGraph-α, complete the processing in another buffer 2 (for distinction, it can be recorded as buffer 2A), and write the processing results into another buffer 1 (for distinction, it can be recorded as buffer 1A). For details, please refer to the description of method two below. They will be introduced separately below.

[0104] Method 1

[0105] When electronic devices use FeatureGraph-α to process images, they usually need to process high-resolution images, which has a high load. It can be seen that when electronic devices use FeatureGraph-α to process images, if the memory usage is very high, such as when the image is first processed by FeatureGraph-β and then by FeatureGraph-α, the memory usage reaches at least 600MB, which will cause the image processing process to be unsmooth, further leading to the phenomenon of shooting freezes.

[0106] Therefore, the electronic device can adopt method 1, and perform memory optimization when processing images with FeatureGraph-β, so as to reduce the memory usage when processing images with FeatureGraph-α, and reduce the possibility of freezes, which is explained in detail below.

[0107] In the description of the first method, the target stage is taken as an example to explain JPEG, and accordingly, the processing result of the target stage is the encoding result:

[0108] Before adopting the embodiments of the present application, Figure 6 Before the improvement, the electronic device completes the JPEG-α processing in the resident buffer 13, such as the 300MB resident memory, and uses the resident buffer 13 to store the encoding result. Also, the electronic device completes the JPEG-β processing in the resident buffer 23, such as the 300MB resident memory, and uses the resident buffer 23 to store the encoding result. The electronic device will not release the resident buffer 13 and the resident buffer 23 until the camera is turned off.

[0109] It can be seen that before the improvement, when the electronic device uses FeatureGraph-α to process images, the resident buffer 13 and the resident buffer 23 may exist at the same time, which will occupy a higher memory. For example, when FeatureGraph-β is used to process the image first and then FeatureGraph-α is used to process the image, the memory occupies at least 600MB, which makes it difficult to successfully complete the image processing using FeatureGraph-α, causing shooting to be stuck.

[0110] After adopting the embodiment of the present application, Figure 6 After the improvement, the electronic device completes the JPEG-α processing in the resident buffer 13, such as the 300MB resident memory, and uses the resident buffer 13 to store the encoding result. However, the difference from before the improvement is that after the improvement, the electronic device completes the JPEG-β processing in the buffer 2, writes the encoding result into the buffer 1, and then releases the buffer 2. The electronic device will not release the resident buffer 13 and the buffer 1 until the camera is turned off.

[0111] The size of buffer 2 is substantially the same as that of resident buffer 23 to ensure that JPEG-β processing can be completed. For example, buffer 2 is also approximately 300 MB. Buffer 1 only needs to meet the requirement of storing the encoding result, and the size of buffer 1 is smaller than that of buffer 2. For example, buffer 1 is approximately 60 MB.

[0112] It can be seen that after the improvement, the electronic device will release buffer 2 in time after completing the JPEG-β processing and writing the encoding result into buffer 1. So, when the electronic device uses FeatureGraph-α to process the image, buffer 2 does not exist, while buffer 1 and resident buffer 13 may exist at the same time. For example, when FeatureGraph-β is used to process the image first, and then FeatureGraph-α is used to process the image, buffer 1 and resident buffer 13 may exist at the same time. Moreover, the sum of the memory of buffer 1 and resident buffer 13 is less than the sum of the memory of resident buffer 13 and resident buffer 23, that is, the memory occupancy is less, so it is more conducive to the smooth completion of FeatureGraph-α image processing and avoid shooting jams.

[0113] Combine the following Figure 7 and Figure 8 , the specific implementation and effect of the above method 1 are explained by using the complete process of using the camera once. Figure 7 In , the electronic device first processes the image using FeatureGraph-α, and then processes the image using FeatureGraph-β; Figure 8In the example, the electronic device first processes the image using FeatureGraph-β, and then processes the image using FeatureGraph-α. The following are the respective descriptions:

[0114] like Figure 7 As shown, the complete process of using the camera once includes:

[0115] S701. The electronic device turns on the camera.

[0116] For example, in response to a user clicking an application icon of a camera application, the electronic device may turn on the camera.

[0117] S702 : In response to receiving the operation of starting high-pixel shooting for the first time after the camera is turned on this time, the electronic device allocates a resident buffer 13 .

[0118] For example, the operation of starting high-pixel shooting may be to press a start shooting button (eg, Figure 1 After the camera is turned on this time, the electronic device receives the click operation of the start shooting button in the high-pixel shooting mode for the first time, and can use the high-pixel shooting mode for the first time.

[0119] It should be noted that the resident buffer 13 can be used not only for JPEG processing when FeatureGraph-α is used, but also for processing in some processing stages when FeatureGraph-β is used. Therefore, during the first shooting process, the electronic device will allocate the resident buffer 13 regardless of whether FeatureGraph-α is used to process the image.

[0120] S703: The electronic device selects FeatureGraph-α based on the brightness environment.

[0121] Specifically, if the brightness environment during the first shooting is a high-brightness environment, such as an outdoor daytime shooting scene or a bright light shooting scene, the electronic device can select FeatureGraph-α to process the image. For details about the selection of processing channels, please refer to the previous section on Figure 2 The relevant explanations will not be repeated here, and the same applies to the following text.

[0122] S704, the electronic device captures an image and processes the image using FeatureGraph-α; wherein the electronic device calls the resident buffer 13 for processing the JPEG in the first shooting and stores the processing result.

[0123] After selecting FeatureGraph-α, the electronic device can use FeatureGraph-α to process the collected image. For example, the electronic device uses FeatureGraph-α to process the collected image using output control-α, algorithm processing-α, and JPEG-α to obtain an image with appropriately improved or no brightness.

[0124] When FeatureGraph-α is used to process images in S704, there is a resident buffer 13, such as about 300 MB, and the memory usage is not large, such as not reaching 600 MB, which is more conducive to the electronic device to successfully complete the image processing of FeatureGraph-α and avoid shooting freezes.

[0125] S705: The electronic device ends the first shooting.

[0126] It is understandable that the electronic device can further process the processing results in the resident buffer 13 to obtain the captured image, and then end the shooting. After the shooting is finished, the electronic device can display the captured image in the preview interface or store the captured image in the gallery.

[0127] Subsequently, the brightness environment is still maintained at a high brightness environment, such as when the operation of starting high-pixel shooting is received for the i1th time (1<i1<k1, i1 and k1 are both integers), the high brightness environment is still maintained, and the electronic device can use the following S706-S708 to shoot. Of course, if the brightness environment changes to a low brightness environment after the first shooting, such as a night scene shooting scene or an indoor shooting scene without lights, the following S706-S708 can be skipped.

[0128] S706 , in response to receiving the operation of starting high-pixel shooting for the i1th time after the camera is turned on this time, the electronic device selects FeatureGraph-α based on the brightness environment, where 1<i1<k1, and i1 and k1 are integers.

[0129] Similarly, after the camera is turned on this time, if the electronic device receives the user's click operation on the start shooting button in the high-pixel shooting mode for the i1th time, the electronic device can use the high-pixel shooting mode for shooting for the i1th time.

[0130] When shooting in the high-pixel shooting mode for the first time, the electronic device has allocated the resident buffer 13, and the resident buffer 13 has not been released. Therefore, when shooting in the high-pixel shooting mode for the i1th time, the electronic device does not need to allocate the resident buffer 13.

[0131] Since the brightness environment is still maintained at a high brightness environment, the electronic device will still choose FeatureGraph-α.

[0132] S707, the electronic device captures an image and processes the image using FeatureGraph-α; wherein the electronic device calls the resident buffer 13 for processing the JPEG in the i1th shooting. For details, please refer to the description of S704, which will not be repeated here.

[0133] S708: The electronic device ends the i1th shooting.

[0134] Similarly, the electronic device may further process the processing result in the resident buffer 13 to obtain the shooting result, thereby ending the shooting.

[0135] It can be seen from the above steps S702 - S708 that when the method 1 is adopted, when the FeatureGraph-α is used to process the image first, there will be no shooting freeze.

[0136] Subsequently, when the brightness environment changes to a low-brightness environment, such as a night scene shooting scene or an indoor shooting scene without lights, the electronic device may execute the following S709-S712.

[0137] S709: In response to receiving the k1th operation of starting high-pixel shooting after the camera is turned on this time, the electronic device selects FeatureGraph-β based on the brightness environment.

[0138] Similarly, after the camera is turned on this time, if the electronic device receives a click operation of the user on the start shooting button in the high-pixel shooting mode for the k1th time, the electronic device can use the high-pixel shooting mode for shooting for the k1th time.

[0139] Since the brightness environment changes to a low brightness environment, the electronic device will select FeatureGraph-β.

[0140] S710 , the electronic device allocates buffer 1 .

[0141] Before the k1th shooting in high-pixel shooting mode, the electronic device selects FeatureGraph-β based on the brightness environment. This is the first time that FeatureGraph-β is selected. Therefore, the electronic device needs to allocate buffer 1, such as 60MB of resident memory, to store JPEG encoding results when using FeatureGraph-β to process images.

[0142] S711, the electronic device captures an image and processes the image using FeatureGraph-β; wherein the electronic device allocates buffer 2 for processing JPEG in the k1th shooting, writes the data in buffer 2 into buffer 1 and releases buffer 2 after JPEG is completed.

[0143] After selecting FeatureGraph-β, the electronic device can use FeatureGraph-β to process the captured image. Exemplarily, when the electronic device uses FeatureGraph-β, it can perform operations such as image output control-β, algorithm processing-β, and JPEG-β on the captured image to obtain an image with greatly improved brightness.

[0144] During the process of processing the image using FeatureGraph-β, in order to meet the memory requirements of the JPEG processing process, before starting JPEG, such as before starting JPEG-β, the electronic device can allocate buffer 2, such as allocating 300MB of temporary memory, to complete JPEG.

[0145] When using FeatureGraph-β to process the image in S711, the resident buffer 13 and buffer 1 exist simultaneously, occupying approximately 360MB of memory (300MB + 60MB). Additionally, during the JPEG processing process, there is also buffer 2, so the total memory occupied is approximately 660MB (360MB + 300MB). Since at this time FeatureGraph-β is used to process the image, and the images processed by FeatureGraph-β are usually downsampled images with lower resolution and lower processing load, even if it is slightly higher than the sum of the memory of the resident buffer 13 and resident buffer 23 before improvement, such as 60MB higher (660MB - 600MB), the electronic device can still successfully use FeatureGraph-β to complete image processing without causing lag.

[0146] S712. The electronic device ends the k1-th shooting.

[0147] It can be understood that the electronic device can further process the processing result in buffer 1 to obtain the shooting result, thereby ending the shooting.

[0148] Subsequently, if the brightness environment remains in a low-brightness environment all the time, such as when shooting in the high-pixel shooting mode for the j1-th time (k1 < j1 < k2, both j1 and k2 are integers) and still maintaining a low-brightness environment, the electronic device can use the following S713 - S715 for shooting. Of course, if after ending the j1-th shooting, the brightness environment changes to a high-brightness environment, the following S713 - S715 can be skipped.

[0149] S713. In response to the j1-th operation of starting high-pixel shooting received after the camera is turned on this time, the electronic device selects FeatureGraph-β based on the brightness environment, where k1 < j1 < k2 and both j1 and k2 are integers.

[0150] Similarly, after the camera is turned on this time, if the electronic device receives the user's click operation on the start shooting button in the high-pixel shooting mode for the j1th time, the electronic device can use the high-pixel shooting mode for the j1th time.

[0151] Since the brightness environment remains low, the electronic device will still choose FeatureGraph-β.

[0152] It should be noted that this is not the first time FeatureGraph-β is selected, and the electronic device has allocated buffer 1 when FeatureGraph-β was selected for the first time, and buffer 1 has not been released. Therefore, when shooting in high-pixel shooting mode for the j1th time, the electronic device does not need to allocate buffer 1 again.

[0153] S714, the electronic device collects an image and processes the image using FeatureGraph-β; wherein the electronic device allocates buffer 2 for JPEG processing in the j1th shooting, writes the data in buffer 2 into buffer 1 after JPEG is completed, and releases buffer 2. For details, please refer to the description of S711 above, which will not be repeated here.

[0154] S715. The electronic device ends the j1th shooting.

[0155] Through the above S709 - S715 , it can be known that, by adopting the first method, when FeatureGraph-α is first used to process the image and FeatureGraph-β is used to process the image later, the electronic device can also ensure that no lag occurs.

[0156] Subsequently, when the electronic device uses the high-pixel shooting mode to shoot again (such as the k2th time and thereafter), the processing channel is also selected based on the brightness environment. If FeatureGraph-α is selected, the electronic device uses steps similar to the aforementioned S707-S708 to complete the shooting; if FeatureGraph-β is selected, the electronic device uses steps similar to the aforementioned S714-S715 to complete the shooting. They are not listed here one by one.

[0157] S716. The electronic device turns off the camera and releases the resident memory.

[0158] Herein, closing the camera includes ending the running of the camera process or the camera exiting the foreground running.

[0159] The resident memory includes a resident buffer 13 and a buffer 1.

[0160] like Figure 8 As shown, the complete process of using the camera once includes:

[0161] S801. The electronic device turns on the camera.

[0162] S802 : In response to receiving the operation of starting high-pixel shooting for the first time after the camera is turned on this time, the electronic device allocates a resident buffer 13 .

[0163] S803: The electronic device selects FeatureGraph-β based on the brightness environment.

[0164] Specifically, when shooting for the first time, the brightness environment is a low-light environment, and the electronic device can select FeatureGraph-β to process the image.

[0165] S804. The electronic device allocates buffer 1.

[0166] Since this is the first time that FeatureGraph-β is selected after the camera is turned on, the electronic device needs to allocate buffer 1, such as 60MB of resident memory, for storing JPEG encoding results when the FeatureGraph-β is used to process the image.

[0167] That is, if FeatureGraph-β is selected for the first shooting, the electronic device will not only allocate resident buffer 13 but also buffer 1.

[0168] S805, the electronic device captures an image and processes the image using FeatureGraph-β; wherein the electronic device allocates buffer 2 for JPEG processing in the first shooting, writes the data in buffer 2 into buffer 1 and releases buffer 2 after JPEG is completed.

[0169] When FeatureGraph-β is used to process an image in S805, resident buffer 13 and buffer 1 exist at the same time, occupying a total of about 360MB (300MB+60MB) of memory. In addition, during the JPEG processing, buffer 2 also exists, occupying a total of about 660MB (360MB+300MB) of memory. Therefore, based on the same reasons as in S711 above, electronic devices can also successfully use FeatureGraph-β to complete image processing without causing lag.

[0170] S806: The electronic device ends the first shooting.

[0171] Subsequently, the brightness environment still maintains a low brightness environment. For example, when shooting in the i2th (1<i2<k3, i2 and k3 are integers) high-pixel shooting mode, the low brightness environment is still maintained, and the electronic device can use the following S807-S809 to shoot. Of course, if the brightness environment changes to a high brightness environment after the first shooting, the following S807-S809 can be skipped.

[0172] S807 . In response to receiving the operation of starting high-pixel shooting for the i2th time after the camera is turned on this time, the electronic device selects FeatureGraph-β based on the brightness environment, where 1<i2<k3, and i2 and k3 are integers.

[0173] Previously, when FeatureGraph-β was selected for the first time, the electronic device had allocated buffer 1, and buffer 1 was not released. Therefore, when shooting in high-pixel shooting mode for the i2th time, the electronic device does not need to allocate buffer 1 even if FeatureGraph-β is selected.

[0174] S808, the electronic device captures an image and processes the image using FeatureGraph-β; wherein the electronic device allocates buffer 2 for JPEG processing in the first shot, writes the data in buffer 2 into buffer 1 after JPEG is completed, and releases buffer 2. For details, please refer to the description of S805, which will not be repeated here.

[0175] S809: The electronic device ends the i2 th shooting.

[0176] It can be known from the above S802 to S809 that, when the first method is adopted, the electronic device can also be guaranteed not to be stuck when the FeatureGraph-β is used to process the image.

[0177] S810: In response to receiving the operation of starting high-pixel shooting for the k3th time after the camera is turned on this time, the electronic device selects FeatureGraph-α based on the brightness environment.

[0178] It should be noted that, although this is the first time FeatureGraph-α is selected, the resident buffer 13 has been allocated when the high-pixel shooting mode was used for the first time, and the resident buffer 13 has not been released. Therefore, the electronic device does not need to allocate the resident buffer 13 this time.

[0179] S811, the electronic device captures an image and processes the image using FeatureGraph-α; wherein the electronic device calls the resident buffer 13 for JPEG processing in the k3th shooting.

[0180] In S811, when FeatureGraph-α is used to process images, resident buffer 13 and buffer 1 exist at the same time, occupying a total of about 360MB (300MB+60MB) of memory. The sum of the two is smaller than the sum of the memory of resident buffer 13 and resident buffer 23, which is more conducive to the electronic device to successfully complete the image processing of FeatureGraph-α and avoid shooting freezes.

[0181] It can be seen that, when the first method is adopted, when FeatureGraph-β is used to process the image first, and then the electronic device uses FeatureGraph-α to process the image, it can also be ensured that no lag occurs.

[0182] S812: The electronic device finishes the k3th shooting.

[0183] Subsequently, when the electronic device uses the high-pixel shooting mode to shoot again (such as the k3+1th time and thereafter), the processing channel is also selected based on the brightness environment. If FeatureGraph-α is selected, the electronic device uses steps similar to the aforementioned S811-S812 to complete the shooting; if FeatureGraph-β is selected, the electronic device uses steps similar to the aforementioned S809-S810 to complete the shooting. They are not listed one by one here.

[0184] S813. The electronic device turns off the camera and releases the resident memory.

[0185] Likewise, the resident memory includes the resident buffer 13 and the buffer 1.

[0186] It should be noted that Figure 8 For the parts not described in detail in the process shown, please refer to the previous article Figure 7 The description of the corresponding steps in will not be repeated here.

[0187] In summary Figure 7 and Figure 8 As described above, in the first approach, during the use of a camera, the electronic device first uses FeatureGraph-α or FeatureGraph-β to process an image, which is more conducive to successfully completing the image processing of FeatureGraph-α. In addition, there will be no lag when using FeatureGraph-β to process an image.

[0188] Furthermore, the specific implementation of a shooting process is described below in combination with the hardware and software structure of the electronic device. Fig. 9 As shown, a shooting process includes:

[0189] S901: In high-pixel shooting mode, a camera application receives an event of starting shooting.

[0190] For example, when a user Figure 1 After the start shooting button 1031 in the preview interface 103 is clicked, the camera application may receive a start shooting event and start shooting in a high-pixel shooting mode.

[0191] S902: The camera application sends a shooting request to the camera service.

[0192] Exemplarily, the shooting request includes streaming information, shooting mode information, etc. The streaming information is used for initialization processing of shooting, such as powering on the image sensor, etc. And the shooting mode information is used to indicate the currently selected shooting mode, such as the shooting mode information is used to indicate the high-pixel shooting mode.

[0193] S903: If it is the first time to shoot in high-pixel shooting mode after the camera is turned on, the camera service allocates a resident buffer 13. For details, please refer to the description of S702.

[0194] After the camera service allocates the resident buffer 13, the resident buffer 13 will exist until the camera is turned off. Therefore, if it is not the first time to use the high-pixel shooting mode after the camera is turned on, the camera service does not need to allocate the resident buffer 13, but can directly call the allocated resident buffer 13 for the JPEG when processing the image using FeatureGraph-α. If it is the first time to use the high-pixel shooting mode after the camera is turned on, it means that the resident buffer 13 is not allocated, so the camera service can allocate the resident buffer 13. For example, the camera service will Fig.10 A resident buffer 13 is allocated in the application framework layer as shown.

[0195] In a specific implementation, the electronic device may record the usage of the high-pixel shooting mode after the camera is turned on this time. For example, the electronic device may record identification 1 to indicate that the usage is: after the camera is turned on this time, an image has been taken using the high-pixel shooting mode. Accordingly, the electronic device may query the recorded usage to determine whether it is the first time that the high-pixel shooting mode is used for shooting after the camera is turned on this time. For example, if the electronic device queries identification 1, it determines that it is not the first time that the high-pixel shooting mode is used for shooting after the camera is turned on this time.

[0196] In another specific implementation, the electronic device can determine whether it is the first time to use the high-pixel shooting mode to shoot after the camera is turned on by querying whether there is a permanent buffer 13. If the electronic device does not query the permanent buffer 13, it is determined that it is the first time to use the high-pixel shooting mode to shoot after the camera is turned on; if the electronic device queries the permanent buffer 13, it is determined that it is not the first time to use the high-pixel shooting mode to shoot after the camera is turned on.

[0197] S904: The camera service sends a shooting request to the FeatureGraph through the interface module and the channel selection management module.

[0198] S905 , FeatureGraph selects a target processing channel according to the brightness environment.

[0199] The target processing channel includes FeatureGraph-α or FeatureGraph-β. If it is a high-brightness environment, the target selection channel is FeatureGraph-α; if it is a low-brightness environment, the target selection channel is FeatureGraph-β.

[0200] S906. FeatureGraph sends indication information of the target processing channel to the image processing module.

[0201] In this way, the image processing module can process the collected image in a corresponding manner based on the instruction information. For details, please refer to the description of S913 and S914 below.

[0202] S907 , if the target processing channel is FeatureGraph-β, and it is the first time that FeatureGraph-β is selected after the camera is turned on, FeatureGraph allocates buffer 1.

[0203] After buffer 1 is allocated, it will exist until the camera is turned off. Therefore, if FeatureGraph-β is not selected for the first time after the camera is turned on, FeatureGraph does not need to allocate buffer 1, but can directly call the allocated buffer 1 to store the JPEG encoding result when the image is processed using FeatureGraph-β. If FeatureGraph-β is selected for the first time after the camera is turned on, it means that buffer 1 is not allocated, so FeatureGraph can allocate resident buffer 13. For example, FeatureGraph will Fig.10 Buffer 1 is allocated in the HAL as shown.

[0204] In a specific implementation, the electronic device may record the selection of FeatureGraph-β after the camera is turned on this time. For example, the electronic device may record identification 2 to indicate that the selection of FeatureGraph-β is: FeatureGraph-β has been selected after the camera is turned on this time. Accordingly, the electronic device may query the recorded selection to determine whether it is the first time that FeatureGraph-β is selected after the camera is turned on this time. For example, if the electronic device queries identification 2, it determines that it is not the first time that FeatureGraph-β is selected after the camera is turned on this time.

[0205] In another specific implementation, the electronic device can determine whether FeatureGraph-β is selected for the first time after the camera is turned on by querying whether there is buffer 1. If the electronic device does not query buffer 1, it is determined that FeatureGraph-β is selected for the first time after the camera is turned on; if the electronic device queries buffer 1, it is determined that it is not the first time FeatureGraph-β is selected after the camera is turned on.

[0206] In actual implementation, there is no strict order between S906 and S907. For example, S907 may be after S906.

[0207] S908 . FeatureGraph sends a shooting request carrying instruction information to the camera abstract device.

[0208] S909: The camera abstract device determines shooting information that matches the target processing channel.

[0209] Among them, the shooting information includes resolution information, frame rate information, camera information, etc.

[0210] S910: The camera abstract device sends a shooting request carrying shooting information to the camera driver.

[0211] S911: The camera driver drives the camera module to collect images based on the shooting information.

[0212] Among them, the camera module includes a camera and an image sensor.

[0213] S912: The camera module transmits the image back to the image processing module through the camera abstract device.

[0214] After receiving the image, the image processing module may acquire the target processing channel based on the instruction information received in S906, and then perform processing using the following S913 or S904.

[0215] S913. If the target processing channel is FeatureGraph-α, the image processing module processes the image using FeatureGraph-α; wherein the image processing module calls the resident buffer 13 for JPEG processing.

[0216] Exemplarily, the image processing module may call Fig.10 The resident buffer 13 allocated in the application framework layer is shown to be used for processing JPEG in images using FeatureGraph-α in the HAL.

[0217] S914. If the target processing channel is FeatureGraph-β, the image processing module processes the image using FeatureGraph-β; wherein the image processing module allocates buffer 2 for JPEG processing, calls buffer 1 after JPEG is completed, writes the encoding result in buffer 2 into buffer 1 and releases buffer 2.

[0218] For example, when using FeatureGraph-β to process JPEG images, the image processing module Fig.10 Buffer 2 is allocated in the HAL layer and used to process JPEG in the image using FeatureGraph-β; and the image processing module calls Fig.10 As shown in the buffer 1 allocated in the HAL layer, the encoding result in the buffer 2 is written into the buffer 1 and the buffer 2 is released.

[0219] S915. The image processing module transmits the processed image back to the camera application through the channel selection management module, the interface module, and the camera service in sequence.

[0220] Finally, the camera app can display the processed image or save it to the gallery.

[0221] Method 2

[0222] Based on the first method mainly for memory optimization when using FeatureGraph-β to process images, the electronic device can also perform memory optimization when using FeatureGraph-α to process images, so that the processing can be completed more smoothly when using FeatureGraph-β to process images.

[0223] In the following description of the second method, the target stage is still taken as an example. In addition, the buffer 1A and the buffer 1 are two different resident buffers, and the sizes of the two can be the same or different. The buffer 2A and the buffer 2 are two different resident buffers, and the sizes of the two can be the same or different.

[0224] See also Fig.11 , before adopting the improved solution (such as Figure 6 Based on the scheme before improvement shown in FIG. 1 , the electronic device can be improved to obtain the following improved method 2: the electronic device completes the JPEG-α processing in buffer 2A, writes the encoding result into buffer 1A, and then releases buffer 2A. The electronic device completes the JPEG-β processing in buffer 2, writes the encoding result into buffer 1, and then releases buffer 2.

[0225] The size of the buffer 2A is substantially the same as that of the resident buffer 13 to ensure that the JPEG-α processing can be completed. For example, the buffer 2A is also approximately 300 MB. The buffer 1A only needs to meet the requirement of storing the encoding result, and the size of the buffer 1A is smaller than that of the buffer 2A. For example, the buffer 1A is approximately 60 MB.

[0226] For buffer 2 and buffer 1, please refer to the previous article Figure 6 The relevant instructions will not be repeated here.

[0227] It can be seen that in the improved method 2, when the electronic device uses FeatureGraph-α to process the image, buffer 1 and buffer 1A may exist at the same time. For example, when FeatureGraph-β is used to process the image first and then FeatureGraph-α is used to process the image, buffer 1 and buffer 1A may exist at the same time. In addition, during the processing of JPEG-α, buffer 2A also exists. The sum of the memory of buffer 1, buffer 1A and buffer 2A is usually less than the sum of the memory of resident buffer 13 and resident buffer 23, that is, the memory occupancy is less, so it is more conducive to the smooth completion of FeatureGraph-α image processing and avoid shooting jams.

[0228] Moreover, when the electronic device uses FeatureGraph-β to process images, buffer 1 and buffer 1A may exist at the same time. In addition, during the processing of JPEG-β, buffer 2 also exists. The sum of the memories of these three is usually smaller than the sum of the memories of resident buffer 13 and resident buffer 23, that is, less memory is occupied, which is more conducive to the smooth completion of FeatureGraph-β image processing and avoids shooting jams.

[0229] Combine the following Fig.12 and Fig.13 , using the complete process of using a camera once to illustrate the specific implementation and effect of the above method 2. Fig.12 In the process, the electronic device first uses FeatureGraph-α to process the image, and then uses FeatureGraph-β to process the image. Fig.13 In the process, the electronic device first processes the image using FeatureGraph-β and then processes the image using FeatureGraph-α.

[0230] The following are respectively described:

[0231] like Fig.12 As shown, the complete process of using the camera once includes:

[0232] S1201. The electronic device turns on the camera.

[0233] In response to a user clicking an application icon of a camera application, the electronic device may turn on the camera.

[0234] S1202: In response to receiving the operation of starting high-pixel shooting for the first time after the camera is turned on, the electronic device allocates a buffer area 1A.

[0235] Similar to the resident buffer 13 mentioned above, the buffer 1A can be used not only for JPEG processing when FeatureGraph-α is used, but also for processing in some processing stages when FeatureGraph-β is used. Therefore, during the first shooting process, the electronic device will allocate the buffer 1A regardless of whether FeatureGraph-α is used to process the image.

[0236] S1203: The electronic device selects FeatureGraph-α based on the brightness environment.

[0237] S1204, the electronic device captures an image and processes the image using FeatureGraph-α; wherein the electronic device allocates buffer 2A for JPEG processing in the first shooting, writes the data in buffer 2A into buffer 1A and releases buffer 2A after JPEG is completed.

[0238] At this time, there is buffer 1A, such as about 60MB, and during the JPEG processing there is also buffer 2A, such as about 300MB, and the total is about 360MB (60MB+300MB), which is about 240MB less than the sum of the memory of resident buffer 13 and resident buffer 23, such as about 600MB, which is more conducive to the electronic device to smoothly complete the processing of FeatureGraph-α without causing jamming.

[0239] S1205: The electronic device ends the first shooting.

[0240] Subsequently, the brightness environment is still maintained at a high brightness environment, such as when shooting in the i1th (1<i1<k1, i1 and k1 are integers) high-pixel shooting mode, the high brightness environment is still maintained, and the electronic device can use the following S1206-S1208 to shoot. Of course, if the brightness environment changes to a low brightness environment after the first shooting, such as a night scene shooting scene or an indoor shooting scene without lights, the following S1206-S1208 can be skipped.

[0241] S1206 . In response to receiving the operation of starting high-pixel shooting for the i1th time after the camera is turned on, the electronic device selects FeatureGraph-α based on the brightness environment, where 1<i1<k1, and i1 and k1 are integers.

[0242] When shooting in the high-pixel shooting mode for the first time, the electronic device has allocated the buffer 1A, and the buffer 1A has not been released. Therefore, when shooting in the high-pixel shooting mode for the i1th time, the electronic device does not need to allocate the buffer 1A.

[0243] S1207, the electronic device collects an image and processes the image using FeatureGraph-α; wherein the electronic device allocates buffer 2A for JPEG processing in the i1th shooting, writes the data in buffer 2A into buffer 1A after JPEG is completed, and releases buffer 2A. For details, please refer to the description of S1204, which will not be repeated here.

[0244] S1208: The electronic device ends the i1th shooting.

[0245] Through the above S1202 - S1208 , it can be known that, by adopting the second method, when FeatureGraph-α is first used to process the image, the electronic device can complete the processing more smoothly and avoid jamming.

[0246] Subsequently, when the brightness environment changes to a low-brightness environment, such as a night scene shooting scene or an indoor shooting scene without lights, the electronic device may execute the following S1209-S1212.

[0247] S1209: In response to receiving the k1th operation of starting high-pixel shooting after the camera is turned on this time, the electronic device selects FeatureGraph-β based on the brightness environment.

[0248] S1210 , the electronic device allocates buffer 1 .

[0249] Before the k1th shooting in high-pixel shooting mode, the electronic device selects FeatureGraph-β based on the brightness environment. This is the first time that FeatureGraph-β is selected. Therefore, the electronic device needs to allocate buffer 1, such as 60MB of resident memory, to store the JPEG encoding results.

[0250] S1211, the electronic device captures an image and processes the image using FeatureGraph-β; wherein the electronic device allocates buffer 2 for processing JPEG in the k1th shooting, writes the data in buffer 2 into buffer 1 and releases buffer 2 after JPEG is completed.

[0251] At this time, buffer 1A and buffer 1 exist simultaneously. Additionally, during the processing of JPEG, buffer 2 also exists, with a total of approximately 420 MB (60 MB + 60 MB + 300 MB). This is approximately 180 MB less than the sum of the memory of resident buffer 13 and resident buffer 23, which is approximately 600 MB. This is more conducive to the electronic device smoothly completing the processing of FeatureGraph-β without causing lag.

[0252] S1212. The electronic device ends the k1-th shooting.

[0253] Subsequently, the brightness environment remains in a low-brightness environment all the time. For example, when shooting in the high-pixel shooting mode for the j1-th time (k1 < j1 < k2, where both j1 and k2 are integers), the low-brightness environment is still maintained. The electronic device can then use the following S713 - S715 for shooting. Of course, if the brightness environment changes to a high-brightness environment after ending the j1-th shooting, the following S1213 - S1215 can be skipped.

[0254] S1213. In response to the j1-th time of receiving the operation to start high-pixel shooting after the camera is turned on this time, the electronic device selects FeatureGraph-β based on the brightness environment, where k1 < j1 < k2 and both j1 and k2 are integers.

[0255] It should be noted that this is not the first time to select FeatureGraph-β. Previously, when FeatureGraph-β was selected for the first time, the electronic device had already allocated buffer 1 and buffer 1 was not released. Then, when shooting in the high-pixel shooting mode for the j1-th time, the electronic device does not need to allocate buffer 1.

[0256] S1214. The electronic device captures an image and processes the image using FeatureGraph-β. Among them, the electronic device allocates buffer 2 for the JPEG processing during the j1-th shooting, and after JPEG is completed, writes the data in buffer 2 into buffer 1 and releases buffer 2. For specific details, please refer to the description of S1211 and will not be elaborated here.

[0257] S1215. The electronic device ends the j1-th shooting.

[0258] From the above S1209 - S1215, it can be seen that by using Method 2, when FeatureGraph-α is used to process the image first and then FeatureGraph-β is used to process the image, no lag will be caused.

[0259] Subsequently, when the electronic device uses the high-pixel shooting mode to shoot again (such as the k2th time and thereafter), the processing channel is also selected based on the brightness environment. If FeatureGraph-α is selected, the electronic device uses steps similar to the aforementioned S1207-S1208 to complete the shooting; if FeatureGraph-β is selected, the electronic device uses steps similar to the aforementioned S1214-S1215 to complete the shooting. They are not listed here one by one.

[0260] S1216. The electronic device turns off the camera and releases the resident memory.

[0261] The resident memory includes buffer 1A and buffer 1.

[0262] It should be noted that Fig.12 For the parts not described in detail in the process shown, please refer to the previous article Figure 7 The description of the corresponding steps in will not be repeated here.

[0263] like Fig.13 As shown, the complete process of using the camera once includes:

[0264] S1301. The electronic device turns on the camera.

[0265] S1302: In response to receiving the operation of starting high-pixel shooting for the first time after the camera is turned on this time, the electronic device allocates a buffer area 1A.

[0266] S1303: The electronic device selects FeatureGraph-β based on the brightness environment.

[0267] S1304. The electronic device allocates buffer 1.

[0268] That is, if FeatureGraph-β is selected for the first shooting, the electronic device will allocate not only buffer 1A but also buffer 1.

[0269] S1305, the electronic device captures an image and processes the image using FeatureGraph-β; wherein the electronic device allocates buffer 2 for JPEG processing in the first shooting, writes the data in buffer 2 into buffer 1 and releases buffer 2 after JPEG is completed.

[0270] At this time, buffer 1A and buffer 1 are both about 120MB (60MB+60MB), and, in the process of JPEG processing, there is also buffer 2, such as about 300MB, so the total is about 420MB (120MB+300MB). Compared with the sum of the memory of resident buffer 13 and resident buffer 23, such as about 600MB, it takes up about 180MB less, which is more conducive to the electronic device to smoothly complete the processing of FeatureGraph-β without causing jamming.

[0271] S1306: The electronic device ends the first shooting.

[0272] Subsequently, the brightness environment still maintains a low brightness environment, such as when shooting in the i2th (1<i2<k3, i2 and k3 are integers) high-pixel shooting mode, the low brightness environment is still maintained, and the electronic device can use the following S1307-S1309 to shoot. Of course, if the brightness environment changes to a high brightness environment after the first shooting, the following S1307-S1309 can be skipped.

[0273] S1307 . In response to receiving the operation of starting high-pixel shooting for the i2th time after the camera is turned on this time, the electronic device selects FeatureGraph-β based on the brightness environment, where 1<i2<k3, and i2 and k3 are integers.

[0274] Previously, when FeatureGraph-β was selected for the first time, the electronic device had allocated buffer 1, and buffer 1 was not released. Therefore, when shooting in high-pixel shooting mode for the i2th time, the electronic device does not need to allocate buffer 1 even if FeatureGraph-β is selected.

[0275] S1308, the electronic device captures an image and processes the image using FeatureGraph-β; wherein the electronic device allocates buffer 2 for JPEG processing in the first shot, writes the data in buffer 2 into buffer 1 after JPEG is completed, and releases buffer 2. For details, please refer to the description of S1305, which will not be repeated here.

[0276] S1309: The electronic device ends the i2th shooting.

[0277] Through the above S1302-S1309, it can be known that when the second method is adopted, when FeatureGraph-β is used to process the image first, the processing can be completed smoothly without causing any lag.

[0278] S1310: In response to receiving the operation of starting high-pixel shooting for the k3th time after the camera is turned on this time, the electronic device selects FeatureGraph-α based on the brightness environment.

[0279] It should be noted that, although this is the first time FeatureGraph-α is selected, buffer 1A has been allocated when the high-pixel shooting mode was used for the first time, and buffer 1A has not been released. Therefore, the electronic device does not need to allocate buffer 1A this time.

[0280] S1311, the electronic device captures an image and processes the image using FeatureGraph-α; wherein the electronic device allocates buffer 2A for JPEG processing in the k3th shooting, writes the data in buffer 2A into buffer 1A and releases buffer 2A after JPEG is completed.

[0281] At this time, buffer 1A and buffer 1 exist at the same time, such as a total of about 120MB (60MB+60MB), and, during the JPEG processing, there is also buffer 2A, such as about 300MB, so the total is about 420MB (120MB+300MB). Compared with the sum of the memory of resident buffer 13 and resident buffer 23, such as about 600MB, it takes up about 180MB less, which is more conducive to the electronic device to smoothly complete the processing of FeatureGraph-α without causing jamming.

[0282] S1312: The electronic device finishes the k3th shooting.

[0283] Through the above S1310 - S1312 , it can be known that by adopting the second method, in the case where FeatureGraph-β is used to process the image first and FeatureGraph-α is used to process the image later, the processing can also be completed smoothly without causing any lag.

[0284] Subsequently, when the electronic device uses the high-pixel shooting mode to shoot again (such as the k3+1th time and thereafter), the processing channel is also selected based on the brightness environment. If FeatureGraph-α is selected, the electronic device uses steps similar to the aforementioned S1311-S1312 to complete the shooting; if FeatureGraph-β is selected, the electronic device uses steps similar to the aforementioned S1308-S1309 to complete the shooting. They are not listed here one by one.

[0285] S1313. The electronic device turns off the camera and releases the resident memory.

[0286] Likewise, the resident memory includes buffer 1A and buffer 1.

[0287] It should be noted that Fig.13 For the parts not described in detail in the process shown, please refer to the previous article Fig.12 The description of the corresponding steps in will not be repeated here.

[0288] Furthermore, the specific implementation of a shooting process is described below in combination with the hardware and software structure of the electronic device. Fig.14 As shown: Fig. 9 The main difference between the processes shown is S1403 and S1413. Specifically, a shooting process includes:

[0289] S1401: In high-pixel shooting mode, the camera application receives an event of starting shooting.

[0290] S1402: The camera application sends a shooting request to the camera service.

[0291] S1403: If it is the first time to shoot in high-pixel shooting mode after the camera is turned on, the camera service allocates buffer 1A.

[0292] After the camera service allocates buffer 1A, buffer 1A will exist until the camera is turned off. Therefore, if it is not the first time to use high-pixel shooting mode after the camera is turned on, the camera service does not need to allocate buffer 1A, but can directly call the allocated buffer 1A for JPEG when processing images using FeatureGraph-α. If it is the first time to use high-pixel shooting mode after the camera is turned on, it means that buffer 1A is not allocated, so the camera service can allocate buffer 1A. For example, the camera service will Fig.10 Buffer 1A is allocated in the application framework layer as shown.

[0293] S1404: The camera service sends a shooting request to the FeatureGraph through the interface module and the channel selection management module.

[0294] S1405, FeatureGraph selects a target processing channel according to the brightness environment.

[0295] S1406. FeatureGraph sends indication information of the target processing channel to the image processing module.

[0296] S1407 , if the target processing channel is FeatureGraph-β, and it is the first time that FeatureGraph-β is selected after the camera is turned on, FeatureGraph allocates buffer 1.

[0297] S1408 . FeatureGraph sends a shooting request carrying instruction information to the camera abstract device.

[0298] S1409: The camera abstract device determines shooting information that matches the target processing channel.

[0299] S1410: The camera abstract device sends a shooting request carrying shooting information to the camera driver.

[0300] S1411. The camera driver drives the camera module to capture images based on the shooting information.

[0301] S1412. The camera module transmits the image back to the image processing module through the camera abstract device.

[0302] S1413. If the target processing channel is FeatureGraph-α, the image processing module processes the image using FeatureGraph-α; wherein the image processing module allocates buffer 2A for JPEG processing, calls buffer 1A after JPEG is completed, writes the encoding result in buffer 2A into buffer 1A and releases buffer 2A.

[0303] For example, when using FeatureGraph-α to process JPEG images, the image processing module Fig.15 The HAL layer shown allocates buffer 2A and is used to process JPEG in the image using FeatureGraph-α; and the image processing module calls Fig.15 As shown, the buffer 1A allocated in the application framework layer writes the encoding result in the buffer 2A into the buffer 1A and releases the buffer 2A.

[0304] S1414. If the target processing channel is FeatureGraph-β, the image processing module uses FeatureGraph-β to process the image; wherein the image processing module allocates buffer 2 for JPEG processing, calls buffer 1 after JPEG is completed, writes the encoding result in buffer 2 into buffer 1 and releases buffer 2.

[0305] S1415. The image processing module transmits the processed image back to the camera application through the channel selection management module, the interface module, and the camera service in sequence.

[0306] It should be noted that Fig.14 For the parts not described in detail, please refer to the previous article Fig. 9 The relevant instructions in will not be repeated here.

[0307] In the foregoing, the two processing channels of the high-pixel shooting mode are mainly used as examples to illustrate the embodiments of the present application, but the present application is not limited to this in practice. Fig.16 In the target scene of the frame shown.

[0308] like Fig.16As shown, the target scene includes at least two processing modules, namely, processing module A (such as the aforementioned FeatureGraph-α) and processing module B (such as the aforementioned FeatureGraph-β). Processing module A and processing module B can be switched to run, but not run at the same time. Exemplarily, the above-mentioned processing module A and processing module B can be two different shooting modes. Alternatively, processing module A and processing module B can be two processing channels in the portrait shooting mode for processing different light effects.

[0309] Among them, the load of processing module A is relatively high, and the operation process of processing module B requires a large memory, but the storage of the processing results of processing module B only requires a small memory.

[0310] It can be seen that in the above Fig.16 In the target scenario, if a large amount of resident memory is allocated for the processing process of the B processing module when the B processing module is running, then when the A processing module is running later, there may be a high memory usage, which may cause the high-load A processing module to fail to run normally, such as freezing during operation.

[0311] Therefore, for the above Fig.16 In the target scenario, when the B processing module is running, the electronic device can allocate a larger temporary memory for the operation of the B processing module, and after the processing of the B processing module is completed, the processing result is written into a smaller resident memory, and then the temporary memory is released. In this way, when the A processing module is running later, there is no large memory occupation for the B processing module, which is more conducive to the smooth operation of the A processing module.

[0312] Further, such as Fig.16 As shown, the B processing module includes multiple processing stages such as stage B1, stage B2, and stage B3 (such as the aforementioned JPEG-β), and the processing process of stage B3 mainly requires a large memory, that is, stage B3 is the target stage. Therefore, the electronic device can allocate a larger temporary memory for stage B3, write the processing result of stage B3 into a smaller resident memory, and then release the temporary memory.

[0313] like Fig.17 As shown, the electronic device can allocate a small resident memory BB (such as the aforementioned buffer 1) to store the processing results of the above-mentioned stage B3 in the B processing module; and the electronic device can allocate a large temporary memory bb (such as the aforementioned buffer 2) for the processing process of the above-mentioned stage B3 in the B processing module.

[0314] Similarly, if the operation process of processing module A also requires a large memory, and the storage of the processing results only requires a small memory. When processing module A is running, the electronic device can also allocate a large temporary memory for the operation of processing module A, and after processing module A completes the processing, write the processing results into a smaller resident memory, and then release the temporary memory. In this way, when processing module B is running later, the memory occupancy will be less, which is more conducive to the operation of processing module B.

[0315] Similar, such as Fig.16 As shown, the A processing module includes multiple processing stages such as stage A1, stage A2, and stage A3 (such as the aforementioned JPEG-α), and the processing process of stage A3 mainly requires a large memory. Therefore, the electronic device can allocate a larger temporary memory for stage A3, write the processing result of stage A3 into a smaller resident memory, and then release the temporary memory.

[0316] like Fig.17 As shown, the electronic device can allocate a small resident memory AA (such as the aforementioned buffer 1A) to store the processing results of the above-mentioned stage A3 in the A processing module; and the electronic device can allocate a large temporary memory aa (such as the aforementioned buffer 2A) for the processing process of the above-mentioned stage A3 in the A processing module.

[0317] The present application also provides an electronic device, which may include: a display screen, a memory, and one or more processors (such as a CPU, a GPU, an NPU, etc.). The display screen, the memory, and the processor are coupled. The memory is used to store computer program code, and the computer program code includes computer instructions. When the processor executes the computer instructions, the electronic device may perform each function or step performed by the device in the above method embodiment.

[0318] The embodiment of the present application also provides a chip system, which includes at least one processor and at least one interface circuit. The processor and the interface circuit can be interconnected by lines. For example, the interface circuit can be used to receive signals from other devices (such as a memory of an electronic device). For another example, the interface circuit can be used to send signals to other devices (such as a processor). Exemplarily, the interface circuit can read instructions stored in the memory and send the instructions to the processor. When the instructions are executed by the processor, the electronic device can perform the various steps in the above embodiments. Of course, the chip system can also include other discrete devices, which are not specifically limited in the embodiment of the present application.

[0319] This embodiment further provides a computer storage medium, in which computer instructions are stored. When the computer instructions are executed on an electronic device, the electronic device executes the above-mentioned related method steps to implement the image processing method in the above-mentioned embodiment.

[0320] This embodiment further provides a computer program product. When the computer program product is run on a computer, the computer is enabled to execute the above-mentioned related steps to implement the image processing method in the above-mentioned embodiment.

[0321] In addition, an embodiment of the present application also provides a device, which can specifically be a chip, component or module, and the device may include a connected processor and memory; wherein the memory is used to store computer-executable instructions, and when the device is running, the processor can execute the computer-executable instructions stored in the memory so that the chip executes the image processing method in the above-mentioned method embodiments.

[0322] Among them, the electronic device, computer storage medium, computer program product or chip provided in this embodiment is used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above and will not be repeated here.

[0323] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0324] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0325] The unit described as a separate component may or may not be physically separated, and the component shown as a unit may be one physical unit or multiple physical units, that is, it may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiment.

[0326] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0327] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of each embodiment method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store program code.

[0328] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present application and are not intended to limit it. Although the present application has been described in detail with reference to the preferred embodiments, a person of ordinary skill in the art should understand that the technical solution of the present application may be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present application.

Claims

1. A shooting method, characterized in that: Applied to an electronic device, wherein a camera application is installed in the electronic device, wherein the camera application provides a first shooting mode, wherein an image processing method corresponding to the first shooting mode includes a first image processing method and a second image processing method, and wherein a brightness environment corresponding to the first image processing method is lower than a brightness environment corresponding to the second image processing method; the method comprises: Open the Camera app; In response to a selection operation of the first shooting mode, displaying a shooting preview interface; In response to a first shooting operation in the preview interface, allocating a running memory corresponding to the second image processing mode; After the first image processing method is determined to be used according to the brightness environment, a first operating memory is allocated, and image processing of a first processing stage is performed on the captured data according to the first image processing method in the first operating memory to obtain a first processing result; Writing the first processing result into a second running memory, and releasing the first running memory; the second running memory is the running memory allocated when the first image processing method is first determined to be adopted after the first shooting mode is selected, and the first running memory is larger than the second running memory; A first shooting result is obtained based on the first processing result in the second running memory.

2. The method according to claim 1, characterized in that The first processing stage includes a JPEG encoding stage, and the first processing result includes an encoding result.

3. The method according to claim 1 or 2, characterized in that: After obtaining the first shooting result based on the first processing result in the second running memory, the method further includes: In response to closing the camera application, the second running memory is released.

4. The method according to any one of claims 1 to 3, characterized in that When the running memory corresponding to the second image processing mode is a fourth running memory, the method further includes: After it is determined to use the second image processing method according to the brightness environment, a third operating memory is allocated, and the image processing of the first processing stage is performed on the captured data according to the second image processing method in the third operating memory to obtain a second processing result; Writing the second processing result into a fourth operation memory, releasing the third operation memory, the third operation memory being larger than the fourth operation memory; A second shooting result is obtained based on the second processing result in the fourth running memory.

5. The method according to any one of claims 1 to 3, characterized in that: When the running memory corresponding to the second image processing mode is a fifth running memory, the method further includes: After it is determined to adopt the second image processing method according to the brightness environment, the image processing of the first processing stage is performed on the captured data according to the second image processing method in the fifth running memory to obtain a third processing result, and the third processing result is stored in the fifth running memory; A third shooting result is obtained based on the third processing result in the fifth running memory.

6. The method according to claim 4 or 5, characterized in that: The determining to use the first image processing method according to the brightness environment includes: In response to the mth shooting operation in the preview interface, determining to adopt the first image processing mode according to the brightness environment; m≥1, m is an integer; The determining to use the second image processing method according to the brightness environment includes: In response to the nth shooting operation in the preview interface, the second image processing method is determined to be adopted according to the brightness environment; n≥1, n≠m, and n is an integer.

7. The method according to any one of claims 1 to 6, characterized in that The electronic device includes a camera service, a channel selection module and an image processing module; After determining to adopt the first image processing method according to the brightness environment, allocating a first running memory, performing image processing of a first processing stage on the captured data according to the first image processing method in the first running memory to obtain a first processing result; Writing the first processing result into the second running memory and releasing the first running memory includes: In response to a shooting operation in the preview interface, the camera application sends a shooting request to the camera service; The camera service sends the shooting request to the channel selection module; The channel selection module selects the first image processing mode that matches the current brightness environment of the electronic device; The channel selection module sends instruction information of the first image processing mode to the image processing module; The image processing module allocates the first running memory based on the indication information, and uses the first image processing method to perform a first processing stage of image processing on the captured data in the first running memory, writes the first processing result into the second running memory, and releases the first running memory.

8. The method according to claim 7, characterized in that Before writing the first processing result into the second running memory, the method further includes: The channel selection module allocates the second running memory; The image processing module writes the processing result into the second running memory, including: The image processing module calls the second running memory and writes the processing result into the second running memory.

9. The method according to claim 7 or 8, characterized in that: The allocating the operating memory corresponding to the second image processing method includes: The camera service allocates a running memory corresponding to the second image processing mode.

10. A chip system, characterized in that: The chip system is applied to an electronic device including a display screen and a memory, and the chip system includes an interface circuit and a processor; the interface circuit and the processor are interconnected through lines; the interface circuit is used to receive signals from the memory of the electronic device and send signals to the processor, and the signals include computer instructions stored in the memory; when the processor executes the computer instructions, the electronic device executes the method described in any one of claims 1-9.

11. An electronic device, characterized in that: include: A display screen, one or more processors, and one or more memories; the one or more processors are coupled to the display screen and the one or more memories; the one or more memories are used to store computer program codes, the computer program codes include computer instructions, and when the one or more processors execute the computer instructions, the electronic device executes the method described in any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that: The method comprises computer instructions, which, when executed on an electronic device, cause the electronic device to execute the method as claimed in any one of claims 1 to 9.

Images