An image processing method, an image processing chip, and an electronic device.

Abstract

This application discloses an image processing method, an image processing chip, and an electronic device. The method acquires operating status information by monitoring the operating status of the image processor, and determines the optimal operating mode matching the current operating status information. When the image processor operates in a first operating mode, it performs frame-by-frame processing on the image data. When the image processor operates in a second operating mode, it performs downsampling processing and frame-by-frame processing on the image data. This achieves adaptive adjustment of the image processor's operating mode, thereby optimizing the image processor's power consumption and solving the high power consumption problem that occurs when processing high frame rate images for extended periods in a single operating mode, thus achieving a balance between image processor power consumption and image processing performance.

Description

Technical Field

[0001] This application relates to image processing technology, and more particularly to an image processing method, an image processing chip, and an electronic device. Background Technology

[0002] With the development of image processing technology, high-resolution and high-frame-rate displays have become commonplace in electronic devices. However, this has also brought about the problem of high power consumption by image processors during image processing. Currently, electronic devices use image processors to preprocess signals, but the high power consumption problem becomes even more severe when processing high-resolution and high-frame-rate images for extended periods. Summary of the Invention

[0003] To address the aforementioned technical problems, embodiments of this application aim to provide an image processing method, an image processing chip, and an electronic device.

[0004] The technical solution of this application is implemented as follows:

[0005] Firstly, an image processing method is provided, including:

[0006] Obtain the operating status information of the image processor;

[0007] Based on the operating status information, it is determined whether the image processor is operating in a first working mode or a second working mode. When the image processor is operating in the first working mode, frame-by-frame processing is performed on the image data. When the image processor is operating in the second working mode, downsampling processing and frame-by-frame processing are performed on the image data.

[0008] Secondly, an image processing chip is provided, comprising: an image processor and an application processor coupled to the image processor, the image processor being used to process received image data, the image processor having a first operating mode and a second operating mode.

[0009] When the image processor is running in the first working mode, the image processor performs frame-by-frame processing on the image data to obtain the first image data;

[0010] When the image processor is running in the second working mode, the image processor performs downsampling processing on the image data and then performs frame-by-frame processing to obtain the first image data;

[0011] The application processor is used to process the first image data.

[0012] Thirdly, an electronic device is provided, the electronic device comprising:

[0013] An image acquisition device used to collect image data;

[0014] An image processor is used to execute the method steps of the first aspect described above, which are implemented by the image processor.

[0015] This application provides an image processing method, an image processing chip, and an electronic device. The method acquires operating status information by monitoring the operating status of the image processor, and determines the optimal operating mode matching the current operating status information. When the image processor operates in a first operating mode, it performs frame-by-frame processing on the image data. When the image processor operates in a second operating mode, it performs downsampling processing and frame-by-frame processing on the image data. This achieves adaptive adjustment of the image processor's operating mode, thereby optimizing the image processor's power consumption and solving the high power consumption problem that occurs when processing high frame rate images for extended periods in a single operating mode, thus achieving a balance between image processor power consumption and image processing performance. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the first process of the image processing method in the embodiments of this application;

[0017] Figure 2 This is a schematic diagram of the second process of the image processing method in the embodiments of this application;

[0018] Figure 3 This is a schematic diagram illustrating the classification of image processor operating modes in the embodiments of this application;

[0019] Figure 4 This is a schematic diagram of the third process of the image processing method in the embodiments of this application;

[0020] Figure 5 This is a schematic diagram of the frame-interval processing flow in the embodiments of this application;

[0021] Figure 6 This is a schematic diagram of the fourth process of the image processing method in the embodiments of this application;

[0022] Figure 7 This is a schematic diagram of the downsampling and upsampling processes in the embodiments of this application;

[0023] Figure 8 This is a schematic diagram of the fifth process of the image processing method in the embodiments of this application;

[0024] Figure 9 This is a schematic diagram of the composition structure of the image processing chip in the embodiments of this application;

[0025] Figure 10 This is a schematic diagram of the composition structure of the electronic device in the embodiments of this application. Detailed Implementation

[0026] In order to gain a more detailed understanding of the features and technical content of the embodiments of this application, the implementation of the embodiments of this application will be described in detail below with reference to the accompanying drawings. The accompanying drawings are for reference and illustration only and are not intended to limit the embodiments of this application.

[0027] This application provides an image processing method for adaptively adjusting the operating mode of an image processor. The electronic device includes an application processor (AP) and an image signal processor (ISP), which can also be called a previous image signal processor (Pre-ISP). The image processor performs front-end image processing operations on the raw image data acquired by the image acquisition device, while the application processor performs back-end image processing operations.

[0028] The electronic devices described in this application have shooting capabilities and may include devices such as mobile phones, tablets, laptops, handheld computers, personal digital assistants (PDAs), portable media players (PMPs), wearable devices, cameras, and smart cars.

[0029] The following provides a detailed example of an image processing method provided by an embodiment of this application. Figure 1 This is a schematic diagram of the first process of the image processing method in the embodiments of this application, such as... Figure 1 As shown, the method may specifically include:

[0030] Step 101: Obtain the image processor's operating status information;

[0031] Here, the operating status information is used to characterize the power consumption of the image processor, and the optimal operating mode matching the current power consumption of the image processor is determined based on the operating status information. For example, the operating status information includes at least one of the following: the temperature of the image processor, and the power consumption of the image processor within a preset operating time. The image processor includes a temperature detection device and a power consumption detection device, wherein the temperature detection device is used to detect the temperature, and the power consumption detection device is used to detect the power consumption.

[0032] Temperature can be correlated with the power consumption of an image processor; higher temperatures indicate higher power consumption. The preset operating times mentioned above can be 30 seconds, 60 seconds, 90 seconds, 120 seconds, 5 minutes, etc.

[0033] Step 102: Determine whether the image processor is running in a first working mode or a second working mode based on the running status information. When the image processor is running in the first working mode, the image data is processed by frame skipping. When the image processor is running in the second working mode, the image data is processed by downsampling and frame skipping.

[0034] The image processor has at least a first operating mode and a second operating mode, and the image sensor operates in different operating modes under different power consumption conditions.

[0035] The first and second operating modes can be referred to as low-power modes. The power consumption of the image processor running in the first operating mode is greater than that running in the second operating mode. Low-power modes can be understood as the ISP simplifying the processing of input image data. Different low-power modes employ different simplification strategies by the ISP to process the image data. In this embodiment, in low-power mode, the ISP performs downsampling and frame skipping processing on high frame rate and high resolution video to reduce the amount of data processed.

[0036] For example, in some embodiments, the method further includes: pre-setting multiple operating conditions, with different operating conditions corresponding to different working modes; correspondingly, the operating conditions include at least one of the following: the temperature of the image processor is within a preset temperature range; the power consumption of the image processor is within a preset power consumption range. Different operating conditions correspond to different temperature ranges and different power consumption ranges.

[0037] From the preset operating conditions, the target operating conditions that the operating status information satisfies are determined; when the target operating condition is a first operating condition, the image processor is determined to be operating in the first working mode; when the target operating condition is a second operating condition, the image processor is determined to be operating in the second working mode.

[0038] For example, in some embodiments, the operating mode also includes a high-performance mode, where the power consumption of the image processor running in any low-power mode is less than that running in the high-performance mode. High-performance mode can be understood as not making any adjustments to the image processor's processing procedure; the image processor is determined to be running in high-performance mode based on the operating status information, and the image processor can process the input image data normally. In practical applications, when the image sensor's current power consumption is determined to be low based on the operating status information, high-performance mode is used to achieve high-quality image processing results.

[0039] By adopting the above technical solution, the image processor's working mode can be adaptively adjusted, thereby optimizing the image processor's power consumption, solving the high power consumption problem that occurs when processing high frame rate images for a long time in a single working mode, and achieving the goal of balancing the image processor's power consumption and image processing effect.

[0040] Based on the above embodiments, the image processing method will be further illustrated below with examples. Figure 2 This is a schematic diagram of the second process of the image processing method in the embodiments of this application, as shown below. Figure 2 As shown, the processing method specifically includes:

[0041] Step 201: Obtain the operating status information of the image processor;

[0042] Step 202: Determine the target operating conditions that the operating status information satisfies from the preset operating conditions;

[0043] For example, in some embodiments, the method further includes: pre-setting multiple operating conditions, with different operating conditions corresponding to different working modes;

[0044] Accordingly, the operating conditions include at least one of the following: the temperature of the image processor is within a preset temperature range; the power consumption of the image processor is within a preset power consumption range. Different operating conditions correspond to different temperature ranges and different power consumption ranges. The temperature threshold within the temperature range can be preset based on experimental data during the product development stage, and the power consumption threshold within the power consumption range can be preset based on experimental data during the product development stage. The temperature range and power consumption range include upper and / or lower limits.

[0045] For example, the operating status information includes at least one of the following: the temperature of the image processor and the power consumption of the image processor within a preset operating time. Correspondingly, the operating status information includes the temperature and power consumption of the image processor (ISP), and the first operating condition corresponding to the first operating mode includes: the temperature being within a first temperature range, or the power consumption being within a first power consumption range. Specifically, if the temperature is within the first temperature range, or the power consumption is within the first power consumption range, the operating status information is determined to meet the first operating condition; if the temperature is outside the first temperature range and the power consumption is outside the first power consumption range, the operating status information is determined not to meet the first operating condition. That is, when either the temperature or the power consumption is within a preset range, the first operating condition is determined to be met; otherwise, the first operating condition is determined not to be met.

[0046] In other embodiments, the first operating condition includes: the temperature being within a first temperature range and the power consumption being within a first power consumption range. That is, when both the temperature and power consumption of the ISP simultaneously meet the first operating condition, it is determined that the operating status information meets the first operating condition; otherwise, it is determined that the operating status information does not meet the first operating condition.

[0047] In other embodiments, the operating status information includes only the temperature of the ISP, and the first operating condition includes: the temperature is within a first temperature range; or the operating status information includes only the power consumption of the ISP, and the first operating condition includes: the power consumption is within a first power consumption range.

[0048] Step 203: When the target operating condition is the first operating condition, determine that the image processor is running in the first working mode;

[0049] When the image processor is running in the first working mode, it performs frame-by-frame processing on the image data.

[0050] Step 204: When the target operating condition is the second operating condition, determine that the image processor is running in the second working mode;

[0051] When the image processor is running in the second working mode, it performs downsampling and frame skipping processing on the image data.

[0052] Referring to the first operating condition, the second operating condition corresponds to the second temperature range and the second power consumption range.

[0053] Step 205: When the target operating condition is the third operating condition, determine that the image processor is operating in the third working mode.

[0054] When the image processor is running in the third working mode, some modules in the image processor are set to bypass mode.

[0055] Referring to the first operating condition, the third operating condition corresponds to the third temperature range and the third power consumption range.

[0056] For example, in some embodiments, the operating mode also includes a high-performance mode. The high-performance mode has corresponding high-performance operating conditions, and the power consumption of the image processor running in any low-power mode is less than the power consumption running in the high-performance mode. The high-performance mode can be understood as not making any adjustments to the image processor's processing procedure. Based on the operating status information, it is determined that the image processor is running in high-performance mode, and the image processor can process the input image data normally. In practical applications, when the operating status information determines that the image sensor's current power consumption is low, running the high-performance mode achieves high-quality image processing results.

[0057] For example, Figure 3 This is a schematic diagram illustrating the classification of image processor operating modes in embodiments of this application, such as... Figure 3As shown, by setting three thresholds—threshold 1, threshold 2, and threshold 3—the image processor is divided into four operating modes: high-performance mode, first operating mode, second operating mode, and third operating mode. The power consumption of the image processor in these four operating modes decreases sequentially. The first, second, and third operating modes can be referred to as low-power modes.

[0058] The control device determines the optimal operating mode of the image processor based on the image processor's operating status information. For example, the operating status information includes temperature and power consumption. Each threshold includes a temperature threshold and a power consumption threshold. When the temperature and power consumption are less than threshold 1, the image processor is controlled to enter a high-performance mode. When the temperature or power consumption is greater than or equal to threshold 1 and both are less than threshold 2, the image processor is controlled to enter a first operating mode. When the temperature or power consumption is greater than or equal to threshold 2 and both are less than threshold 3, the image processor is controlled to enter a second operating mode. When the temperature or power consumption is greater than or equal to threshold 3, the image processor is controlled to enter a third operating mode.

[0059] Here, threshold 1, threshold 2 and threshold 3 are used to set the operating conditions corresponding to different working modes. The operating status information is compared with the operating conditions to determine the target operating conditions that the operating status information meets, thereby determining the target working mode corresponding to the target operating conditions.

[0060] The control device can be part of the image processor, meaning that the control device inside the image processor controls the switching of the working mode. The control device can be part of the application processor or other processor coupled to the image processor.

[0061] It should be noted that when the image processor also includes a fourth operating mode, the fourth operating mode corresponds to a fourth operating condition, which in turn corresponds to a fourth temperature range and a fourth power consumption range. Here, the fourth operating mode refers to the fact that the image processor may include other operating modes, not to limit the number of operating modes.

[0062] It should be noted that this application uses the terms first, second, third, etc., to describe various types of information, but this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another, and are not necessarily used to describe a specific order or sequence.

[0063] By adopting the above technical solution, the image processor's working mode can be adaptively adjusted, thereby optimizing the image processor's power consumption, solving the high power consumption problem that occurs when processing high frame rate images for a long time in a single working mode, and achieving the goal of balancing the image processor's power consumption and image processing effect.

[0064] Based on the above embodiments, the image processing method of the ISP when operating in the first working mode will be illustrated below. Figure 4This is a schematic diagram of the third process of the image processing method in the embodiments of this application, as shown below. Figure 4 As shown, when the ISP operates in the first working mode, the image processing method specifically includes:

[0065] Step 401: Obtain the image processor's operating status information;

[0066] Step 402: Determine that the image processor is running in the first working mode based on the running status information;

[0067] by Figure 3 For example, when the ISP's operating status information is detected to be between threshold 1 and threshold 2, the optimal working mode of the ISP is determined to be the first working mode.

[0068] Step 403: Perform frame-by-frame processing on the image data to obtain the first image data and the 3A statistical data of the processed image;

[0069] Here, the image data refers to the raw image data acquired by the image acquisition device.

[0070] For example, during inter-frame processing, the interval can be one frame or two frames. Depending on the algorithm requirements, the 3A statistics of the unprocessed image can be replaced with the 3A statistics of the preceding and following processed images, without needing to perform frame-by-frame statistics or calculations, thus reducing the pressure on the ISP. However, to improve video image quality, the AP side compensates for the unprocessed image. For example, in Lens Shading Correction (LSC) processing, only the actual LSC processing is performed on the inter-frame, without calculating a checklist based on that image. The checklist information used for LSC processing on the inter-frame can be the checklist information of the previous adjacent frame, thereby ensuring the effectiveness of the LSC processing.

[0071] Step 404: Send the first image data and the 3A statistical data to the application processor so that the application processor can perform image compensation on the unprocessed image in the first image data using the 3A statistical data of the processed image to obtain the second image data.

[0072] It should be noted that if image compensation is not performed after the ISP performs frame skipping, the video quality will be affected. Therefore, when the AP determines that the ISP is operating in the first working mode, the AP will perform corresponding compensation processing.

[0073] For example, in some embodiments, the first image data and the 3A statistical data are sent to the application processor so that the application processor can perform image compensation processing on the unprocessed image of the current frame using the 3A statistical data of the processed image of the previous frame; or, the first image data and the 3A statistical data are sent to the application processor so that the application processor can perform weighted processing on the 3A statistical data of the processed image of the previous frame and the processed image of the next frame to obtain weighted 3A statistical data; and perform image compensation processing on the unprocessed image of the current frame using the weighted 3A statistical data.

[0074] In other words, when the AP side compensates for the unprocessed image in the current frame, it can use the 3A statistical data of the processed image in the previous frame for compensation. Alternatively, to improve the accuracy of compensation, it can first perform weighted processing on the 3A statistical data of the processed image in the previous frame and the processed image in the next frame, and then use the weighted 3A statistical data for compensation.

[0075] For example, during weighted processing, the weight of the 3A statistics of the processed image in the previous frame is higher than that in the subsequent frame.

[0076] For example, Figure 5 This is a schematic diagram of the frame-interval processing flow in an embodiment of this application, such as... Figure 5 As shown,

[0077] The image data are: D1, D2, D3, D4, D5, D6, D7...

[0078] After image data enters the ISP, all data is first processed by the front-end module (FE), and then by the back-end module (BE) for frame-interval processing. The data processed by frame-interval processing is... Figure 4 The gray blocks (D1, D3, D5, D7) and the data that has not been processed by the BE module (D2, D4, D6) are sent directly to the AP side;

[0079] The 3A statistics of the data (D1, D3, D5, D7) processed by the ISP in every other frame are transmitted to the AP side through the Mobile Industry Processor Interface (MIPI). On the AP side, through data sharing, the 3A statistics of adjacent frames are shared by unprocessed images, so that the ISP can perform frame-by-frame processing, reduce the read and write operations of the ISP's internal memory, and reduce the ISP's power consumption.

[0080] For example, when shooting high frame rate and high resolution video (e.g., 4K video), image quality requirements are not high during video preview. The AP can use the 3A statistics of the previously processed image to perform image compensation, reducing computation and saving ISP power consumption. When saving the video, image quality requirements are higher. In order to achieve a balance between power consumption and image processing effect, the AP can use weighted 3A statistics for image compensation.

[0081] Based on the above embodiments, the image processing method of the ISP when operating in the second working mode will be illustrated below. Figure 6 This is a schematic diagram of the fourth process of the image processing method in the embodiments of this application, as shown below. Figure 6 As shown, when the ISP operates in the second working mode, the image processing method specifically includes:

[0082] Step 601: Obtain the image processor's operating status information;

[0083] Step 602: Determine that the image processor is operating in the second working mode based on the operating status information;

[0084] by Figure 3 For example, when the ISP's operating status information is detected to be between threshold 2 and threshold 3, the optimal working mode of the ISP is determined to be the second working mode.

[0085] Step 603: Downsample the image data, and then perform frame-by-frame processing on the downsampled image data to obtain the first image data and the 3A statistical data of the processed image;

[0086] Here, image data refers to raw image data acquired by the image acquisition device. The image acquisition device includes a camera and an image sensor. For example, when the electronic device is a mobile phone, the camera can be a rear main camera, a rear wide-angle camera, a rear telephoto camera, or a front-facing camera.

[0087] For example, during downsampling, the sampling coefficient k can be either 1 or 2.

[0088] For example, during inter-frame processing, the interval can be one frame or two frames. Depending on the algorithm requirements, the 3A statistics of the unprocessed image can be replaced with the 3A statistics of the preceding and following processed images, without needing to perform frame-by-frame statistics or calculations, thus reducing the pressure on the ISP. However, to improve video image quality, the AP side compensates for the unprocessed image. For example, in Lens Shading Correction (LSC) processing, only the actual LSC processing is performed on the inter-frame, without calculating a checklist based on that image. The checklist information used for LSC processing on the inter-frame can be the checklist information of the previous adjacent frame, thereby ensuring the effectiveness of the LSC processing.

[0089] Step 603: Send the first image data and the 3A statistical data to the application processor, so that the application processor can perform image compensation on the unprocessed image in the first image data using the 3A statistical data of the processed image, and perform upsampling processing on the compensated image data to obtain the second image data.

[0090] It should be noted that if upsampling and image compensation are not performed after the ISP performs downsampling and frame skipping, the video quality will be affected. Therefore, when the AP determines that the ISP is operating in the second working mode, the AP will perform corresponding compensation and upsampling. The compensation process is to improve image quality, and the upsampling process is to restore the original image size.

[0091] For example, upsampling can be implemented based on bilinear interpolation or as a super-resolution (SR) technique.

[0092] Figure 7 This is a schematic diagram of the downsampling and upsampling processes in the embodiments of this application, as shown below. Figure 7 As shown, the ISP downsamples the received image (height x width) with a downsampling coefficient of 1, resulting in an image size of height / 2 x width / 2. This means that every other point in each row and column of the original image is taken to form an image. The sampled image is then processed with frame skipping to further reduce the computational load on the ISP and save ISP energy. Similarly, to ensure image display quality, the AP first upsamples the image to restore its original height x width before performing image compensation processing.

[0093] For example, in some embodiments, the first image data and the 3A statistical data are sent to the application processor so that the application processor can perform image compensation processing on the unprocessed image of the current frame using the 3A statistical data of the processed image of the previous frame; or, the first image data and the 3A statistical data are sent to the application processor so that the application processor can perform weighted processing on the 3A statistical data of the processed image of the previous frame and the processed image of the next frame to obtain weighted 3A statistical data; and perform image compensation processing on the unprocessed image of the current frame using the weighted 3A statistical data.

[0094] In other words, when the AP side compensates for the unprocessed image in the current frame, it can use the 3A statistical data of the processed image in the previous frame for compensation. Alternatively, to improve compensation accuracy, it can first weight the 3A statistical data of the processed images in the previous and next frames, and then use the weighted 3A statistical data for compensation. For details on frame skipping and compensation processing, please refer to [reference needed]. Figure 5 .

[0095] For example, during weighted processing, the weight of the 3A statistics of the processed image in the previous frame is higher than that in the subsequent frame.

[0096] Based on the above embodiments, the image processing method of the ISP when operating in the third working mode will be illustrated below with an example. Figure 8 This is a schematic diagram of the fifth process of the image processing method in the embodiments of this application.

[0097] For example, such as Figure 8 As shown, the image processor includes a front-end module (FE) and a back-end module (BE). The front-end module performs front-end processing on the image data acquired by the image sensor and stores the processed data in the memory. The back-end module reads image data from the memory for back-end processing. The image processor operates in a third working mode, retaining the functionality of the front-end module (FE) and setting the back-end module (BE) to bypass mode. Image data is processed... Figure 8 After being processed by the front-end module (FE), the 82 processing path is sent to the AP side.

[0098] The image processor runs in high-performance mode, and the image data is processed... Figure 8 In the middle processing path 83, the image data is first processed by the front-end module (FE) and stored in the memory. The back-end module (BE) reads the image data from the memory for back-end processing and then sends the processed image data to the AP side.

[0099] In some embodiments, the operating mode further includes a fourth operating mode, wherein when the image processor is operating in the third operating mode, the image processor is set to a bypass mode.

[0100] The fourth operating mode is also a low-power mode, and the ISP consumes less power running in the fourth operating mode than in the third operating mode. Running the fourth operating mode sets the image processor to bypass mode. Figure 8 For example, setting the image processor to bypass mode includes setting both the front-end module (FE) and the back-end module (BE) to bypass mode, and the image data is then processed... Figure 8 The processing path 81 is sent directly to the AP side.

[0101] Compared to the fourth working mode, setting the backend module (BE) to bypass mode in the third working mode results in better image processing performance because it retains some of the ISP's processing capabilities.

[0102] For example, when the ISP operates in the first and second modes, the image data can be the data processed by the FF module, and the image data is downsampled and interleaved in the back-end module (BE). When the ISP operates in the third mode, the ISP sends the image data directly to the AP.

[0103] It should be noted that an ISP may also include other modules. Figure 8 Not shown in the image.

[0104] By adopting the above technical solution, the computational load of the ISP is reduced by sharing intermediate processing results, thereby reducing power consumption. The 3A statistical data of the unprocessed image can be replaced with the 3A statistical data of the processed images before and after, according to the algorithm requirements, without the need for frame-by-frame statistics or calculation, thus reducing the processing pressure on the ISP. However, in order to improve the video image quality, the AP side compensates for the unprocessed image.

[0105] To implement the method of the embodiments of this application, based on the same inventive concept, the embodiments of this application also provide an image processor chip, such as... Figure 9 As shown, the image processing chip 90 includes:

[0106] An image processor 901a and an application processor 901b coupled to the image processor 901a, the image processor 901a being used to process received image data, the image processor 901a having a first operating mode and a second operating mode.

[0107] When the image processor 901a is running in the first working mode, the image processor 901a performs frame-by-frame processing on the image data to obtain the first image data.

[0108] When the image processor 901a is running in the second working mode, the image processor 901a performs downsampling processing on the image data and then performs frame-by-frame processing to obtain the first image data.

[0109] The application processor 901b is used to process the first image data.

[0110] In some embodiments, when the image processor 901a operates in a first working mode, it performs frame-by-frame processing on the image data to obtain first image data and 3A statistical data of the processed image; the first image data and the 3A statistical data are sent to the application processor so that the application processor can perform image compensation on the unprocessed image in the first image data using the 3A statistical data of the processed image to obtain second image data.

[0111] In some embodiments, when the image processor 901a operates in the second working mode, it performs downsampling processing on the image data, performs frame-by-frame processing on the downsampled image data, and obtains first image data and 3A statistical data of the processed image; the first image data and the 3A statistical data are sent to the application processor, so that the application processor can perform image compensation on the unprocessed image in the first image data using the 3A statistical data of the processed image, and perform upsampling processing on the compensated image data to obtain second image data.

[0112] In some embodiments, the application processor 901b is configured to send the first image data and the 3A statistical data to the application processor, so that the application processor can perform image compensation processing on the unprocessed image of the current frame using the 3A statistical data of the processed image of the previous frame.

[0113] Alternatively, the application processor 901b is used to send the first image data and the 3A statistical data to the application processor, so that the application processor can perform weighted processing on the 3A statistical data of the processed image in the previous frame and the processed image in the next frame to obtain weighted 3A statistical data; and perform image compensation processing on the unprocessed image in the current frame using the weighted 3A statistical data.

[0114] In some embodiments, the image processor 901a operates in a third operating mode, setting some modules in the image processor to bypass mode.

[0115] In some embodiments, the image processor 901a determines a working mode based on the operating status information. Specifically, it determines a target operating condition satisfied by the operating status information from preset operating conditions. When the target operating condition is a first operating condition, the image processor is determined to be operating in the first working mode. When the target operating condition is a second operating condition, the image processor is determined to be operating in the second working mode. When the target operating condition is a third operating condition, the image processor is determined to be operating in the third working mode.

[0116] In some embodiments, the operating status information includes at least one of the following: the temperature of the image processor, the power consumption of the image processor for a preset operating time, and...

[0117] The operating conditions include at least one of the following:

[0118] The temperature of the image processor is within a preset temperature range;

[0119] The power consumption of the image processor is within a preset power consumption range.

[0120] Optionally, such as Figure 9 As shown, the image processing chip 90 may further include a memory 902. The image processor 901a and application processor 901b can call and run computer programs from the memory 902 to implement the methods in the embodiments of this application.

[0121] The memory 902 can be a separate device independent of the image processor 901a and the application processor 901b, or it can be integrated into the image processor 901a and the application processor 901b.

[0122] Optionally, the image processing chip 90 may also include an input interface 903. The input interface 903 allows communication with other devices or chips; specifically, it allows the acquisition of information or data sent by other devices or chips.

[0123] Optionally, the image processing chip 90 may also include an output interface 904. The output interface 904 enables communication with other devices or chips; specifically, it can output information or data to other devices or chips.

[0124] Optionally, the image processing chip 90 can be applied to the electronic device in the embodiments of this application.

[0125] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0126] In practical applications, the aforementioned processor can be at least one of the following: Application-Specific Integrated Circuit (ASIC), Digital Signal Processing Device (DSPD), Programmable Logic Device (PLD), Field-Programmable Gate Array (FPGA), controller, microcontroller, and microprocessor. It is understood that, for different devices, the electronic devices used to implement the functions of the aforementioned processor can also be other types, and the embodiments of this application do not specifically limit this.

[0127] The aforementioned memory can be volatile memory, such as random-access memory (RAM); or non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid-state drive (SSD); or a combination of the above types of memory, and provides instructions and data to the processor.

[0128] This application also provides an electronic device, which has a shooting function. The electronic device may include mobile phones, tablets, laptops, handheld computers, personal digital assistants (PDAs), portable media players (PMPs), wearable devices, cameras, smart cars, etc.

[0129] like Figure 10 As shown, the electronic device 100 includes:

[0130] Image acquisition device 1001 is used to acquire image data;

[0131] The image processor 1002 is used to execute the method steps implemented by the image processor in the above embodiments.

[0132] Application processor 1003 is configured to receive first image data processed by the image processor and process the first image data.

[0133] Of course, in practical applications, as shown in Figure 10, the various components in the electronic device 100 are coupled together via a bus system 1004. It is understood that the bus system 1004 is used to achieve communication between these components. In addition to a data bus, the bus system 1004 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 10 The general labeled all buses as Bus System 1004.

[0134] In an exemplary embodiment, this application also provides a computer-readable storage medium, such as a memory including a computer program, which can be executed by an image processor to perform the steps of the aforementioned method.

[0135] This application also provides a computer program product, including computer program instructions.

[0136] Optionally, the computer program product can be applied to the image processor in the embodiments of this application, and the computer program instructions cause the computer to execute the corresponding processes implemented by the image processor in the various methods of the embodiments of this application. For the sake of brevity, they will not be described in detail here.

[0137] This application also provides a computer program.

[0138] Optionally, the computer program can be applied to the image processor in the embodiments of this application. When the computer program is run on a computer, it causes the computer to execute the corresponding processes implemented by the image processor in the various methods of the embodiments of this application. For the sake of brevity, it will not be described in detail here.

[0139] It should be understood that the terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items. The expressions “having,” “may have,” “comprising,” and “including,” or “may include” and “may contain” used herein may be used to indicate the presence of a corresponding feature (e.g., an element such as a number, function, operation, or component), but do not exclude the presence of additional features.

[0140] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another, and are not necessarily used to describe a specific order or sequence. For example, without departing from the scope of this invention, first information may also be referred to as second information, and similarly, second information may also be referred to as first information.

[0141] The technical solutions described in the embodiments of this application can be combined arbitrarily without conflict.

[0142] In the several embodiments provided in this application, it should be understood that the disclosed methods, apparatus, and devices can be implemented in other ways. The embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical, or other forms.

[0143] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0144] In addition, each functional unit in the various embodiments of this application can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0145] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. An image processing method, applied to an image processor, characterized in that, The method includes: Obtain the operating status information of the image processor; Based on the operating status information, it is determined whether the image processor is operating in a first working mode or a second working mode. When the image processor is operating in the first working mode, frame-by-frame processing is performed on the image data. When the image processor is operating in the second working mode, downsampling processing and frame-by-frame processing are performed on the image data. Wherein, when the image processor is running in the first working mode, performing frame-interval processing on the image data includes: The image data is processed by frame-by-frame processing to obtain the first image data and the 3A statistical data of the processed image; The first image data and the 3A statistical data are sent to the application processor so that the application processor can perform image compensation on the unprocessed image in the first image data by sharing the 3A statistical data of the processed image to obtain the second image data; Wherein, when the image processor is running in the second working mode, the downsampling processing and frame-interval processing of the image data include: The image data is downsampled, and the downsampled image data is then subjected to frame-by-frame processing to obtain the first image data and the 3A statistical data of the processed image. The first image data and the 3A statistical data are sent to the application processor, so that the application processor can perform image compensation on the unprocessed image in the first image data by sharing the 3A statistical data of the processed image, and perform upsampling processing on the compensated image data to obtain the second image data.

2. The method according to claim 1, characterized in that, The first image data and the 3A statistical data are sent to the application processor so that the application processor can perform image compensation processing on the unprocessed image of the current frame using the 3A statistical data of the processed image of the previous frame. Alternatively, the first image data and the 3A statistical data can be sent to the application processor, so that the application processor can perform weighted processing on the 3A statistical data of the processed images in the previous frame and the processed images in the next frame to obtain weighted 3A statistical data; and perform image compensation processing on the unprocessed image in the current frame using the weighted 3A statistical data.

3. The method according to claim 1, characterized in that, The method further includes: Based on the operating status information, it is determined that the image processor is running in a third working mode, wherein when the image processor is running in the third working mode, some modules in the image processor are set to bypass mode.

4. The method according to claim 3, characterized in that, The method further includes: From the preset operating conditions, determine the target operating conditions that the operating status information satisfies; When the target operating condition is the first operating condition, the image processor is determined to be operating in the first working mode; When the target operating condition is the second operating condition, the image processor is determined to be operating in the second working mode; When the target operating condition is the third operating condition, it is determined that the image processor is operating in the third working mode.

5. The method according to claim 4, characterized in that, The operating status information includes at least one of the following: the temperature of the image processor, the preset working time, and the power consumption of the image processor. The operating conditions include at least one of the following: The temperature of the image processor is within a preset temperature range; The power consumption of the image processor is within a preset power consumption range.

6. An image processing chip, characterized in that, include: An image processor and an application processor coupled to the image processor, the image processor being used to process received image data, the image processor having a first operating mode and a second operating mode. When the image processor is running in the first working mode, the image processor performs frame-by-frame processing on the image data to obtain first image data and 3A statistical data of the processed image; the first image data and the 3A statistical data are sent to the application processor; the application processor is used to perform image compensation on the unprocessed image in the first image data by sharing the 3A statistical data of the processed image to obtain second image data; When the image processor is running in the second working mode, the image processor performs downsampling processing on the image data and then performs frame-by-frame processing to obtain first image data and 3A statistical data of the processed image; the first image data and the 3A statistical data are sent to the application processor; the application processor is used to perform image compensation on the unprocessed image in the first image data by sharing the 3A statistical data of the processed image, and to perform upsampling processing on the compensated image data to obtain second image data.

7. An electronic device, characterized in that, The electronic device includes: An image acquisition device used to collect image data; An image processor for performing the method steps of any one of claims 1 to 5 implemented by the image processor.

8. The electronic device as claimed in claim 7, characterized in that, It also includes the application processor. When the image processor is running in the first working mode, the application processor is used to receive the first image data processed by the image processor and the 3A statistics of the processed image. Image compensation is performed on the unprocessed images in the first image data by sharing the 3A statistical data of the processed images, and the compensated image data is upsampled to obtain the second image data; When the image processor is running in the second working mode, the application processor is configured to receive first image data processed by the image processor and 3A statistics of the processed image. Image compensation is performed on the unprocessed image in the first image data by sharing the 3A statistical data of the processed image, and the compensated image data is upsampled to obtain the second image data.

Images