An image processing method, apparatus, device and storage medium

Abstract

Embodiments of the present application provide an image processing method, device, equipment and storage medium. The method comprises: in response to a shooting operation, determining shooting information; determining a target working mode of an image sensor based on the shooting information; the image sensor has at least a first working mode and a second working mode; the resolution of an image generated by the image sensor in the first working mode is less than the resolution of an image generated by the image sensor in the second working mode; when the current working mode of the image sensor is different from the target working mode of the image sensor, switching the current working mode of the image sensor to the target working mode of the image sensor; and obtaining and saving an image generated by the image sensor in the target working mode. The present application simplifies the switching process when the image shooting device needs to change the output image mode, and effectively improves the output image efficiency when the image shooting device needs to change the output image mode.

Description

[Technical Field]

[0001] This application relates to the field of image processing, and in particular to an image processing method, apparatus, device, and storage medium. [Background Technology]

[0002] Image capturing devices utilize image sensors to capture and enhance images during the shooting process. To adapt to different shooting environments and output high-quality images, existing image capturing devices employ two sensors: one in binning mode and the other in remosaic mode. When a high-resolution image is required, the device needs to switch from binning mode to remosaic mode. This sensor switching process typically involves: first, shutting down the sensor operating in binning mode, then activating the sensor in remosaic mode; after taking the picture, shutting down the remosaic mode sensor again, and then activating the binning mode sensor to continue previewing. This switching method is complex, time-consuming, and inefficient in terms of image output. [Summary of the Invention]

[0003] In view of this, the present application provides an image processing method that simplifies the switching process when the shooting device needs to change the output mode and can effectively improve the output efficiency when the image shooting device needs to change the output mode.

[0004] This application provides an image processing method, including: determining image capture information in response to a shooting operation; determining a target operating mode of an image sensor based on the image capture information; the image sensor having at least a first operating mode and a second operating mode; the resolution of an image generated by the image sensor in the first operating mode being lower than the resolution of an image generated by the image sensor in the second operating mode; switching the current operating mode of the image sensor to the target operating mode when the current operating mode of the image sensor differs from the target operating mode; and acquiring and saving an image generated by the image sensor in the target operating mode.

[0005] In one embodiment of this application, determining the target operating mode of the image sensor based on the captured image information includes: determining whether the captured image information meets preset conditions. If the captured image information meets the preset conditions, then the second operating mode is determined as the target operating mode of the image sensor.

[0006] In one embodiment of this application, the image processing method further includes: if the captured information does not meet the preset conditions, then determining the first working mode as the target working mode of the image sensor.

[0007] In one embodiment of this application, before determining the photographing information in response to the shooting operation, the method further includes: in response to the image acquisition start operation, acquiring and displaying an image generated by the image sensor in a first operating mode.

[0008] In one embodiment of this application, before switching the current operating mode of the image sensor to the target operating mode of the image sensor, the method further includes: when the target operating mode of the image sensor is a second operating mode, acquiring the frame rate of the image sensor in the second operating mode; determining whether the frame rate of the image sensor in the second operating mode is less than a preset frame rate threshold; if the frame rate of the image sensor in the second operating mode is less than the preset frame rate threshold, stopping the acquisition of images generated by the image sensor in the current operating mode within the target time period. The target time period is the time period between a first moment and a second moment; the first moment is the moment when the shooting operation is received; the second moment is the start moment when the image sensor generates the first frame image in the second operating mode.

[0009] In one embodiment of this application, the image processing method further includes: if the frame rate of the image sensor in the second working mode is not less than a preset frame rate threshold, acquiring and displaying the image generated by the image sensor in the current working mode within the target time period.

[0010] In one embodiment of this application, the first working mode is binning mode; the second working mode is remosaic mode.

[0011] In one embodiment of this application, the photographic information includes at least one of the current scene brightness value or zoom level.

[0012] In one embodiment of this application, when the photo information includes the current scene brightness value and the zoom level, the preset condition is that the current scene brightness value is greater than a preset brightness threshold and the zoom level is greater than a preset zoom level threshold.

[0013] In one embodiment of this application, the step of switching the current operating mode of the image sensor to the target operating mode of the image sensor includes: detecting the status information of a first register; when the status information of the first register is a first target value, configuring the shooting parameters of the digital camera interface module to a preset value; after the shooting parameters are configured, configuring the status information of a second register to a second target value; and sending the status information of the second register to the camera serial interface dynamic switch module, so that the camera serial interface dynamic switch module switches the current operating mode of the image sensor to the target operating mode.

[0014] This application also provides an image processing apparatus, comprising: a determining unit, a processing unit, and a saving unit. The determining unit determines image capture information in response to a shooting operation. The determining unit is further configured to determine a target operating mode of an image sensor based on the image capture information; the image sensor has at least a first operating mode and a second operating mode; the resolution of the image generated by the image sensor in the first operating mode is lower than the resolution of the image generated by the image sensor in the second operating mode. When the current operating mode of the image sensor differs from the target operating mode, the processing unit switches the current operating mode of the image sensor to the target operating mode. The saving unit acquires and saves the image generated by the image sensor in the target operating mode.

[0015] This application also provides an apparatus, which includes a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for enabling communication between the processor and the memory. When the program is executed by the processor, it implements the steps of the image processing method described in the foregoing embodiments.

[0016] This application also provides a storage medium storing one or more programs, which can be executed by one or more processors to implement the steps of the image processing method in any of the foregoing embodiments.

[0017] In this embodiment, the image sensor has at least two operating modes: a first operating mode and a second operating mode, capable of generating images at different resolutions. The image sensor's operating mode can be switched between the first and second operating modes. Compared to existing methods that switch between different sensors, the image processing method provided in this embodiment switches the sensor's operating mode, a process that is simpler and faster. Therefore, the image processing method provided in this embodiment can effectively improve the image output efficiency of the image capturing device.

[0018] Furthermore, the image processing method provided in this application embodiment can also determine the target working mode of the sensor based on the photographed information, thus enabling automatic adjustment of the target working mode of the sensor, making the image processing method more intelligent and more flexible. [Attached Image Description]

[0019] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1A schematic flowchart of an image processing method provided in an embodiment of this application;

[0021] Figure 2 A schematic diagram illustrating the principles of generating images for binning and remosaic modes;

[0022] Figure 3 A schematic diagram illustrating the principle of determining the target operating mode of an image sensor, provided in an embodiment of this application.

[0023] Figure 4 A flowchart illustrating the process of determining a target operating mode of an image sensor, provided in an embodiment of this application;

[0024] Figure 5 A schematic flowchart of an image processing method provided in an embodiment of this application;

[0025] Figure 6 This is a schematic diagram of the structure of an image processing apparatus provided in an embodiment of this application;

[0026] Figure 7 This is a schematic diagram illustrating the switching of the image sensor's operating mode, as provided in an embodiment of this application.

[0027] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0028] Label Explanation

[0029] 100. Determining unit; 200. Processing unit; 300. Storage unit; 400. Electronic device; 401. Processor; 402. Memory; 403. Communication unit.

Detailed Implementation Methods

[0030] To better understand the technical solution of this application, the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0031] It should be understood that the described embodiments are merely some, not all, of the embodiments in this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0032] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. The singular forms “a,” “the,” and “the” used in the embodiments of this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0033] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0034] In the description of this specification, it should be understood that the terms "substantially", "approximately", "about", "about", "generally", "largely" used in the claims and embodiments of this application refer to values ​​that can be generally agreed upon within a reasonable range of process operations or tolerances, rather than a precise value.

[0035] It should be understood that although terms such as first, second, third, etc., may be used to describe regions in the embodiments of this application, these regions, etc., should not be limited to these terms. These terms are only used to distinguish regions, etc., from one another. For example, without departing from the scope of the embodiments of this application, a first working mode may also be referred to as a second working mode, and similarly, a second working mode may also be referred to as a first working mode.

[0036] Through meticulous and in-depth research, the applicant in this case has provided a solution to the problems existing in the prior art.

[0037] The inventors discovered that in existing image processing methods, when using a mobile phone camera for digital zoom, the output image from the image sensor first needs to be cropped according to the scaling factor, and then super-resolution calculations are performed on the cropped image. This operation results in a loss of image quality, especially at high-magnification digital zoom, where the magnification calculations lead to the loss of image details and a significant reduction in image quality.

[0038] Figure 1 This is a flowchart illustrating an image processing method provided in an embodiment of this application.

[0039] Please see Figure 1 This application provides an image processing method, including:

[0040] Step S100: In response to the shooting operation, determine the photo information.

[0041] The shooting operation can be achieved by pressing the shutter button on the image capturing device. This means that pressing the shutter button will cause the image capturing device to take a picture. Before the image capturing device can take a picture, it needs to determine the operating mode of the image sensor, which is determined based on the shooting information. Therefore, upon receiving a shooting operation, the shooting information needs to be determined first. At this point, the image processing device can determine the shooting information based on the current environmental information of the image capturing device and the values ​​of various shooting parameters applied by the camera in the image capturing device.

[0042] In some embodiments, the image capture information includes at least one of the current scene brightness value or zoom level. When the image capture information includes the current scene brightness value, the image processing device can determine the current scene brightness value based on its current environment and through an optical sensor. When the image capture information includes a zoom level, the image processing device can determine the specific value of the zoom level based on the zoom level set by the operator. If the image capturing device does not require the operator to set the zoom level (e.g., in automatic mode) or the operator has not set the zoom level, the specific value of the zoom level can be determined based on the zoom level preset by the image capturing device or a zoom level calculated by the device itself.

[0043] It should be noted that the time for acquiring the photo information is not limited. For example, the photo information can be acquired before the shutter button is pressed or within the time period after the shutter button is pressed but before the image capturing device takes the picture. In this step, confirming the photo information can be understood as verifying the photo information to provide a basis for judgment in the next step S200.

[0044] Step S200: Determine the target operating mode of the image sensor based on the captured image information.

[0045] In step S200, since the operating mode of the image sensor needs to be determined based on the captured information, preset conditions need to be pre-defined based on the parameters contained in the captured information. The preset conditions include the parameters and their preset values ​​or preset ranges. The target operating mode of the image sensor is determined by comparing the captured information with the preset conditions.

[0046] The image sensor has at least a first operating mode and a second operating mode. The resolution of the image generated by the image sensor in the first operating mode is lower than the resolution of the image generated by the image sensor in the second operating mode. The lower resolution of the image generated by the image sensor in the first operating mode is sufficient for cropping and super-resolution operations within its applicable range. Simultaneously, due to the lower image resolution, the image sensor has lower hardware requirements and consumes less memory when generating images in the first operating mode, effectively ensuring the smooth operation of the image capturing device. The higher resolution of the image generated by the image sensor in the second operating mode effectively ensures that the image will not be distorted when cropping and super-resolution operations are performed according to a larger scaling factor, resulting in better display effects.

[0047] In step S200, the target operating mode is either the first operating mode or the second operating mode. The first operating mode is the binning mode; the second operating mode is the remosaic mode.

[0048] Figure 2 A schematic diagram illustrating the principles of generating images for binning and remosaic modes.

[0049] Please see Figure 2 Image sensors output lossless images in both binning and remosaic modes. Binning mode offers advantages such as high sensitivity and a high signal-to-noise ratio, making it suitable for low-light and high-frame-rate scenarios. However, its image resolution is only 1 / 4 of the full-size image, resulting in slightly lower resolution compared to full-size images. Remosaic mode outputs images at full-size resolution, offering excellent resolution, but it requires more powerful hardware, has lower sensitivity, and is less suitable for low-light conditions. The scaling principles of binning and remosaic modes are as follows: Figure 2 As shown in the comparison above, in order to ensure system performance and user experience, the system typically operates in binning mode (e.g., during previews, when the current scene brightness is insufficient, or when the scaling factor is not large), and only switches to remosaic mode under specific conditions.

[0050] Figure 3 A schematic diagram illustrating the principle of determining the target operating mode of an image sensor, provided in an embodiment of this application. Figure 4 This is a schematic flowchart illustrating a process for determining the target operating mode of an image sensor, as provided in an embodiment of this application. Figure 3In this context, T represents telephoto photography, W represents wide-angle photography, and UW represents ultra-wide-angle photography; DCAM (digital camera) represents digital camera; and ISP (Image Signal Processing) represents image processing device (also known as image processor).

[0051] Please see Figure 3 and Figure 4 In step S200, determining the target operating mode of the image sensor based on the captured information includes:

[0052] Step S201: Determine whether the photographed information meets the preset conditions.

[0053] In the application embodiment, after obtaining the shooting information, it is necessary to compare the corresponding parameters in the shooting information with the parameters included in the preset conditions one by one to determine whether the shooting information meets the preset conditions. Here, the corresponding parameters in the shooting information refer to parameters of the same type as those included in the preset conditions. For example, if the preset conditions include a scene brightness value, then the corresponding parameters in the shooting information also include the scene brightness value.

[0054] In some embodiments, when the image capture information includes the current scene brightness value and scaling factor, the preset conditions are that the current scene brightness value is greater than a preset brightness threshold, and the scaling factor is greater than a preset scaling factor threshold. The preset brightness threshold and the preset scaling factor threshold can be manually preset, preset by default, or automatically calculated by the image capturing device according to its specific operating mode (e.g., photo mode) (e.g., automatic mode). For example, the preset scaling factor threshold is 2x. To ensure good image quality, when the current scene brightness value meets the standard and the scaling factor of the image capturing device is greater than the preset scaling factor threshold, cropping and scaling are performed based on a nearby lossless image (an image generated in remosaic mode), ensuring that the final image maintains image quality as much as possible.

[0055] It should be noted that the steps to be executed differ depending on the determination result of step S201. When the determination result of step S201 is yes, step S202 is executed. When the determination result of step S201 is no, step S203 is executed.

[0056] Step S202: If the image capture information meets the preset conditions, then the second working mode is determined as the target working mode of the image sensor.

[0057] For example, if the current scene brightness value is greater than the preset brightness threshold and the scaling factor is greater than the preset threshold, in response to the shooting operation, the second working mode is determined as the target working mode of the image sensor, that is, the image sensor will output the image in the second working mode with higher resolution.

[0058] Step S203: If the image capture information does not meet the preset conditions, then the first working mode is determined as the target working mode of the image sensor.

[0059] For example, if the current scene brightness value is less than or equal to a preset brightness threshold, and / or the scaling factor is less than or equal to a preset threshold, in response to the shooting operation, the second working mode is determined as the target working mode of the image sensor, that is, the image sensor will output the image in the first working mode with a lower resolution.

[0060] Step S300: When the current operating mode of the image sensor differs from the target operating mode of the image sensor, the current operating mode of the image sensor is switched to the target operating mode. After determining the target operating mode based on the captured image information, the image processing device needs to determine whether the target operating mode is the same as the current operating mode. For example, this can be done by comparing the flag information of the target operating mode with the identifier information of the current operating mode. When the comparison results are different, the current operating mode of the image sensor is switched to the target operating mode via control hardware.

[0061] The switching of the image sensor's operating mode is achieved by triggering the camera serial interface dynamic switching module. Therefore, when the image sensor's current operating mode differs from its target operating mode, triggering the camera serial interface dynamic switching module can switch the current operating mode to the target mode. Based on this, it can be determined whether the camera serial interface dynamic switching module needs to be triggered by checking if the target and current operating modes are the same. If the target and current operating modes differ, it is determined that the camera serial interface dynamic switching module needs to be triggered to switch the image sensor's current operating mode to the target mode. If the target and current operating modes are the same, it is determined that the camera serial interface dynamic switching module does not need to be triggered, and the image sensor remains in its current operating mode.

[0062] The camera serial interface dynamic switch module is used to switch the operating mode of the image sensor. When the camera serial interface switch module is triggered, the operating mode of the image sensor will change.

[0063] As one possible implementation, in step S300, switching the current operating mode of the image sensor to the target operating mode of the image sensor includes:

[0064] Step S301: Detect the status information of the first register.

[0065] The image processing device can detect the status information of the first register by detecting the flag bit of the status information of the first register. The first register is the dis_eff register, which is used to indicate whether the current process has been completed. For example, the current process is the preview image data of the image capturing device entering BVBLANK (vertical blank). The status information of the first register is configured by the image processing device. The specific process is as follows: when the user clicks the shutter button, the image processing device configures the status information of the fast_start register to 1. After the dynamic switch module of the camera serial interface detects that the status information of the fast_start register is 1, the image processing device waits for the current preview image data to enter BVBLANK, and then configures the status information of the first register to the first status target value, which is, for example, 1.

[0066] Step S302: When the status information of the first register is the first status target value, configure the shooting parameters of the digital camera interface module (dcam_ip) to the first preset value.

[0067] Shooting parameters refer to the parameters required during shooting. These parameters include: image size, image buffer parameters, camera tuning parameters, etc. The first preset value includes the preset values ​​for each shooting parameter. The image processing device can detect the status information of the first register by detecting the flag bit of the first register's status information. When the image processing device detects that the flag bit of the first register's status information is 1, it configures the first preset value of the shooting parameters to the digital camera interface module.

[0068] In step S302, configuring the shooting parameters of the data camera interface module to the first preset value includes: after the data camera interface module completes the current frame processing, initializing the data camera interface module and configuring the first preset value of the shooting parameters to the shooting parameters.

[0069] The image processing device can perform initialization by resetting the data camera interface module. After initialization, the first preset value of the shooting parameters is configured to the shooting parameters.

[0070] When executing step S302, if the status information of the first register is the first target value, the image detection device will wait for the digital camera interface module to finish processing the current frame before configuring the shooting parameters of the digital camera interface module. For example, when the image processing device detects that the flag bit of the status information of the first register is 1, it will detect the working status of the digital camera interface module. While the digital camera interface module is processing the current frame, the image processing device will keep the flag bit of the status information of the first register at 1. After the digital camera interface module finishes processing the current frame, the image processing device will configure the first preset value of the shooting parameters to the digital camera interface module.

[0071] Step S303: After the shooting parameters are configured, configure the status information of the second register to the second status target value.

[0072] The target value for the second state can be 1. The second register is the cfg_done register, which is used to check whether the parameters (including flag shooting parameters or preview parameters) have been configured.

[0073] In step S303, the shooting parameters can be determined to be configured successfully by detecting the flag indicating the completion of the shooting parameter configuration.

[0074] Step S304: Send the status information of the second register to the camera serial interface dynamic switch module so that the camera serial interface dynamic switch module switches the current working mode of the image sensor to the target working mode.

[0075] In this embodiment, whether the captured image information meets preset conditions determines whether the camera serial interface dynamic switch module needs to be triggered. When the camera serial interface dynamic switch module needs to be triggered, the state information of the second register determines whether the camera serial interface dynamic switch is triggered. The camera serial interface dynamic switch module is used to switch the operating mode of the image sensor. It can be understood that when the camera serial interface dynamic switch module is triggered, the operating mode of the image sensor will change. This change can be a switch from the first operating mode to the second operating mode, or a switch from the second operating mode to the first operating mode.

[0076] For example, when the status information of the second register is the second status information, and when the image capture information meets the preset conditions, the image processing device will trigger the camera serial interface dynamic switch module. After the camera serial interface dynamic switch is triggered, it will change the working mode of the image sensor. In this step, specifically, the image sensor is switched from the current working mode to the target working mode (remosaic mode).

[0077] Step S305: Reset the status information of the first register and the status information of the second register.

[0078] Step S400: Acquire and save the image generated by the image sensor in the target working mode.

[0079] The image sensor generates image frames in the target operating mode, and the image processing device acquires and saves these image frames. Acquiring images generated by the image sensor in the target operating mode includes: acquiring images generated by the image sensor in the target operating mode that reach the target number of frames.

[0080] Figure 5 This is a flowchart illustrating an image processing method provided in an embodiment of this application.

[0081] Please see Figure 5 An image processing method provided in one embodiment of this application includes:

[0082] Step 501: In response to the image acquisition start operation, acquire and display the image generated by the image sensor in the first working mode.

[0083] In this embodiment, when the image acquisition function of the image capturing device is activated, the image capturing device is in preview mode while its image acquisition component is capturing the scene within the current viewing angle. At this time, the image sensor operates in a default working mode (e.g., a first working mode) and generates an image in the first working mode. The image processing device acquires the image generated by the image sensor in the first working mode and displays it through the display component of the image capturing device.

[0084] Step S502: In response to the shooting operation, determine the photo information.

[0085] In the embodiments of this application, the specific implementation process of step S502 can be referred to step S100, and will not be repeated here.

[0086] Step S503: Determine the target operating mode of the image sensor based on the captured image information.

[0087] In the embodiments of this application, the specific implementation process of step S503 can be referred to step S200, and will not be repeated here.

[0088] Step 504: Determine whether the current operating mode of the image sensor is the same as the target operating mode of the image sensor.

[0089] In the embodiments of this application, the specific implementation process of step S504 can be referred to step S300, and will not be repeated here.

[0090] It should be noted that the steps performed below will differ depending on the determined result. When the current operating mode of the image sensor is the same as the target operating mode of the image sensor, step S504a is executed. When the current operating mode of the image sensor is different from the target operating mode of the image sensor, step S504b is executed.

[0091] Step S504a: When the current working mode of the image sensor is the same as the target working mode of the image sensor, maintain the image sensor in the current working mode.

[0092] This can be understood as the current operating mode of the image sensor being the target operating mode, and the image sensor's operating mode remaining unchanged. For example, when the image capturing device is in preview mode, the image sensor generates an image in the first operating mode (at this time, it is in preview state, and the image is either a real-time image or a preview image; when the image capturing device shakes, the real-time image will change in accordance with the changes in the scene within the viewing angle) and outputs it. When the image capture information does not meet the preset conditions, in response to the image capture operation, the image sensor generates an image in the first operating mode (at this time, the image is the captured photograph and will not change due to the shaking of the image capturing device) and saves it.

[0093] Step 504b: Determine whether the target working mode of the image sensor is the second working mode.

[0094] It should be noted that the subsequent steps will differ depending on the result of step S504b. If the result of step S504b is yes, then step S504c will be executed. If the result of step S504b is no, then steps S504d-S504g will be executed.

[0095] Step S504c: Switch the current working mode of the image sensor to the target working mode of the image sensor.

[0096] Step S504d: When the current working mode of the image sensor is different from the target working mode of the image sensor and the target working mode of the image sensor is the second working mode, obtain the frame rate of the image sensor in the second working mode.

[0097] Because the second operating mode has a higher resolution, the image sensor places higher demands on related hardware performance when generating images in this mode. Therefore, to ensure smooth operation of the image capturing device, it is necessary to obtain the frame rate of the image sensor in the second operating mode. The frame rate can be calculated from the time required for the image sensor to generate one frame in the second operating mode.

[0098] Step S504e: Determine whether the frame rate of the image sensor in the second working mode is less than the frame rate preset threshold.

[0099] The frame rate of the image sensor in the second operating mode is compared with a preset frame rate threshold. If the frame rate of the image sensor in the second operating mode is less than the preset frame rate threshold, acquisition of images generated by the image sensor in the current operating mode within the target time period is stopped. The target time period is the time interval between the first moment and the second moment; the first moment is the moment the shooting operation is received; the second moment is the starting moment when the image sensor generates the first frame image in the second operating mode. The preset frame rate threshold can be 15fps.

[0100] In step S504e, during the target time period, the image displayed on the screen of the image capturing device is a still image.

[0101] Among them, a static image can be an image of a frame up to the first moment (e.g., the last frame).

[0102] Step S504f: If the frame rate of the image sensor in the second working mode is not less than the frame rate preset threshold, acquire and display the image generated by the image sensor in the current working mode within the target time period.

[0103] Step S504g: Switch the current working mode of the image sensor to the target working mode of the image sensor.

[0104] In this embodiment of the application, the specific implementation of steps S504c and S504g is described in step S300, and will not be repeated here.

[0105] After step S504a, or step S504c, or step S504g is completed, proceed to steps S505-S506.

[0106] Step S505: Acquire and save the image generated by the image sensor in the target working mode.

[0107] In the embodiments of this application, the specific implementation process of step S505 can be referred to step S400, and will not be repeated here.

[0108] Step S506: When the number of frames of the image generated by the image sensor in the target working mode reaches the target frame value, the current working mode of the image sensor is determined as the first working mode, and the image generated by the image sensor in the first working mode is acquired and displayed.

[0109] In this embodiment, when the number of frames generated by the image sensor in the target operating mode reaches the target frame value, it means that the shooting operation in step S501 has been completed, i.e., the shooting operation is over. When the shooting ends, the image sensor's operating mode needs to be restored to the initial operating mode. In some embodiments, the initial operating mode of the image sensor is the first operating mode. In this case, if the image sensor's operating mode is the second operating mode when shooting, the image sensor's operating mode needs to be switched back to the first operating mode after the shooting ends. If the image sensor's operating mode is the first operating mode when shooting, the image sensor needs to be maintained in the first operating mode after the shooting ends.

[0110] In some possible implementations, determining the current operating mode of the image sensor as the first operating mode in step S506 includes the following steps:

[0111] Step S5061: When the number of frames of the image generated by the image sensor in the target working mode reaches the target frame value, configure the status signal of the first register (dis_eff register) to the first status target value.

[0112] The target frame rate can be obtained either by setting a preset or by configuring it according to the specific working method of the shooting mode.

[0113] Step S5062: Detect the status information of the first register. When the status information of the first register is the first status target value, after the digital camera interface module finishes processing the current frame, configure the preview parameters of the digital camera interface module to the second preset value.

[0114] Preview parameters refer to the parameters required when the image capturing device is in preview mode.

[0115] Step S5063: After the preview parameters are configured, configure the status information of the second register to the second status target value.

[0116] Step S5064: Send the status information of the second register to the camera serial interface dynamic switch module to determine the current working mode of the image sensor as the first working mode.

[0117] When the status information of the second register is the second status information, and when the image capture information meets the preset conditions, the image processing device triggers the camera serial interface dynamic switch module. After the camera serial interface dynamic switch is triggered, it will change the working mode of the image sensor.

[0118] When the target operating mode of the image sensor is different from the current operating mode, in step S5064, the camera serial interface dynamic switch module will be triggered, thereby switching the image sensor from the current operating mode (remosaic mode) to the first operating mode (binning mode).

[0119] When the target operating mode of the image sensor is the same as the current operating mode, the camera serial interface dynamic switch module will not be triggered in step S5064, therefore, the operating mode of the image sensor will not change. For example, the image sensor's operating mode is always in the first operating mode in this step.

[0120] In this embodiment, the image sensor has at least two operating modes: a first operating mode and a second operating mode, capable of generating images at different resolutions. The image sensor's operating mode can be switched between the first and second operating modes. Compared to existing methods that switch between different sensors, the image processing method provided in this embodiment switches the sensor's operating mode, a process that is simpler and faster. Therefore, the image processing method provided in this embodiment can effectively improve the image output efficiency of the image capturing device.

[0121] This application implements rapid switching of image sensor operating modes. During preview, binning mode is used for image output, and scene brightness and zoom levels are acquired in real time to determine the sensor's operating mode for the current capture moment. When capturing in a bright scene (i.e., when the scene brightness value is greater than a preset brightness threshold, which is adjustable) and the zoom level is between 2.0x and (T-0.1x), the image sensor's operating mode is switched from binning mode to remosaic mode for frame capture. Here, T is the focal length multiplier of the telephoto lens. If, during the capture process, the image sensor's frame rate in remosaic mode is too low (less than 15fps) due to hardware performance issues, acquiring preview frames within the target time period can be stopped. At this time, the image capturing device stops displaying the real-time image. After capturing the frames, it switches back from remosaic mode to binning mode, and the image capturing device resumes normal preview.

[0122] Meanwhile, to enable rapid switching of the image sensor's operating mode, two registers are introduced in the hardware: the fast_start register and the cfg_done register. Configuring fast_start indicates that the user has clicked the capture button, while configuring the cfg_done signal in the image processing device informs the camera's serial interface dynamic switching module that the relevant configuration has been completed and the corresponding image can be connected. Furthermore, this solution can simultaneously configure the number of frames captured. After the hardware completes capturing the target frames, it automatically switches back to binning mode, thus avoiding the time consumption caused by switching the image sensor on and off.

[0123] In summary, the beneficial effects of the image processing method provided in this application can be summarized as follows:

[0124] First, this application implements fast switching of image sensor operating modes for switching the operating modes of image sensors in image capturing devices (such as mobile phone cameras). The purpose is to improve the image effect at high magnification, reduce the time spent on switching image sensor operating modes and pattern conversion, and improve the user experience.

[0125] Secondly, in order to improve the image quality at a zoom level of 2.0x and above in bright environments in fully automatic mode, the binning mode is used for previewing, and information on ambient light brightness and zoom level is acquired in real time to determine the image sensor working mode to be used at the current shooting time. When the ambient light brightness and zoom level reach a certain value during shooting, the image is switched to remosaic mode. After the shooting and frame capture are completed, the image is switched back to binning mode to restore normal preview.

[0126] Third, this application can also configure the number of frames captured during photography, so that the hardware can automatically count and switch, reducing the time spent on switching the working mode of the image sensor and converting patterns.

[0127] Figure 6 This is a schematic diagram of the structure of an image processing device provided in an embodiment of this application.

[0128] Please see Figure 6This application also provides an image processing apparatus, comprising: a determining unit 100, a processing unit 200, and a saving unit 300. The determining unit 100 determines image capture information in response to a shooting operation. The determining unit 100 is further configured to determine a target operating mode of an image sensor based on the image capture information; the image sensor has at least a first operating mode and a second operating mode; the resolution of the image generated by the image sensor in the first operating mode is lower than the resolution of the image generated by the image sensor in the second operating mode. When the current operating mode of the image sensor differs from the target operating mode, the processing unit 200 switches the current operating mode of the image sensor to the target operating mode. The saving unit 300 acquires and saves the image generated by the image sensor in the target operating mode.

[0129] In one embodiment of this application, the determining unit 100 is further configured to determine whether the photographing information meets preset conditions. If the photographing information meets the preset conditions, the second operating mode is determined as the target operating mode of the image sensor.

[0130] In one embodiment of this application, the determining unit 100 is further configured to determine the first working mode as the target working mode of the image sensor if the photographing information does not meet the preset conditions.

[0131] In one embodiment of this application, the processing unit 200 is further configured to acquire and display an image generated by the image sensor in a first operating mode in response to the image acquisition start operation.

[0132] In one embodiment of this application, the processing unit 200 is further configured to acquire the frame rate of the image sensor in the second operating mode when the target operating mode of the image sensor is the second operating mode. It then determines whether the frame rate of the image sensor in the second operating mode is less than a preset frame rate threshold. If the frame rate of the image sensor in the second operating mode is less than the preset frame rate threshold, it stops acquiring images generated by the image sensor in the current operating mode within the target time period. The target time period is the time interval between a first moment and a second moment; the first moment is the moment the shooting operation is received; the second moment is the start time when the image sensor generates the first frame image in the second operating mode.

[0133] In one embodiment of this application, the processing unit 200 is further configured to acquire and display the image generated by the image sensor in the current working mode within the target time period if the frame rate of the image sensor in the second working mode is not less than a preset frame rate threshold.

[0134] In one embodiment of this application, the processing unit 200 is further configured to detect the status information of the first register. When the status information of the first register is a first target value, the shooting parameters of the digital camera interface module are configured to preset values. After the shooting parameters are configured, the status information of the second register is configured to a second target value. The status information of the second register is sent to the camera serial interface dynamic switch module so that the camera serial interface dynamic switch module switches the current working mode of the image sensor to the target working mode.

[0135] Figure 7 This is a schematic diagram illustrating the switching of the image sensor's operating mode, as provided in an embodiment of this application.

[0136] Please see Figure 7 To better understand the image processing method and apparatus provided in the above embodiments, let's take a 48MP image acquired by a QCFA sensor, magnified 5 times, and captured in 2 frames as an example for explanation. Figure 3 The timing diagram shown illustrates the specific execution steps as follows.

[0137] Step 1: The user clicks the camera button, and the image processing device configures the fast_start register;

[0138] Step 2: After the camera serial interface dynamic switch module detects the configuration of the fast_start register, it first waits for the current preview image data to enter VBLAND, and then sets the dis_eff register status signal to 1;

[0139] Step 3: When the image processing device detects that the dis_eff register status signal is 1, and after dcam_ip has completed the processing of the current frame, it configures dcam_ip with a reset signal and sets the data related to taking pictures (i.e., shooting parameters).

[0140] Step 4: When the parameter configuration is complete, the image processing device configures the cfg_done register and informs the camera serial interface dynamic switch module that the relevant configuration has been completed and it can be connected to capture images.

[0141] Step 5: The image sensor outputs the relevant captured images. When the target frame is output according to the configured number of captured frames, the dis_eff register status signal is set to 1 again.

[0142] Step 6: When the image processing device detects that the dis_eff register status signal is 1, and after dcam_ip has completed the processing of the current frame, it configures the dcam_ip with a reset signal and the preview-related data (i.e., preview parameters);

[0143] Step 7: When the parameter configuration is complete, the image processing device configures the cfg_done register to inform the camera serial interface dynamic switch module that the relevant configuration has been completed and the relevant images can be connected for preview.

[0144] Step 8: When the image processing device detects that the dis_eff register status signal is 0 again, it indicates that the image sensor has completed the output of the image in remosaic mode and the normal preview state has been restored.

[0145] This application also provides an apparatus, which includes a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for communication between the processor and the memory. When the program is executed by the processor, it implements the steps of the image processing method described in the foregoing embodiments. The apparatus can be an electronic device such as a camera, mobile phone, or tablet.

[0146] The device provided in this application embodiment can take photos in fully automatic mode, at a zoom level of 2.0x or higher (the main zoom level range is 2.0x to (T-0.1)x), and in bright environments. The image sensor can quickly and efficiently switch from binning mode, which outputs 1 / 4 full-size preview data, to remosaic mode, which outputs full-size photo data, and captures frames according to the set frame rate. After capturing the photos and frames, it automatically switches back to binning mode to restore the normal preview state. This not only improves the image quality at high zoom levels but also reduces the time spent switching image sensor operating modes and converting patterns, further enhancing the user experience.

[0147] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device 400 may include a processor 401, a memory 402, and a communication unit 403. These components communicate through one or more buses. Those skilled in the art will understand that the structure of the electronic device shown in the figure does not constitute a limitation on the embodiments of the present invention. It may be a bus-shaped structure or a star-shaped structure, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0148] The communication unit 403 is used to establish a communication channel, enabling the electronic device to communicate with other devices. It receives user data from other devices or sends user data to other devices.

[0149] The processor 401 serves as the control center of the electronic device, connecting various parts of the device via various interfaces and lines. It executes software programs and / or modules stored in the memory 402, and calls data stored in the memory to perform various functions and / or process data. The processor can be composed of integrated circuits (ICs), such as a single packaged IC or multiple packaged ICs with the same or different functions connected together. For example, the processor 401 may consist only of a central processing unit (CPU). In this embodiment, the CPU may have a single processing core or include multiple processing cores.

[0150] The memory 402 is used to store the execution instructions of the processor 401. The memory 402 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0151] When the execution instructions in memory 402 are executed by processor 401, the electronic device 400 is able to perform operations. Figure 5 Some or all of the steps in the illustrated embodiments.

[0152] This application also provides a storage medium storing one or more programs, which can be executed by one or more processors to implement the steps of the image processing method in any of the foregoing embodiments.

[0153] Furthermore, the image processing method provided in this application embodiment can also determine the target working mode of the sensor based on the photographed information, thus enabling automatic adjustment of the target working mode of the sensor, making the image processing method more intelligent and more flexible.

[0154] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. An image processing method, characterized in that, include: In response to the shooting operation, determine the photo information; The target operating mode of the image sensor is determined based on the captured information; The image sensor has at least a first operating mode and a second operating mode; the resolution of the image generated by the image sensor in the first operating mode is lower than the resolution of the image generated by the image sensor in the second operating mode. When the current operating mode of the image sensor is different from the target operating mode of the image sensor, the current operating mode of the image sensor is switched to the target operating mode of the image sensor. Acquire and save the image generated by the image sensor in the target operating mode; The step of switching the current operating mode of the image sensor to the target operating mode of the image sensor includes: Detect the status information of the first register; When the status information of the first register is the first status target value, the shooting parameters of the digital camera interface module are configured to the preset value; After the shooting parameters are configured, the status information of the second register is configured with the second status target value; The status information of the second register is sent to the serial interface dynamic switch module of the serial interface camera, so that the camera serial interface dynamic switch module switches the current working mode of the image sensor to the target working mode. Before the step of switching the current operating mode of the image sensor to the target operating mode of the image sensor, the method further includes: When the target operating mode of the image sensor is the second operating mode, the frame rate of the image sensor in the second operating mode is obtained; Determine whether the frame rate of the image sensor in the second working mode is less than a preset frame rate threshold; If the frame rate of the image sensor in the second working mode is less than the frame rate preset threshold, the acquisition of images generated by the image sensor in the current working mode within the target time period is stopped; the target time period is the time period between the first moment and the second moment; the first moment is the moment when the shooting operation is received; the second moment is the starting moment when the image sensor generates the first frame image in the second working mode.

2. The image processing method according to claim 1, characterized in that, Determining the target operating mode of the image sensor based on the captured information includes: Determine whether the photographed information meets preset conditions; If the captured image information meets the preset conditions, then the second working mode is determined as the target working mode of the image sensor.

3. The image processing method according to claim 2, characterized in that, Also includes: If the captured information does not meet the preset conditions, then the first working mode is determined as the target working mode of the image sensor.

4. The image processing method according to claim 1, characterized in that, Before determining the image capture information in response to the shooting operation, the following is also included: In response to the start operation of image acquisition, the image generated by the image sensor in the first working mode is acquired and displayed.

5. The image processing method according to claim 1, characterized in that, Also includes: If the frame rate of the image sensor in the second working mode is not less than the frame rate preset threshold, the image generated by the image sensor in the current working mode within the target time period is acquired and displayed.

6. The image processing method according to claim 1, characterized in that, The first working mode is binning mode; the second working mode is remosaic mode.

7. The image processing method according to claim 1, characterized in that, The photo information includes at least one of the current scene brightness value or zoom level.

8. The image processing method according to claim 7, characterized in that, When the photo information includes the current scene brightness value and the zoom level, the preset condition is that the current scene brightness value is greater than a preset brightness threshold, and the zoom level is greater than a preset zoom level threshold.

9. An image processing apparatus, characterized in that, include: The unit is identified in response to the shooting operation, and the photo information is determined. The determining unit is further configured to determine the target operating mode of the image sensor based on the captured information; the image sensor has at least a first operating mode and a second operating mode; the resolution of the image generated by the image sensor in the first operating mode is less than the resolution of the image generated by the image sensor in the second operating mode; The processing unit switches the current operating mode of the image sensor to the target operating mode of the image sensor when the current operating mode of the image sensor is different from the target operating mode of the image sensor. The storage unit acquires and saves the image generated by the image sensor in the target working mode. Specifically, the processing unit is used for: Detect the status information of the first register; When the status information of the first register is the first status target value, the shooting parameters of the digital camera interface module are configured to the preset value; After the shooting parameters are configured, the status information of the second register is configured with the second status target value; The status information of the second register is sent to the serial interface dynamic switch module of the serial interface camera, so that the camera serial interface dynamic switch module switches the current working mode of the image sensor to the target working mode. The processing unit is further configured to: When the target operating mode of the image sensor is the second operating mode, the frame rate of the image sensor in the second operating mode is obtained; Determine whether the frame rate of the image sensor in the second working mode is less than a preset frame rate threshold; If the frame rate of the image sensor in the second working mode is less than the frame rate preset threshold, the acquisition of images generated by the image sensor in the current working mode within the target time period is stopped; the target time period is the time period between the first moment and the second moment; the first moment is the moment when the shooting operation is received; the second moment is the starting moment when the image sensor generates the first frame image in the second working mode.

10. A device, characterized in that, The device includes a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for establishing communication between the processor and the memory. When executed by the processor, the program implements the steps of the image processing method as described in any one of claims 1 to 8.

11. A storage medium, characterized in that, The storage medium stores one or more programs, which can be executed by one or more processors to implement the steps of the image processing method according to any one of claims 1 to 8.

Images