Image processing method, device, chip, communication equipment, and storage medium
By separating control parameter configuration and smoothing processing between the application processor and the image processor, dynamically adjusting the number of smoothing processing images, the problem of inflexible switching of the shooting mode of electronic devices is solved, and the switching speed and efficiency are improved.
Patent Information
- Application Number
- CN202211610994.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-12-12
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2042-12-12
AI Technical Summary
When switching shooting modes, existing electronic devices need to complete scene parameter configuration and preview images smoothing processing, resulting in inflexible switching methods and taking a long time.
By sending specific instructions between the application processor and the image processor, the parameter configuration process and smoothing process for controlling the shooting mode switching are separated, and the separation control of parameter configuration and smoothing process is realized, and the number of images of smoothing process is dynamically adjusted according to the environmental parameters. , optimize the flexibility and speed of shooting mode switching.
It improves the flexibility and speed of shooting mode switching, takes into account the smoothing effect and mode switching speed, and provides a fast stop mechanism for smoothing processing to optimize processing efficiency.
Smart Images

Figure CN115988317B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of electronic technology, and in particular to an image processing method and apparatus, a chip, a communication device, and a storage medium. Background Art
[0002] As users' photography needs increase, electronic devices usually set shooting modes suitable for different scenes. For example, normal mode, night scene mode, portrait mode, etc. By turning on the corresponding shooting mode in different shooting scenes, you can get the ideal shooting effect. At present, a complete shooting mode switch requires the completion of scene parameter configuration and smooth processing of the preview image, resulting in inflexible switching. Summary of the invention
[0003] Embodiments of the present application provide an image processing method and apparatus, a chip, a communication device, and a storage medium.
[0004] The technical solution of this application is implemented as follows:
[0005] In a first aspect, an embodiment of the present application provides an image processing method, which is applied to an application processor, including:
[0006] Responding to the mode switching instruction, switching the shooting mode; the shooting mode switching process includes a parameter configuration stage and a smoothing processing stage;
[0007] When the shooting mode switching process is in a smoothing processing stage, sending a first instruction to the image processor, wherein the first instruction is used to instruct to perform smoothing processing; and
[0008] When the shooting mode switching process is in the parameter configuration stage, a second instruction is sent to the image processor, where the second instruction is used to instruct the image processor to perform parameter configuration of the target shooting mode.
[0009] In a second aspect, an embodiment of the present application further provides an image processing method, which is applied to an image processor, comprising:
[0010] When the shooting mode switching process is in a smoothing processing stage, receiving a first instruction sent by the application processor, wherein the first instruction is used to instruct to perform smoothing processing; and / or,
[0011] When the shooting mode switching process is in a parameter configuration stage, a second instruction sent by the application processor is received, where the second instruction is used to instruct the image processor to perform parameter configuration of the target shooting mode.
[0012] In a third aspect, an embodiment of the present application provides an image processing device, including:
[0013] A sending unit is configured to switch a shooting mode in response to a mode switching instruction; the shooting mode switching process includes a parameter configuration stage and a smoothing processing stage; when the shooting mode switching process is in the smoothing processing stage, a first instruction is sent to an image processor, and the first instruction is used to instruct the execution of smoothing processing; and when the shooting mode switching process is in the parameter configuration stage, a second instruction is sent to the image processor, and the second instruction is used to instruct the image processor to perform parameter configuration of the target shooting mode.
[0014] In a fourth aspect, an embodiment of the present application further provides an image processing device, including:
[0015] The receiving unit is configured to receive a first instruction sent by the application processor when the shooting mode switching process is in a smoothing processing stage, wherein the first instruction is used to instruct the execution of smoothing processing; and / or, when the shooting mode switching process is in a parameter configuration stage, receive a second instruction sent by the application processor, wherein the second instruction is used to instruct the image processor to perform parameter configuration of the target shooting mode.
[0016] In a fifth aspect, the present embodiment provides an image processing system, including: an application processor and an image processor; wherein:
[0017] The application processor is configured to switch the shooting mode in response to a mode switching instruction; the shooting mode switching process includes a parameter configuration stage and a smoothing processing stage; when the shooting mode switching process is in the smoothing processing stage, a first instruction is sent to the image processor, the first instruction is used to instruct the execution of smoothing processing; and when the shooting mode switching process is in the parameter configuration stage, a second instruction is sent to the image processor, the second instruction is used to instruct the image processor to perform parameter configuration of the target shooting mode;
[0018] The image processor is configured to, when the shooting mode switching process is in a smoothing processing stage, receive a first instruction sent by the application processor, wherein the first instruction is used to instruct the execution of smoothing processing; and / or, when the shooting mode switching process is in a parameter configuration stage, receive a second instruction sent by the application processor, wherein the second instruction is used to instruct the image processor to perform parameter configuration of the target shooting mode.
[0019] In a sixth aspect, an embodiment of the present application provides a chip, comprising: a processor, configured to call and run a computer program from a memory, so that a device equipped with the chip executes the image processing method described in the first aspect or the second aspect.
[0020] In a seventh aspect, an embodiment of the present application provides a communication device, comprising an application processor and an image processor, wherein the application processor is used to execute the method described in the first aspect, and the image processor is used to execute the method described in the second aspect.
[0021] In an eighth aspect, an embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the image processing method described in the first aspect or the second aspect.
[0022] In the image processing method provided in the embodiment of the present application, the application processor can control the image processor to perform smoothing processing through the first instruction, and control the image processor to perform parameter configuration through the second instruction, thereby realizing separate control of parameter configuration and smoothing processing. In this way, the application processor can control the parameter configuration process and the smoothing process of the shooting mode switching respectively, thereby improving the flexibility of the shooting mode switching. BRIEF DESCRIPTION OF THE DRAWINGS
[0023] Figure 1 A schematic diagram of an image processing method provided in an embodiment of the present application Figure 1 ;
[0024] Figure 2 A schematic diagram of an image processing method provided in an embodiment of the present application Figure 2 ;
[0025] Figure 3 A schematic diagram of an image processing method provided in an embodiment of the present application Figure 3 ;
[0026] Figure 4 A schematic diagram of an image processing method provided in an embodiment of the present application Figure 4 ;
[0027] Figure 5 A schematic diagram of an image processing method provided in an embodiment of the present application Figure 5 ;
[0028] Figure 6 A schematic diagram of an image processing method provided in an embodiment of the present application Figure 6 ;
[0029] Figure 7 A schematic diagram of an image processing method provided in an embodiment of the present application Figure 7 ;
[0030] Figure 8 A schematic diagram of an image processing method provided in an embodiment of the present application Figure 8 ;
[0031] Fig. 9A schematic diagram of an image processing method provided in an embodiment of the present application Figure 9 ;
[0032] Fig.10 A schematic diagram of the structure of an image processing device provided in an embodiment of the present application Figure 1 ;
[0033] Fig.11 A schematic diagram of the structure of an image processing device provided in an embodiment of the present application Figure 2 ;
[0034] Fig.12 A schematic diagram of a chip structure provided in an embodiment of the present application;
[0035] Fig.13 A schematic diagram of the hardware structure of a communication device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0036] The following will describe the technical solutions in the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0037] The image processing method provided in the embodiment of the present application can be applied to an electronic device provided with a pre-image processor (PreImage Signal Processing, PreISP). The PreISP is used to process the preview image collected by the image acquisition device in the image preview stage. When the PreISP is provided in the electronic device, a variety of functional combinations can be implemented at the same time. For example, the shooting mode can be automatically adjusted according to the shooting scene during the process of the electronic device performing the spatial alignment transform (Spatial Alignment Transform, SAT).
[0038] In practical applications, traditional zoom solutions are divided into two types: optical zoom and digital zoom. Optical zoom relies on the structure of optical lenses to achieve zoom. Specifically, optical zoom enlarges and reduces the scene to be photographed by moving the lens. Digital zoom uses a processor to increase the area of each pixel in the image to achieve the purpose of magnification.
[0039] However, due to the thickness limitation of electronic devices, it is difficult to integrate zoom lenses with complex optical structures. Currently, electronic devices can use spatial alignment transform (SAT) for zooming. In simple terms, SAT is a technology that uses fixed-focus lens relay to achieve zooming. When SAT is switched, the camera device can achieve continuous zooming by switching fixed-focus lenses and combining digital zoom.
[0040] It should be understood that, in the case of PreISP, in order to reduce the image jump during the shooting mode switching process, a complete shooting mode switching process may include a parameter configuration stage and a smoothing stage. Among them, the parameter configuration operation stage may be a stage for adjusting the shooting parameters. Usually, the parameter configuration operation stage may be a parameter adjustment of the back-end (BE) hardware module of PreISP. The smoothing operation stage can be understood as filtering and denoising the images before and after the switching, so that the images before and after the shooting mode switching will not jump.
[0041] In actual applications, the shooting mode switching is usually completed through the firmware of PreISP. The switching process cannot be interrupted, and each shooting mode switching takes a long time.
[0042] Based on this, an embodiment of the present application provides an image processing method, which can be applied to an application processor (AP) and an image signal processing (ISP) of an electronic device. Figure 1 As shown, the image processing method provided in the embodiment of the present application may include the following steps:
[0043] Step 110: The AP responds to the mode switching instruction and switches the shooting mode. The shooting mode switching process includes a parameter configuration stage and a smoothing processing stage.
[0044] Step 120: When the shooting mode switching process is in the smoothing processing stage, the AP sends a first instruction to the ISP, where the first instruction is used to instruct the execution of smoothing processing; and, when the shooting mode switching process is in the parameter configuration stage, the AP sends a second instruction to the ISP, where the second instruction is used to instruct the ISP to perform parameter configuration of the target shooting mode.
[0045] Correspondingly, the ISP can receive a first instruction when the shooting mode switching process is in the smoothing processing stage, and / or receive a second instruction when the shooting mode switching process is in the parameter configuration stage, and respond to the first instruction and / or the second instruction respectively.
[0046] Optionally, the mode switching instruction can be manually triggered by the user, for example, the user manually triggers the switching of the shooting mode when shooting at night. The mode switching instruction can also be automatically triggered by the AP. Specifically, the AP can obtain relevant parameters of the shooting in real time and automatically trigger the mode switching instruction after the relevant parameters meet the preset conditions. For example, while performing SAT switching, the AP can automatically trigger the mode switching instruction according to the brightness of the scene to be shot.
[0047] It should be understood that a complete shooting mode switching process may include a parameter configuration phase and a smoothing processing phase. In the embodiment of the present application, the AP may control the parameter configuration phase and the smoothing processing phase separately to achieve separation of the parameter configuration phase and the smoothing processing phase.
[0048] Specifically, the AP can determine the stage of the current shooting mode switching process. When the AP determines that the shooting mode switching process is in the parameter configuration stage, the AP can control the image processor through the second instruction to perform parameter configuration for the target shooting mode. When the AP determines that the shooting mode switching process is in the smoothing processing stage, the AP can control the image processor through the first instruction to perform smoothing on the images before and after the switching.
[0049] It should be noted that the ISP in the embodiment of the present application may be the PreISP in the above embodiment. The parameter configuration stage of the shooting mode switching process may be the configuration of the hardware parameters of the BE module of the PreISP.
[0050] Optionally, the above processing may be implemented through a hardware abstraction layer (HAL) in the AP.
[0051] It is understandable that the AP can control the PreISP to perform smoothing through the first instruction and control the PreISP to perform parameter configuration through the second instruction, thereby realizing separate control of parameter configuration and smoothing. In this way, the AP can control the parameter configuration process and the smoothing process of the shooting mode switching respectively, thereby improving the flexibility of the shooting mode switching.
[0052] In actual applications, the processing operations for different mode switching directions will be different. For example, for the scene switching from Normal Mode (NM) to Super Night Mode (SN) / Ultra Dark Mode (UD), the parameters of the BE module can be configured first, and then smoothing can be performed. For the scene switching from SN / UD to NM, smoothing can be performed first, and then the parameters of the BE module can be configured.
[0053] In one embodiment of the present application, refer to Figure 2 As shown in the flowchart, before step 120, the image processing method provided in the embodiment of the present application may further include the following steps:
[0054] Step 130: The AP obtains environmental parameters of the scene to be photographed;
[0055] Step 140: The AP determines the number of images that match the environmental parameters, where the number of images refers to the number of images that need to be smoothed during the switching of the shooting mode.
[0056] Step 150: The AP sends a third instruction to the ISP, where the third instruction is used to instruct the ISP to perform smoothing processing based on the number of images to complete the switching of the shooting mode.
[0057] In an embodiment of the present application, the AP may respond to the received mode switching instruction, obtain the environmental parameters of the scene to be captured, and determine the number of images that need to be smoothed based on the environmental parameters. In an embodiment of the present application, the number of images that need to be smoothed may also be referred to as the number of smooth image frames. Here, different environmental parameters may correspond to different numbers of smooth image frames.
[0058] It is understandable that the AP can dynamically adjust the number of smooth image frames in the smooth processing process according to the actual requirements of the environmental parameters of the current scene.
[0059] Optionally, the environmental parameter may include but is not limited to at least one of the following: luminous flux, brightness, and illumination. After receiving the mode switching instruction, the AP may control the light sensor to collect the environmental parameters of the current scene to be photographed.
[0060] In a possible implementation, the AP may pre-set the number of smooth image frames corresponding to different environmental parameters. In this way, when the AP switches modes, it may determine the number of smooth image frames according to the collected environmental parameters.
[0061] In another possible implementation, the AP may input the collected environmental parameters into a pre-trained neural network model for processing to obtain the number of smooth image frames corresponding to the current environmental parameters. It should be noted that the pre-trained neural network model may be trained according to a machine learning algorithm.
[0062] Optionally, due to the different configuration parameters when switching between different shooting modes, the AP can also determine the number of smooth image frames in combination with the switching direction of the shooting mode and the environmental parameters. It can be understood that when the switching direction of the shooting mode is different, the AP can determine the number of images that actually need to be smoothed according to different corresponding relationships. Exemplarily, for the scene switching from the normal shooting mode (Normal Mode, NM) to the night scene shooting mode, where the night scene mode can include the super night shooting mode (Super Night Mode, SN) / ultra night shooting mode (Ultra Dark Mode, UD), Table 1 shows the correspondence between the illuminance index value (lux index) and the number of smooth image frames.
[0063] Table 1
[0064]
[0065] As can be seen from Table 1, in the scenario of switching from NM to SN / UD, when the lux index is greater than 350 and less than or equal to 370, the AP can determine that the number of images that need to be smoothed is 3 frames; when the lux index is greater than 370 and less than or equal to 390, the AP can determine that the number of images that need to be smoothed is 1 frame; when the lux index is greater than 390, the AP can determine that the number of images that need to be smoothed is 0 frames.
[0066] In addition, for the scene switching from SN / UD to NM, Table 2 shows the corresponding relationship between the illuminance index value (lux index) and the number of smooth image frames.
[0067] Table 2
[0068]
[0069] It can be seen from Table 2 that in the scenario of switching from SN / UD to NM, when the lux index is less than or equal to 320 and greater than 310, the AP can determine that the number of images that need to be smoothed is 3 frames; when the lux index is less than or equal to 310 and greater than 290, the AP can determine that the number of images that need to be smoothed is 1 frame; when the lux index is less than or equal to 290, the AP can determine that the number of images that need to be smoothed is 0 frames.
[0070] In an embodiment of the present application, after the AP determines the current number of smooth image frames, three instructions can be sent to the image processor PreISP to instruct the PreISP to perform a smoothing operation based on the determined number of smooth image frames to complete the switching of the shooting mode.
[0071] It is understandable that the third instruction may include the number of smooth image frames determined by the AP. The image processor may combine the first instruction and the third instruction and perform smoothing according to the number of smooth image frames indicated by the third instruction.
[0072] Optionally, the third instruction may be the same instruction as the first instruction, or may be two independent instructions, which is not limited in the embodiment of the present application.
[0073] In summary, in the image processing method provided by the embodiment of the present application, the application processor can dynamically adjust the number of images that need to be smoothed according to the environmental parameters of the scene to be shot, and control the image processor to perform smoothing during the shooting mode switching process based on the determined number of images. That is to say, in an environment where multiple images need to be smoothed, the application processor can dynamically increase the number of images to be smoothed to improve the smoothing effect during the shooting mode switching process, and in an environment where only a small number of images need to be smoothed, the application processor can dynamically reduce the number of images to be smoothed to optimize the speed of shooting mode switching. In this way, both the smoothing effect and the mode switching speed during the shooting mode switching process can be taken into account.
[0074] In one embodiment of the present application, refer to Figure 3 As shown, the image processing method provided in the embodiment of the present application may also include the following steps:
[0075] Step 160, the AP determines the cumulative number of images that have undergone the smoothing operation;
[0076] Step 170: If the accumulated number of images matches the number of images, the AP determines that the smoothing process in the process of switching the shooting mode is completed.
[0077] It should be understood that the AP can send a first instruction to the PreISP to trigger the PreISP to perform smoothing during the shooting mode switching process. The end of the smoothing process during the shooting mode switching process can be controlled by the FW in the PreISP. Considering the influence of the delay of whether the AP and PreISP complete the smoothing process, in the embodiment of the present application, the PreISP and the AP can respectively control the number of smoothing processing frames to reduce the delay.
[0078] That is, the AP can count the cumulative number of images that have been smoothed. If the cumulative number of images matches the number of images determined by the AP, it can be determined that the smoothing process has been completed. The AP can count the number of frames that have been smoothed according to the image frame rate when the PreISP performs the smoothing process.
[0079] Furthermore, after determining that the smoothing process has been completed, the AP can also determine whether the shooting mode switching process is completed according to the switching direction of the current shooting mode. Exemplarily, for the scenario of switching from NM to SN / UD, the PreISP can first perform parameter configuration of the BE module and then perform smoothing. Therefore, after determining that the smoothing process has been completed, the AP can determine that the current shooting mode switching is completed. For the scenario of switching from SN / UD to NM, the PreISP can first perform smoothing and then perform parameter configuration of the BE module. Therefore, after determining that the smoothing process has been completed, the AP can determine that the current shooting mode switching still requires parameter configuration processing of the BE module, and control the PreISP to perform parameter configuration processing of the BE module.
[0080] In summary, in the image processing method provided in the embodiment of the present application, the application processor can count the number of images that have been smoothed so as to determine the progress of the current shooting mode switching, thereby performing subsequent control processing. The application processor itself counts the number of images that have been smoothed so as to avoid the delay problem caused by the image processor sending a completion instruction to it.
[0081] In one embodiment of the present application, refer to Figure 4 As shown, the image processing method provided in the embodiment of the present application may also include the following steps:
[0082] Step 180: During the process of switching shooting modes, if a pending task request instruction is received, the AP sends a fourth instruction to the PreISP, where the fourth instruction is used to instruct to stop smoothing during the shooting mode switching process. The priority of the pending task is higher than the priority of switching the shooting mode.
[0083] It is understandable that the AP can separate and control the parameter configuration process and the smoothing process in the process of switching the shooting mode. Based on this, the AP can also provide a quick stop mechanism for smoothing in the embodiment of the present application. When the AP receives a task with a higher priority, the quick stop mechanism for smoothing can be called to stop the more time-consuming smoothing process, thereby optimizing the switching time of the shooting mode and the processing time of the higher priority task, and improving the processing efficiency.
[0084] In the embodiment of the present application, the AP can pre-set the priority of each type of task. In this way, after receiving the pending tasks, the AP can determine whether to send the fourth instruction to the image processor to quickly terminate the smoothing process based on the relationship between the priority of each task and the priority of the shooting mode switching task. After determining that the shooting mode switching is completed, the pending tasks can continue to be executed, thereby optimizing the processing time of the pending tasks and improving the processing efficiency.
[0085] Optionally, if a pending task request instruction is received in step 180, a fourth instruction is sent to the image processor, which can also be implemented in the following manner:
[0086] Step 1801: If a task to be processed is received, the AP determines whether the shooting mode switching process is in the initial stage of smooth processing;
[0087] Step 1802: If the shooting mode switching process is in the initial stage of smoothing, the AP sends a fourth instruction to the PreISP.
[0088] It is understandable that the AP may send the fourth instruction to the PreISP at the initial stage of the smoothing process so as to stop the smoothing process at the initial stage of the smoothing process, thereby further shortening the time for switching shooting modes and improving processing efficiency.
[0089] Optionally, the initial stage of the smoothing process may be the first frame of the image to be smoothed, or the first N frames of the image, where N may be a preset value, such as 2 or 3.
[0090] Optionally, after sending the fourth instruction, the image processing method provided in the embodiment of the present application may further include the following steps:
[0091] If the shooting mode is switched from normal mode to night mode, the AP determines that the shooting mode switch is complete;
[0092] If the shooting mode is switched from the night scene mode to the normal mode, the AP determines that the shooting mode switching ends in advance.
[0093] It is understandable that, since the switch from normal mode to night scene mode (such as SN / UD) is to configure the parameters of the BE module first and then perform smoothing, in this scenario, after the AP quickly terminates the smoothing process in the shooting mode switch through the fourth instruction, it can directly determine that the current shooting mode switch has been completed.
[0094] Optionally, after determining that the current shooting mode is switched, the AP may record in the log the relevant information that the smoothing process ends early.
[0095] In addition, since the switch from night scene mode (such as SN / UD) to NM is to perform smoothing first and then configure the parameters of the BE module, in this scenario, after the AP quickly terminates the smoothing process in the shooting mode switch through the fourth instruction, the parameter configuration of the BE module will also end in advance.
[0096] Optionally, after determining that the current shooting mode switching is completed ahead of schedule, the AP may output notification information of shooting mode switching failure.
[0097] It should be noted that the task to be processed may be any task that is intended to shorten the shooting mode switching time.
[0098] Optionally, the above-mentioned tasks to be processed may include a SAT switching task. That is, the priority of SAT switching is higher than the priority of smoothing processing. When the HAL end of the AP determines to perform STA switching, the AP may send a fourth instruction to the PreISP to instruct to stop the smoothing processing during the shooting mode switching process.
[0099] In one embodiment of the present application, refer to Figure 5 As shown, in the image processing method provided in the embodiment of the present application, in step 110, the AP responds to the mode switching instruction and switches the shooting mode, which can be implemented in the following manner:
[0100] Step 1101: receiving a mode switching instruction, determining first identification information of a target shooting mode and second identification information of a target image sensor for executing the current mode switching;
[0101] Step 1102: When the first identification information is different from the identification information of the current shooting mode of the image processor, determine whether the identification information of the current image sensor is the second identification information;
[0102] Step 1103: When the identification information of the current image sensor is the second identification information, switch the shooting mode.
[0103] It should be understood that the target switching mode may be a shooting mode to be switched. The first identification information may uniquely identify the target switching mode.
[0104] In the embodiment of the present application, the first identification information of the target shooting identification can be carried by a mode switching instruction. The mode switching instruction can be triggered by the user's hand or by the collected ambient brightness, which is not limited in the embodiment of the present application.
[0105] After receiving the mode switching instruction, the AP may determine whether the current shooting mode is the target shooting mode. If the current shooting mode is the target shooting mode, it means that no mode switching is required, and the AP may not respond to the received mode switching instruction. If the current shooting mode is not the target shooting mode, it can be determined that the AP needs to switch the current shooting mode to the target shooting mode.
[0106] It should be noted that the AP can determine whether the identification information of the current PreISP shooting mode is the first identification information of the target shooting mode in each frame time according to the frame rate.
[0107] In the embodiment of the present application, the AP can compare the identification information of the current shooting mode with the first identification information of the target shooting mode to determine whether the current shooting mode is the target shooting mode. If the identification information of the current shooting mode is consistent with the first identification information of the target shooting mode, the current shooting mode is the target shooting mode. If the identification information of the current shooting mode is inconsistent with the first identification information of the target shooting mode, the current shooting mode is not the target shooting mode.
[0108] In addition, the switching of the shooting mode needs to be targeted at the same image sensor. Therefore, before switching the shooting mode, it is necessary to determine whether the current sensor is the target sensor for which the shooting mode switching needs to be implemented.
[0109] In the embodiment of the present application, when receiving the mode switching instruction, the AP may determine the second identification information of the target image sensor for executing the current mode switching.
[0110] Optionally, when receiving the mode switching instruction, the AP may initialize relevant setting information for the current shooting mode switching, wherein the relevant setting information may include the second identification information of the target image sensor for the current shooting mode switching.
[0111] In an embodiment of the present application, if the identification information of the current sensor is the same as the second identification information, it means that the current sensor is the target sensor for implementing shooting mode switching. In this case, the AP can execute the mode switching instruction to implement commands related to mode switching. Specifically, when the shooting mode switching process is in the smoothing processing stage, the AP can send a first instruction to the ISP, and the first instruction is used to instruct the execution of smoothing processing; or, when the shooting mode switching process is in the parameter configuration stage, the AP can send a second instruction to the ISP, and the second instruction is used to instruct the ISP to configure the parameters of the target shooting mode. In addition, when the AP receives a request instruction for a task to be processed with a higher priority during the process of switching the shooting mode, it can also send a fourth instruction to the ISP to instruct the ISP to stop the smoothing processing in the shooting mode switching process.
[0112] In addition, if the identification information of the current sensor is different from the second identification information, it means that the current sensor is not the target sensor for implementing the shooting mode switching. At this time, the AP can exit the mode switching process.
[0113] The image processing method provided in the embodiment of the present application is described in detail below in conjunction with specific application scenarios.
[0114] refer to Figure 6-Figure 8 As shown in the flowchart, the image processing method provided in the embodiment of the present application can be implemented in the following manner.
[0115] Step S1: AP determines first identification information of a target shooting mode.
[0116] It should be understood that the first identification information identifier can uniquely identify the target shooting mode that needs to be switched.
[0117] Step S2: The AP determines whether the first identification information of the target shooting mode is the same as the identification information of the shooting mode of the current PreISP.
[0118] It should be noted that the AP can determine whether the first identification information of the target shooting mode is the same as the identification information of the shooting mode of the current PreISP at each frame time according to the frame rate.
[0119] If the two are not the same, continue to step 3. If the two are the same, the AP determines that no shooting mode switching occurs, and ends this processing flow.
[0120] Step S3: Initialize the relevant setting information of the current shooting mode switching.
[0121] The relevant setting information may include second identification information (sensor ID) of the image sensor for switching the shooting mode this time.
[0122] Step S4: Determine whether the current sensor ID is an initialized sensor ID.
[0123] It should be noted that the switching of the shooting mode needs to be for the same image sensor. Therefore, before switching the shooting mode, it is necessary to determine whether the current sensor ID is the sensor ID for which the shooting mode switching needs to be implemented.
[0124] If the current sensor ID is the initialized sensor ID, continue to execute step S5.
[0125] If the current sensor ID is not the initialized sensor ID, it is determined that the current sensor does not match the sensor for this shooting mode switch or the shooting mode is not switched, and this processing flow ends.
[0126] Step S5: Execute commands related to switching the shooting mode.
[0127] The commands related to the shooting mode switching may include the shooting mode switching execution command “SCENE_SWITCH_AND_SMOOTH_PROCESS” and the fast stop command “SCENE_SWITCH_AND_SMOOTH_FASTSTOP”.
[0128] Optionally, the AP may execute "SCENE_SWITCH_AND_SMOOTH_PROCESS" in a normal shooting mode switching situation.
[0129] Optionally, during the process of switching the shooting mode, if the AP receives a request instruction for a high-priority task to be processed, it can execute "SCENE_SWITCH_AND_SMOOTH_FASTSTOP".
[0130] In this example, "SCENE_SWITCH_AND_SMOOTH_PROCESS" corresponds to step S6 to step S21. "SCENE_SWITCH_AND_SMOOTH_FASTSTOP" corresponds to step S22 to step S25.
[0131] refer to Figure 7 As shown, the processing flow of the "SCENE_SWITCH_AND_SMOOTH_PROCESS" command may include the following steps.
[0132] Step S6: The AP determines to execute "SCENE_SWITCH_AND_SMOOTH_PROCESS" to initialize the relevant parameters of this shooting mode switching.
[0133] Exemplarily, the parameters related to the switching of the shooting mode may include the number of smooth image frames. The AP may determine the number of smooth image frames according to at least one of the collected luminous flux, brightness, and illumination.
[0134] Step S7: The AP determines whether the target shooting mode currently set by the HAL is consistent with the target shooting mode currently being switched.
[0135] If they are consistent, continue to step 8. If they are inconsistent, end this processing flow.
[0136] Step S8: The AP determines the stage of the shooting mode switching process.
[0137] If it is in the parameter configuration stage of the BE module, step S9 to step S14 are executed.
[0138] If it is in the smoothing processing stage, step S15 to step S21 are executed.
[0139] Step S9: When it is determined that the BE module is in the parameter configuration stage, the AP sends a second instruction to the PreISP.
[0140] The second instruction is used to instruct PreISP to configure parameters of the target shooting mode.
[0141] Step S10: The AP determines whether the second instruction is executed successfully.
[0142] If the second instruction is executed successfully, a notification message of successful switching is fed back to the HAL end, and step S11 is continued.
[0143] If the second instruction fails to execute, the failed communication information is fed back to the HAL end, and the error handling process is executed.
[0144] Step S11: The AP determines whether the current shooting mode switching process includes one stage or two stages.
[0145] If the current shooting mode switching process has only one stage, step S12 is executed.
[0146] If the current shooting mode switching process includes two stages, step S13 is executed.
[0147] Step S12: The AP sets the current shooting mode to be switched successfully.
[0148] Step S13: The AP determines the direction of the current shooting mode switching.
[0149] If switching from SN / UD to NM, jump to step S14.1.
[0150] If switching from NM to SN / UD, jump to step S14.2.
[0151] Step S14.1: The AP determines that the current shooting mode switch is completed.
[0152] Since the switch from SN / UD to NM is to first perform smoothing and then perform parameter configuration of the BE module, after the parameter configuration of the BE module is successful, it can be determined that the current shooting mode switch is completed.
[0153] Step S14.2: The AP determines that the current shooting mode switching stage is a smoothing processing stage.
[0154] Since the switch from NM to SN / UD is to configure the parameters of the BE module first and then perform smoothing, after the parameters of the BE module are successfully configured, smoothing is required. Based on this, the AP can set the current shooting mode switching stage as the smoothing stage. The processing flow of the smoothing stage can continue to be executed.
[0155] Step S15: When it is determined that the smoothing process is in progress, the AP may determine whether the number of smooth image frames currently set by the HAL end is a non-zero value.
[0156] If the number of smooth image frames is 0, it means that the number of smooth image frames is not set. At this time, the AP can determine to end the processing flow of the smoothing processing stage.
[0157] If the smooth image frame number is a valid value other than 0, continue to execute step S16.
[0158] Step S16: The AP determines whether the current stage is the initial stage of the smoothing process.
[0159] Specifically, the AP may determine whether the current image is the first frame of the smoothing process, and if so, continue to execute step S17, and if not, continue to execute step S18.
[0160] It should be noted that the AP may perform the above determination in each frame time according to the frame rate.
[0161] Step S17: The AP sends the first instruction or the third instruction to the PreISP.
[0162] The first instruction is used to instruct the execution of smoothing processing, and the third instruction is used to instruct the execution of smoothing processing based on the number of smooth image frames determined in step S6.
[0163] Step S18: AP adds 1 to the number of image frames that have been smoothed.
[0164] Step S19: The AP determines whether the accumulated number of smoothed images is the same as the preset number of smooth image frames.
[0165] If the two are the same, it can be determined that the smoothing process is completed, and the AP can continue to execute step S20.
[0166] If the two are different, the smoothing process of the next frame of image will be continued.
[0167] Step S20: Determine the direction of the current shooting mode switch.
[0168] If switching from SN / UD to NM, jump to step S21.1.
[0169] If switching from NM to SN / UD, jump to step S21.2.
[0170] Step S21.1: The AP determines that the current shooting mode switching stage is a smoothing processing stage.
[0171] Since the switch from SN / UD to NM is to perform smoothing first and then configure the parameters of the BE module, the parameters of the BE module need to be configured after the smoothing is completed. Based on this, the AP can set the current shooting mode switching stage as the parameter configuration stage. The processing flow of the parameter configuration stage can continue to be executed.
[0172] Step S21.2: The AP determines that the current shooting mode switch is completed.
[0173] Since the switch from NM to SN / UD is to configure the parameters of the BE module first and then perform the smoothing process, after the parameters of the BE module are successfully configured, it can be determined that the current shooting mode switch is completed.
[0174] refer to Figure 8 As shown, the processing flow of the "SCENE_SWITCH_AND_SMOOTH_FASTSTOP" command may include the following steps.
[0175] Step S22, the AP determines to execute "SCENE_SWITCH_AND_SMOOTH_FASTSTOP", and the AP determines whether it is in the process of switching the shooting mode and is currently in the initial stage of smoothing processing.
[0176] If the AP is not in the process of switching the shooting mode or is not in the initial stage of the smoothing process, the AP determines to end the current processing flow. It should be understood that the AP determines that the shooting mode switching process is not in progress or is not in the smoothing process stage, and can determine that the shooting mode switching does not occur.
[0177] If the shooting mode is being switched and the current state is in the initial stage of smoothing, the process jumps to step S23.
[0178] Step S23: The AP sends a fourth instruction to the PreISP.
[0179] It should be understood that the AP can quickly stop the smoothing operation through the fourth instruction to increase the processing speed.
[0180] Step S24: The AP determines the current shooting mode switching direction.
[0181] If switching from NM to SN / UD, jump to step S25.1.
[0182] If switching from SN / UD to NM, jump to step S25.2.
[0183] Step S25.1: The AP determines that the current shooting mode switching has been completed, and records in the log the relevant information that the smoothing process has ended early.
[0184] Since the switch from NM to SN / UD is to configure the parameters of the BE module first and then perform the smoothing process, in this scenario, after the AP quickly terminates the smoothing process in the shooting mode switch through the fourth instruction, it can directly determine that the current shooting mode switch has been completed.
[0185] Step S25.2: The AP determines that the current shooting mode switching ends early, and outputs a notification message indicating that the shooting mode switching fails.
[0186] Since the switch from SN / UD to NM is smoothing first and then parameter configuration of BE module, in this scenario, after AP quickly terminates the smoothing process in the shooting mode switch through the fourth instruction, the parameter configuration of BE module will also be terminated in advance.
[0187] To sum up, the embodiment of the present application can split the shooting mode switching process into two stages and control them separately. When there is a higher priority task to be processed, the more time-consuming smoothing stage can be stopped, thereby optimizing the time required for switching when the shooting mode switching occurs simultaneously, thereby improving the user experience.
[0188] The following takes the scenario of automatically switching the shooting mode during the SAT switching process as an example for explanation.
[0189] In order to optimize the performance of SAT switching, the "SCENE_SWITCH_AND_SMOOTH_FASTSTOP" command can be called during the SAT switching process of the image sensor to shorten the shooting mode switching time and the SAT switching time, thereby optimizing the processing time. Fig. 9 As shown, the method of switching the shooting mode during the SAT switching process may include the following steps:
[0190] Step 1: The HAL end of the AP determines to perform SAT switching.
[0191] Step 2: The AP determines whether the shooting mode is currently being switched.
[0192] Optionally, the AP may obtain environmental parameters of the scene to be photographed during the SAT switching process, and trigger the switching of the shooting mode according to the environmental parameters.
[0193] If the shooting mode is currently being switched, proceed to step 3. If the shooting mode is not currently being switched, jump to step 7.
[0194] Step 3: The AP calls the "SCENE_SWITCH_AND_SMOOTH_FASTSTOP" command to control the PreISP to quickly stop the smoothing operation.
[0195] Step 4: The AP determines whether it is currently in the smoothing processing stage.
[0196] If it is in the smoothing processing stage, execute step 5. If it is not in the smoothing processing stage, end the current processing flow.
[0197] Step 5: Determine the switching direction of the shooting mode.
[0198] If switching from NM to SN / UD, jump to step 6.1.
[0199] If switching from SN / UD to NM, jump to step 6.2.
[0200] Step 6.1: The AP determines that the current shooting mode switching has been completed, and records the relevant information that the smoothing process ends early in the log.
[0201] Since the switch from NM to SN / UD is to configure the parameters of the BE module first and then perform the smoothing process, in this scenario, after the AP quickly terminates the smoothing process in the shooting mode switch through the fourth instruction, it can directly determine that the current shooting mode switch has been completed.
[0202] Step 6.2: The AP determines that the current shooting mode switching ends early, and outputs a notification message indicating that the shooting mode switching fails.
[0203] Since the switch from SN / UD to NM is smoothing first and then parameter configuration of BE module, in this scenario, after AP quickly terminates the smoothing process in the shooting mode switch through the fourth instruction, the parameter configuration of BE module will also be terminated in advance.
[0204] Step 7: The AP determines to start SAT switching.
[0205] The embodiment of the present application can implement a quick-stop mechanism for smooth processing. When a shooting mode switch is encountered during the SAT switching process, the quick-stop mechanism for smooth processing can be used to accelerate the switching speed of the SAT.
[0206] The preferred embodiments of the present application are described in detail above in conjunction with the accompanying drawings. However, the present application is not limited to the specific details in the above embodiments. Within the technical concept of the present application, the technical solution of the present application can be subjected to a variety of simple modifications, and these simple modifications all belong to the protection scope of the present application. For example, the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, the present application will not further explain various possible combinations. For another example, the various different embodiments of the present application can also be arbitrarily combined, as long as they do not violate the idea of the present application, they should also be regarded as the contents disclosed in the present application. For another example, under the premise of no conflict, the various embodiments and / or the technical features in the various embodiments described in the present application can be arbitrarily combined with the prior art, and the technical solution obtained after the combination should also fall within the protection scope of the present application.
[0207] An embodiment of the present application also provides an image processing device, which is applied to an application processor. Fig.10 As shown, the image processing device 1000 may include:
[0208] The sending unit 1001 is configured to respond to a mode switching instruction and switch the shooting mode; the shooting mode switching process includes a parameter configuration stage and a smoothing processing stage; when the shooting mode switching process is in the smoothing processing stage, a first instruction is sent to the image processor, and the first instruction is used to instruct the execution of smoothing processing; and when the shooting mode switching process is in the parameter configuration stage, a second instruction is sent to the image processor, and the second instruction is used to instruct the image processor to perform parameter configuration of the target shooting mode.
[0209] Optionally, the image processing device 1000 may further include an acquiring unit and a determining unit.
[0210] The acquisition unit may be configured to acquire environmental parameters of the scene to be photographed;
[0211] The determining unit may be configured to determine the number of images matching the environmental parameter, where the number of images refers to the number of images that need to be smoothed during the switching of the shooting mode;
[0212] The sending unit 1001 is further configured to send a third instruction to the image processor, where the third instruction is used to instruct the image processor to perform the smoothing process based on the number of images to complete the switching of the shooting mode.
[0213] Optionally, the environmental parameter includes at least one of the following:
[0214] Luminous flux, brightness, illumination.
[0215] Optionally, the determination unit may also be configured to determine the cumulative number of images that have undergone the smoothing process; if the cumulative number of images matches the number of images, it is determined that the smoothing process in the shooting mode switching process is completed.
[0216] Optionally, the sending unit 1001 is also configured to send a fourth instruction to the image processor if a request instruction for a task to be processed is received during the process of switching the shooting mode, and the fourth instruction is used to instruct the image processor to stop the smoothing processing during the shooting mode switching process, and the priority of the task to be processed is higher than the priority of the switching shooting mode.
[0217] Optionally, the sending unit 1001 is further configured to, if the task to be processed is received, determine whether the shooting mode switching process is in the initial stage of the smoothing processing; if the shooting mode switching process is in the initial stage of the smoothing processing, send the fourth instruction to the image processor.
[0218] Optionally, the determining unit is further configured to determine that the current shooting mode switching is completed if the shooting mode switching direction is switched from the normal mode to the night scene mode;
[0219] If the shooting mode switching direction is switched from the night scene mode to the normal mode, it is determined that the shooting mode switching ends in advance.
[0220] Optionally, the tasks to be processed include SAT tasks.
[0221] Optionally, the image processing device 1000 further includes an instruction receiving unit configured to receive a mode switching instruction, determine first identification information of a target shooting mode and second identification information of a target image sensor for performing the current mode switching;
[0222] The determination unit is also configured to determine whether the identification information of the current image sensor is the second identification information when the first identification information is different from the identification information of the current shooting mode of the image processor; and switch the shooting mode when the identification information of the current image sensor is the second identification information.
[0223] An embodiment of the present application further provides an image processing device applied to an image processor, referring to Fig.11 As shown, the image processing device 1100 may include:
[0224] The receiving unit 1101 is configured to receive a first instruction sent by the application processor when the shooting mode switching process is in a smoothing processing stage, wherein the first instruction is used to instruct the execution of smoothing processing; and / or, when the shooting mode switching process is in a parameter configuration stage, receive a second instruction sent by the application processor, wherein the second instruction is used to instruct the image processor to perform parameter configuration of the target shooting mode.
[0225] Optionally, the receiving unit 1101 is further configured to receive a third instruction sent by the application processor, wherein the third instruction is used to instruct to perform the smoothing processing based on the number of images to complete the switching of the shooting mode; the number of images is determined by the application processor based on environmental parameters of the scene to be shot.
[0226] Optionally, the environmental parameter includes at least one of the following:
[0227] Luminous flux, brightness, illumination.
[0228] Optionally, the receiving unit 1101 is further configured to receive a fourth instruction sent by the application processor, wherein the fourth instruction is used to instruct to stop the smoothing process in the process of switching the shooting mode.
[0229] The fourth instruction is sent by the application processor when receiving a request instruction for a task to be processed during the process of switching the shooting mode, and the priority of the task to be processed is higher than the priority of switching the shooting mode.
[0230] Optionally, the tasks to be processed include SAT tasks.
[0231] Fig.12 : is a schematic structural diagram of a chip in an embodiment of the present application. The chip can be an application processor in the above embodiment or an image processor. Fig.12 The chip 1200 shown includes a processor 1210, which can call and run a computer program from a memory to implement the image processing method in the embodiment of the present application.
[0232] Alternatively, if Fig.12 As shown, the chip 1200 may further include a memory 1220. The processor 1210 may call and run a computer program from the memory 1220 to implement the method in the embodiment of the present application.
[0233] The memory 1220 may be a separate device independent of the processor 1210 , or may be integrated into the processor 1210 .
[0234] Optionally, the chip 1200 may further include an input interface 1230. The processor 1210 may control the input interface 1230 to communicate with other devices or chips, and specifically, may obtain information or data sent by other devices or chips.
[0235] Optionally, the chip 1200 may further include an output interface 1240. The processor 1210 may control the output interface 1240 to communicate with other devices or chips, and specifically, may output information or data to other devices or chips.
[0236] Optionally, the chip can be applied to the application processor in the embodiments of the present application, and the chip can implement the corresponding processes implemented by the application processor in the various methods of the embodiments of the present application. For the sake of brevity, they will not be repeated here.
[0237] Optionally, the chip can be applied to the image processor in the embodiment of the present application, and the chip can implement the corresponding processes implemented by the image processor in each method of the embodiment of the present application, which will not be described here for the sake of brevity.
[0238] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.
[0239] Fig.131 is a schematic block diagram of a communication system / communication device 1300 provided in an embodiment of the present application. Fig.13 As shown, the communication device 1300 includes an application processor 1301 and an image processor 1302 .
[0240] The application processor 1301 is configured to switch the shooting mode in response to a mode switching instruction; the shooting mode switching process includes a parameter configuration stage and a smoothing processing stage; when the shooting mode switching process is in the smoothing processing stage, a first instruction is sent to the image processor 1302, and the first instruction is used to instruct the execution of smoothing processing; and when the shooting mode switching process is in the parameter configuration stage, a second instruction is sent to the image processor 1302, and the second instruction is used to instruct the image processor to perform parameter configuration of the target shooting mode;
[0241] The image processor 1302 is configured to receive a first instruction sent by the application processor 1301 when the shooting mode switching process is in a smoothing processing stage, wherein the first instruction is used to instruct the execution of smoothing processing; and / or, when the shooting mode switching process is in a parameter configuration stage, receive a second instruction sent by the application processor 1301, wherein the second instruction is used to instruct the image processor to perform parameter configuration of the target shooting mode.
[0242] Optionally, the application processor 1301 is further configured to obtain environmental parameters of the scene to be photographed; determine the number of images that match the environmental parameters, and the number of images refers to the number of images that need to be smoothed during the shooting mode switching process; send a third instruction to the image processor 1302, and the third instruction is used to instruct the image processor 1302 to perform the smoothing process based on the number of images to complete the switching of the shooting mode.
[0243] Correspondingly, the image processor 1302 is further configured to receive a third instruction sent by the application processor 1301 .
[0244] Optionally, the environmental parameter includes at least one of the following:
[0245] Luminous flux, brightness, illumination.
[0246] Optionally, the application processor 1301 is further configured to determine the cumulative number of images that have undergone the smoothing process; if the cumulative number of images matches the number of images, it is determined that the smoothing process in the shooting mode switching process is completed.
[0247] Optionally, the application processor 1301 is further configured to send a fourth instruction to the image processor 1302 if it receives a request instruction for a task to be processed during the process of switching the shooting mode. The fourth instruction is used to instruct the image processor 1302 to stop the smoothing processing during the shooting mode switching process, and the priority of the task to be processed is higher than the priority of the switching shooting mode.
[0248] Optionally, the application processor 1301 is also configured to, if receiving the task to be processed, determine whether the shooting mode switching process is in the initial stage of the smoothing processing; if the shooting mode switching process is in the initial stage of the smoothing processing, send the fourth instruction to the image processor 1302.
[0249] Correspondingly, the image processor 1302 is further configured to receive a fourth instruction sent by the application processor, where the fourth instruction is used to instruct to stop the smoothing process during the shooting mode switching process.
[0250] Optionally, the application processor 1301 is further configured to determine that the shooting mode switch is completed if the shooting mode switch direction is switched from normal mode to night mode; if the shooting mode switch direction is switched from night mode to normal mode, determine that the shooting mode switch is ended early.
[0251] Optionally, the tasks to be processed include SAT tasks.
[0252] Optionally, the application processor 1301 is further configured to receive a mode switching instruction, determine first identification information of the target shooting mode and second identification information of the target image sensor for performing this mode switching; when the first identification information is different from the identification information of the current shooting mode of the image processor, determine whether the identification information of the current image sensor is the second identification information; and switch the shooting mode when the identification information of the current image sensor is the second identification information.
[0253] It should be noted that the application processor 1301 can be used to implement the corresponding functions implemented by the application processor AP in the above method, and the image processor 1302 can be used to implement the corresponding functions implemented by the image processor ISP in the above method. For the sake of brevity, they are not repeated here.
[0254] It should be understood that the processor of the embodiment of the present application may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method embodiment can be completed by the hardware integrated logic circuit or software instructions in the processor. The above processor can be a general processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to perform, or the hardware and software modules in the decoding processor are combined and performed. The software module can be located in a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, and other mature storage media in the art. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.
[0255] It can be understood that the memory in the embodiments of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAM bus random access memory (DR RAM). It should be noted that the memory of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.
[0256] It should be understood that the above-mentioned memory is exemplary but not restrictive. For example, the memory in the embodiments of the present application may also be static random access memory (static RAM, SRAM), dynamic random access memory (dynamic RAM, DRAM), synchronous dynamic random access memory (synchronous DRAM, SDRAM), double data rate synchronous dynamic random access memory (double data rate SDRAM, DDR SDRAM), enhanced synchronous dynamic random access memory (enhanced SDRAM, ESDRAM), synchronous link dynamic random access memory (synch link DRAM, SLDRAM) and direct memory bus random access memory (Direct Rambus RAM, DR RAM), etc. That is to say, the memory in the embodiments of the present application is intended to include but not limited to these and any other suitable types of memory.
[0257] The present application also provides a computer storage medium, specifically a computer-readable storage medium, on which computer instructions are stored. When the computer storage medium is located in an application processor and / or an image processor, the computer instructions are executed by the processor to implement any step of the above-mentioned image processing method in the present application.
[0258] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.
[0259] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.
[0260] In addition, all functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may be separately configured as a unit, or at least two units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.
[0261] A person of ordinary skill in the art can understand that: all or part of the steps of implementing the above-mentioned method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium, which, when executed, executes the steps of the above-mentioned method embodiment; and the aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, disks or optical disks.
[0262] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.
[0263] It should be noted that the technical solutions described in the embodiments of the present application can be combined arbitrarily without conflict.
[0264] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
Claims
1. An image processing method, characterized in that: Applied to an application processor, the method comprises: Responding to the mode switching instruction, switching the shooting mode; the shooting mode switching process includes a parameter configuration stage and a smoothing processing stage; When the shooting mode switching process is in the smoothing processing stage, sending a first instruction to the image processor, wherein the first instruction is used to instruct to perform smoothing processing; and When the shooting mode switching process is in the parameter configuration stage, sending a second instruction to the image processor, where the second instruction is used to instruct the image processor to perform parameter configuration of the target shooting mode; During the process of switching the shooting mode, if a request instruction of a task to be processed is received, and the priority of the task to be processed is higher than the priority of the switching shooting mode, then determining whether the shooting mode switching process is in the initial stage of the smoothing process; if the shooting mode switching process is in the initial stage of the smoothing process, sending a fourth instruction to the image processor; the fourth instruction is used to instruct the image processor to stop the smoothing process during the shooting mode switching process; If the shooting mode is switched from the normal mode to the night mode, it is determined that the shooting mode switching is completed; If the shooting mode switching direction is switched from the night scene mode to the normal mode, determining that the current shooting mode switching ends in advance; Wherein, when the shooting mode switching process is in the smoothing processing stage, before sending the first instruction to the image processor, it includes: Obtain environmental parameters of the scene to be shot; The number of images is determined in combination with the shooting mode switching direction and the environmental parameters, where the number of images refers to the number of images that need to be smoothed during the shooting mode switching process.
2. The method according to claim 1, wherein: After determining the number of images by combining the shooting mode switching direction with the environmental parameters, the method further includes: A third instruction is sent to the image processor, wherein the third instruction is used to instruct the image processor to perform the smoothing process based on the number of images to complete the switching of the shooting mode.
3. The method according to claim 1, characterized in that The environmental parameters include at least one of the following: Luminous flux, brightness, illumination.
4. The method according to claim 1, characterized in that: The method further comprises: Determining the cumulative number of images that have undergone the smoothing process; If the accumulated number of images matches the number of images, it is determined that the smoothing process in the shooting mode switching process is completed.
5. The method according to claim 1, characterized in that The tasks to be processed include spatial alignment change (SAT) tasks.
6. The method according to any one of claims 1 to 5, characterized in that: The responding mode switching instruction and switching the shooting mode include: receiving the mode switching instruction, and determining first identification information of the target shooting mode and second identification information of the target image sensor for executing the current mode switching; In a case where the first identification information is different from the identification information of the current shooting mode of the image processor, determining whether the identification information of the current image sensor is the second identification information; When the identification information of the current image sensor is the second identification information, the shooting mode is switched.
7. An image processing method, characterized in that: Applied to an image processor, the method comprises: When the shooting mode switching process is in a smoothing processing stage, receiving a first instruction sent by an application processor, the first instruction is used to instruct to perform smoothing processing; wherein the application processor is used to obtain environmental parameters of a scene to be shot; and determining the number of images in combination with the shooting mode switching direction and the environmental parameters, the number of images referring to the number of images that need to be smoothed during the shooting mode switching process; When the shooting mode switching process is in a parameter configuration stage, receiving a second instruction sent by the application processor, where the second instruction is used to instruct the image processor to perform parameter configuration of a target shooting mode; Receive a fourth instruction sent by the application processor; the fourth instruction is used to instruct the image processor to stop the smoothing process in the shooting mode switching process; the fourth instruction is sent to the image processor if the application processor receives a request instruction for a task to be processed, and the priority of the task to be processed is higher than the priority of switching the shooting mode; if the shooting mode switching process is in the initial stage of the smoothing process, the fourth instruction is sent to the image processor; the application processor is also used to determine that the shooting mode switching is completed if the shooting mode switching direction is switched from the normal mode to the night scene mode; if the shooting mode switching direction is switched from the night scene mode to the normal mode, determine that the shooting mode switching is ended early.
8. The method according to claim 7, characterized in that The method further comprises: A third instruction sent by an application processor is received, where the third instruction is used to instruct to perform the smoothing process based on the number of images to complete the switching of the shooting mode.
9. The method according to claim 8, characterized in that The environmental parameters include at least one of the following: Luminous flux, brightness, illumination.
10. The method according to claim 7, characterized in that The tasks to be processed include spatial alignment change (SAT) tasks.
11. An image processing device, characterized in that: For application processors, including: a sending unit configured to respond to a mode switching instruction, the mode switching instruction being used to instruct switching of a shooting mode; the shooting mode switching process comprising a parameter configuration stage and a smoothing processing stage; when the shooting mode switching process is in the smoothing processing stage, sending a first instruction to an image processor, the first instruction being used to instruct execution of smoothing processing; and, when the shooting mode switching process is in the parameter configuration stage, sending a second instruction to the image processor, the second instruction being used to instruct the image processor to perform parameter configuration of a target shooting mode; An acquisition unit, configured to acquire environmental parameters of a scene to be photographed; a determination unit configured to determine the number of images in combination with the shooting mode switching direction and the environmental parameter, wherein the number of images refers to the number of images that need to be smoothed during the shooting mode switching process; The sending unit is further configured to, during the process of switching the shooting mode, if a request instruction of a task to be processed is received, and the priority of the task to be processed is higher than the priority of the switching shooting mode, determine whether the shooting mode switching process is in the initial stage of the smoothing process; if the shooting mode switching process is in the initial stage of the smoothing process, send a fourth instruction to the image processor; the fourth instruction is used to instruct the image processor to stop the smoothing process during the shooting mode switching process; The determination unit is further configured to determine that the current shooting mode switching is completed if the shooting mode switching direction is switched from the normal mode to the night scene mode; and to determine that the current shooting mode switching is ended early if the shooting mode switching direction is switched from the night scene mode to the normal mode.
12. An image processing device, characterized in that: Applied to image processors, including: A receiving unit is configured to receive a first instruction sent by an application processor when the shooting mode switching process is in a smoothing processing stage, wherein the first instruction is used to instruct the execution of smoothing processing; wherein the application processor is used to obtain environmental parameters of a scene to be shot; determine the number of images in combination with the shooting mode switching direction and the environmental parameters, wherein the number of images refers to the number of images that need to be smoothed during the shooting mode switching process; and receive a second instruction sent by the application processor when the shooting mode switching process is in a parameter configuration stage, wherein the second instruction is used to instruct the image processor to perform parameter configuration of a target shooting mode; The receiving unit is further configured to receive a fourth instruction sent by the application processor; the fourth instruction is used to instruct the image processor to stop the smoothing process in the shooting mode switching process; the fourth instruction is used by the application processor to determine whether the shooting mode switching process is in the initial stage of the smoothing process if it receives a request instruction for a task to be processed, and the priority of the task to be processed is higher than the priority of switching the shooting mode; if the shooting mode switching process is in the initial stage of the smoothing process, it is sent to the image processor; the application processor is also used to determine that the shooting mode switching is completed if the shooting mode switching direction is switched from the normal mode to the night scene mode; if the shooting mode switching direction is switched from the night scene mode to the normal mode, determine that the shooting mode switching is ended early.
13. An image processing system, characterized in that: include: Application processor and image processor; among them, The application processor is configured to respond to a mode switching instruction and switch the shooting mode; the shooting mode switching process includes a parameter configuration stage and a smoothing processing stage; obtain environmental parameters of the scene to be shot; determine the number of images in combination with the shooting mode switching direction and the environmental parameters, the number of images referring to the number of images that need to be smoothed during the shooting mode switching process; when the shooting mode switching process is in the smoothing processing stage, send a first instruction to the image processor, the first instruction is used to instruct the execution of smoothing processing; and when the shooting mode switching process is in the parameter configuration stage, send a second instruction to the image processor, the second instruction is used to instruct the image processor to perform a target shooting mode ; in the process of switching the shooting mode, if a request instruction of a task to be processed is received, and the priority of the task to be processed is higher than the priority of the switching shooting mode, then it is determined whether the shooting mode switching process is in the initial stage of the smoothing process; if the shooting mode switching process is in the initial stage of the smoothing process, a fourth instruction is sent to the image processor; the fourth instruction is used to instruct the image processor to stop the smoothing process in the shooting mode switching process; if the shooting mode switching direction is switched from the normal mode to the night scene mode, it is determined that the current shooting mode switching has been completed; if the shooting mode switching direction is switched from the night scene mode to the normal mode, it is determined that the current shooting mode switching is ended in advance; The image processor is configured to receive a first instruction sent by the application processor when the shooting mode switching process is in a smoothing processing stage, the first instruction being used to instruct the execution of smoothing processing; to receive a second instruction sent by the application processor when the shooting mode switching process is in a parameter configuration stage, the second instruction being used to instruct the image processor to perform parameter configuration of the target shooting mode; and to receive the fourth instruction sent by the application processor.
14. A chip, comprising: A processor, configured to call and run a computer program from a memory, so that a device equipped with the chip executes a method as claimed in any one of claims 1 to 6, or a method as claimed in any one of claims 7 to 10.
15. A communication device, characterized in that: The method comprises an application processor and an image processor, wherein the application processor is used to execute the method according to any one of claims 1 to 6, and the image processor is used to execute the method according to any one of claims 7 to 10.
16. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 or the steps in the image processing method according to any one of claims 7 to 10 are implemented.
