Image processing method, device and equipment

Abstract

The invention discloses an image processing method, device and equipment, and the method comprises the steps: obtaining first information; the first information represents information related to image acquisition; under the condition that the first information does not meet the preset condition, the electronic equipment is controlled to operate in a first mode; and under the condition that the first information meets a preset condition, controlling the electronic equipment to operate in a second mode, wherein the electronic equipment represents equipment for operating the image processing method; the first mode represents that statistical information of the collected image is determined by using an image information statistical algorithm solidified on hardware of the electronic equipment; the second mode represents that the statistical information of the collected image is determined by using a software image information statistical algorithm running in the electronic equipment.

Description

Technical Field

[0001] The present application relates to but is not limited to the field of computer technology, and in particular to an image processing method, device and equipment. Background Art

[0002] During the image acquisition process, the camera's image signal processor (ISP) is required to post-process the signal output by the front-end image sensor.

[0003] However, the ISP logic unit is a hardware unit. Therefore, when the ISP logic unit is used to count the real-time information of the current image, the information counting method cannot be adjusted in real time to adapt to different image shooting scenes, which leads to poor image shooting quality and a bad user experience. Summary of the invention

[0004] In view of this, the present application at least provides an image processing method, device and equipment.

[0005] The technical solution of this application is implemented as follows:

[0006] On the one hand, the present application provides a data processing method, comprising:

[0007] Acquire first information; the first information represents information related to image acquisition;

[0008] When the first information does not meet the preset condition, controlling the electronic device to operate in a first mode;

[0009] When the first information meets the preset condition, the electronic device is controlled to operate in the second mode; wherein,

[0010] Electronic device characterizes the device running the image processing method;

[0011] The first method is characterized by using an image information statistical algorithm embedded in the hardware of the electronic device to determine the statistical information of the collected image; the second method is characterized by using a software image information statistical algorithm running in the electronic device to determine the statistical information of the collected image.

[0012] In some embodiments, the first information includes at least one of the second information, the third information, and the fourth information; wherein,

[0013] The second information represents the image acquisition scene information; the third information represents the interactive information obtained by the electronic device; and the fourth information represents the feature information detected from the acquired image.

[0014] In some embodiments, the first information includes second information; the second information includes at least one of the following: color information, ambient light information, and motion status information;

[0015] When the first information satisfies a preset condition, controlling the electronic device to operate in a second mode includes:

[0016] When the second information satisfies a preset image acquisition scenario condition, the electronic device is controlled to determine statistical information of the acquired image in a second manner.

[0017] In some embodiments, the first information includes third information;

[0018] When the first information satisfies a preset condition, controlling the electronic device to operate in a second mode includes:

[0019] When the third information includes obtaining an interactive operation of the user on the first area in the image preview interface of the electronic device, the electronic device is controlled to determine statistical information of the acquired image in a second manner.

[0020] In some embodiments, the first information includes fourth information; the fourth information represents object feature information detected from the acquired image;

[0021] When the first information satisfies a preset condition, controlling the electronic device to operate in a second mode includes:

[0022] In a case where the fourth information includes feature information of the target object, the electronic device is controlled to determine statistical information of the acquired image in a second manner.

[0023] In some embodiments, the image processing method further includes:

[0024] Based on the second method, determining first statistical information corresponding to a first area or a target object in a first image captured by the electronic device, and second statistical information corresponding to an area in the first image other than the first area or an area containing the target object;

[0025] Determining target statistical information corresponding to the first image based on the first statistical information and the second statistical information;

[0026] The statistical accuracy of the first statistical information is higher than the statistical accuracy of the second statistical information.

[0027] In some embodiments, the first information includes fourth information; the fourth information represents focus value statistics detected from the acquired image;

[0028] When the first information satisfies a preset condition, controlling the electronic device to operate in a second mode includes:

[0029] When the focus value statistical information satisfies a preset focus value statistical information condition, the electronic device is controlled to operate in a second mode.

[0030] In some embodiments, obtaining the first information includes:

[0031] In response to the trigger information, acquiring focus value statistical information;

[0032] The trigger information includes at least one of the following:

[0033] Detecting that the electronic device enters a low-light scene;

[0034] Detecting that the electronic device enters a monochrome scene;

[0035] It is detected that the electronic device enters a non-stationary state from a stationary state.

[0036] On the other hand, the present application also provides an image processing device, which is provided in an electronic device, comprising:

[0037] An information acquisition module, used to acquire first information; the first information represents information related to image acquisition;

[0038] An operation control module, used to control the electronic device to operate in a first mode when the first information does not meet the preset condition; and to control the electronic device to operate in a second mode when the first information meets the preset condition;

[0039] Among them, the first method represents the use of an image information statistical algorithm solidified in the hardware of the electronic device to determine the statistical information of the collected image; the second method represents the use of a software image information statistical algorithm running in the electronic device to determine the statistical information of the collected image.

[0040] On the other hand, the present application also provides an electronic device, including a decision component, an image signal processor and an auxiliary processor; wherein,

[0041] A decision-making component, configured to obtain first information; if the first information does not satisfy a preset condition, control the electronic device to operate in a first manner; if the first information satisfies a preset condition, control the electronic device to operate in a second manner; wherein the first information represents information related to image acquisition;

[0042] The first method is characterized by using an image information statistical algorithm solidified on an image signal processor to determine the statistical information of the collected image;

[0043] The second method is characterized by using a software image information statistics algorithm running on an auxiliary processor to determine the statistical information of the acquired image.

[0044] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and are not intended to limit the technical solutions of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and are used together with the specification to illustrate the technical solution of the present application.

[0046] Figure 1 It is a system framework diagram of an image acquisition system;

[0047] Figure 2 A schematic diagram of an implementation flow of an image processing method provided in this application;

[0048] Figure 3 A schematic diagram of an implementation flow of an image processing method according to an embodiment of the present application;

[0049] Figure 4 A schematic diagram of the structure of an image processing device provided by the present application;

[0050] Figure 5 A schematic diagram of a device for realizing auto-focusing of an image processing device provided by the present application;

[0051] Figure 6 A hardware entity schematic diagram of an electronic device provided in this application. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solutions and advantages of the present application clearer, the technical solutions of the present application are further elaborated in detail below in conjunction with the drawings and embodiments. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0053] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0054] The terms "first / second / third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It is understandable that "first / second / third" can be interchanged with a specific order or sequence where permitted so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0055] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing this application and are not intended to limit this application.

[0056] Before further explaining the present application in detail, the nouns and terms involved in the present application are explained first. The nouns and terms involved in the present application are subject to the following interpretations:

[0057] 1) Auto Exposure (AE) refers to automatically determining the exposure according to the light conditions to achieve the appropriate brightness level in different lighting conditions and scenes, so that the captured video or image is neither too dark nor too bright. To achieve this goal, the lens aperture, sensor exposure time, sensor analog gain, and sensor / ISP digital gain need to be adjusted.

[0058] 2) Automatic Focus (AF) means that the camera measures the distance in a specific area (for example, the central area, or any corner seen in the viewfinder) and adjusts the focal length of the lens to achieve focusing, so as to improve the clarity of the captured image.

[0059] 3) Automatic white balance (AWB) refers to changing the amplification ratio of the RGB photosensitive circuit signal to keep the image color affected by ambient light consistent with the real color of the object. The core of this process is to make white objects appear white under any light source, thereby achieving accurate color restoration. The implementation of automatic white balance relies on estimating the color temperature of the ambient light source and adjusting the color of the captured image by calculating the gain to ensure that it is close to the real color of the object. Specific implementation algorithms include gray world algorithm, perfect reflection algorithm, dynamic threshold algorithm, etc.

[0060] 4) ISP pipeline is a complex image processing process, which consists of a series of processing modules, which are driven by a clock of several hundred MHz and run at high speed at the same time, ensuring that image data is continuously transferred from one module to the next until all algorithm processing is completed and finally flows out from the final stage of the pipeline in the form of YUV image or RGB image. This process involves multiple key steps and algorithms, including but not limited to color space conversion, de-mosaicing, lens shading correction, noise reduction, automatic white balance, automatic exposure, automatic focus, etc.

[0061] In order to better understand the image processing method provided by the present application, the process of performing image acquisition using an image acquisition system in the related art is first described below.

[0062] Figure 1 An image acquisition system is shown. Figure 1As shown, the image acquisition system 100 includes a lens 110, a sensor 120, an ISP pipeline module 130, an AE algorithm module 140, an AWB algorithm module 150, an AF algorithm module 160 and a motor 170; wherein,

[0063] First, the unprocessed image data (RAW image data) collected by the lens 110 and the sensor 120 is transmitted to the ISP pipeline module 130;

[0064] Then, the ISP pipeline module 130 uses the AE statistical algorithm, AWB statistical algorithm and AF statistical algorithm solidified therein to determine the statistical information of the currently acquired image data based on the RAW image data; wherein the statistical information includes AE statistical information related to automatic exposure, AWB statistical information related to automatic white balance and AF statistical information related to automatic focus;

[0065] Afterwards, the ISP pipeline module 130 sends the AE statistical information to the AE algorithm module 140, sends the AWB statistical information to the AWB algorithm module 150, and sends the AF statistical information to the AF algorithm module 160. In this way, the AE algorithm module 140 uses the preset AE algorithm to determine the gain and exposure of the sensor 120 based on the AE statistical information, and sends the determined gain and exposure information to the sensor 120; the AWB algorithm module 150 uses the preset AWB algorithm to determine the white balance gain and color temperature and other information based on the AWB statistical information, and sends the determined white balance gain and color temperature and other information to the ISP pipeline module 130; the AF algorithm module 160 uses the preset AF algorithm to determine the current focus status based on the AF statistical information, and calculates the moving step length of the motor 170, and sends the moving step length to the motor 170, so that the motor 170 drives the lens 110 to move based on the moving step length information.

[0066] For example, the AF algorithm module 160 may include an AF algorithm submodule and an AF control module. The AF algorithm submodule is used to determine the moving step and moving direction of the motor 170 based on the AF statistical information using the built-in AF algorithm; the AF algorithm submodule sends the calculated moving step and moving direction information to the AF control module; the AF control module generates a motor control signal based on the received moving step and moving direction information, and sends it to the motor 170, so that the motor 170 performs the corresponding moving action.

[0067] As mentioned above, the AE statistical algorithm, AWB statistical algorithm and AF statistical algorithm are statistical algorithms fixed on the ISP hardware, and there is a problem that the hardware cannot be modified after the board is sealed. At the same time, there are many technical problems with the statistical algorithms currently fixed on the ISP hardware. For example, the hardware AE statistical algorithm, the hardware AWB statistical algorithm and the hardware AF algorithm all perform information statistics on the global image collected, and cannot perform information statistics separately for the area of ​​interest to the user.

[0068] In this way, if these statistical algorithms are found to have defects or need to be enhanced or customized after the hardware is sealed, the implementation steps of the algorithm enhancement or customization will be very complicated.

[0069] To solve this problem, the related art proposes to implement the above statistical algorithm by software, that is, to use the algorithm calculation unit in the electronic device to run the software statistical algorithm. However, using the software statistical algorithm to count the statistical information related to AE, AWB and AF in the image in real time will cause high power consumption problems for the electronic device, affecting the basic power consumption of the electronic device in the camera scene.

[0070] Based on this, the present application provides an image processing method, which can be executed by a processor of an electronic device. The electronic device may refer to a server, a laptop, a tablet computer, a desktop computer, a smart TV, a set-top box, a mobile device (such as a mobile phone, a portable video player, a personal digital assistant, a dedicated messaging device, a portable gaming device) and other devices with image processing capabilities. In some embodiments, the electronic device may be a monitoring electronic device, a consumer electronic device, or any type of electronic device that supports a front-end image processing (Pre-ISP) scenario, and the like.

[0071] Figure 2 A schematic diagram of the implementation process of an image processing method provided in this application is shown in FIG. Figure 2 As shown, the method includes the following steps S201 to S203:

[0072] Step S201, obtaining first information; the first information represents information related to image acquisition.

[0073] Here, the first information refers to information determined during the image acquisition process and used to determine the image acquisition method.

[0074] In some embodiments, the first information may include image acquisition scene information, lighting condition information, color complexity information of the image acquisition object, and other information that may affect the image acquisition effect.

[0075] Step S202, when the first information does not meet the preset condition, controlling the electronic device to operate in a first mode;

[0076] Step S203, when the first information meets the preset condition, control the electronic device to operate in a second mode; wherein,

[0077] The electronic device represents a device for executing the image processing method;

[0078] The first method represents the use of an image information statistical algorithm embedded in the hardware of the electronic device to determine the statistical information of the collected image; the second method represents the use of a software image information statistical algorithm running in the electronic device to determine the statistical information of the collected image.

[0079] Here, the preset condition refers to a condition for switching the operation mode of the electronic device that is preset based on the type of the first information. That is, when the first information is at least one type of information, a corresponding preset condition is set for the at least one type of information.

[0080] For example, when the first information is image acquisition scene information, the preset condition may be a specified image acquisition scene, such as a monochrome scene with a single or similar color, a low illumination scene with a light intensity less than 1 lux, a mixed light scene with a mixture of natural light and artificial light, and so on.

[0081] For another example, when the first information is feature information of an image acquisition object, the preset condition may be specified feature information of the object, such as specified facial features of a person, specified shape features, and so on.

[0082] The first mode and the second mode correspond to different modes for the electronic device to determine image statistical information. That is, in the first mode, the electronic device determines the statistical information of the collected image using an image information statistical algorithm fixed on the hardware of the electronic device (i.e., a hardware statistical algorithm); in the second mode, the electronic device determines the statistical information of the collected image using a software image information statistical algorithm running in the electronic device (i.e., a software statistical algorithm).

[0083] Here, the statistical information of the image refers to information obtained by statistically analyzing designated information in the image data output by the image sensor of the electronic device, for example, AE statistical information, AWB statistical information or AF statistical information determined based on the RAW image, YUV image or RGB image output by the image sensor.

[0084] Hardware statistical algorithm refers to an image information statistical algorithm in the form of firmware running on the hardware of an electronic device.

[0085] In some embodiments, the hardware for running the hardware statistics algorithm may be an ISP of an electronic device or other processor with image processing capabilities.

[0086] Software statistical algorithm refers to an image information statistical algorithm that runs in the computing unit of an electronic device in the form of software.

[0087] In some embodiments, the computing unit used to run the software statistical algorithm can be a computing unit with data processing capabilities commonly used in the field, such as a central processing unit (CPU), a graphics processor (GPU), a neural network processor (NPU) or a digital signal processor (DSP) in an electronic device.

[0088] In some embodiments, under the software statistical algorithm, a specific image information statistical method can be determined based on the first information, that is, the software statistical algorithm is not a fixed algorithm, but can be switched to a corresponding image information statistical algorithm according to specific image acquisition information. It can be seen that compared with the hardware statistical algorithm, the software statistical algorithm has higher flexibility and can better adapt to changes in image acquisition information.

[0089] In some embodiments, the software statistical algorithm can be defined according to user needs to make the software statistical algorithm more in line with the user's usage habits.

[0090] In the image processing method provided by the present application, by obtaining the first information related to the image acquisition, it is determined whether the first information meets the preset conditions, and the electronic device is controlled to operate in the first mode or in the second mode according to the judgment result; wherein the first mode represents the use of the image information statistical algorithm solidified on the hardware of the electronic device to determine the statistical information of the acquired image, and the second mode represents the use of the software image information statistical algorithm running in the electronic device to determine the statistical information of the acquired image. In this way, according to the first information, it is possible to choose to use a hardware statistical algorithm or a software statistical algorithm to determine the statistical information of the acquired image, so that the method of determining the image statistical information can better adapt to the current image acquisition conditions, with diversity and flexibility, that is, without increasing the hardware cost, it can greatly increase the scope of use of the hardware product and bring a better user experience to the user; in addition, compared with only using the software statistical algorithm to determine the statistical information of the image, the method provided by the present application can switch between the use of the hardware statistical algorithm and the software statistical algorithm according to the first information, thereby achieving the beneficial effect of reducing the power consumption of the electronic device and increasing the battery life.

[0091] In some embodiments, the first information includes at least one of the second information, the third information and the fourth information; wherein,

[0092] The second information represents image acquisition scene information; the third information represents interaction information acquired by the electronic device; and the fourth information represents feature information detected from the acquired image.

[0093] Here, the second information, that is, the image acquisition scene information, refers to information related to the image acquisition environment. For example, the image acquisition scene information may be ambient light information, color information of the image acquisition object, and so on.

[0094] The image acquisition environment directly affects the signal acquisition effect of the image sensor in the electronic device. For example, the color information of the acquired object directly affects the focus effect of the image sensor, the ambient light intensity directly affects the exposure effect of the image sensor, etc. Therefore, switching between the hardware statistical algorithm and the software statistical algorithm based on the image acquisition scene information can improve the matching degree between the image information statistical algorithm and the image acquisition scene while controlling the power consumption of the electronic device.

[0095] The third information, that is, the interaction information obtained by the electronic device, refers to the user interaction information obtained by the electronic device through the function keys, touch screen, image acquisition device or audio acquisition device of the electronic device.

[0096] Since the interactive information can represent the user's real-time adjustment needs for the image acquisition effect, switching between the hardware statistical algorithm and the software statistical algorithm based on the interactive information and then adjusting the image acquisition effect can obtain images that better meet the user's expectations and improve the user's experience.

[0097] The fourth information, i.e., feature information detected from the collected image, refers to information determined by extracting features from the collected image. For example, the fourth information may include face information, color information, etc. in the image.

[0098] In this way, the image information statistical algorithm can be adjusted in real time based on the feature information preset by the user, so as to improve the intelligence of the electronic device.

[0099] In some embodiments, the first information may also include application scenario information corresponding to the application currently running in the electronic device. For example, the application scenario may include a camera scenario, a video conference scenario, a video chat scenario, and the like.

[0100] Here, according to the requirements of the application scenario for image clarity, the application scenario conditions may be preset, so that when the first information meets the preset application scenario conditions, the electronic device is controlled to operate in the second mode.

[0101] In some embodiments, the preset application scenario condition includes a video conferencing scenario, that is, when the application scenario information indicates that the application currently running in the electronic device is a video conferencing application, the electronic device is controlled to run in the second mode.

[0102] In some embodiments, when the application scenario information indicates that the application currently running in the electronic device does not meet the preset application scenario conditions, it is further determined whether at least one of the second information, third information or fourth information corresponding to the electronic device meets the corresponding preset conditions.

[0103] In some embodiments, when the first information includes the second information, the second information includes at least one of the following: color information, ambient light information and / or motion status information.

[0104] Here, color information refers to the color information of the acquisition object during the image acquisition process; wherein the acquisition object refers to all objects within the image acquisition field of view of the electronic device.

[0105] In some embodiments, the color information may include color complexity information. For example, when the color information indicates that the color of the current acquisition object is single, for example, only one color, or multiple similar colors, or colors in the same color system, the current image acquisition scene is determined to be a monochrome scene; correspondingly, when the color information indicates that the current acquisition object presents rich color changes, the current image acquisition scene is determined to be a complex scene.

[0106] In some embodiments, the electronic device has a scene detection module, which can obtain color features in the current image acquisition scene and determine whether the current image acquisition scene is a monochrome scene or a complex scene based on the color features.

[0107] In some embodiments, an artificial intelligence (AI) model running in an electronic device may be used to assist in identifying the current scene.

[0108] In some embodiments, ambient light information refers to the light intensity information during the image acquisition process. For example, when the light intensity in the current image acquisition scene is low (e.g., less than 1 lux), the current image acquisition scene is determined as a low-light scene or a night scene; when the light intensity in the current image acquisition scene is high (e.g., greater than 1000 lux), the current image acquisition scene is determined as a bright light scene; when the main light source in the image acquisition environment is located behind the subject, the current image acquisition scene is determined as a backlit scene.

[0109] In some embodiments, the ambient light information refers to the color temperature information during the image acquisition process. For example, the color temperature of light produced by artificial lighting is different from the color temperature produced by natural lighting. Based on this, indoor scenes (i.e., only artificial lighting), outdoor scenes (i.e., only natural lighting), and mixed light scenes (i.e., both artificial lighting and natural lighting) can be determined.

[0110] In some embodiments, the electronic device has a scene detection module, which can obtain information such as light intensity and color temperature in the current image acquisition scene, and determine the current image acquisition scene based on the information such as light intensity and color temperature.

[0111] In some embodiments, the exposure (lux) or sensitivity (ISO) output by the AE algorithm module in the electronic device can be used to determine the light intensity information. For example, when the exposure or sensitivity is high, it means that the light intensity is low, which is a low-light scene or a night scene; when the exposure or sensitivity is low, it means that the light intensity is high, which is a strong light scene.

[0112] The motion state information refers to the information that the electronic device enters a non-static state from a static state, or enters a static state from a non-static state, during the image acquisition process.

[0113] In some embodiments, the motion state information of the electronic device is determined using hardware devices such as a gyroscope or an acceleration sensor in the electronic device.

[0114] In this way, when the first information meets the preset condition, the electronic device is controlled to operate in the second mode, that is, the above step S203 can be implemented as the following step S2031:

[0115] Step S2031: When the second information satisfies a preset image acquisition scenario condition, control the electronic device to determine statistical information of the acquired image in the second manner.

[0116] Here, the preset image acquisition scene condition may be a system default image acquisition scene condition or an image acquisition scene condition determined based on user configuration.

[0117] In some embodiments, the preset image acquisition scene includes at least one of the following: a monochrome scene, a low-light scene or a night scene, a backlight scene, and a scene in which the electronic device enters a non-static state from a static state.

[0118] In some embodiments, when the second information includes color information, if the color information represents that the current image acquisition scene is a monochrome scene, the electronic device is controlled to operate in the second mode, that is, the electronic device is controlled to switch to a software statistical algorithm; otherwise, the electronic device is controlled to operate in the first mode, that is, the electronic device is controlled to switch to a hardware statistical algorithm.

[0119] In this way, because the colors in the image acquisition environment are the same or similar in a monochrome scene, the edges of the objects in the acquisition scene are unclear and the focus is inaccurate, which leads to low image clarity. Therefore, in a monochrome scene, a software statistical algorithm can be used to improve the statistical accuracy of image information. For example, in a monochrome scene, at least one of the AE statistical algorithm, the AWB statistical algorithm, and the AF statistical algorithm is configured as a software statistical algorithm.

[0120] In some embodiments, when the second information includes ambient light information, if the ambient light information indicates that the current image acquisition scene is a low-light scene, a backlight scene, or a mixed light scene, the electronic device is controlled to operate in the second mode, that is, the electronic device is controlled to switch to a software statistical algorithm; otherwise, the electronic device is controlled to determine the statistical information of the acquired image in the first mode, that is, the electronic device is controlled to switch to a hardware image information statistical algorithm. For example, in a low-light scene, a backlight scene, or a mixed light scene, at least one of the AE statistical algorithm and the AF statistical algorithm is configured as a software statistical algorithm; for another example, in a mixed light scene or a green plant and sky scene, the AWB statistical algorithm is configured as a software statistical algorithm.

[0121] In this way, due to the low light intensity, low light intensity reflected by the subject or complex light color temperature in low-light scenes, backlit scenes or mixed light scenes, the image sensor has difficulty focusing, insufficient exposure time or difficulty estimating the color temperature of the ambient light. Therefore, in low-light scenes, backlit scenes or mixed light, software statistical algorithms can be used to improve the statistical accuracy of image information.

[0122] In some embodiments, when the second information includes motion state information, if the electronic device enters a non-static state from a static state, the electronic device is controlled to operate in the second mode, that is, the electronic device is controlled to switch to a software statistical algorithm; otherwise, the electronic device is controlled to operate in the first mode, that is, the electronic device is controlled to switch to a hardware statistical algorithm. For example, in a scenario where the electronic device enters a non-static state from a static state, or the electronic device is in a non-static state, at least one of the AE statistical algorithm, the AWB statistical algorithm, and the AF statistical algorithm is configured as a software statistical algorithm.

[0123] Compared with the static state, it is more difficult for the image sensor of the electronic device to achieve focusing in the non-static state. Therefore, when it is detected that the electronic device enters the non-static state from the static state, the software statistical algorithm is switched to improve the statistical accuracy of the image information by using the software image information statistical algorithm.

[0124] In the above embodiments, in response to the second information satisfying a preset image acquisition scene condition, for example, a monochrome scene, a low illumination scene, a backlight scene, a mixed light scene, or entering a non-static state from a static state, the electronic device is controlled to operate in the second mode to improve the statistical accuracy of the image information, and then more accurate control information can be determined based on more accurate statistical information, such as more accurate exposure information, white balance information, and motor movement step information, thereby achieving better automatic exposure, automatic white balance and automatic focus effects.

[0125] In some embodiments, the first information includes the third information;

[0126] In this way, when the first information meets the preset condition, the electronic device is controlled to operate in the second mode, that is, the above step S203 can be implemented as the following step S2032:

[0127] Step S2032: When the third information includes obtaining an interactive operation of the user on the first area in the image preview interface of the electronic device, control the electronic device to determine statistical information of the collected image in the second manner.

[0128] Here, the third information represents the interaction information acquired by the electronic device.

[0129] The preset condition refers to obtaining an interactive operation of the user on the first area in the image preview interface of the electronic device.

[0130] In some embodiments, an image preview interface is displayed on a touch screen of an electronic device; and the user's interactive operation on the first area is determined by detecting the user's touch operation on the touch screen.

[0131] In some embodiments, an image preview interface is displayed on a display screen of an electronic device; in response to a user's single-click, double-click, slide, or frame selection operation on a designated area on the display screen using an input / output device such as a mouse, a touchpad, or function keys, the user's interactive operation on the first area is determined.

[0132] In this way, in response to the user's interactive operation, the electronic device is controlled to operate in the second mode, that is, the electronic device is controlled to switch to the software statistical algorithm. For example, at least one of the AE statistical algorithm, the AWB statistical algorithm, and the AF statistical algorithm is configured as a software statistical algorithm.

[0133] In the above embodiment, the electronic device can receive the user's interactive instructions in real time and switch the image information statistical algorithm based on the interactive instructions, thereby allowing the user to adjust the image acquisition method in real time according to the shooting requirements during the image acquisition process to obtain the desired image shooting effect, thereby improving the user experience.

[0134] In some embodiments, the first information includes the fourth information; the fourth information represents object feature information detected from the acquired image;

[0135] In this way, when the first information meets the preset condition, the electronic device is controlled to operate in the second mode, that is, the above step S203 can be implemented as the following step S2033:

[0136] Step S2033: When the fourth information includes feature information of the target object, control the electronic device to determine statistical information of the collected image in the second manner.

[0137] Here, feature extraction is performed on the currently captured image, and the extracted features are used as the fourth information.

[0138] The target object refers to an object pre-specified by the user, for example, a person, animal, plant, object, or color pre-specified by the user.

[0139] In some embodiments, the features of the target object may include facial features of a person pre-specified by the user, facial features or body features of a specified animal, shape features of a specified object, or pre-specified color features, etc.

[0140] In some embodiments, feature information corresponding to the target object may be determined based on at least one image of the target object pre-specified by the user.

[0141] In this way, when the characteristic information of the target object is detected in the current image, the electronic device is controlled to operate in the second mode, that is, the electronic device is controlled to switch to the software statistical algorithm. For example, at least one of the AE statistical algorithm, the AWB statistical algorithm, and the AF statistical algorithm is configured as a software statistical algorithm.

[0142] In the above embodiment, the electronic device can detect the characteristic information in the collected image in real time, and switch to the software statistical algorithm when the characteristic information of the target object is detected, thereby automatically adjusting the image information statistical method according to the collected object characteristics, thereby improving the intelligence of the electronic device.

[0143] In some embodiments, the image processing method provided by the present application further includes the following steps S204 to S205:

[0144] Step S204, based on the second method, determine the first statistical information corresponding to the first area or target object in the first image captured by the electronic device, and the second statistical information corresponding to the area in the first image other than the first area or the area containing the target object; wherein the statistical accuracy of the first statistical information is higher than the statistical accuracy of the second statistical information.

[0145] Here, when the first information includes the third information (i.e., the interaction information acquired by the electronic device), and the third information includes an interaction operation of the user on the first area acquired in the image preview interface of the electronic device, the electronic device determines the first statistical information corresponding to the first area in the first image and the second statistical information corresponding to the area other than the first area in the first image by using a software statistical algorithm;

[0146] When the first information includes fourth information (i.e., object feature information detected from the captured image) and the fourth information includes feature information of a target object preset by a user, the electronic device uses a software statistical algorithm to determine first statistical information corresponding to an area in the first image containing the target object, and second statistical information corresponding to an area in the first image excluding the area containing the target object.

[0147] In some embodiments, the first area in the first image or the area containing the target object is divided into multiple information statistical areas with a smaller division granularity relative to other areas in the first image, and image information statistics are performed on each information statistical sub-area, so as to obtain higher statistical accuracy, that is, the statistical accuracy of the first statistical information is higher than the statistical accuracy of the second statistical information.

[0148] Step S205, determining target statistical information corresponding to the first image based on the first statistical information and the second statistical information;

[0149] Here, the first statistical information and the second statistical information are fused to obtain target statistical information corresponding to the first image.

[0150] In the above embodiment, when the captured image includes a user-specified area or contains target object features, after the electronic device is switched to a software statistical algorithm, higher statistical accuracy is used to determine the statistical information of the user's area of ​​interest, so that the electronic device can pay more attention to the area of ​​interest to the user; at the same time, lower statistical accuracy is used to determine the statistical information of non-user-interested areas, which can reduce the demand for computing resources and reduce the power consumption of the electronic device.

[0151] In some embodiments, the first information includes the fourth information; the fourth information represents focus value statistics detected from the acquired image;

[0152] In this way, when the first information meets the preset condition, the electronic device is controlled to operate in the second mode, that is, the above step S203 can be implemented as the following step S2034:

[0153] Step S2034: When the focus value statistical information satisfies a preset focus value statistical information condition, control the electronic device to operate in the second mode.

[0154] Here, the fourth information is the focus value statistical information of the image acquired when the electronic device operates in the first mode, that is, the FV (focus value, FV) value in the AF statistical information.

[0155] The preset condition refers to the condition for switching the image information statistical algorithm determined based on the FV value.

[0156] In some embodiments, the preset FV statistical information condition includes that the clarity curves of the multiple images determined based on the FV values ​​do not have a single peak feature.

[0157] Here, the unimodal feature refers to the calculation of the clarity curves corresponding to the multiple images based on the FV values ​​of the multiple images during the autofocus process. The curve changes with the movement of the lens and is usually expressed as a unimodal curve that first increases and then decreases, and reaches the highest clarity at the peak.

[0158] In some embodiments, the method for calculating the clarity curves corresponding to multiple images can be: obtaining FV values ​​corresponding to multiple images; for each image, adding the horizontal FV value and the vertical FV value, and dividing the accumulated sum by 2 to obtain the clarity detection value corresponding to the image; based on the clarity detection values ​​corresponding to the multiple images, generating clarity curves corresponding to the multiple images.

[0159] It can be seen that the clarity curves corresponding to multiple images do not have a single peak feature, indicating that focusing cannot be achieved using the current hardware statistical algorithm. Therefore, it is necessary to switch to a software statistical algorithm.

[0160] In some embodiments, the preset FV statistical information condition includes that the absolute value of the FV value of the plurality of images is less than a preset threshold.

[0161] Here, the absolute value of the FV value of the multiple images is less than the preset threshold, indicating that the clarity of the currently acquired image is poor. At this time, even if the clarity curve corresponding to the multiple images has a single peak, since the absolute value of the FV value is less than the preset threshold, it still indicates that the focus effect of the focus position determined by the current image information statistical algorithm is not ideal.

[0162] Therefore, when the absolute value of the FV value of a plurality of images is less than a preset threshold, the electronic device is switched to a software statistical algorithm to improve the accuracy of the determined motor movement step by improving the statistical accuracy of the FV value.

[0163] In some embodiments, the preset FV statistical information condition includes determining whether the image contains noise based on the FV value corresponding to the acquired image.

[0164] Here, the clarity of the image can be determined based on the FV value corresponding to the image using a specified clarity evaluation function, and further, whether the image contains noise.

[0165] In the case where it is determined based on the FV value that the image contains noise, it is necessary to switch the electronic device to a software statistical algorithm to improve the accuracy of the determined motor movement step length by improving the statistical accuracy of the FV value.

[0166] In some embodiments, when the FV values ​​corresponding to the specified number of images do not meet the preset focus value statistical information condition, the electronic device is controlled to operate in the second mode. That is, when the number of images that do not meet the preset focus value statistical information condition reaches a preset number threshold, the statistical algorithm is switched.

[0167] In some embodiments, the obtaining of the first information, that is, the above step S201, can be implemented as the following step S2011:

[0168] Step S2011, in response to trigger information, obtaining the focus value statistical information;

[0169] The trigger information includes at least one of the following:

[0170] Detecting that the electronic device enters a low-light scene;

[0171] detecting that the electronic device enters a monochrome scene;

[0172] It is detected that the electronic device enters a non-stationary state from a stationary state.

[0173] Here, the low illumination scene refers to an image acquisition scene in which the illumination intensity is lower than the illumination intensity threshold.

[0174] A monochrome scene refers to a scene in which the color within the image acquisition range of the image sensor is single or similar.

[0175] To non-stationary device from a stationary state refers to the change of motion state of an electronic device determined by an acceleration sensor in the electronic device.

[0176] In this way, after the electronic device detects entering a low-illumination scene, a monochrome scene, or entering a non-static state from a static state, it triggers the acquisition of focus value statistical information and determines whether the focus value statistical information meets the preset focus value statistical information conditions. If the focus value statistical information meets the preset focus value statistical information conditions, the electronic device is controlled to operate in the second mode.

[0177] In the above embodiment, after obtaining the preset trigger information, the focus value statistical information is obtained and judged whether it satisfies the preset focus value statistical information condition. In this way, since the computing resources occupied by detecting whether the trigger condition occurs are small and the power consumption is low, determining whether to further detect the focus value statistical information based on whether the trigger condition is detected can save the computing resources of the electronic device and reduce the power consumption of the electronic device.

[0178] In some embodiments, for each trigger information, a corresponding clarity threshold is determined, and when the clarity determined based on the FV value is lower than the corresponding clarity threshold, the electronic device is controlled to operate in the second mode.

[0179] For example, when the trigger information is a low-light scene, a first clarity threshold is set. In this way, in response to detecting that the electronic device enters a low-light scene, the FV value of at least one image collected from the current time point is obtained; based on the FV value of at least one image, the corresponding clarity value is determined; it is determined whether the clarity value corresponding to at least one image is greater than the first clarity threshold; if it is not greater than the first clarity threshold, the electronic device is controlled to operate in the second mode, that is, the electronic device is controlled to switch to the software statistical algorithm; otherwise, the electronic device is controlled to continue to operate in the first mode.

[0180] For another example, when the trigger information is a monochrome scene, a second clarity threshold is set. In this way, in response to detecting that the electronic device has entered a monochrome scene, the FV value of at least one image collected from the current time point is obtained; based on the FV value corresponding to the at least one image, the corresponding clarity value is determined; it is determined whether the clarity value corresponding to the at least one image is greater than the second clarity threshold; if it is not greater than the second clarity threshold, the electronic device is controlled to operate in the second mode; otherwise, the electronic device is controlled to continue to operate in the first mode.

[0181] In some embodiments, when the electronic device detects that the image acquisition scene has changed, for example, from a low-illuminance scene to a well-lit high-illuminance scene, from a monochrome scene to a colorful complex scene, or from a non-static state to a static state, the FV value of at least one image is obtained to determine whether to switch the operating mode of the electronic device based on the FV value.

[0182] For example, a corresponding third clarity threshold is set for a high-illuminance scene; wherein the third clarity threshold is higher than the first clarity threshold. In this way, after the electronic device enters a high-illuminance scene from a low-illuminance scene, first, the FV value of at least one image collected from the scene switching time point is obtained; based on the FV value corresponding to the at least one image, the corresponding clarity value is determined; it is determined whether the clarity value corresponding to the at least one image is greater than the third clarity threshold; if it is not greater than the third clarity threshold, the electronic device is controlled to operate in the second mode; otherwise, the electronic device is controlled to operate in the first mode.

[0183] In the above embodiment, when a change in the scene corresponding to the electronic device is detected, the FV value corresponding to at least one image is obtained, and the corresponding clarity value is determined based on the FV value, so as to determine the operation mode of the electronic device based on the clarity value; at the same time, a lower clarity threshold is set for scenes with poor image acquisition environments (for example, the electronic device is in operation, low-illuminance scenes, or monochrome scenes), and a higher clarity threshold is set for scenes with better image acquisition environments (for example, the electronic device is in a stationary state, high-illuminance scenes, or complex scenes).

[0184] In some embodiments, when the operating mode of the electronic device is switched from a first mode to a second mode, or from a second mode to a first mode, a smooth transition strategy is used to implement the switching of the operating mode.

[0185] For example, after determining to switch the operating mode of the electronic device from the first mode to the second mode, continue to use the hardware statistical algorithm to determine and output the statistical information of the collected image, and at the same time use the software statistical algorithm to determine the statistical information of the collected image, but do not output the statistical information determined by the software statistical algorithm; after the image statistical information determined by the software statistical algorithm converges, switch the hardware statistical algorithm to the software statistical algorithm, that is, stop using the hardware statistical algorithm to determine the image statistical information, and only use the software statistical algorithm to determine and output the statistical information of the collected image.

[0186] Accordingly, after determining to switch the operation mode of the electronic device from the second mode to the first mode, the same smooth transition strategy may be adopted.

[0187] Next, combine Figure 3 , taking the image statistical information as AF statistical information as an example, an embodiment of the image processing method provided by the present application is described in detail. Figure 3 As shown, this embodiment includes the following steps S301 to S305:

[0188] Step S301, obtaining AF statistical information of the captured image; then, executing step S302;

[0189] In this embodiment, the image information statistical algorithm currently executed by the electronic device is a hardware AF statistical algorithm. Here, the AF statistical information of the image is the hardware AF statistical information of the image.

[0190] Step S302, based on the AF statistical information, determine whether the clarity of the current image meets the clarity condition; if so, execute step S303; if not, execute step S305;

[0191] Here, the clarity value of the current image is determined based on the FV value in the AF statistical information; and then it is determined whether the clarity value of the current image is lower than the clarity threshold, whether it has a single peak, whether it contains noise, and so on.

[0192] Step S303, determining the current image acquisition scene; then, executing step S304;

[0193] Here, the current image acquisition scene may include a monochrome scene, a complex scene, a low illumination scene, a strong light scene, a scene where the electronic device enters a non-static state from a static state, and the like.

[0194] Step S304, whether the clarity of the current image is higher than the clarity threshold preset for the current image acquisition scene; if so, execute step S301; if not, execute step S305;

[0195] Here, if the clarity value of the current image is higher than the clarity threshold preset for the current image acquisition scene, it proves that the current AF statistical algorithm can meet the preset clarity requirements, therefore, returns to step S301 to continue detecting the AF statistical information; on the contrary, if it is proved that the current AF statistical algorithm cannot meet the preset clarity requirements, therefore, the electronic device is switched to the second mode of operation.

[0196] Step S305: controlling the electronic device to operate in a second mode.

[0197] From the above, it can be seen that in the image processing method provided by the present application, the software statistical algorithm and the hardware statistical algorithm for image information can be dynamically switched according to the image acquisition scenario, which can not only meet the usage requirements of image acquisition scenarios with poor image acquisition environments or scenarios with high image quality requirements, but also meet the needs of static scenes or scenarios with low image quality requirements, taking into account both image quality experience and power consumption requirements; at the same time, the software statistical algorithm can be customized according to user needs, thereby improving the flexibility and intelligence of image information statistics.

[0198] Based on the foregoing embodiments, the present application provides an image processing device, which includes the units included and the modules included in the units, which can be implemented by a processor in an electronic device; of course, it can also be implemented by a specific logic circuit; in the implementation process, the processor can be a CPU, NPU, MPU, DSP or Field Programmable Gate Array (FPGA), etc.

[0199] Figure 4 A schematic diagram of the structure of an image processing device provided in this application is shown in FIG. Figure 4 As shown, the image processing device 400 includes: an information acquisition module 410 and an operation control module 420, wherein:

[0200] The information acquisition module 410 is used to acquire first information; the first information represents information related to image acquisition;

[0201] An operation control module 420, configured to control the electronic device to operate in a first mode if the first information does not satisfy a preset condition; and to control the electronic device to operate in a second mode if the first information satisfies a preset condition;

[0202] Among them, the first method represents the use of an image information statistical algorithm solidified on the hardware of the electronic device to determine the statistical information of the collected image; the second method represents the use of a software image information statistical algorithm running in the electronic device to determine the statistical information of the collected image.

[0203] In some embodiments, the first information includes at least one of the second information, the third information and the fourth information; wherein,

[0204] The second information represents image acquisition scene information; the third information represents interaction information acquired by the electronic device; and the fourth information represents feature information detected from the acquired image.

[0205] In some embodiments, the first information includes the second information; the second information includes at least one of the following: color information, ambient light information, and motion status information;

[0206] The operation control module 420 is used to control the electronic device to determine the statistical information of the collected image in the second manner when the second information meets the preset image collection scene condition.

[0207] In some embodiments, the first information includes the third information;

[0208] The operation control module 420 is used to control the electronic device to determine the statistical information of the collected image in the second manner when the third information includes obtaining the user's interactive operation on the first area in the image preview interface of the electronic device.

[0209] In some embodiments, the first information includes the fourth information; the fourth information represents object feature information detected from the acquired image;

[0210] The operation control module 420 is used to control the electronic device to determine the statistical information of the collected image in the second manner when the fourth information includes feature information of the target object.

[0211] In some embodiments, the image processing apparatus 400 further includes a statistical module 430; the statistical module 430 is used to:

[0212] Based on the second manner, determining first statistical information corresponding to a first area or a target object in a first image captured by the electronic device, and second statistical information corresponding to an area in the first image other than the first area or an area including the target object;

[0213] Determining target statistical information corresponding to the first image based on the first statistical information and the second statistical information;

[0214] The statistical accuracy of the first statistical information is higher than the statistical accuracy of the second statistical information.

[0215] In some embodiments, the first information includes the fourth information; the fourth information represents focus value statistics detected from the acquired image;

[0216] The operation control module 420 is used to control the electronic device to operate in the second mode when the focus value statistical information meets a preset focus value statistical information condition.

[0217] In some embodiments, the information acquisition module 410 is used to:

[0218] In response to the trigger information, acquiring the focus value statistical information;

[0219] The trigger information includes at least one of the following:

[0220] Detecting that the electronic device enters a low-light scene;

[0221] detecting that the electronic device enters a monochrome scene;

[0222] It is detected that the electronic device enters a non-stationary state from a stationary state.

[0223] Next, combine Figure 5 , a system embodiment for realizing auto-focusing according to the image processing device provided by the present application is described.

[0224] like Figure 5 As shown, the auto focus system 500 includes an image sensor 510, an ISP chip 520, a computing unit 530, an AF management module 540, and a motor 550; wherein:

[0225] The image sensor 510 is used to convert the collected optical signal into an electronic image signal and RAW image data; and transmit the RAW image data to the ISP chip 520.

[0226] The ISP chip 520 includes an AF statistics switching module 521 and a hardware AF statistics module 522 .

[0227] The AF statistics switching module 521 is used to switch the current AF statistics mode from the hardware AF statistics algorithm to the software AF statistics algorithm, or from the software AF statistics algorithm to the hardware AF statistics algorithm according to the control signal of the AF statistics algorithm decision module 548 in the AF management module 540. For example, when the default AF statistics algorithm in the electronic device is the hardware AF statistics algorithm, when the electronic device starts the camera function, the AF statistics switching module 521 turns on the hardware AF statistics algorithm and configures the corresponding data path.

[0228] The hardware AF statistics module 522 is used to determine the hardware AF statistics information of the RAW image data based on the RAW image data by using the hardware AF statistics algorithm solidified on the ISP chip; at the same time, the hardware AF statistics module 522 can send the hardware AF statistics information to the AF algorithm module 541 and the AF statistics value acquisition module 544 in the AF management module 540.

[0229] The computing unit 530 may be implemented as a CPU, a GPU, an NPU, a DSP or an FPGA, and is used to run the software AF statistical algorithm. When the electronic device switches to using the software AF statistical algorithm to determine the AF statistical information, the RAW image data is sent to the computing unit 530 to determine the software AF statistical information. Afterwards, the computing unit 530 may send the software AF statistical information to the AF algorithm module 541 and the AF statistical value acquisition module 544 in the AF management module 540.

[0230] The AF management module 540 includes an AF algorithm module 541 , an AF control module 542 , and an algorithm decision module 549 .

[0231] The AF algorithm module 541 is used to receive hardware AF statistical information from the hardware AF statistical module 522, or receive software AF statistical information from the computing unit 530; use the built-in AF algorithm to calculate the moving step and moving direction information of the motor 550 based on the hardware AF statistical information or the software AF statistical information; and send the calculated moving step and moving direction information to the AF control module 542.

[0232] The AF control module 542 is used to generate control information for the motor 550 based on the moving step length and the moving direction information, and send the control information to the motor 550.

[0233] The motor 550 is used to move in response to the moving direction and moving step in the control information sent by the AF control module 542, so as to drive the image sensor 510 to move forward and backward, thereby adjusting the focusing effect.

[0234] The algorithm decision module 549 is used to determine information related to image acquisition through various modules, and determine whether to switch the current AF statistical algorithm based on the information. Here, the algorithm decision module 549 includes a scene detection module 543, an AF statistical value acquisition module 544, an AF statistical value control module 545, a motion information acquisition module 546, a motion detection module 547 and an AF statistical algorithm decision module 548.

[0235] The scene detection module 543 is used to run the trained AI model to determine the current image acquisition scene based on the AI ​​model. For example, the scene detection module 543 can determine whether the current image acquisition scene is a monochrome scene, a low-light scene, a strong light scene, a backlight scene, a mixed light scene, a scene entering a non-static state, etc. The scene detection module 543 sends the determined scene information to the AF statistical algorithm decision module 548.

[0236] The AF statistics acquisition module 544 is used to obtain AF statistics information of the currently acquired image from the hardware AF statistics module 522 or the calculation unit 530. Here, the AF statistics acquisition module can receive type information about the currently adopted AF statistics algorithm from the AF statistics switching module 521.

[0237] The AF statistic control module 545 is used to receive the AF statistic information from the AF statistic acquisition module 544, and determine the clarity value of the currently acquired image, whether the clarity curve has a single peak, whether the clarity is less than the clarity threshold, whether there is noise, etc. based on the FV value in the AF statistic information. At the same time, the AF statistic control module 545 sends the clarity and noise of the currently acquired image determined based on the AF statistic information to the AF statistic algorithm decision module 548.

[0238] The motion information acquisition module 546 can be implemented as a gyroscope or an acceleration sensor, etc., for acquiring the motion state information of the electronic device. At the same time, the motion information acquisition module 546 sends the acquired motion state information of the electronic device to the motion detection module 547.

[0239] The motion detection module 547 is used to determine whether the electronic device is currently in a static state, or is entering a non-static state from a static state, etc. based on the motion state information of the electronic device sent by the motion information acquisition module 546; at the same time, the motion detection module 547 sends the determined motion state information of the electronic device to the AF statistical algorithm decision module 548.

[0240] The AF statistical algorithm decision module 548 is used to comprehensively determine whether to switch the current AF statistical algorithm based on the scene information sent by the scene detection module 543, the clarity and noise information of the currently collected image sent by the AF statistical value control module 545, and the motion state information of the electronic device sent by the motion detection module 547. Here, in the case where the AF statistical algorithm decision module 548 determines to switch the current AF statistical algorithm, the AF statistical algorithm decision module 548 generates a corresponding switching instruction and sends the switching instruction to the AF statistical switching module 521.

[0241] Here, after receiving the switching instruction sent by the AF statistics algorithm decision module 548, the AF statistics switching module 521 switches the transmission path of the RAW image data. For example, the AF statistics switching module 521 configures the RAW image data or the YUV image data to be stored in the Double Data Rate Synchronous Dynamic Random Access Memory (DDR SDRAM) cache corresponding to the calculation unit 530, so that the calculation unit 530 can obtain the RAW image data or the YUV image data from the cache and perform AF information statistics, and output software AF statistics information.

[0242] Accordingly, if the AF statistical algorithm decision module 548 determines that there is no need to switch the current AF statistical algorithm, no processing is performed.

[0243] If the AF statistics algorithm decision module 548 switches the software AF statistics algorithm to the hardware AF statistics algorithm, the AF statistics algorithm decision module 548 sends a switching instruction to the AF statistics switching module 521; thereafter, the AF statistics switching module 521 closes the data path for outputting the RAW image data or YUV image data to the cache of the computing unit 530, and configures the RAW image data or YUV image data to be sent to the hardware AF statistics module 522, so that the hardware AF statistics module 522 performs AF information statistics based on the RAW image data or YUV image data, and outputs hardware AF statistics information.

[0244] The description of the above device embodiment is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiment. For technical details not disclosed in the device embodiment of the present application, please refer to the description of the method embodiment of the present application for understanding.

[0245] On the other hand, based on the above image processing method, the present application also provides an electronic device. Figure 6 As shown, the electronic device 600 includes a decision component 610, an image signal processor 620 and an auxiliary processor 630; wherein,

[0246] The decision component 610 is used to obtain first information; if the first information does not meet the preset condition, control the electronic device 600 to operate in a first mode; if the first information meets the preset condition, control the electronic device 600 to operate in a second mode; wherein the first information represents information related to image acquisition;

[0247] The first method is characterized by using the image information statistical algorithm solidified on the image signal processor 620 to determine the statistical information of the collected image;

[0248] The second method represents the use of a software image information statistical algorithm running on the auxiliary processor 630 to determine the statistical information of the acquired image.

[0249] The decision component 610 , the image signal processor 620 and the auxiliary processor 630 may transmit data via a bus 640 .

[0250] It should be noted that the description of the electronic device embodiment provided in this application is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. In some embodiments, the functions or various units or components included in the electronic device provided in the embodiment of the present disclosure can be used to execute the method described in the above method embodiment. For technical details not disclosed in the device embodiment of this application, please refer to the description of the method embodiment of this application for understanding.

[0251] It should be noted that in the embodiment of the present application, if the above-mentioned image processing method is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present application can be essentially or partly embodied in the form of a software product that contributes to the relevant technology. The software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a disk or an optical disk. In this way, the embodiment of the present application is not limited to any specific hardware, software or firmware, or any combination of hardware, software, and firmware.

[0252] An embodiment of the present application provides an electronic device, including a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the program, some or all of the steps in the above method are implemented.

[0253] The embodiment of the present application provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, some or all of the steps in the above method are implemented. The computer-readable storage medium can be transient or non-transient.

[0254] An embodiment of the present application provides a computer program, including a computer-readable code. When the computer-readable code is run in a computer device, a processor in the computer device executes some or all of the steps for implementing the above method.

[0255] The embodiment of the present application provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and when the computer program is read and executed by a computer, some or all of the steps in the above method are implemented. The computer program product can be implemented specifically by hardware, software or a combination thereof. In some embodiments, the computer program product is specifically embodied as a computer storage medium, and in other embodiments, the computer program product is specifically embodied as a software product, such as a software development kit (SDK) and the like.

[0256] It should be noted here that the description of the various embodiments above tends to emphasize the differences between the various embodiments, and the same or similar aspects can be referenced to each other. The description of the above device, storage medium, computer program and computer program product embodiments is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the embodiments of the device, storage medium, computer program and computer program product of this application, please refer to the description of the method embodiment of this application for understanding.

[0257] It should be understood that "one embodiment" or "an embodiment" mentioned throughout the specification means that specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present application, the size of the serial number of each step / process mentioned above does not mean the order of execution, and the execution order of each step / process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application. The serial numbers of the embodiments of the present application mentioned above are for description only and do not represent the advantages and disadvantages of the embodiments.

[0258] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0259] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0260] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0261] In addition, all functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may be a separate unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0262] A person skilled in the art can understand that all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM), magnetic disks or optical disks, etc., various media that can store program codes.

[0263] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can essentially or in other words, the part that contributes to the relevant technology can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0264] The above is only an implementation method of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. An image processing method, comprising: Obtaining first information; The first information represents information related to image acquisition; When the first information does not meet the preset condition, controlling the electronic device to operate in a first mode; When the first information meets a preset condition, the electronic device is controlled to operate in a second mode; wherein, The electronic device represents a device for executing the image processing method; The first method represents the use of an image information statistical algorithm embedded in the hardware of the electronic device to determine the statistical information of the collected image; the second method represents the use of a software image information statistical algorithm running in the electronic device to determine the statistical information of the collected image.

2. The method according to claim 1, wherein the first information includes at least one of the second information, the third information and the fourth information; wherein, The second information represents image acquisition scene information; the third information represents interaction information acquired by the electronic device; and the fourth information represents feature information detected from the acquired image.

3. The method according to claim 2, wherein the first information includes the second information; the second information includes at least one of the following: color information, ambient light information and / or motion status information; When the first information satisfies a preset condition, controlling the electronic device to operate in a second mode includes: When the second information satisfies a preset image acquisition scenario condition, the electronic device is controlled to determine statistical information of the acquired image in the second manner.

4. The method according to claim 2, wherein the first information includes the third information; When the first information satisfies a preset condition, controlling the electronic device to operate in a second mode includes: When the third information includes obtaining an interactive operation of the user on the first area in the image preview interface of the electronic device, the electronic device is controlled to determine statistical information of the acquired image in the second manner.

5. The method according to claim 2, wherein the first information includes the fourth information; the fourth information represents object feature information detected from the acquired image; When the first information satisfies a preset condition, controlling the electronic device to operate in a second mode includes: In a case where the fourth information includes feature information of the target object, the electronic device is controlled to determine statistical information of the acquired image in the second manner.

6. The method according to claim 4 or 5, further comprising: Based on the second manner, determining first statistical information corresponding to a first area or a target object in a first image captured by the electronic device, and second statistical information corresponding to an area in the first image other than the first area or an area including the target object; Determining target statistical information corresponding to the first image based on the first statistical information and the second statistical information; The statistical accuracy of the first statistical information is higher than the statistical accuracy of the second statistical information.

7. The method according to claim 2, wherein the first information includes the fourth information; the fourth information represents focus value statistics detected from the acquired image; When the first information satisfies a preset condition, controlling the electronic device to operate in a second mode includes: When the focus value statistical information satisfies a preset focus value statistical information condition, the electronic device is controlled to operate in the second mode.

8. The method according to claim 7, wherein obtaining the first information comprises: In response to acquiring trigger information, acquiring the focus value statistical information; The trigger information includes at least one of the following: Detecting that the electronic device enters a low-light collection scene; Detecting that the electronic device enters a monochrome acquisition scene; It is detected that the electronic device enters a non-stationary state from a stationary state.

9. An image processing device, provided in an electronic device, comprising: An information acquisition module, used to acquire first information; The first information represents information related to image acquisition; an operation control module, configured to control the electronic device to operate in a first mode if the first information does not satisfy a preset condition; and to control the electronic device to operate in a second mode if the first information satisfies a preset condition; The first method is characterized by using an image information statistical algorithm solidified in the hardware of the electronic device to determine the statistical information of the collected image; The second method is characterized by using a software image information statistical algorithm running in an electronic device to determine the statistical information of the collected image.

10. An electronic device comprising a decision component, an image signal processor and an auxiliary processor; wherein: The decision component is used to obtain first information; if the first information does not meet the preset condition, control the electronic device to operate in a first mode; if the first information meets the preset condition, control the electronic device to operate in a second mode; wherein the first information represents information related to image acquisition; The first method is characterized by using an image information statistical algorithm solidified on the image signal processor to determine the statistical information of the collected image; The second mode is characterized by using a software image information statistical algorithm running on the auxiliary processor to determine the statistical information of the acquired image.

Images