Image processing method and image processing device

Abstract

The invention provides an image processing method and an image processing device. The display effect of an image is improved. The method comprises the steps that a target area is determined based on a scene recognition result, and the target area is an area for executing preset image processing operation; under the condition that target image frame data is collected, executing the preset image processing operation on the target image frame data in the target area, and obtaining and displaying a target image frame; wherein the preset image processing operation comprises at least one of automatic focusing (AF), automatic exposure (AE) and automatic white balance (AWB).

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method and an image processing device. Background Art

[0002] 3A refers to auto focus (AF), auto exposure (AE) and auto white balance (AWB), which are currently an important focus for improving image quality. Among them, AE refers to adjusting the shutter, aperture and sensitivity to make the image reach the appropriate brightness; AWB refers to adjusting the gain value of the R / B channel of the image to adjust the color temperature / hue of the image; AF refers to controlling the displacement of the focus motor to make the lens focus at the appropriate position. Auto focus is the process of adjusting the focal length of the camera to automatically obtain a clear image.

[0003] However, the image obtained by the above processing may have a problem of poor display effect. Summary of the invention

[0004] The present application provides an image processing method and an image processing device, which improve the display effect of the image.

[0005] In a first aspect, an image processing method is provided, the method comprising: determining a target area based on a scene recognition result, the target area being an area for performing a preset image processing operation; when target image frame data is collected, performing the preset image processing operation on the target image frame data in the target area to obtain and display the target image frame; wherein the preset image processing operation comprises at least one of auto focus AF, automatic exposure AE and automatic white balance AWB.

[0006] In the present application, when the target image frame data is collected, the preset image processing operation can be performed on the target image frame data in the target area to obtain and display the target image frame. The preset image processing operation includes at least one of auto focus AF, automatic exposure AE and automatic white balance AWB, and the target area is determined based on the scene recognition result. In other words, the present application can selectively perform preset image processing on an area in the image frame (such as a target area) based on the scene recognition result to improve the display effect of the image.

[0007] In a second aspect, an image processing device is provided, which includes: a determination module and a processing module, wherein the determination module is used to determine a target area based on a scene recognition result, and the target area is an area for performing a preset image processing operation; the processing module is used to perform the preset image processing operation on the target image frame data in the target area when the target image frame data is collected, so as to obtain and display the target image frame; wherein the preset image processing operation includes at least one of auto focus AF, automatic exposure AE and automatic white balance AWB.

[0008] In a third aspect, another image processing device is provided, the image processing device includes a processor coupled to a memory, and can be used to execute instructions in the memory to implement the method in any possible implementation of the first aspect. Optionally, the image processing device also includes a memory. Optionally, the electronic device also includes a communication interface, and the processor is coupled to the communication interface.

[0009] In a fourth aspect, a processor is provided, comprising: an input circuit, an output circuit and a processing circuit. The processing circuit is used to receive a signal through the input circuit and transmit a signal through the output circuit, so that the processor executes the method in any possible implementation of the first aspect.

[0010] In the specific implementation process, the processor can be a chip, the input circuit can be an input pin, the output circuit can be an output pin, and the processing circuit can be a transistor, a gate circuit, a trigger, and various logic circuits. The input signal received by the input circuit can be, for example, but not limited to, received and input by a receiver, and the signal output by the output circuit can be, for example, but not limited to, output to a transmitter and transmitted by the transmitter, and the input circuit and the output circuit can be the same circuit, which is used as an input circuit and an output circuit at different times. The embodiments of the present application do not limit the specific implementation methods of the processor and various circuits.

[0011] In a fifth aspect, a processing device is provided, comprising a processor and a memory. The processor is used to read instructions stored in the memory, and can receive signals through a receiver and transmit signals through a transmitter to execute the method in any possible implementation of the first aspect.

[0012] Optionally, there are one or more processors and one or more memories.

[0013] Optionally, the memory may be integrated with the processor, or the memory may be provided separately from the processor.

[0014] In the specific implementation process, the memory can be a non-transitory memory, such as a read-only memory (ROM), which can be integrated with the processor on the same chip or can be set on different chips respectively. The embodiments of the present application do not limit the type of memory and the setting method of the memory and the processor.

[0015] It should be understood that the relevant data interaction process, such as sending indication information, can be a process of outputting indication information from a processor, and receiving capability information can be a process of receiving input capability information from a processor. Specifically, the processed output data can be output to a transmitter, and the input data received by the processor can come from a receiver. Among them, the transmitter and the receiver can be collectively referred to as a transceiver.

[0016] The processing device in the fifth aspect mentioned above can be a chip. The processor can be implemented by hardware or by software. When implemented by hardware, the processor can be a logic circuit, an integrated circuit, etc.; when implemented by software, the processor can be a general-purpose processor, which is implemented by reading the software code stored in the memory. The memory can be integrated in the processor or can be located outside the processor and exist independently.

[0017] In a sixth aspect, a computer program product is provided, the computer program product comprising: a computer program (also referred to as code, or instruction), which, when executed, enables a computer to execute a method in any possible implementation of the first aspect.

[0018] In a seventh aspect, a computer-readable storage medium is provided, which stores a computer program (also referred to as code, or instructions) which, when executed on a computer, enables the computer to execute a method in any possible implementation of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 is a schematic diagram of the system architecture of an electronic device provided in an embodiment of the present application;

[0020] Figure 2 is a flow chart of an image processing method provided by an embodiment of the present application;

[0021] Figure 3 is a flowchart of a first specific example of an image processing method provided by an embodiment of the present application;

[0022] Figure 4 is a schematic diagram of a target area provided in an embodiment of the present application;

[0023] Figure 5is a flowchart of a second specific example of an image processing method provided by an embodiment of the present application;

[0024] Figure 6 is a structural block diagram of an example of an image processing device provided in an embodiment of the present application;

[0025] Figure 7 It is a structural diagram of another example of an image processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0026] The technical solution in this application will be described below in conjunction with the accompanying drawings.

[0027] 3A refers to auto focus (AF), auto exposure (AE) and auto white balance (AWB), which are currently an important focus for improving image quality. Among them, AE refers to adjusting the shutter, aperture and sensitivity to make the image reach the appropriate brightness; AWB refers to adjusting the gain value of the R / B channel of the image to adjust the color temperature / hue of the image; AF refers to controlling the displacement of the focus motor to make the lens focus at the appropriate position. Auto focus is the process of adjusting the focal length of the camera to automatically obtain a clear image.

[0028] However, the image obtained by the above processing may have a problem of poor display effect.

[0029] For example, with the difference in sensor capabilities and upgrades, larger sensor output sizes need to be adapted and supported. However, if some hardware is not updated in a timely manner, it is easy for the demand to exceed the hardware's processing capabilities. If the configuration is forced, it is easy to have unexpected risks and problems, such as abnormal data volume in 3A statistics, unequal output of some data, and thus cause the center area of ​​the image to be better than the surrounding areas.

[0030] In view of this, the embodiments of the present application provide an image processing method and an image processing device, which can perform the above-mentioned preset image processing operation on the above-mentioned target image frame data in the above-mentioned target area when the target image frame data is collected, and obtain and display the target image frame. Among them, the preset image processing operation includes at least one of auto focus AF, automatic exposure AE and automatic white balance AWB, and the target area is determined based on the scene recognition result. In other words, the present application can selectively perform preset image processing on an area of ​​the image frame (such as the target area) based on the scene recognition result to improve the display effect of the image.

[0031] It should be understood that the image processing method provided in the present application can be executed by an electronic device with an image processing function, and the electronic device can also be called: user equipment (UE), mobile station (MS), mobile terminal (MT), access terminal, user unit, user station, mobile station, mobile station, remote station, remote terminal, mobile device, user terminal, terminal, wireless communication device, user agent or user device, etc. The electronic device involved in the embodiment of the present application can be a device that provides voice / data connectivity to a user, for example, a handheld device with a wireless connection function, a vehicle-mounted device, etc. At present, some examples of terminals are: mobile phones, tablet computers, laptop computers, PDAs, mobile internet devices (MIDs), wearable devices, virtual reality (VR) devices, augmented reality (AR) devices, wireless terminals in industrial control, wireless terminals in self driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, cellular phones, cordless phones, session initiation protocol (SIP) phones, wireless local loop (WLL) stations, personal digital assistants (PDAs), handheld devices with wireless communication capabilities, computing devices or other processing devices connected to wireless modems, vehicle-mounted devices, wearable devices, electronic devices in 5G networks or future evolved public land mobile communication networks (public land mobile The electronic devices in the network (PLMN) and the like are not limited to this in the embodiments of the present application.

[0032] As an example but not limitation, in the embodiments of the present application, the electronic device may also be a wearable device. Wearable devices may also be referred to as wearable smart devices, which are a general term for wearable devices that are intelligently designed and developed using wearable technology for daily wear, such as glasses, gloves, watches, clothing, and shoes. A wearable device is a portable device that is worn directly on the body or integrated into the user's clothes or accessories. Wearable devices are not only hardware devices, but also powerful functions achieved through software support, data interaction, and cloud interaction. Broadly speaking, wearable smart devices include full-featured, large-sized, and fully or partially independent of smartphones, such as smart watches or smart glasses, as well as devices that only focus on a certain type of application function and need to be used in conjunction with other devices such as smartphones, such as various types of smart bracelets and smart jewelry for vital sign monitoring.

[0033] In addition, in the embodiments of the present application, the electronic device can also be an electronic device in the Internet of Things (IoT) system. IoT is an important part of the future development of information technology. Its main technical feature is to connect objects to the network through communication technology, so as to realize the intelligent network of human-machine interconnection and object-to-object interconnection. The electronic device of the present application can also be a vehicle-mounted unit, a vehicle-mounted module, a vehicle-mounted component, a vehicle-mounted chip or a vehicle-mounted unit built into the vehicle as one or more components or units. The vehicle can implement the method of the present application through the built-in vehicle-mounted unit, vehicle-mounted module, vehicle-mounted component, vehicle-mounted chip or vehicle-mounted unit. Therefore, the embodiments of the present application can be applied to vehicle networking, such as vehicle to everything (V2X), long-term evolution-vehicle (LTE-V), vehicle-to-vehicle (V2V), etc.

[0034] For example, Figure 1 A schematic diagram of a system architecture of an electronic device provided in an embodiment of the present application.

[0035] like Figure 1 As shown, the electronic device includes a processor 110, a transceiver 120, and a display unit 170. The display unit 170 may include a display screen.

[0036] Optionally, the electronic device may further include a memory 130. The processor 110, the transceiver 120 and the memory 130 may communicate with each other through an internal connection path to transfer search data. The memory 130 is used to store computer programs, and the processor 110 is used to call and run the computer programs from the memory 130.

[0037] The processor 110 and the memory 130 may be combined into a processing device, or more commonly, they are independent components, and the processor 110 is used to execute the program code stored in the memory 130 to implement the above functions. In specific implementation, the memory 130 may also be integrated into the processor 110, or may be independent of the processor 110.

[0038] In addition, in order to make the functions of the electronic device more complete, the electronic device may further include one or more of an input unit 160 , an audio circuit 180 , and a sensor 101 .

[0039] Optionally, the electronic device may further include a power supply 150 for providing power to various devices or circuits in the electronic device.

[0040] Understandably, Figure 1 The operations and / or functions of each module in the electronic device shown are respectively for implementing the corresponding processes in the following method embodiments. For details, please refer to the description in the following method embodiments, and the detailed description is appropriately omitted here to avoid repetition.

[0041] Understandably, Figure 1 The processor 110 in the electronic device shown may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units may be independent devices or integrated in one or more processors.

[0042] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0043] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0044] It is understandable that the interface connection relationship between the modules illustrated in the embodiments of the present application is only a schematic illustration and does not constitute a structural limitation on the electronic device. In other embodiments of the present application, the electronic device may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0045] Understandably, Figure 1 The power supply 150 shown is used to supply power to the processor 110 , the memory 130 , the display unit 170 , the input unit 160 , the transceiver 120 , and the like.

[0046] The transceiver 130 can provide wireless communication solutions for electronic devices including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc. The transceiver 120 can be one or more devices integrating at least one communication processing module.

[0047] The electronic device implements the display function through a GPU, a display unit 170, and an application processor. The GPU is a microprocessor for image processing, which connects the display unit 170 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0048] The display unit 170 is used to display images, videos, etc. The display unit 170 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light emitting diode (QLED), etc.

[0049] The memory 130 may be used to store computer executable program codes, which include instructions. The memory 130 may include a program storage area and a data storage area. The program storage area may store an operating system, at least one application required for a function, and the like. The data storage area may store data created during the use of the electronic device, and the like. In addition, the memory 130 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), and the like. The processor 110 executes various functional applications and data processing of the electronic device by running instructions stored in the memory 130 and / or instructions stored in a memory provided in the processor.

[0050] The electronic device can implement audio functions such as music playing and recording through the audio circuit 180 and the application processor.

[0051] It should be understood that the electronic device 100 is a device with image processing capabilities, so it can also be called an image processing device, and this application does not limit this.

[0052] The technical solution in this application will be described below in conjunction with the accompanying drawings.

[0053] In order to clearly describe the technical solutions of the embodiments of the present application, in the embodiments of the present application, words such as "first" and "second" are used to distinguish the same or similar items with substantially the same functions and effects. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit the difference.

[0054] It should be noted that, in this application, words such as "exemplarily" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplarily" or "for example" in this application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplarily" or "for example" is intended to present related concepts in a specific way.

[0055] In addition, "at least one" means one or more, and "plurality" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b and c can mean: a, or b, or c, or a and b, or a and c, or b and c, or a, b and c, where a, b, c can be single or multiple.

[0056] In order to make the purpose and technical solution of this application clearer and more intuitive, an image processing method and an image processing device provided by the embodiment of this application will be described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not used to limit this application.

[0057] Figure 2 2 is a schematic flow chart of an image processing method 200 provided in an embodiment of the present application. Figure 2 As shown, the method 200 may include the following steps:

[0058] S201, the electronic device determines a target area based on a scene recognition result, where the target area is an area where a preset image processing operation is performed.

[0059] It should be understood that the scene recognition result may be a scene recognition result of an image frame preceding the target image frame, which may be referred to as a reference image frame in the present application.

[0060] In a possible implementation, the electronic device may determine the target center point of the target region based on the above scene recognition result, determine the target area of ​​the target region based on the resolution, and further determine the target region based on the target center point and the target area.

[0061] S202, when the electronic device collects the target image frame data, it performs a preset image processing operation on the target image frame data in the above-mentioned target area to obtain and display the target image frame; wherein the preset image processing operation includes at least one of auto focus AF, automatic exposure AE and automatic white balance AWB.

[0062] In a possible implementation, the electronic device may perform at least one of automatic focus AF, automatic exposure AE, and automatic white balance AWB on the target image frame data in the target area based on the target image processing parameters.

[0063] It should be understood that the target image processing parameters may be obtained by the electronic device based on reference statistical data before performing the above-mentioned preset image processing. The reference statistical data may also be referred to as 3A statistical data, which may be obtained by the electronic device based on the statistics of the previous image frame (reference image frame) displaying the target image frame in the above-mentioned target area, and the reference statistical data may include AF statistical data, AE statistical data and AWB statistical data of the previous image frame in the above-mentioned target area.

[0064] In the present application, when the target image frame data is collected, the preset image processing operation can be performed on the target image frame data in the target area to obtain and display the target image frame. The preset image processing operation includes at least one of auto focus AF, automatic exposure AE and automatic white balance AWB, and the target area is determined based on the scene recognition result. In other words, the present application can selectively perform preset image processing on an area of ​​the image frame (such as the target area) based on the scene recognition result to improve the display effect of the image.

[0065] The following uses a mobile phone as an example of an electronic device to describe the image processing method provided by the present application in detail.

[0066] Figure 3 3 is a schematic flow chart of an image processing method 300 provided in an embodiment of the present application. Figure 3 As shown, the method 300 may include the following steps:

[0067] S301, when displaying a reference image frame, the mobile phone obtains a scene recognition result and a focus parameter of the reference image frame.

[0068] It should be understood that the scene recognition result may be a portrait scene or a landscape scene, etc. The portrait scene may also include a multi-person portrait scene, a child scene, an adult scene, a male portrait scene, a female portrait scene, etc., which is not limited in this application.

[0069] In one possible case, the scene recognition result is obtained by the mobile phone when a user operation is detected.

[0070] Exemplarily, the user operation may be a user operation on a scene selection button. Multiple scene selection buttons may be displayed on the mobile phone interface, such as a selection button for a portrait scene, a selection button for a landscape scene, and the like. The user may click the corresponding scene selection button according to their own needs and the current actual scene, so that the mobile phone determines the scene recognition result based on the user operation on the scene selection button. If the user has high requirements for the processing effect of the area where the portrait is located in the image, the user may click the selection button for the portrait scene. When the mobile phone detects the user operation, it may determine the scene recognition result as a portrait scene, so as to prepare for the subsequent positioning of the target area in the area where the portrait is located to meet the user's needs.

[0071] Optionally, the above user operation may be a user focusing operation in addition to the user operation on the scene selection button, wherein the user focusing operation may be an operation on any area on the currently displayed image frame.

[0072] In another possible case, the scene recognition result may be obtained by the mobile phone without detecting any user operation.

[0073] S302: The mobile phone determines a target center point of the target area based on the focus parameters of the reference image frame.

[0074] Exemplarily, the target center may be the focus center point corresponding to the above focus parameters. In other words, the mobile phone may determine the focus center point corresponding to the focus parameters based on the focus parameters, and determine the focus center point as the target center point of the above target area. That is, based on the focus parameters, the area that the user may be interested in is located to ensure the image quality of the target area of ​​the image (such as the area where the portrait is located) under limited hardware resources, so that the final image effect is more in line with user expectations and needs.

[0075] Optionally, the mobile phone may also determine the target center point of the target area based on the scene recognition result and the focus parameter. That is, the mobile phone may determine the target object based on the scene recognition result and the focus parameter, and determine the target center point of the target area based on the target object. The target center may be the object center point corresponding to the target object, and may also be called the centroid of the target object.

[0076] Exemplarily, when the scene recognition result is a multi-portrait scene, the mobile phone can determine the multiple portraits in the multi-portrait scene as reference objects, and determine the target object in the reference objects based on the focus parameters. In other words, the target object can be any one of the multiple portraits.

[0077] It should be understood that when no user operation is detected, the above focus parameters can be understood as active focus parameters. Alternatively, when a user focus operation is detected, the above focus parameters can also be understood as passive focus parameters, that is, the focus parameters are obtained by the mobile phone according to the user's focus operation.

[0078] S303: The mobile phone determines the target area of ​​the target area based on the resolution.

[0079] It should be understood that the target area mentioned above can be understood as the maximum area of ​​the target area. In order to improve the user experience, the target area can also be smaller than the above maximum area (also called area threshold).

[0080] In a possible implementation manner, the mobile phone may obtain a current power consumption state, and determine a target area of ​​the target region based on the current power consumption and the above-mentioned resolution.

[0081] Exemplarily, the mobile phone can determine that the target area is smaller than the area threshold when the current power consumption state is greater than or equal to the power consumption threshold state, so as to avoid excessive power consumption, causing the mobile phone to shut down and affecting the user's continuity of use of the mobile phone.

[0082] Exemplarily, the mobile phone may also determine that the target area is equal to the area threshold when the current power consumption state is less than the power consumption threshold state, so as to improve the display effect and user experience.

[0083] S304: The mobile phone determines the density of the target area based on the scene recognition result of the reference image frame.

[0084] It should be understood that the target region may be composed of multiple sub-regions, and the density may be used to indicate the number and size of blocks contained in each of the multiple sub-regions included in the target region.

[0085] It should also be understood that the multiple blocks can also be understood as regions in each of the above sub-regions whose area is smaller than the above sub-region.

[0086] Exemplarily, when the scene recognition result is a preset scene, such as a portrait scene, the mobile phone determines that the target area includes a sub-area for displaying portraits and a sub-area for displaying non-portraits, and determines that the sub-area for displaying portraits can have more blocks. Correspondingly, the sub-area for displaying portraits can have a smaller area, and the gaps between the blocks can be smaller, so that the image processing accuracy is improved, and then the target image display effect is improved. Conversely, when the mobile phone determines that the above scene recognition result is a non-preset scene, it can also be determined that the number of blocks of the sub-area included in the target area can be less, the area can be larger, and the gaps between the sub-blocks can be larger, so as to reduce power consumption.

[0087] S305, the mobile phone determines the target area based on the target center point, target area and density.

[0088] Exemplarily, the mobile phone may determine the starting point of the target area based on the target area and the target center point, and further determine the target area based on the target center point, the target area, the density and the starting point.

[0089] It should be understood that the starting point can be a point in any direction of the target area, such as a point at the upper right corner or a point at the upper left corner, etc., and this application does not limit this.

[0090] S306: The mobile phone obtains reference statistical data based on the reference image frame displayed in the target area.

[0091] It should be understood that the reference statistical data includes AF statistical data, AE statistical data, and AWB statistical data of the reference image frame within the target area.

[0092] S307, the mobile phone obtains target image processing parameters based on the above reference statistical data.

[0093] It should be understood that the target image processing parameters can be used to instruct the mobile phone to perform at least one of auto focus AF, auto exposure AE and auto white balance AWB, and achieve a specific value or effect. In other words, the mobile phone can determine whether to perform preset image processing based on the above reference statistical data, and determine the target image processing parameters to improve the image display effect.

[0094] S308, when the mobile phone collects the target image frame data, based on the target image processing parameters, performs a preset image processing operation on the target image frame data in the target area to obtain and display the target image frame.

[0095] Exemplarily, the mobile phone may perform at least one of auto focus AF, auto exposure AE and auto white balance AWB on the target area based on the target image processing parameters, so as to realize image processing on the key area and improve the image display effect.

[0096] It should be understood that the above-mentioned reference image frame can be an initial image frame or a non-initial image frame, and this application does not limit this.

[0097] Exemplarily, when the reference image frame is an initial image frame, the electronic device may acquire the reference image frame data, and perform a preset image processing operation on the reference image frame data based on preset image processing parameters to obtain and display the reference image frame. When displaying the reference image frame, the electronic device may perform S201 or S301 to obtain the scene recognition result.

[0098] It should be understood that the above-mentioned preset image processing parameters may be image processing parameters initially configured for the electronic device.

[0099] Exemplarily, the electronic device may also determine the target area based on the scene recognition result of the historical image frame when the reference image frame is not the initial image frame, and when the reference image frame data is acquired, perform the preset image processing operation on the reference image frame data in the new target area to obtain and display the reference image frame. The electronic device may perform S201 or S301 to obtain the scene recognition result when displaying the reference image frame.

[0100] It should be understood that the above-mentioned historical image frame may be the previous image frame of the above-mentioned reference image frame, and the target area determined by the scene recognition result based on the reference image frame may be the same as or different from the target area determined here based on the scene recognition result based on the historical image frame. It can be specifically determined based on the actual scene recognition result, and this application does not impose any limitation on this.

[0101] Figure 4 A schematic diagram of the target area provided for this application. Figure 4 As shown in FIG. 1 , when displaying a historical image frame, the electronic device can determine the target area based on the scene recognition result of the historical image frame. Figure 4 As shown in FIG. 2 , when displaying a reference image frame, the electronic device can determine the target area based on the scene recognition result and focus parameters of the reference image frame. Figure 4 As shown, Figure 4 a and Figure 4 In other words, when receiving new image frame data, the electronic device can determine a new target area based on the current scene recognition result (i.e., the scene recognition result of the reference image frame) and the focus parameter, that is, locate the target area to the region of interest, ensure the image quality of the main area of ​​the image (the target area), improve the image display effect, and make the final image effect more in line with user expectations and needs.

[0102] Optionally, following the above S202 or S308, the electronic device may also perform a preset image processing operation on the non-target area when determining that the target image frame is a preset image frame, so as to reduce the difference in display effects between the target area and the non-target area and improve the image display effect and user experience.

[0103] Exemplarily, the electronic device can obtain new statistical data based on the above-mentioned target image frame in the non-target area, and the new statistical data include AF statistical data, AE statistical data and AWB statistical data in the non-target area, and obtain new image processing parameters based on the new statistical data. When the next image frame of the target image frame is captured, based on the new image processing parameters, a preset image processing operation is performed on the above-mentioned next image frame data in the non-target area to obtain and display the next image frame.

[0104] It should be understood that the above-mentioned preset image frames are image frames with a preset number of frames, and the preset image frames are used to instruct to perform a preset image processing operation on the non-target area of ​​the next image frame.

[0105] Exemplarily, the preset image frame may be multiple, such as even-numbered frames such as the second image frame, the fourth image frame, and the sixth image frame. In other words, the electronic device may determine that the area to be subjected to the preset image processing operation next is a non-target area when displaying even-numbered frames such as the second image frame, the fourth image frame, or the sixth image frame, so as to further improve the image display effect.

[0106] It should also be understood that the above non-target area can be determined when the electronic device determines the target area based on the scene recognition result of the target image frame. That is, when the target area is determined, the remaining area can be determined as the non-target area.

[0107] Figure 5 A schematic diagram of an electronic device 500 is shown, Figure 5 As described above, the electronic device 500 may also include a sensor 501, a 3A data statistics module 502, a 3A control parameter calculation module 503, a 3A control module 504 and a scene recognition module 505, and the image processing method provided in this application is implemented through the above modules.

[0108] Exemplarily, the electronic device 500 can display the reference image frame on the screen through the sensor 501. Among them, the optical image can be projected onto the surface of the sensor 501, and then converted into an electrical signal, which is converted into a digital image signal after A / D (analog-to-digital conversion), and then sent to the digital signal processing chip (digital signal processing, DSP) for processing, and then transmitted to the central processing unit (entral processing unit, CPU) through the (Input output system, I / O) interface for processing, and then displayed on the screen. The electronic device 500 can obtain a scene recognition result based on the reference image frame through the scene recognition module 505, and determine the target area (i.e., the area of ​​interest to the user) through the scene recognition module 505 based on the above scene recognition result. The scene recognition result can be the portrait scene, landscape scene, etc. shown in the above embodiment, and this application does not limit this. When the electronic device 500 collects the target image frame data, it can perform a preset image processing operation on the target image frame data in the target area based on the target image processing parameters through the 3A control module 504, and obtain and display the target image frame on the screen through the sensor 501. The target image processing parameters may be obtained by the electronic device 500 through the 3A control parameter calculation module 503 based on the above-mentioned reference statistical data. The reference statistical data may be obtained by the electronic device 500 through the 3A data statistics module 502 based on the reference image frame displayed in the target area.

[0109] It should be understood that the modules shown above are merely exemplary. In addition, the electronic device may also include other modules, such as other image processing modules, etc., which is not limited in this application.

[0110] It should be understood that the above embodiments may be coupled to each other, and this application does not limit this. Moreover, the size of the sequence number of each process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0111] Combined with the above Figures 1 to 5 , describes in detail the image processing method of the embodiment of the present application, and will now be combined with Figure 6 to Figure 7 , describes in detail the image processing device of an embodiment of the present application.

[0112] Figure 6 An image processing device 600 provided in an embodiment of the present application is shown. The image processing device 600 includes: a determination module 601 and a processing module 602 .

[0113] Among them, the determination module 601 is used to: determine the target area based on the scene recognition result, and the above-mentioned target area is the area for performing preset image processing operations; the processing module 602 is used to: when the target image frame data is collected, perform the above-mentioned preset image processing operation on the above-mentioned target image frame data in the above-mentioned target area, and obtain and display the target image frame; wherein the above-mentioned preset image processing operation includes at least one of auto focus AF, automatic exposure AE and automatic white balance AWB.

[0114] Optionally, the processing module 602 is used to: obtain focus parameters; determine the above-mentioned target area based on the scene recognition result and the focus parameters; wherein the focus parameters and scene recognition results are the focus parameters and scene recognition results of a reference image frame, and the reference image frame is an image frame previous to the above-mentioned target image frame.

[0115] Optionally, the determination module 601 is used to: determine the target center point of the target area based on the above-mentioned focus parameters, and the above-mentioned target center point is the focus center point corresponding to the above-mentioned focus parameters; determine the density of the above-mentioned target area based on the scene recognition result; determine the above-mentioned target area based on the target center point and the density; wherein the target area includes multiple sub-areas, and the density is used to indicate the number and size of blocks contained in each of the multiple sub-areas included in the above-mentioned target area.

[0116] Optionally, the determination module 601 is used to: determine the target object based on the above-mentioned scene recognition result and the above-mentioned focus parameters; determine the target center point of the above-mentioned target area based on the above-mentioned target object, and the above-mentioned target center is the object center point corresponding to the above-mentioned target object; determine the density of the target area based on the above-mentioned scene recognition result; determine the above-mentioned target area based on the target center point and the density; wherein the target area includes multiple sub-areas, and the density is used to indicate the number and size of blocks contained in each of the multiple sub-areas included in the above-mentioned target area.

[0117] Optionally, the determination module 601 is used to: determine the multiple portraits in the multiple portrait scene as reference objects; and determine the target object from the reference objects based on the focus parameters, where the target object is any one of the multiple portraits.

[0118] Optionally, the determination module 601 is used to: determine the target area of ​​the above-mentioned target area based on the resolution; determine the starting point of the above-mentioned target area based on the above-mentioned target area and the above-mentioned target center point; determine the above-mentioned target area based on the above-mentioned target center point, the above-mentioned density, the above-mentioned target area and the above-mentioned starting point.

[0119] Optionally, the processing module 602 is used to: obtain the current power consumption state; determine the target area of ​​the target area based on the current power consumption and the resolution; wherein the target area is less than or equal to a threshold area, and the threshold area is used to indicate the maximum area of ​​the target area, and when the current power consumption state is greater than or equal to a power consumption threshold state, the target area is less than the area threshold, and when the current power consumption state is less than the power consumption threshold state, the target area is equal to the area threshold.

[0120] Optionally, the scene recognition result is a scene recognition result of a reference image frame, and the reference image frame is a previous image frame of the target image frame. The processing module 602 is used to: obtain reference statistical data based on the reference image frame displayed in the target area, and the reference statistical data includes AF statistical data, AE statistical data and AWB statistical data in the target area; obtain target image processing parameters based on the reference statistical data; and perform the preset image processing operation on the target image frame data in the target area based on the target image processing parameters to obtain and display the target image frame.

[0121] Optionally, the reference image frame is an initial image frame, and the processing module 602 is used to: acquire reference image frame data; perform preset image processing operations on the reference image frame data based on preset image processing parameters, and obtain and display the reference image frame.

[0122] Optionally, the reference image frame is a non-initial image frame, and the processing module 602 is used to: determine the new target area based on the scene recognition results of the historical image frame; when the reference image frame data is collected, perform the preset image processing operation on the reference image frame data in the new target area to obtain and display the reference image frame.

[0123] Optionally, the processing module 602 is used to: when it is determined that the target image frame is a preset image frame, obtain new statistical data based on the target image frame in the non-target area, the new statistical data including AF statistical data, AE statistical data and AWB statistical data in the non-target area; obtain new image processing parameters based on the new statistical data; when the next image frame of the target image frame is acquired, perform the preset image processing operation on the next image frame data in the non-target area based on the new image processing parameters, and obtain and display the next image frame; wherein the preset image frame is an image frame with a preset number of frames, there are multiple preset image frames, and the preset image frame is used to indicate that the preset image processing operation is performed on the non-target area of ​​its next image frame.

[0124] It should be understood that the image processing device 600 here is embodied in the form of a functional module. The term "module" here may refer to an application specific integrated circuit (ASIC), an electronic circuit, a processor (such as a shared processor, a proprietary processor or a group processor, etc.) and a memory for executing one or more software or firmware programs, a merged logic circuit and / or other suitable components that support the described functions. In an optional example, those skilled in the art may understand that the image processing device 600 may be specifically an electronic device in the above-mentioned embodiment, or the functions of the electronic device in the above-mentioned embodiment may be integrated in the image processing device 600, and the image processing device 600 may be used to execute the various processes and / or steps corresponding to the electronic device in the above-mentioned method embodiment. To avoid repetition, it will not be repeated here.

[0125] The image processing device 600 has the function of implementing the corresponding steps performed by the electronic device in the above method; the above functions can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions. In the embodiment of the present application, Figure 6 The image processing device 600 in the embodiment may also be a chip or a chip system, for example, a system on chip (SoC).

[0126] Figure 7 Another image processing device 700 provided by an embodiment of the present application is shown. The image processing device 700 includes: a processor 701 and a memory 702, a communication interface 703 and a bus 704. The memory 702 is used to store instructions, and the processor 701 is used to execute the instructions stored in the memory 702. The processor 701, the memory 702 and the communication interface 703 are connected to each other through the bus 704.

[0127] Among them, the processor 701 is used to: determine the target area based on the scene recognition result, and the target area is the area for performing the preset image processing operation; when the target image frame data is collected, perform the above-mentioned preset image processing operation on the target image frame data in the above-mentioned target area to obtain and display the target image frame; wherein, the preset image processing operation includes at least one of auto focus AF, automatic exposure AE and automatic white balance AWB.

[0128] It should be understood that the image processing device 700 can specifically be the electronic device in the above-mentioned embodiments, the functions of the electronic device in the above-mentioned embodiments can be integrated in the image processing device 700, and the image processing device 700 can be used to execute the various steps and / or processes corresponding to the electronic device in the above-mentioned method embodiments.

[0129] Optionally, the memory 702 may include a read-only memory and a random access memory, and provide instructions and data to the processor 701. A portion of the memory 702 may also include a non-volatile random access memory. For example, the memory 702 may also store information about the device type. The processor 701 may be used to execute the instructions stored in the memory, and when the processor executes the instructions, the processor 701 may execute the various steps and / or processes corresponding to the electronic device in the above method embodiment.

[0130] It should be understood that in the embodiments of the present application, the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0131] In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in a processor or an instruction in the form of software. The steps of the method disclosed in conjunction with the embodiment of the present application can be directly embodied as a hardware processor for execution, or a combination of hardware and software modules in a processor for execution. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor executes the instructions in the memory, and completes the steps of the above method in conjunction with its hardware. To avoid repetition, it is not described in detail here.

[0132] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0133] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0134] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0135] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0136] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0137] If the above functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server, or electronic device, etc.) to perform all or part of the steps of the above methods in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program codes.

[0138] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the above claims.

Claims

1. An image processing method, characterized in that: include: Based on the scene recognition result, determining a target area, wherein the target area is an area where a preset image processing operation is performed; When target image frame data is acquired, performing the preset image processing operation on the target image frame data in the target area to obtain and display the target image frame; The preset image processing operation includes at least one of auto focus AF, automatic exposure AE and automatic white balance AWB.

2. The method according to claim 1, characterized in that Before determining the target area based on the scene recognition result, the method further includes: Get focus parameters; The determining of the target area based on the scene recognition result includes: Determining the target area based on the scene recognition result and the focus parameter; The focus parameter and the scene recognition result are the focus parameter and the scene recognition result of a reference image frame, and the reference image frame is an image frame previous to the target image frame.

3. The method according to claim 2, characterized in that The determining the target area based on the scene recognition result and the focus parameter includes: Based on the focus parameter, determining a target center point of the target area, the target center point being a focus center point corresponding to the focus parameter; Based on the scene recognition result, determining the density of the target area; Determining the target area based on the target center point and the sparseness; The target area includes multiple sub-areas, and the density is used to indicate the number and size of blocks contained in each of the multiple sub-areas included in the target area.

4. The method according to claim 2, characterized in that: The determining the target area based on the scene recognition result and the focus parameter includes: Determining a target object based on the scene recognition result and the focus parameter; Based on the target object, determining a target center point of the target area, the target center being an object center point corresponding to the target object; Based on the scene recognition result, determining the density of the target area; Determining the target area based on the target center point and the sparseness; The target area includes multiple sub-areas, and the density is used to indicate the number and size of blocks contained in each of the multiple sub-areas included in the target area.

5. The method according to claim 4, wherein the scene recognition result is a multi-person portrait scene, and determining the target object based on the scene recognition result and the focus parameter comprises: determining a plurality of portraits in the multi-portrait scene as reference objects; Based on the focus parameter, the target object is determined in the reference object, and the target object is any one of the multiple portraits.

6. The method according to claim 3 or 4, characterized in that: Before determining the target area based on the target center point and the sparseness, the method further includes: determining a target area of ​​the target region based on the resolution; Based on the target area and the target center point, determining a starting point of the target area; The determining the target area based on the target center point and the sparseness includes: The target area is determined based on the target center point, the density, the target area and the starting point.

7. The method according to claim 6, characterized in that Before determining the target area of ​​the target region based on the resolution, the method further includes: Get the current power consumption status; The determining the target area of ​​the target region based on the resolution includes: determining the target area of ​​the target region based on the current power consumption and the resolution; Among them, the target area is less than or equal to the threshold area, the threshold area is used to indicate the maximum area of ​​the target area, when the current power consumption state is greater than or equal to the power consumption threshold state, the target area is less than the area threshold, when the current power consumption state is less than the power consumption threshold state, the target area is equal to the area threshold.

8. The method according to claim 1, characterized in that The scene recognition result is a scene recognition result of a reference image frame, the reference image frame is an image frame previous to the target image frame, and in the case where the target image frame data is acquired, the preset image processing operation is performed on the target image frame data in the target area, before obtaining and displaying the target image frame, the method further includes: acquiring reference statistical data based on the reference image frame displayed in the target area, the reference statistical data comprising AF statistical data, AE statistical data and AWB statistical data in the target area; In the case where the target image frame data is acquired, performing the preset image processing operation on the target image frame data in the target area to obtain and display the target image frame includes: Based on the reference statistical data, obtaining target image processing parameters; Based on the target image processing parameters, the preset image processing operation is performed on the target image frame data in the target area to obtain and display the target image frame.

9. The method according to claim 8, characterized in that The reference image frame is an initial image frame. Before obtaining reference statistical data based on the reference image frame displayed on the target area, the method further includes: Acquiring reference image frame data; A preset image processing operation is performed on the reference image frame data based on preset image processing parameters to obtain and display the reference image frame.

10. The method according to claim 8, characterized in that The reference image frame is a non-initial image frame. Before obtaining reference statistical data based on the reference image frame displayed on the target area, the method further includes: Determine the new target area based on the scene recognition results of the historical image frames; When the reference image frame data is acquired, the preset image processing operation is performed on the reference image frame data in the new target area to obtain and display the reference image frame.

11. The method according to claim 1, characterized in that: The method further comprises: In the case where it is determined that the target image frame is a preset image frame, new statistical data are acquired based on the target image frame in the non-target area, the new statistical data including AF statistical data, AE statistical data and AWB statistical data in the non-target area; Based on the new statistical data, new image processing parameters are obtained; When the next image frame of the target image frame is acquired, based on the new image processing parameters, the preset image processing operation is performed on the next image frame data in the non-target area to obtain and display the next image frame; The preset image frame is an image frame with a preset number of frames, there are multiple preset image frames, and the preset image frame is used to indicate to perform the preset image processing operation on the non-target area of ​​the next image frame.

12. An image processing device, characterized in that: include: A determination module, used to determine a target area based on the scene recognition result, wherein the target area is an area where a preset image processing operation is performed; A processing module is used to perform the preset image processing operation on the target image frame data in the target area when the target image frame data is collected, so as to obtain and display the target image frame; wherein the preset image processing operation includes at least one of auto focus AF, automatic exposure AE and automatic white balance AWB.

13. An image processing device, characterized in that: The system comprises a processor and a memory, wherein the memory is used to store code instructions, and the processor is used to run the code instructions to execute the method according to any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that: Used to store a computer program, the computer program comprising instructions for implementing the method according to any one of claims 1 to 11.

15. A computer program product, comprising computer program code, characterized in that: When the computer program code is executed on a computer, the computer is caused to implement the method according to any one of claims 1 to 11.

Images