Electronic device and image processing method thereof

Abstract

An embodiment of the present application provides an electronic device and an image processing method for the electronic device, wherein the electronic device includes: a processor for selecting a first image processing model from a plurality of image processing models based on scene information, and performing first image signal processing on a first image signal using the first image processing model to obtain a second image signal, wherein the first image signal is obtained based on first image data output by an image sensor, and the scene information reflects a feature classification of the first image signal; and an image signal processor ISP for performing second image signal processing on the second image signal to obtain a first image processing result. The electronic device provided by the embodiment of the present application can improve the image processing effect.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of electronic technology, and in particular, to an electronic device and an image processing method of the electronic device. Background Art

[0002] With the advancement of electronic science and technology, smart terminals are integrating more and more functions. Thanks to the development of image processing technology, more and more users like to use smart terminal devices to take photos, record videos, and make video calls.

[0003] Due to the limitations of the computing power of the image signal processor (ISP) algorithms in smart terminals, in order to improve image processing effects, the industry has proposed a technology that combines traditional image processing algorithms with artificial intelligence (AI) algorithms for image processing. In this solution, the same network model is usually used to process image signals collected in various scenarios, which increases the complexity of the model structure and the complexity of the model training process. Due to the limitations of the memory capacity and operating speed of the terminal device, this network model is difficult to deploy and implement in the terminal device. As a result, traditional technologies have not yet solved the problem of poor image processing effects using traditional ISPs in terminal devices. Summary of the Invention

[0004] The electronic device and image processing method of the electronic device provided in this application can improve the image processing effect. To achieve the above purpose, this application adopts the following technical solutions.

[0005] In a first aspect, an embodiment of the present application provides an electronic device, comprising: an artificial intelligence (AI) processor for selecting a first image processing model from a plurality of image processing models based on scene information, and performing first image signal processing on a first image signal using the first image processing model to obtain a second image signal, wherein the first image signal is obtained based on first image data output by an image sensor, and the scene information reflects a feature classification of the first image signal; and an image signal processor (ISP) for performing second image signal processing on the second image signal to obtain a first image processing result.

[0006] By running multiple image processing models to process image data collected in various scenarios, the AI ​​processor can reduce the complexity of the structure of each image processing model. For example, each image processing model can be implemented with fewer convolutional layers and a smaller number of nodes, making the image processing model easier to deploy and run in terminal devices. Since the complexity of the image processing model structure is reduced, the running speed of the AI ​​processor, that is, the image processing speed, can be increased. In addition, since each image processing model is dedicated to processing image data in one scenario, the image processing effect can also be improved compared to using the same image processing model to process image data collected in multiple scenarios.

[0007] In a possible implementation, the first image signal processing includes at least one of the following processing procedures: noise elimination, black level correction, shading correction, white balance correction, demosaicing, chromatic aberration correction, or gamma correction.

[0008] In a possible implementation, the scene information includes at least one of first ambient light brightness information and first motion state information of the electronic device.

[0009] In one possible implementation, the AI ​​processor is further used to: when the first motion state information is used to indicate that the electronic device is moving at a speed lower than a preset threshold, process the first image signal based on a previous frame image signal and an image processing result of the previous frame image signal.

[0010] By processing the current frame image signal with reference to the previous frame image and the image processing result of the previous frame image, the processing effect of the image signal can be further improved.

[0011] In one possible implementation, the ISP is used to: select a first parameter from multiple groups of parameters of an image processing algorithm based on the scene information; obtain an updated image processing algorithm based on the first parameter; and perform the second image signal processing on the second image signal using the updated image processing algorithm.

[0012] In one possible implementation, the second image signal processing includes at least one of the following processing processes: noise elimination, black level correction, shadow correction, white balance correction, demosaicing, color difference correction, gamma correction, color difference correction or RGB to YUV domain conversion.

[0013] In a possible implementation, the ISP is further configured to: receive the first image data from the image sensor, and perform third image signal processing on the first image data to obtain the first image signal.

[0014] In a possible implementation, the ISP is further configured to: perform the third image signal processing on the first image data using the updated image processing algorithm.

[0015] In a possible implementation, the electronic device further includes: a controller configured to generate the scene information based on data collected by at least one sensor, where the at least one sensor includes at least one of the following: an acceleration sensor, a gravity sensor, and the image sensor.

[0016] In a possible implementation, the third image signal processing includes at least one of the following processing procedures: noise elimination, black level correction, shading correction, white balance correction, demosaicing, chromatic aberration correction, or gamma correction.

[0017] In one possible implementation, the multiple image processing models are trained based on multiple training sample sets corresponding to multiple scenes, wherein each of the multiple training sample sets includes a preprocessed image signal generated by processing sample image data collected in the corresponding scene, and a reference image signal generated by processing the sample image data.

[0018] In a second aspect, an embodiment of the present application provides an image processing method for an electronic device, the image processing method comprising: based on scene information, controlling an artificial intelligence (AI) processor to select a first image processing model from a plurality of image processing models, and using the first image processing model to perform a first image signal processing on a first image signal to obtain a second image signal, wherein the first image signal is obtained based on first image data output by an image sensor, and the scene information reflects a feature classification of the first image signal; controlling an image signal processor (ISP) to perform a second image signal processing on the second image signal to obtain a first image processing result.

[0019] Based on the second aspect, in a possible implementation method, the image signal processor ISP is controlled to perform second image signal processing on the second image signal to obtain an image processing result, including: based on the scene information, controlling the ISP to select a first parameter from multiple groups of parameters for running an image processing algorithm; controlling the ISP to obtain an updated image processing algorithm based on the first parameter; and controlling the ISP to perform the second image signal processing on the second image signal using the updated image processing algorithm.

[0020] In a third aspect, an embodiment of the present application provides an image processing device, which includes: an AI processing module for selecting a first image processing model from a plurality of image processing models, and using the first image processing model to perform first image signal processing on a first image signal to obtain a second image signal, wherein the first image signal is obtained based on first image data output by an image sensor, and the scene information reflects the feature classification of the first image signal; an image signal processing module for performing second image signal processing on the second image signal to obtain a first image processing result.

[0021] In a possible implementation manner, the scene information includes at least one of first ambient light brightness information and first motion state information of the electronic device.

[0022] In one possible implementation, the image signal processing module is used to: select a first parameter from multiple groups of parameters for running an image processing algorithm based on the scene information; obtain an updated image processing algorithm based on the first parameter; and perform the second image signal processing on the second image signal using the updated image processing algorithm.

[0023] In one possible implementation, the AI ​​processing module is further used to: when the first motion state information is used to indicate that the electronic device is moving at a speed lower than a preset threshold, process the first image signal based on a previous frame image signal and an image processing result of the previous frame image signal.

[0024] In a possible implementation, the first image signal processing includes at least one of the following processing procedures: noise elimination, black level correction, shading correction, white balance correction, demosaicing, chromatic aberration correction, or gamma correction.

[0025] In one possible implementation, the second image signal processing includes at least one of the following processing processes: noise elimination, black level correction, shadow correction, white balance correction, demosaicing, color difference correction, gamma correction, color difference correction or RGB to YUV domain conversion.

[0026] In one possible implementation, the multiple image processing models are trained based on multiple training sample sets corresponding to multiple scenes, wherein each of the multiple training sample sets includes a preprocessed image signal generated by processing sample image data collected in the corresponding scene, and a reference image signal generated by processing the sample image data.

[0027] In a fourth aspect, an embodiment of the present application provides an electronic device, comprising a memory and at least one processor, wherein the memory is used to store computer programs, and the at least one processor is configured to call all or part of the computer programs stored in the memory to execute the method described in the second aspect above. The at least one processor includes the AI ​​processor and the ISP. Optionally, the electronic device also includes the image sensor.

[0028] In a fifth aspect, embodiments of the present application provide a system on chip, comprising at least one processor and an interface circuit, wherein the interface circuit is configured to obtain a computer program from outside the chip system; the computer program, when executed by the at least one processor, is configured to implement the method described in the second aspect above. The at least one processor comprises the AI ​​processor and the ISP.

[0029] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, which, when executed by at least one processor, is used to implement the method according to the second aspect. The at least one processor includes the AI ​​processor and the ISP.

[0030] In a seventh aspect, an embodiment of the present application provides a computer program product, which, when executed by at least one processor, is used to implement the method described in the second aspect. The at least one processor includes the AI ​​processor and the ISP.

[0031] It should be understood that the second to seventh aspects of this application are consistent with the technical solutions of the first aspect of this application, and the beneficial effects achieved by each aspect and the corresponding feasible implementation methods are similar, so they will not be repeated. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0033] Figure 1 is a hardware structure diagram of an electronic device provided in an embodiment of the present application;

[0034] Figure 2 This is a schematic diagram of an application scenario provided by an embodiment of the present application;

[0035] Figure 3 is another hardware structure diagram of the electronic device provided in an embodiment of the present application;

[0036] Figure 4 is another hardware structure diagram of the electronic device provided in an embodiment of the present application;

[0037] Figure 5 is another hardware structure diagram of the electronic device provided in an embodiment of the present application;

[0038] Figure 6 is another hardware structure diagram of the electronic device provided in an embodiment of the present application;

[0039] Figure 7 This is a schematic flowchart of a training method for an image processing model running in an AI processor provided in an embodiment of the present application;

[0040] Figure 8 is a schematic flow chart of the image processing method provided in an embodiment of the present application;

[0041] Figure 9 It is a schematic diagram of the software structure of the electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0042] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0043] The terms "first," "second," and similar terms used herein do not denote any order, quantity, or importance, but are merely used to distinguish one component from another. Similarly, terms such as "a" or "an" do not denote a limitation of quantity, but rather denote the presence of at least one component. Terms such as "coupled" and similar terms are not limited to direct physical or mechanical connections but may include electrical connections, whether direct or indirect, and are equivalent to connectivity in a broad sense.

[0044] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a concrete way. In the description of the embodiments of the present application, unless otherwise specified, the meaning of "multiple" refers to two or more. For example, a plurality of processors refers to two or more processors.

[0045] The electronic device provided in the embodiments of the present application may be an electronic device or a module, chip, chipset, circuit board or component integrated into an electronic device. The electronic device may be a user equipment (UE), such as a mobile phone, tablet computer, smart screen or image capture device and other types of devices. The electronic device may be provided with an image sensor for collecting image data. The electronic device may also be installed with various software applications such as camera applications, video call applications or online video capture applications for driving image sensors to collect images. Users can use the image sensor to take photos or videos by launching the above-mentioned applications. Users can also use such applications to perform various personalized settings for image beautification. Taking video call applications as an example, users can choose to automatically adjust the screen image (such as the facial avatar presented or the background image presented) during a video call (such as "one-click beautification"). When the user launches the above-mentioned applications or launches the above-mentioned applications and selects image beautification, the image processing service supported by the above-mentioned applications in the electronic device can trigger the electronic device to process the image data collected by the image sensor, thereby presenting the processed image on the screen of the electronic device to achieve the effect of image beautification. The image beautification may include, for example, but is not limited to: increasing the brightness of a part or the entire image, changing the display color of the image, smoothing the facial objects presented in the image, adjusting the saturation of the image, adjusting the exposure of the image, adjusting the vividness of the image, adjusting the highlights of the image, adjusting the contrast of the image, adjusting the sharpness of the image, or adjusting the clarity of the image. The image processing described in the embodiment of the present application may include, but is not limited to: noise removal, black level correction, shadow correction, white balance correction, demosaicing, color difference correction, gamma correction or red, green and blue (RGB) to YUV (YCrCb) domain, so as to achieve the above-mentioned image beautification effect. Based on the electronic device described in the embodiment of the present application, in a specific application scenario, when user A and user B make a video call, the image presented on the screen of the electronic device used by user A and the image of user A presented on the screen of the electronic device used by user B may be the image processed by the electronic device described in the embodiment of the present application, and the processed image will be presented until user A and user B terminate the video call or user A turns off the image processing service.

[0046] Based on the application scenario described above, please refer to Figure 1, which shows a hardware structure diagram of the electronic device provided in an embodiment of the present application. The electronic device 100 can be, for example, a chip or a chipset or a circuit board equipped with a chip or chipset or an electronic device including the circuit board, but it is not used to limit the embodiment. The specific electronic device is as described above and is omitted here. The chip or chipset or the circuit board equipped with the chip or chipset can work under software drive. The electronic device 100 includes one or more processors, such as an AI processor 101 and an ISP 102. Optionally, the one or more processors can be integrated into one or more chips, and the one or more chips can be regarded as a chipset. When one or more processors are integrated into the same chip, the chip is also called a system on a chip (System on a Chip, SOC). In addition to the one or more processors, the electronic device 100 also includes one or more other components, such as a memory 104 and an image sensor 105. In one possible implementation, the memory 104 can be located in the same system on chip in the electronic device 100 as the AI ​​processor 101 and ISP 102, that is, the memory 104 is integrated into the above Figure 1 The SOC shown.

[0047] like Figure 1The AI ​​processor 101 shown may include a dedicated neural processor such as a neural network processor (NPU), including but not limited to a convolutional neural network processor, a tensor processor, or a neural processing engine. The AI ​​processor can be used as a separate component or integrated into other digital logic devices, which include but are not limited to: a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or a DSP (Digital Signal Processing). Exemplarily, the CPU, GPU, and DSP are all processors within a system-on-chip. The AI ​​processor 101 can run multiple image processing models, which are used to perform image processing operations in multiple scenarios. One of the image processing models is used to perform image processing operations in one of the scenarios. For example, the multiple scenarios include a scene with high external ambient light brightness and a scene with low external ambient light brightness. The AI ​​processor 101 can run two image processing models, one of which is used to perform image processing operations in a scene with high ambient light brightness, and the other is used to perform image processing operations in a scene with low ambient light brightness. In one possible implementation, the above-mentioned multiple scenarios can be divided based on pre-set scene information. Scene information may include, but is not limited to, at least one of the following: ambient light brightness information and the motion state information of the electronic device. In embodiments of the present application, scene information reflects the feature classification of the image signal to be processed by the AI ​​processor. Scene information may include ambient light brightness information and the motion state information of the electronic device. Specifically, image signal features may include, but are not limited to, noise features, shadow features, and white balance features. Feature classification includes motion features and ambient light brightness features, for example, which can be divided into low ambient light brightness and high ambient light brightness, or into high-speed motion and low-speed motion states. This feature classification can be used to indicate, for example, the magnitude of image signal noise or the size of shadows in the image signal. When the scene in which the image data is collected varies, the feature categories of the image signal corresponding to the collected image data vary. The AI ​​processor can determine the features of the image signal corresponding to the image data through scene information. Therefore, the AI ​​processor 101 can execute one of the image processing models based on the scene information to perform image processing. Each of the multiple image processing models described above can execute one or more image processing processes. The one or more image processing processes may include, but are not limited to, noise reduction, black level correction, shadow correction, white balance correction, demosaicing, color difference correction, gamma correction, or RGB to YUV conversion. Each image processing model is trained using machine learning methods based on sample image data collected in the corresponding scene.Among them, specific reference is made to the training method of the image processing model. Figure 7 The embodiment shown. It should be noted that the multiple image processing models running in the AI ​​processor 101 are all used to perform the same image processing operation. For example, the multiple image processing models running in the AI ​​processor 101 are all used to perform image processing operations for noise reduction. Based on different scene information, the noise reduction level performed by each image processing model is different. For example, in a high ambient light brightness scene, the noise of the image signal is low, and the noise reduction level performed by the image processing model corresponding to the high ambient light brightness scene is relatively weak; in a low ambient light brightness scene, the noise of the image signal is high, and the noise reduction level performed by the image processing model corresponding to the low ambient light brightness scene is relatively strong. In an embodiment of the present application, the AI ​​processor 101 can reduce the complexity of each image processing model by running multiple image processing models to process image data collected in various scenes. For example, each image processing model can be implemented with fewer convolutional layers and a smaller number of nodes, thereby improving the running speed of the AI ​​processor 101, that is, the image processing speed. In addition, since each image processing model is dedicated to processing image data in one scene, the image processing effect can be improved compared to using the same image processing model to process image data collected from multiple scenes.

[0048] The following takes the scene information including ambient light brightness information and the motion state information of the electronic device as an example to describe in more detail the various image processing models run by the AI ​​processor 101. In an embodiment of the present application, the motion state of the electronic device can be divided into multiple motion state intervals according to the order of the motion speed of the electronic device from high to low, for example, the motion state of the electronic device is divided into five motion state intervals from the first motion state to the fifth motion state. Similarly, the ambient light brightness can be divided into multiple brightness intervals according to the order of the ambient light brightness from low brightness to high brightness, for example, the ambient light brightness is divided into five brightness intervals from the first brightness to the fifth brightness. Then, the motion state intervals and brightness intervals are arbitrarily combined to obtain multiple combinations of motion state and brightness. For each of the multiple combinations, there is an image processing model. That is, multiple combinations correspond to multiple image processing models. In the embodiment of the present application, the motion state of the electronic device is divided into two categories: high-speed motion state and low-speed motion state (wherein the stationary state can be divided into the low-speed motion state), and the ambient light brightness is divided into two categories: low ambient light brightness and high ambient light brightness. Figure 2 The application scenario shown in the following is described. Figure 2As shown, the motion state of the electronic device is divided into a low-speed motion state and a high-speed motion state, and the ambient light brightness is divided into low ambient light brightness and high ambient light brightness. The AI ​​processor 101 can run four image processing models. Among them, image processing model 01 is used to perform image processing operations on image data collected in a low-speed motion state and low ambient light brightness scene; image processing model 02 is used to perform image processing operations on image data collected in a low-speed motion state and high ambient light brightness scene; image processing model 03 is used to perform image processing operations on image data collected in a high-speed motion state and low ambient light brightness scene; image processing model 04 is used to perform image processing operations on image data collected in a high-speed motion state and high ambient light brightness scene. Based on the scene information, the AI ​​processor 101 runs one of the four image processing models to perform image processing. As an example, assuming that the scene information is used to indicate a high-speed motion state and high ambient light brightness, the AI ​​processor 101 runs image processing model 04 to process the image data.

[0049] In one possible implementation of an embodiment of the present application, when the scene information includes the motion state information of the electronic device, the image processing model corresponding to the low-speed motion state (that is, the electronic device moves at a speed lower than a preset threshold) can be obtained by training a recurrent neural network based on training samples. In the low-speed motion state scenario, when the AI ​​processor runs the image processing model to process the current frame image signal, the AI ​​processor can also input the previous frame image signal and at least one of the image processing results of the previous frame image signal, as well as the current frame image signal, into the image processing model, and the image processing model can process the current frame image signal with reference to the previous frame image signal and the image processing result of the previous frame image signal.

[0050] The scene information described in the embodiments of the present application may be sent to the AI ​​processor 101 by the controller running in the electronic device 100. In one possible implementation, the AI ​​processor 101 may pre-store a first mapping relationship table between scene information and storage address information of the image processing model. After obtaining the scene information, the AI ​​processor 101 may query the first mapping relationship table to obtain the address information of the corresponding image processing model. Finally, the AI ​​processor 101 may load the image processing model from the address indicated by the obtained address information. In other possible implementations, the above-mentioned first mapping relationship table may also be pre-stored in the controller. After obtaining the scene information, the controller may directly send the storage address information of the image processing model to the AI ​​processor 101 based on the first mapping relationship table. The specific method of determining the scene information is described below. Figure 3 The relevant descriptions in the illustrated embodiments are omitted here for brevity.

[0051] like Figure 1 The illustrated ISP 102 can be configured with multiple hardware modules or run software programs to process images. ISP 102 executes image processing algorithms to perform multiple image processing processes, including but not limited to tone mapping, contrast enhancement, edge enhancement, noise reduction, and color correction. Certain parameters in the image processing algorithms executed by ISP 102 are adjustable. For example, the spatial domain Gaussian kernel parameters and the pixel value domain Gaussian kernel parameters in the image processing algorithm used to perform noise reduction are examples. In one possible implementation, ISP 102 can be pre-configured with multiple sets of adjustable parameter values ​​corresponding to image processing in various scenarios. One set of adjustable parameter values ​​corresponds to image processing in one of the scenarios. These multiple scenarios can also be divided based on scene information, which is the same as the scene information used to set up the image processing model. For details, please refer to the relevant description and will not be elaborated here. Based on the scene information, ISP 102 can select a set of adjustable parameter values ​​and update the corresponding portion of the image processing algorithm based on the selected adjustable parameter values. For example, among the multiple image processing processes performed by ISP 102, only the parameters of the noise reduction image processing algorithm are adjustable, while the parameters of the remaining image processing processes do not need to be adjusted. ISP 102 can then update only the noise reduction image processing algorithm based on the selected parameter values. The updated image processing algorithm is then used to process the image signal. Continuing with the example of scene information including ambient light brightness information and the motion state of the electronic device, the motion state of the electronic device including high-speed and low-speed motion, and the ambient light brightness including low brightness and high brightness, the correspondence between the values ​​of the adjustable parameters in ISP 102 and the scene will be described. ISP 102 can be pre-set with four sets of adjustable parameter values: the first set of adjustable parameter values ​​corresponds to low-speed motion and low-brightness scenes, the second set of adjustable parameter values ​​corresponds to low-speed motion and high-brightness scenes, the third set of adjustable parameter values ​​corresponds to high-speed motion and low-brightness scenes, and the fourth set of adjustable parameter values ​​corresponds to high-speed motion and high-brightness scenes. As an example, assuming the scene information indicates high-speed motion and high brightness, ISP 102 uses the values ​​of the fourth set of adjustable parameters to update the relevant image processing algorithms being executed. Then, the image signal is processed using the updated image processing algorithm. In one possible implementation, ISP 102 may pre-store a second mapping table between scene information and adjustable parameter values. Based on the scene information, ISP 102 may query the second mapping table to obtain the corresponding adjustable parameter value.

[0052] In an embodiment of the present application, the AI ​​processor 101 and the ISP 102 can cooperate with each other to process image data collected in the same scene. Specifically, the image data obtained from the image sensor 105 can undergo multiple image processing processes to generate the final image processing result. The multiple image processing processes may include but are not limited to: noise reduction, black level correction, shadow correction, white balance correction, demosaicing, color difference correction, gamma correction or RGB to YUV domain conversion. The AI ​​processor 101 can execute one or more of the above-mentioned image processing processes by running the image processing model, that is, corresponding to one or more of the above-mentioned image processing operations, and the ISP 102 can also execute one or more of the above-mentioned image processing processes by running the image processing algorithm. Therefore, the entire image processing flow includes multiple processing processes, which are assigned as tasks to the AI ​​processor 101 and the ISP 102. The AI ​​processor 101 and the ISP 102 can perform different image processing processes, or the AI ​​processor 101 and the ISP 102 can perform the same image processing process. When the AI ​​processor 101 and ISP102 perform the same image processing process, the image processing performed by the AI ​​processor 101 can serve as an enhancement or supplement to the image processing process. For example, when the AI ​​processor 101 and ISP102 perform the noise elimination process at the same time, ISP102 is used to perform the initial denoising, and the AI ​​processor 101 is used to perform the secondary denoising based on the initial denoising of ISP102. In one possible implementation, ISP102 and the AI ​​processor 101 can communicate through an electronic circuit connection. The electronic circuit connection between the AI ​​processor 101 and ISP102 is also called a physical connection or an interrupt connection. The interrupt connection includes an interrupt signal processing hardware circuit for realizing the interrupt signal sending and receiving functions and a connecting line for transmitting signals to realize the sending and receiving of interrupt signals. The interrupt signal processing hardware circuit includes but is not limited to a traditional interrupt controller circuit. Regarding the specific implementation scheme of the interrupt signal processing hardware circuit, reference can be made to the relevant description of the interrupt controller in the prior art, which will not be repeated here. Among them, the specific connection between the AI ​​processor 101 and ISP102 and the specific implementation of the mutual cooperation to process the image are referred to. Figure 4-Figure 6 Description of the illustrated embodiment.

[0053] In the embodiment of the present application, the electronic device 100 further includes a controller 103, such as Figure 3 As shown. The controller 103 can be an integrated controller. In a specific implementation, the controller 103 can be various digital logic devices or circuits, including but not limited to: CPU, GPU, microcontroller, microprocessor or DSP, etc. The controller 103 can be located in the same system on chip in the electronic device 100 with the AI ​​processor 101 and ISP102, that is, the controller 103 is integrated into the system on chip. Figure 1 In the SOC shown. In addition, the controller 103 may also be separately provided from the AI ​​processor 101, ISP102 and memory 10, which is not limited in this embodiment. Furthermore, the controller 103 may also be integrated with the AI ​​processor 101 in the same logic operation device (such as a CPU), and the functions performed by the controller 103 and the AI ​​processor 101 described in the embodiment of the present application may be implemented by the same logic operation device. The controller 103 runs a software program or a software plug-in to drive the controller 103 to obtain the above-mentioned scene information, and then sends the obtained scene information to the AI ​​processor 101 and ISP102 respectively. When the scene information includes ambient light brightness information, in one possible implementation, the ambient light brightness information may be generated by the controller 103 based on the sensitivity information of the image data, wherein the sensitivity information of the image data may be calculated by the exposure compensation module in the ISP102 by running the corresponding algorithm. The ambient light brightness information may be a bit signal. The controller 103 may be pre-set with multiple sensitivity intervals (e.g., a low sensitivity interval and a high sensitivity interval), and the controller 103 may compare the obtained sensitivity information with the threshold values ​​of the multiple sensitivity intervals, and generate a bit signal based on the comparison result. When the scene information includes the motion state information of the electronic device, in one possible implementation, the motion state information of the electronic device may be generated by the controller 103 based on the acceleration data of the electronic device and the three-axis component (X-axis, Y-axis, and Z-axis) data of the electronic device. The motion state information of the electronic device may be a bit signal. The controller 103 may also be pre-set with multiple motion speed intervals (e.g., a low motion speed interval and a high motion speed interval), and the controller 103 may generate motion state data based on the acceleration data and the three-axis component data, compare the generated motion state data with the threshold values ​​of the multiple motion speed intervals, and generate a bit signal based on the comparison result. The above-mentioned acceleration data may be collected by an acceleration sensor, and the three-axis component data of the electronic device may be collected by a gravity sensor. At this time, the electronic device 100 may further include an acceleration sensor 106 and a gravity sensor 107. Figure 3As shown. The scene information can use a two-bit signal, the first bit indicates the ambient light brightness, and the second bit indicates the motion state of the electronic device. For example, "00" indicates low ambient light brightness and low motion state; "01" indicates low ambient light brightness and high motion state; "10" indicates high ambient light brightness and low motion state; "11" indicates high ambient light brightness and high motion state. It should be noted that the number of bits used to indicate the scene information shown in the embodiment of the present application is only schematic, and the number of bits may include more bits or fewer bits depending on the specific scene information included and the intervals divided by each scene. For example, when the brightness interval includes low brightness, medium brightness and high brightness, the bits used to indicate the brightness may include three bits. It should also be noted that in the multiple numerical interval segments pre-set in the controller 103 for dividing different scenes, an overlapping interval is set between each two adjacent numerical interval segments. Assuming that the scene information currently generated by the controller 103 falls into the overlapping interval, the controller 103 can refer to the scene information generated last time. If the difference between the currently generated scene information and the last generated scene information is less than or equal to a preset threshold, the AI ​​processor 101 can maintain the currently running image processing model unchanged, and the ISP 102 can maintain the currently running image processing algorithm unchanged; if the difference between the currently generated scene information and the last generated scene information is greater than the preset threshold, the currently generated scene information can be resent to the AI ​​processor 101 and ISP 102, so that the AI ​​processor 101 changes the image processing model and ISP 102 changes the parameters of the image processing algorithm. This embodiment of the present application prevents frequent switching of the image processing model running in the AI ​​processor 101 by setting an overlapping interval between each two adjacent numerical intervals, thereby improving the stability of the image processing model running in the AI ​​processor 101.

[0054] In an embodiment of the present application, the controller 103 can obtain scene information in real time or periodically. When it is detected that the current scene information is different from the previously obtained scene information (for example, the scene changes from a high-brightness, low-motion state to a high-brightness, high-motion state), the scene information indicating the current scene is promptly sent to the ISP 102 and the AI ​​processor 101 respectively. The AI ​​processor 101 promptly replaces the image processing model being run based on the currently received scene information, so that the replaced image processing model is run when the image processing is performed in the next image processing cycle. The ISP 102 can also promptly replace the parameters of the image processing algorithm being run based on the currently received scene information, so that the image processing algorithm with updated parameters is run when the image processing is performed in the next image processing cycle. Thus, the electronic device described in the embodiment of the present application can dynamically adjust the image processing model used and the parameters of the image processing algorithm run by the ISP 102 based on the scene information, so that when the user uses the electronic device described in the embodiment of the present application to change the scene (for example, from a bright light area to a dim light area or the electronic device changes from a stationary state to a moving state), the captured image can be processed in a targeted manner, thereby improving the image processing effect and improving the user experience.

[0055] Please continue to refer to Figure 4 , which shows a schematic diagram of a structure in which the ISP 102 and the AI ​​processor 101 provided in an embodiment of the present application are connected via electronic circuits. Figure 4 In the electronic device 100 shown, ISP102 may include a plurality of cascaded image processing modules, and the plurality of cascaded image processing modules include image processing module 01, image processing module 02, image processing module 03…image processing module N and image processing module N+1. Each image processing module may include a plurality of logic devices or circuits to perform specific image processing functions. For example, image processing module 01 is used to perform image processing for black level correction, image processing module 02 is used to perform image processing for shadow correction, image processing module 03 is used to perform image processing for shadow correction…, and image processing module N+1 is used to perform RGB to YUV processing. Based on the image processing requirements, any one of the plurality of cascaded image processing modules may be provided with an output port and an input port, the output port being used to provide image signal A to the AI ​​processor 101, and the input port being used to obtain image signal B from the AI ​​processor 101. Figure 4 The figure schematically shows that the image processing module 02 is provided with an output port Vpo1 and the image processing module 03 is provided with an input port Vpi1. Figure 4In the structure shown in FIG. 1 , in a possible implementation, the electronic device 100 may also be provided with an on-chip RAM, which is integrated with the ISP 102 and the AI ​​processor 101 in a chip in the electronic device 100. The image signal provided by the ISP 102 to the AI ​​processor 101 and the image signal provided by the AI ​​processor 101 to the ISP 102 may be stored in the on-chip RAM. In addition, the on-chip RAM is also used to store intermediate data generated during the operation of the AI ​​processor 101 and the weight data of each network node in the neural network operated by the AI ​​processor 101. In a specific implementation, the on-chip RAM may be provided as follows: Figure 1 or Figure 3 In the memory 104 shown.

[0056] In a specific scenario, ISP 102 acquires image data from image sensor 105. Image processing modules 01 and 02 sequentially perform shading correction and white balance correction on the image data, generating image signal A, which is then stored in on-chip RAM. After storing image signal A in the on-chip RAM, image processing module 02 sends an interrupt signal Z1 to AI processor 101. In response to interrupt signal Z1, AI processor 101 retrieves image signal A from the on-chip RAM. AI processor 101 performs demosaicing on image signal A to generate image signal B, which it then stores in the on-chip RAM. After storing image signal B in the on-chip RAM, AI processor 101 sends the aforementioned interrupt signal Z2 to image processing module 03. In response to interrupt signal Z2, image processing module 03 reads image signal B from the on-chip RAM. Image processing module 03 then sequentially performs chromatic aberration correction, gamma correction, and RGB-to-YUV conversion on image signal B in ISP 102, generating the final image processing result. It should be noted that more image processing modules may be included before the image processing module 01, so that the ISP 102 performs more image processing processes on the image data.

[0057] exist Figure 4 In the embodiment shown, the image processing process performed by the AI ​​processor 101 is set between the multiple image processing processes performed by the ISP 102 to replace or supplement certain intermediate image processing processes performed by the ISP 102. In some other possible implementations, the AI ​​processor 101 can directly obtain image data from the image sensor 105 and perform the front-end image processing process. In this implementation, the AI ​​processor 101 can replace and supplement certain image processing modules in the front end of the ISP 102 to perform the corresponding image processing process. At this time, the AI ​​processor 101 can directly communicate with the image processing module behind the ISP 102. The hardware structure of this implementation refers to Figure 5 ,like Figure 5 The connection and interaction between the AI ​​processor 101 and ISP 102 shown are similar to Figure 4 The connection and interaction between the AI ​​processor 101 and ISP 102 are similar, see Figure 4 The relevant descriptions in the illustrated embodiments will not be repeated here.

[0058] exist Figure 4 and Figure 5 In the embodiment shown, the AI ​​processor 101 interacts with the ISP 102 once, and the AI ​​processor 101 performs one image processing process or performs multiple continuous image processing processes to process image data or image signals. In some other possible implementations, the AI ​​processor 101 can perform multiple discontinuous image processing processes, that is, the AI ​​processor 101 and the ISP 102 can perform image processing alternately, so that both parties complete the image processing process together to obtain the processing result, thereby replacing the image processing process of the traditional ISP. In this case, the ISP 102 may also include more output ports and input ports. Figure 6 The structure of the electronic device shown in FIG. Figure 6 In ISP 102, image processing modules 02 and 03 are provided with output ports Vpo1 and Vpo2, respectively, and image processing modules 03 and N are provided with input ports Vpi1 and Vpi2, respectively. The output ports of each module are used to provide image signals to the AI ​​processor, and the input ports of each module are used to obtain image signals from the AI ​​processor. Image data collected by image sensor 105 is processed by image processing modules 01 and 02 to generate image signal A, which is provided to AI processor 101. AI processor 101 processes image signal A to generate image signal B, which is provided to image processing module 03. Image signal B is processed by image processing module 03 to generate image signal C, which is provided to AI processor 101. The AI ​​processor processes image signal C to generate image signal D, which is provided to image processing module N. Image signal D is processed by image processing modules N and image processing module N+1 to generate the final image processing result.

[0059] based on Figure 6In the structural diagram shown, among the multiple image processing models running in the AI ​​processor 101, at least one first image processing model performs a first image processing operation, and at least one second image processing model performs a second image processing operation. Here, when there are multiple first image processing models, the multiple first image processing models are used to process image data collected in different scenarios, and the first image processing operations performed by the multiple first image processing models are the same image processing operations. Similarly, when there are multiple second image processing models, the multiple second image processing models are used to process image data collected in different scenarios, and the second image processing operations performed by the multiple second image processing models are the same image processing operations. For example, assume that the AI ​​processor 101 can run two first image processing models and two second image processing models. Among them, one of the first image processing models is used to perform noise reduction processing on image data collected in high ambient light brightness scenes, and another first image processing model is used to perform noise reduction processing on image data collected in low ambient light brightness scenes. One of the second image processing models is used to perform demosaicing processing on image data collected in high ambient light brightness scenes, and another first image processing model is used to perform demosaicing processing on image data collected in low ambient light brightness scenes.

[0060] In a possible implementation of the embodiment of the present application, the electronic device further includes an off-chip memory 108, such as Figure 3 As shown. Since the off-chip memory 108 has a larger storage space, it can replace the on-chip RAM to store larger units of image data. The off-chip memory 108 can be used to store multiple frames of images, which can be the previous frame of image, the previous two frames of image, or the previous multiple frames of image before the current image. In addition, the off-chip memory 108 can also be used to store feature maps of each of the above-mentioned multiple frames of image. The feature map is generated after the image processing model running in the AI ​​processor 101 performs operations such as convolution and pooling on the image signal. When the image processing model generated by the recurrent neural network is run in the AI ​​processor 101 to process the current image signal, the previous frame of image signal or the feature map of the previous frame of image signal can also be obtained from the off-chip memory 108, and then the previous frame of image signal or the feature map of the previous frame of image signal is used as reference data to process the current image signal. In addition, the AI ​​processor 101 can also store the processed image signal in the off-chip memory 108. The off-chip memory 108 may include a random access memory (RAM), which may include a volatile memory (such as SRAM, DRAM, DDR (Double Data Rate SDRAM) or SDRAM, etc.) and a non-volatile memory.

[0061] In this embodiment, the electronic device 100 may further include a communication unit (not shown), which may include but is not limited to a short-range communication unit or a cellular communication unit. The short-range communication unit exchanges information with a terminal located outside the mobile terminal for accessing the internet by running a short-range wireless communication protocol. The short-range wireless communication protocol may include but is not limited to various protocols supported by radio frequency identification technology, Bluetooth communication technology protocols, or infrared communication protocols. The cellular communication unit accesses the internet through a wireless access network by running a cellular wireless communication protocol, thereby enabling information exchange between the mobile communication unit and servers supporting various applications on the internet. The communication unit may be integrated into the same SOC as the AI ​​processor 101 and ISP 102 described in the above embodiments, or may be separately provided. Furthermore, the electronic device 100 may optionally include a bus, input / output (I / O) ports, or a memory controller. The memory controller is used to control the memory 103 and the off-chip memory 108. The bus, input / output (I / O) ports, and memory controller may all be integrated into the same SOC as the ISP 102 and AI processor 101 described above. It should be understood that in actual applications, the electronic device 100 may include Figure 1 or Figure 3 The embodiments of the present application are not limited to the more or fewer components shown.

[0062] In the embodiment of the present application, each of the multiple image processing models running in the AI ​​processor is based on sample image data collected in the corresponding scene, and is deployed in the electronic device after training multiple neural networks offline using machine learning methods. Figure 7 , which shows a schematic process 700 of a training method for an image processing model running in an AI processor, combined with Figure 7 , describes the training of image processing models.

[0063] Step 701: Generate multiple training sample sets. The step of generating multiple training sample sets may include the following sub-steps: Step 7011: Generate a first model. This first model is an end-to-end model generated offline and can process image data collected from any scene. The first model can be trained using traditional model training methods based on training samples. Step 7012: Based on the divided scenes, sample image data is collected from different scenes. Step 7013: The collected sample image data is input into the first model to generate reference image signals for the different scenes. Step 7014: Based on the image processing flow executed by the AI ​​processor, the sample image data is pre-processed to generate pre-processed image signals to be input into the image processing model. After steps 7011-7014, multiple training sample sets are obtained. A training sample set corresponds to a scene one-to-one. Each training sample set includes a pre-processed image signal generated by processing sample image data collected from that scene, as well as a reference image signal generated by processing sample image data collected from that scene using the first model.

[0064] Step 702: Train multiple neural networks using multiple training sample sets. Based on the training results, generate multiple image processing models. The neural networks may include, but are not limited to, recurrent neural networks, convolutional neural networks, or deep neural networks. In a specific implementation, for scenarios where the electronic device is stationary or in slow motion, any one of these can be trained to generate an image processing model. For scenarios where the electronic device is in high-speed motion, any one of these can be trained to generate an image processing model. Preferably, for scenarios where the electronic device is stationary or in slow motion, to further improve image signal processing, a recurrent neural network can be trained to generate an image processing model. The following describes in detail the example of training a convolutional neural network using one of the training sample sets. A preprocessed image signal is input into the neural network to generate an output image signal. The output image signal is compared with a reference image signal, and a loss function is constructed based on the difference between the output image signal and the reference image signal. The loss function includes the weight parameters of the neural network. The weight parameters of the neural network are iteratively adjusted using a backpropagation algorithm and a gradient descent algorithm. When preset conditions are met, the parameters of the neural network are saved. The neural network that meets the preset conditions is then designated as the image processing model. The above-mentioned preset conditions may include at least one of the following: a loss value of a preset loss function is less than or equal to a preset threshold and the number of iterative adjustments of the neural network is greater than or equal to a preset threshold.

[0065] Based on the above embodiments, the present application also provides an image processing method. The image processing method can be applied to Figure 1、 Figure 3-Figure 6 In any of the electronic devices 100 shown. Below, the scene information includes the ambient light brightness information and the motion state information of the electronic device as an example, combined with Figure 3 and Figure 4 The electronic device 100 shown in FIG. 1 is used to describe the image processing method provided by the embodiment of the present application. Figure 8 , Figure 8 This is a process 800 of an image processing method provided in an embodiment of the present application. The image processing method includes: step 801, the image sensor 105 collects image data and provides the collected image data to the ISP 102.

[0066] In step 802, the controller 103 obtains the sensitivity information of the image data from the ISP 102, the acceleration data of the electronic device from the acceleration sensor, and the three-axis component data of the electronic device from the gravity sensor. In step 803, the controller 103 generates the motion state data of the electronic device based on the acceleration data and the three-axis component data. In step 804, the controller 103 compares the sensitivity information with a plurality of pre-set sensitivity intervals, and compares the motion state data with a plurality of pre-set motion speed intervals. Based on the comparison results, the controller 103 generates scene information including ambient light brightness information and motion state information, and provides them to the AI ​​processor 101 and ISP 102 respectively. Among them, the ambient light brightness information is used to indicate low ambient light brightness, and the motion state information is used to indicate low-speed motion of the electronic device.

[0067] In step 805, ISP 102 updates the parameters of the image processing algorithm based on the scene information. In step 806, the updated image processing algorithm is used to process the image data to generate image signal A. In step 807, AI processor 101 selects one of multiple image processing models based on the scene information to process image signal A and generate image signal B. In step 808, ISP 102 processes image signal B to generate the final image processing result.

[0068] It should be understood that Figure 8 The steps or operations of the image processing method shown are only examples, and the embodiments of the present application may also perform other operations or Figure 8 The embodiment of the present application may also include the following: Figure 8 For example, when the ISP 102 does not have a parameter adjustment unit and the ISP 102 uses the same parameters to process images captured from different scenes, the controller 103 in step 804 may not need to provide the scene information to the ISP 102, and step 805 may be omitted. For another example, when the image processing method described in the embodiment of the present application is applied to Figure 6In the electronic device 100 shown, step 808 is replaced by ISP102 processing the image signal B to generate image signal C. After step 808, the steps of AI processor 101 processing image signal C to generate image signal D and ISP102 processing image signal D to generate the final image processing result are also included.

[0069] It is understandable that, in order to implement the above functions, the electronic device includes hardware and / or software modules that perform the corresponding functions. In combination with the steps of the various examples described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in combination with the embodiments, but such implementation should not be considered to be beyond the scope of this application.

[0070] In this embodiment, one or more processors can be divided into functional modules according to the above method examples. For example, different processors can be divided according to different functions, or two or more processors with different functions can be integrated into a single processor module. The above integrated modules can be implemented in the form of hardware. It should be noted that the division of modules in this embodiment is illustrative and is only a logical functional division. In actual implementation, other division methods may be used.

[0071] In the case of dividing each functional module into corresponding functional modules, Figure 9 FIG. 9 is a schematic diagram showing a possible embodiment of the apparatus 900 involved in the above embodiment, which can be further expanded. Figure 9 As shown, the apparatus 900 may include: an AI processing module 901 and an image signal processing module 902. The AI ​​processing module 901 is configured to select a first image processing model from a plurality of image processing models, and perform a first image signal processing on a first image signal using the first image processing model to obtain a second image signal, wherein the first image signal is obtained based on first image data output by an image sensor, and the scene information reflects a feature classification of the first image signal; and the image signal processing module 902 is configured to perform a second image signal processing on the second image signal to obtain a first image processing result.

[0072] In a possible implementation manner, the scene information includes at least one of first ambient light brightness information and first motion state information of the electronic device.

[0073] In one possible implementation, the image signal processing module 902 is used to: select a first parameter from multiple groups of parameters for running an image processing algorithm based on the scene information; obtain an updated image processing algorithm based on the first parameter; and perform the second image signal processing on the second image signal using the updated image processing algorithm.

[0074] In one possible implementation, the AI ​​processing module 901 is further used to: when in response to the first motion state information indicating that the electronic device is moving at a speed lower than a preset threshold, process the first image signal based on a previous frame image signal and an image processing result of the previous frame image signal.

[0075] In a possible implementation, the first image signal processing includes at least one of the following processing procedures: noise elimination, black level correction, shading correction, white balance correction, demosaicing, chromatic aberration correction, or gamma correction.

[0076] In one possible implementation, the second image signal processing includes at least one of the following processing processes: noise elimination, black level correction, shadow correction, white balance correction, demosaicing, color difference correction, gamma correction, color difference correction or RGB to YUV domain conversion.

[0077] In one possible implementation, the multiple image processing models are trained based on multiple training sample sets corresponding to multiple scenes, wherein each of the multiple training sample sets includes a preprocessed image signal generated by processing sample image data collected in the corresponding scene, and a reference image signal generated by processing the sample image data.

[0078] The image processing device 900 provided in this embodiment is used to execute the image processing method executed by the electronic device 100, and can achieve the same effect as the above-mentioned implementation method or device. Figure 9 The corresponding modules can be implemented in software, hardware or a combination of both. For example, each module can be implemented in software, corresponding to Figure 1 The corresponding processor corresponding to the module is used to drive the corresponding processor to work. Alternatively, each module may include a corresponding processor and corresponding driver software, that is, it is implemented in combination with software or hardware. Therefore, the image processing device 900 can be considered to logically include Figure 1 、 Figure 3-Figure 6 In the device shown, each module at least includes a driver software program of corresponding functions, which will not be elaborated in this embodiment.

[0079] Exemplarily, the image processing apparatus 900 may include at least one processor and a memory, specifically referring to Figure 1. Among them, at least one processor can call all or part of the computer program stored in the memory to control and manage the actions of the electronic device 100, for example, it can be used to support the electronic device 100 to execute the steps performed by the above modules. The memory can be used to support the electronic device 100 to execute stored program codes and data, etc. At least one processor can implement or execute various exemplary multiple logic modules described in conjunction with the contents disclosed in this application, which can be a combination of one or more microprocessors that implement computing functions, for example, including but not limited to Figure 1 The AI ​​processor 101 and image signal processor 102 are shown. In addition, at least one processor may also include other programmable logic devices, transistor logic devices, or discrete hardware components. The memory described in this embodiment may include but is not limited to Figure 3 Off-chip memory 108 or memory 104 is shown.

[0080] This embodiment further provides a computer-readable storage medium, in which computer instructions are stored. When the computer instructions are executed on a computer, the computer executes the above-mentioned related method steps to implement the image processing method in the above-mentioned embodiment.

[0081] This embodiment further provides a computer program product. When the computer program product is run on a computer, it enables the computer to execute the above-mentioned related steps to implement the image processing method in the above-mentioned embodiment.

[0082] Among them, the computer-readable storage medium or computer program product provided in this embodiment is used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above, and will not be repeated here.

[0083] Through the description of the above implementation methods, technical personnel in the relevant field can understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0084] In addition, each functional unit in each embodiment of the present application may be integrated into one product, or each unit may exist physically separately, or two or more units may be integrated into one product. Figure 9, if the above modules are implemented in the form of software functional units and sold or used as independent products, they can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to execute all or part of the steps of the various embodiments of the present application. The aforementioned readable storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk and other media that can store program code.

[0085] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. An electronic device, characterized in that: include: an artificial intelligence (AI) processor, configured to select a first image processing model from a plurality of image processing models based on scene information, and perform first image signal processing on a first image signal using the first image processing model to obtain a second image signal, wherein the first image signal is obtained based on first image data output by an image sensor, and the scene information reflects a feature classification of the first image signal; an image signal processor ISP, configured to perform second image signal processing on the second image signal to obtain a first image processing result; Wherein, the scene information includes first motion state information of the electronic device; When the first motion state information is used to indicate that the electronic device moves at a speed lower than a preset threshold, the first image signal is processed using the first image processing model based on a previous frame image signal and an image processing result of the previous frame image signal.

2. The electronic device according to claim 1, wherein: The scene information also includes first ambient light brightness information.

3. The electronic device according to claim 1 or 2, characterized in that: Said ISP is used to: selecting a first parameter from a plurality of sets of parameters of an image processing algorithm based on the scene information; Based on the first parameter, obtaining an updated image processing algorithm; The second image signal processing is performed on the second image signal using the updated image processing algorithm.

4. The electronic device according to claim 1 or 2, characterized in that: The electronic device further comprises: The controller is configured to generate the scene information based on data collected by at least one sensor, wherein the at least one sensor includes at least one of the following: an acceleration sensor, a gravity sensor, and the image sensor.

5. The electronic device according to claim 1 or 2, characterized in that: The first image signal processing includes at least one of the following processing procedures: noise elimination, black level correction, shading correction, white balance correction, demosaicing, chromatic aberration correction or gamma correction.

6. The electronic device according to claim 1 or 2, characterized in that: The second image signal processing includes at least one of the following processing processes: noise elimination, black level correction, shadow correction, white balance correction, demosaicing, color difference correction, gamma correction, color difference correction or RGB to YUV domain conversion.

7. The electronic device according to claim 1 or 2, characterized in that: The multiple image processing models are trained based on multiple training sample sets corresponding to multiple scenes, wherein each of the multiple training sample sets includes a preprocessed image signal generated by processing sample image data collected in the corresponding scene, and a reference image signal generated by processing the sample image data.

8. An image processing method, applied to an electronic device, characterized in that: The method comprises: Based on the scene information, control the artificial intelligence (AI) processor to select a first image processing model from a plurality of image processing models, and perform first image signal processing on a first image signal using the first image processing model to obtain a second image signal, where the first image signal is obtained based on first image data output by an image sensor, and the scene information reflects a feature classification of the first image signal; controlling an image signal processor ISP to perform second image signal processing on the second image signal to obtain a first image processing result; Wherein, the scene information includes first motion state information of the electronic device; When the first motion state information is used to indicate that the electronic device moves at a speed lower than a preset threshold, the first image signal is processed using the first image processing model based on a previous frame image signal and an image processing result of the previous frame image signal.

9. The image processing method according to claim 8, characterized in that: The controlling the image signal processor ISP to perform second image signal processing on the second image signal to obtain an image processing result includes: Based on the scene information, controlling the ISP to select a first parameter from a plurality of sets of parameters of an image processing algorithm; controlling the ISP to obtain an updated image processing algorithm based on the first parameter; The ISP is controlled to perform the second image signal processing on the second image signal using the updated image processing algorithm.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which is used to implement the method according to claim 8 or 9 when executed by at least one processor.

11. A computer program product, characterized in that When the computer program product is executed by at least one processor, it is used to implement the method according to claim 8 or 9.

Images