Image processing method, training method, image processing device, equipment and medium

By using images acquired by cameras with different focal lengths, feature extraction and high-frequency feature superposition are solved, and the problem of insufficient image details in the prior art is achieved, and the efficient super-segment reconstruction of the image and the imaging effect are improved.

CN119006278APending Publication Date: 2024-11-22GUANGDONG OPPO MOBILE TELECOMMUNICATIONS CORP LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310554877.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-16
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

The existing image super-scoring technology has shortcomings in restoring details, especially the details of low-resolution images are still limited and cannot effectively improve the sharpness of the image.

Method used

By acquiring images taken by cameras with different focal lengths, a second image with a smaller field of view but rich details is used to enhance the first input image with a larger field of view but less details, and feature extraction and high-frequency feature superposition are performed to generate More detailed target image.

Benefits of technology

The image imaging effect is improved, and the generated target image details are relatively complete, which improves the clarity and detail richness of the image.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119006278A_ABST
    Figure CN119006278A_ABST
Patent Text Reader

Abstract

The invention discloses an image processing method, a training method, an image processing device, electronic equipment and a nonvolatile computer readable storage medium. The image processing method comprises the steps that a first input image and a second input image are obtained, the first input image is generated according to the first image, the second input image is generated according to the second image, and the focal length of a first camera collecting the first image is smaller than that of a second camera collecting the second image; the view field ranges corresponding to the first input image and the second image are at least partially overlapped; performing feature extraction on the first input image to obtain a first image feature, and performing feature extraction on the second input image to obtain a second image feature; and generating a third image feature according to the first image feature and the second image feature, and superposing a high-frequency feature of the third image feature and the first input image to generate a target image. Therefore, the first input image can be enhanced by using the second image, so that the details of the obtained target image are relatively perfect.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of imaging technology, and in particular to an image processing method, a training method, an image processing device, an electronic device, and a non-volatile computer-readable storage medium. Background Art

[0002] Image super-resolution reconstruction, referred to as image super-resolution, is an image processing technology that reconstructs low-resolution images into high-resolution images. It can effectively improve the clarity of images, so it is often used in a wide range of fields such as mobile phone camera photography and video surveillance. Existing technical solutions usually use multi-frame information or input information such as gradients for restoration, but the details of the super-resolution image are still limited. Summary of the invention

[0003] Embodiments of the present application provide an image processing method, a training method, an image processing device, an electronic device, and a non-volatile computer-readable storage medium.

[0004] The image processing method of the embodiment of the present application includes acquiring a first input image and a second input image, wherein the first input image is generated based on the first image, and the second input image is generated based on the second image, the focal length of the first camera that captures the first image is smaller than the focal length of the second camera that captures the second image, and the field of view corresponding to the first input image and the second image at least partially overlap; performing feature extraction on the first input image to obtain a first image feature, and performing feature extraction on the second input image to obtain a second image feature; and generating a third image feature based on the first image feature and the second image feature, and superimposing the high-frequency features of the third image feature and the first input image to generate a target image.

[0005] The training method of the embodiment of the present application includes downsampling a first input image and a second input image respectively to obtain a first sample image and a second sample image respectively, the first input image is generated according to the first image, the second input image is generated according to the second image, the focal length of the first camera for acquiring the first image is smaller than the focal length of the second camera for acquiring the second image, and the field of view corresponding to the first input image and the second image at least partially overlaps; extracting features of the first sample image to obtain first training image features, and extracting features of the second sample image to obtain second training image features; generating third training image features according to the first training image features and the second training image features, and superimposing high-frequency features of the third training image features and the first sample image to generate a first training image; and calculating a first loss value based on a first loss function, the first training image, the first input image and the second input image, and adjusting model parameters of an image processing model according to the first loss value to train the image processing model to convergence.

[0006] The image processing device of the embodiment of the present application includes an acquisition module, a feature extraction module and a feature processing module. The acquisition module is used to acquire a first input image and a second input image, the first input image is generated according to the first image, the second input image is generated according to the second image, the focal length of the first camera that captures the first image is smaller than the focal length of the second camera that captures the second image, and the field of view corresponding to the first input image and the second image at least partially overlap; the feature extraction module is used to perform feature extraction on the first input image to obtain first image features, and perform feature extraction on the second input image to obtain second image features; the feature processing module is used to generate third image features according to the first image features and the second image features, and superimpose the high-frequency features of the third image features and the first input image to generate a target image.

[0007] The electronic device of the embodiment of the present application includes a processor, and the processor is used to execute the image processing method and the training method. The image processing method includes acquiring a first input image and a second input image, the first input image is generated according to the first image, the second input image is generated according to the second image, the focal length of the first camera that collects the first image is smaller than the focal length of the second camera that collects the second image, and the field of view corresponding to the first input image and the second image at least partially overlap; feature extraction is performed on the first input image to obtain a first image feature, and feature extraction is performed on the second input image to obtain a second image feature; and based on the first image feature and the second image feature, a third image feature is generated, and the high-frequency feature of the third image feature and the first input image are superimposed to generate a target image. The training method includes downsampling a first input image and a second input image respectively to obtain a first sample image and a second sample image respectively, wherein the first input image is generated according to the first image, and the second input image is generated according to the second image, the focal length of a first camera for acquiring the first image is smaller than the focal length of a second camera for acquiring the second image, and the fields of view corresponding to the first input image and the second image at least partially overlap; extracting features from the first sample image to obtain first training image features, and extracting features from the second sample image to obtain second training image features; generating third training image features according to the first training image features and the second training image features, and superimposing high-frequency features of the third training image features and the first sample image to generate a first training image; and calculating a first loss value based on a first loss function, the first training image, the first input image, and the second input image, and adjusting model parameters of an image processing model according to the first loss value to train the image processing model to convergence.

[0008] The non-volatile computer-readable storage medium of the embodiment of the present application includes a computer program, and when the computer program is executed by a processor, the processor executes an image processing method and a training method. The image processing method includes acquiring a first input image and a second input image, the first input image is generated according to the first image, the second input image is generated according to the second image, the focal length of the first camera that captures the first image is smaller than the focal length of the second camera that captures the second image, and the field of view corresponding to the first input image and the second image at least partially overlap; feature extraction is performed on the first input image to obtain a first image feature, and feature extraction is performed on the second input image to obtain a second image feature; and a third image feature is generated according to the first image feature and the second image feature, and the high-frequency feature of the third image feature and the first input image are superimposed to generate a target image. The training method includes downsampling a first input image and a second input image respectively to obtain a first sample image and a second sample image respectively, wherein the first input image is generated according to the first image, and the second input image is generated according to the second image, the focal length of a first camera for acquiring the first image is smaller than the focal length of a second camera for acquiring the second image, and the fields of view corresponding to the first input image and the second image at least partially overlap; extracting features from the first sample image to obtain first training image features, and extracting features from the second sample image to obtain second training image features; generating third training image features according to the first training image features and the second training image features, and superimposing high-frequency features of the third training image features and the first sample image to generate a first training image; and calculating a first loss value based on a first loss function, the first training image, the first input image, and the second input image, and adjusting model parameters of an image processing model according to the first loss value to train the image processing model to convergence.

[0009] The image processing method, training method, image processing device, electronic device and non-volatile computer-readable storage medium of the embodiment of the present application respectively control the first camera and the second camera to shoot, so as to obtain the first image and the second image respectively, wherein the focal length of the first camera is smaller than the focal length of the second camera, so that the field of view of the first image is larger but the details are less, and the field of view of the second image is smaller but the details are richer. Then, a first input image with a larger zoom ratio is generated according to the first image, and a second input image is generated according to the second image, and the field of view corresponding to the first input image and the second image is at least partially the same, so that the first input image with a larger field of view but richer details can be enhanced by using the second image with a smaller field of view but richer details. After obtaining the first input image and the second input image, feature extraction is performed on the first input image and the second input image respectively to obtain the first image feature and the second image feature respectively. Then, according to the first image feature and the second image feature, a third image feature containing two image features is generated, so that the details of the third image feature are richer, and finally the high-frequency features of the third image feature are superimposed on the first input image, so as to generate a target image with richer details. In this way, based on the images acquired by two cameras with different focal lengths, the first input image with a larger field of view but richer details can be enhanced by using the second image with a smaller field of view but richer details, thereby achieving super-resolution reconstruction of the first input image, and the details of the obtained target image are more complete, thereby improving the imaging effect of the target image.

[0010] Additional aspects and advantages of the embodiments of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0012] Figure 1 is a flowchart of an image processing method of certain embodiments of the present application;

[0013] Figure 2 is a planar structural diagram of an electronic device according to some embodiments of the present application;

[0014] Figure 3 is a schematic diagram of a usage scenario of an image processing method in some embodiments of the present application;

[0015] Figure 4 is a flowchart of an image processing method of certain embodiments of the present application;

[0016] Figure 5 is a schematic diagram of a usage scenario of an image processing method in some embodiments of the present application;

[0017] Figure 6 is a schematic diagram of a usage scenario of an image processing method in some embodiments of the present application;

[0018] Figure 7 is a schematic diagram of a usage scenario of an image processing method in some embodiments of the present application;

[0019] Figure 8 is a flowchart of an image processing method of certain embodiments of the present application;

[0020] Fig. 9 is a flowchart of an image processing method of certain embodiments of the present application;

[0021] Fig.10 is a flowchart of an image processing method of certain embodiments of the present application;

[0022] Fig.11 is a flowchart of an image processing method of certain embodiments of the present application;

[0023] Fig.12 is a flowchart of an image processing method of certain embodiments of the present application;

[0024] Fig.13 is a schematic diagram of a usage scenario of an image processing method in some embodiments of the present application;

[0025] Fig.14 is a flowchart of an image processing method of certain embodiments of the present application;

[0026] Fig.15 is a schematic diagram of a usage scenario of an image processing method in some embodiments of the present application;

[0027] Fig.16 is a schematic diagram of a module of an image processing device in some embodiments of the present application; and

[0028] Fig.17 It is a schematic diagram of the interaction between the non-volatile computer-readable storage medium and the processor in certain embodiments of the present application. DETAILED DESCRIPTION

[0029] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions from beginning to end. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the embodiments of the present application, and cannot be understood as limiting the embodiments of the present application.

[0030] In the present application, unless otherwise clearly specified and limited, a first feature being “above” or “below” a second feature may mean that the first and second features are in direct contact, or the first and second features are in indirect contact through an intermediate medium. Moreover, a first feature being “above”, “above”, and “above” a second feature may mean that the first feature is directly above or obliquely above the second feature, or simply means that the first feature is higher in level than the second feature. A first feature being “below”, “below”, and “below” a second feature may mean that the first feature is directly below or obliquely below the second feature, or simply means that the first feature is lower in level than the second feature.

[0031] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one feature. In the description of this application, the meaning of "plurality" is at least two, such as two, three, unless otherwise clearly and specifically defined.

[0032] See also Figures 1 to 3 , the image processing method of the embodiment of the present application includes:

[0033] Step 011: Acquire a first input image S11 and a second input image S21. The first input image S11 is generated according to the first image S1, and the second input image S21 is generated according to the second image S2. The focal length of the first camera 20 for acquiring the first image S1 is smaller than the focal length of the second camera 30 for acquiring the second image S2. The fields of view corresponding to the first input image S11 and the second image S2 at least partially overlap.

[0034] Specifically, cameras can be divided into different types of cameras according to the type of lens used. For example, according to the length of the focal length of the lens, that is, the field of view during shooting, the lens may include a macro lens, a wide-angle lens, and a telephoto lens, etc., and the corresponding cameras may include a macro camera, a wide-angle camera, and a telephoto camera. The imaging effects of images taken by different types of cameras are different. For example, the focal length of a wide-angle lens is smaller, the photographic angle of view is wider, and it is suitable for shooting scenes that are close and have a large range. Then, the field of view of the image taken by the wide-angle camera is larger, but the details are less; the focal length of a telephoto lens is larger, and it is suitable for shooting distant scenes, and the depth of field is smaller. Then, the field of view of the image taken by the telephoto camera is smaller, but the details are more. Then, when the fields of view corresponding to the images taken by cameras with lenses of different focal lengths have overlapping parts, the details of the image with a larger field of view but fewer details can be enhanced by using the image with a smaller field of view but more details, so that the imaging effect of the enhanced image is better.

[0035] In order to improve the image imaging effect, the electronic device 100 (such as a mobile phone and a tablet computer) currently provided with a camera can also be provided with two or more cameras at the same time, and the types of different cameras can be different, so that the user can select the corresponding camera type for shooting according to the shooting needs. For example, most mobile phone manufacturers use an asymmetric camera system composed of multiple fixed focal length lenses. The most common configuration is a wide-angle camera (main camera) and a telephoto lens with different fields of view. Therefore, the electronic device 100 provided with two or more cameras can use cameras with different focal lengths to shoot the same scene. After shooting the same scene with cameras with different focal lengths, the imaging effect of the acquired image cannot be guaranteed. For example, after the user uses the electronic device 100 to perform zoom shooting, the image quality of the image acquired at this time is poor. Then, at this time, the images acquired by cameras with different focal lengths can be processed based on the image super-resolution technology so that the imaging effect of the finally generated image is better.

[0036] First, it is necessary to control the first camera 20 and the second camera 30 to acquire images respectively, so as to obtain the first image S1 and the second image S2 respectively. The specific combination type of the first camera 20 and the second camera 30 is not limited, and it is only necessary to ensure that the focal length of the first camera 20 is smaller than the focal length of the second camera 30. For example, the first camera 20 is a wide-angle camera and the second camera 30 is a telephoto camera; for another example, the first camera 20 is an ultra-wide-angle camera and the second camera 30 is a wide-angle camera; for another example, the first camera 20 is a wide-angle camera and the second camera 30 is a black and white camera or an infrared camera with a focal length larger than that of the first camera 20. Therefore, Figure 2 As shown, the first image S1 has a larger field of view but fewer details, and the second image S2 has a smaller field of view but richer details, so that the details of the first input image S11 can be subsequently enriched according to the details of the second input image S21.

[0037] After acquiring the first image S1, an imaging target area can be selected from the first image S1 according to imaging needs, and a first input image S11 with a larger zoom factor is generated according to the imaging target area, and a second input image S21 is generated according to the second image S2. In order to facilitate enhancing the details of the first input image S11 according to the second input image S21, the field of view ranges corresponding to the first input image S11 and the second image S2 at least partially overlap, so that the overlapping parts in the second image S2 can be used to enhance the details of the overlapping parts in the first input image S11.

[0038] Furthermore, the resolutions of the first input image S11 and the second input image S21 may be different to save the step of keeping the resolutions of the two input images the same, thereby improving the efficiency of image processing; the resolutions of the first input image S11 and the second input image S21 may also be the same to facilitate the subsequent improvement of the details of the first input image S11 based on the second input image S21.

[0039] Step 012: performing feature extraction on the first input image S11 to obtain first image features, and performing feature extraction on the second input image S21 to obtain second image features;

[0040] Specifically, image features are the basic information that distinguishes one image from another, and are used to identify and mark the image content. Image features are usually divided into two categories: natural features and artificial features. Natural features are some information contained in the image content itself, such as image size, color distribution, contour, edge gradient, texture and other information. Artificial features are the recognized information mined out after later analysis and processing of the image. For example: color histogram, moment features, entropy and other information. Therefore, when processing the image, the image can also be feature extracted, such as using a 3*3 convolutional layer to map the input image space to a high-order feature space, and then using N residual modules for deep feature extraction. The nonlinear activation function in this process can use the linear rectified function (Rectified Linear Unit, ReLU).

[0041] Then, after acquiring the first input image S11 and the second input image S21, feature extraction can be performed on the first input image S11 and the second input image S21 respectively to obtain the first image feature and the second image feature respectively, so as to improve the imaging effect of the target image S3 generated in the end according to the first image feature and the second image feature. Among them, since the focal length of the first camera 20 is smaller than the focal length of the second camera 30, the first image feature contains fewer details, while the second image feature contains richer details.

[0042] Furthermore, feature extraction can be performed based on one or more frames of input images. In one embodiment, feature extraction can be performed based on a frame of first input image S11 and a frame of second input image S21 to reduce the workload of feature extraction, thereby improving the efficiency of image processing. In another embodiment, feature extraction can be performed based on multiple frames of first input image S11 and multiple frames of second input image S21. At this time, a frame of first basic frame can be determined in the multiple frames of first input image S11, and a frame of second basic frame can be determined in the multiple frames of second input image S21. During feature extraction, feature extraction is performed on multiple frames of input images based on the basic frame to enrich the corresponding image features, and to avoid the situation where the image of a certain frame of input image is blurred and it is difficult to extract accurate image features, so that the generated target image S3 has more complete details, and the temporal and spatial continuity is also taken into consideration when generating the target image S3, so that the imaging effect of the target image S3 is better.

[0043] Step 013: Generate a third image feature based on the first image feature and the second image feature, and superimpose the high-frequency features of the third image feature and the first input image S11 to generate a target image S3.

[0044] Specifically, after obtaining the first image feature and the second image feature, the third image feature can be generated according to the first image feature and the second image feature, so that the third image feature contains the details of the first image feature and the details of the second image feature. Then, the third feature image is processed to obtain the high-frequency features of the third image feature, wherein the high-frequency features correspond to the parts of the image that change dramatically, that is, the edges (contours) or noise and details of the image. Then, after superimposing the high-frequency features of the third image feature and the first input image S11, the high-frequency features can be used to improve the fewer details in the first input image S11, thereby generating a target image S3 with richer details.

[0045] In addition, since the target image S3 is generated by superimposing the high-frequency features of the third image features and the first input image S11, the structures of the target image S3 and the first input image S11 are consistent. At the same time, the structures of the first input image S11 and the target image S3 can also be kept consistent through low-frequency features, wherein the low-frequency features represent areas in the image where the brightness or grayscale value changes slowly, that is, large flat areas in the image, which describe the main part of the image and are a comprehensive measure of the intensity of the entire image.

[0046] The image processing method of the embodiment of the present application controls the first camera 20 and the second camera 30 to shoot respectively, so as to obtain the first image S1 and the second image S2 respectively, wherein the focal length of the first camera 20 is smaller than the focal length of the second camera 30, so that the field of view of the first image S1 is larger but the details are less, and the field of view of the second image S2 is smaller but the details are richer. Then, a first input image S11 with a larger zoom ratio is generated according to the first image S1, and a second input image S21 is generated according to the second image S2, and the field of view corresponding to the first input image S11 and the second image S2 is at least partially the same, so that the first input image S11 with a larger field of view but fewer details can be enhanced by using the second image S2 with a smaller field of view but richer details. After obtaining the first input image S11 and the second input image S21, feature extraction is performed on the first input image S11 and the second input image S21 respectively to obtain the first image feature and the second image feature respectively. Then, based on the first image feature and the second image feature, a third image feature including the two image features is generated, so that the third image feature has richer details, and finally the high-frequency features of the third image feature are superimposed on the first input image S11, so that a target image S3 with richer details can be generated. In this way, based on the images acquired by two cameras with different focal lengths, the second image S2 with a smaller field of view but richer details can be used to enhance the first input image S11 with a larger field of view but fewer details, thereby realizing super-resolution reconstruction of the first input image, and the details of the obtained target image S3 are more complete, thereby improving the imaging effect of the target image S3.

[0047] See also Figures 3 to 5 , optionally, the image processing method may include:

[0048] Step 014: cropping the first image S1 according to the current zoom ratio, and upsampling the first image S1 to obtain a first input image S11, wherein the resolution of the first input image S11 is the same as that of the first image S1; and

[0049] Step 015: Determine a target image region S12 corresponding to the second image S2 in the first input image S11, and adjust the target image region S12 according to the second image S2 to generate a second input image S21.

[0050] Specifically, after obtaining the first image S1, the user can select a zoom ratio in the user interface (UI) according to imaging needs, wherein the zoom ratio of the first image S1 and the zoom ratio of the second image S2 correspond to the minimum zoom ratio and the maximum zoom ratio, respectively. The zoom ratio is generally between the zoom ratio of the first image and the zoom ratio of the second image S2, and the zoom ratio can be equal to the maximum zoom ratio or the minimum zoom ratio. Then, according to the current zoom ratio, the first image S1 is cropped, for example Figure 6, when the magnification of the first image S1 is 1 times, if the current zoom magnification is 2 times, half of the image can be cropped from the first image S1; for example Figure 7 , when the magnification of the first image S1 is 0.5 times, if the current zoom magnification is 2 times, a quarter of the image is cropped from the first image S1. It should be noted that the cropping center during cropping can be centered, for example Figure 6 The crop center is centered, and can also be offset based on the camera relationship, such as Figure 7 The cropping center is shifted to a certain extent so that the overlap between the cropped first image S1 and the second image S2 is greater. Then the cropped image is upsampled, for example, bilinear interpolation is performed on the cropped image to obtain a first input image S11 having the same resolution and size as the first image S1.

[0051] Please combine Figure 5 , the field of view corresponding to the first input image S11 and the second image S2 at least partially overlap, then the target image area S12 corresponding to the second image S2 in the first input image S11 can be determined, and the target image area S12 can be adjusted according to the second image S2, for example, the second image S2 is processed in a certain way, and the processed second image S2 is used to replace the target image area S12 in the first input image S11, or the second image S2 is directly used to replace the target image area S12, so as to generate a second input image S21 corresponding to the second image S2.

[0052] In this way, the size and resolution of the second input image S21 and the first input image S11 can be made the same through a corresponding generation method, thereby facilitating subsequent image registration and alignment based on the first input image S11 and the second input image S21.

[0053] See also Figure 2 , Figure 5 and Figure 8 Optionally, step 015: determining a target image region S12 corresponding to the second image S2 in the first input image S11, and adjusting the target image region S12 according to the second image S2 to generate the second input image S21 may include:

[0054] Step 0151: Determine a target image area S12 corresponding to the second image S2 in the first input image S11 according to the field of view range and relative position of the first camera 20 and the second camera 30;

[0055] Step 0152: Based on a preset histogram matching algorithm, obtain histogram statistical information of the target image area S12, and process the second image S2 according to the histogram statistical information to generate a first mapping image;

[0056] Step 0153: aligning the first mapping image and the first input image S11 to adjust the target image area S12;

[0057] Step 0154: down-sampling the first mapped image to obtain a second mapped image matching the adjusted target image area S12; and

[0058] Step 0155: Use the second mapped image to replace the adjusted target image area S12 to generate a second input image S21.

[0059] Specifically, multi-camera images exhibit different distortions due to the use of different lenses, sensors and image signal processing (ISP). There are certain brightness and color differences between images. Therefore, when performing image super-resolution, it is also necessary to perform image registration on the first input image S11 and the second image S2.

[0060] The field of view range and relative position of the first camera 20 and the second camera 30 are known, so the target image area S12 corresponding to the second image S2 in the first input image S11 can be determined according to the field of view range and relative position of the first camera 20 and the second camera 30. Then, based on the preset histogram matching algorithm, the histogram statistical information of the target image area S12, such as the color information and brightness information of the target image area S12, can be obtained, and the second image S2 can be processed according to the histogram statistical information, such as adjusting the color of the second image S2 according to the color information of the target image area S12, and adjusting the brightness of the second image S2 according to the brightness information of the target image area S12, so as to generate a first mapping image, thereby realizing the registration and alignment of the first input image S11 and the second image S2.

[0061] After the first mapping image is generated, since the information of the first mapping image is changed compared with the information of the second image S2, the first mapping image and the first input image S11 may be aligned so that the target image area S12 is aligned with the first mapping image. For example, the first mapping image and the first input image S11 may be first subjected to feature point extraction and matching. At this time, scale-invariant feature transform (SIFT) feature points or oriented fast rotation (ORB) feature points may be extracted. SIFT feature points are obtained by SIFT algorithm, which is a computer vision (CV) algorithm used to detect and describe local features in images. It searches for extreme points in scale space and extracts their position, scale, and rotation invariants. Similarly, ORB feature points are obtained by ORB algorithm, which is an algorithm for accelerating feature point extraction and description. It extracts feature points based on FAST algorithm and constructs feature point descriptors based on BRIEF algorithm. They are modified on their original basis to achieve scale invariance and rotation invariance of feature points, that is, feature points after scaling and rotation can still produce descriptors similar to the original ones. The full name of FAST algorithm is Features from accelerated segment. test is a corner detection method that can be used to extract feature points and complete tracking and mapping of objects; BRIEF algorithm is called Binary Robust Independent Elementary Features, which is a fast method for calculating and matching feature point descriptors. The extracted feature points are then used for image registration and alignment, for example, directly using a neural network (such as a RANSAC-Flow neural network) for alignment; or first using an estimated homography matrix for coarse alignment, and then using a stereo matching algorithm or an optical flow algorithm for fine alignment. Among them, the homography of a plane is defined as a projection mapping from one plane to another. The estimated homography matrix can be calculated based on the extracted feature points, and then the image is aligned based on the estimated homography matrix. Stereo matching algorithms, such as the Semi-Global Matching (SGM) algorithm, achieve image alignment by finding the corresponding pixel point of each pixel point in the image from another perspective.Optical flow algorithms, such as the DIS (full name: Dense Inverse Search-based method) optical flow algorithm, are an algorithm that strikes a balance between optical flow quality and calculation time. Optical flow describes the flow of pixel intensity in an image, and the purpose of the optical flow algorithm is to estimate the speed and direction of an object's movement based on the change in the grayscale value intensity of pixels in the image, thereby achieving image alignment.

[0062] After the first input image S11 and the first mapping image are aligned, the similarity of the color information and brightness information of the two becomes higher, and the accuracy of the target image area S12 determined on this basis will also become higher. Therefore, the image area in the first input image S11 corresponding to the first mapping image can be determined again, and the image area can be reconfirmed as the target image area S12, thereby completing the targeted adjustment of the target image area S12, thereby improving the confirmation accuracy of the target image area S12.

[0063] After the first mapping image is generated, the first mapping image may be downsampled to obtain a second mapping image matching the adjusted target image area S12, for example, the first mapping image may be reduced to the size of the target image area S12 to obtain the second mapping image. Figure 3 and Figure 5 , the adjusted target image area S12 can be replaced by a second mapping image matching the target image area S12, and the pixels of the area other than the target image area S12 in the first input image S11 can be set to 0 to generate the second input image S21, and the area other than the target image area S12 in the second input image S21 is prevented from interfering with the subsequent feature extraction, thereby affecting the imaging effect of the target image S3.

[0064] The size of the target image area S12 is usually fixed, so the second image S2 may be downsampled according to the target image area S12, and then the downsampled second image S2 and the first input image S11 may be registered to generate the first mapping image. In this way, after aligning the first mapping image and the first input image S11 and adjusting the target image area S12, since the first mapping image and the adjusted target image area S12 match, the first mapping image may be directly used to replace the adjusted target image area S12 to generate the second input image S21.

[0065] In this way, the second input image S21 is generated according to the target image area S12 and the second image S2, so that the details of the first input image S11 are subsequently enhanced according to the details of the target image area S12 of the second mapped image.

[0066] See also Fig. 9Optionally, step 013: generating a third image feature according to the first image feature and the second image feature may include:

[0067] Step 0131: Determine a correlation matrix between the first image feature and the second image feature;

[0068] Step 0132: Calculate the weights of each channel of the second image feature according to the correlation matrix and the preset activation function;

[0069] Step 0133: Generate a fourth image feature according to the weights of each channel and the second image feature; and

[0070] Step 0134: Superimpose the first image feature and the fourth image feature to generate a third image feature.

[0071] Specifically, please combine Figure 3 After obtaining the first image feature and the second image feature, the correlation matrix of the first image feature and the second image feature can be determined. For example, if both the first image feature and the second image feature are in the form of feature matrices, the feature matrix of the first image feature and the feature matrix of the second image feature can be dot-producted, that is, one of the two feature matrices is transposed, and the transposed feature matrix is ​​multiplied by the other feature matrix, and the result obtained is the correlation matrix of the first image feature and the second image feature.

[0072] Then, the weight of each channel of the second image feature is calculated according to the correlation matrix and a preset activation function, such as a softmax function. Then, according to the weights of each channel and the second image feature, each channel of the second image feature is enhanced, for example, the pixel value of each channel is multiplied by the corresponding weight to generate a fourth image feature, so that the features of the fourth image feature that are more correlated with the first image feature are clearer or more prominent. Then, the first image feature and the fourth image feature after feature enhancement can be superimposed to generate a third image feature, so that the features of the third image feature that are more correlated are clearer or more prominent, so as to facilitate the subsequent enhancement of the details of the first input image S11 according to the third image feature.

[0073] In another embodiment, the first image feature and the second image feature may be superimposed to generate a third image feature without enhancing the features in the second image feature that are highly correlated with the first image feature. In this way, on the one hand, the generated third image feature still includes the first image feature and the second image feature, and on the other hand, the step of enhancing features can be reduced, thereby improving image processing efficiency.

[0074] Furthermore, image features can be divided into three dimensions: w dimension (rows), h dimension (columns) and c dimension (number of channels). The dual-camera image is highly symmetrical under the epipolar constraint. Therefore, the dimension to be used for calculating the correlation matrix can be determined based on the current input image alignment error constraint and the arrangement of the two cameras. For example, when the two cameras are arranged vertically (that is, the two cameras are arranged in a direction perpendicular to the horizontal line), the correlation matrix can be calculated along the horizontal line w dimension. For another example, when the two cameras are arranged horizontally (that is, the two cameras are arranged in the direction of the horizontal line), the correlation matrix can be calculated along the h dimension. In this way, please combine Figure 2 , the dimension referenced when calculating the correlation matrix can be determined in a targeted manner based on the correlation parameters between the first camera 20 and the second camera 30, thereby improving the accuracy of the correlation matrix.

[0075] See also Fig.10 Optionally, step 0131: determining a correlation matrix between the first image feature and the second image feature may include:

[0076] Step 01311: performing layer normalization on the first image feature to obtain a first layer normalized feature, and performing layer normalization on the second image feature to obtain a second layer normalized feature;

[0077] Step 01312: Determine a correlation matrix based on the first layer normalized features and the second layer normalized features.

[0078] Specifically, layer normalization is a normalization technique for deep neural networks, and layer normalization can normalize all neurons in a certain layer. After layer normalization of the first image feature, the first layer normalized feature can be obtained, and after layer normalization of the second image feature, the second layer normalized feature can be obtained. Then, the correlation matrix can be determined based on the first layer normalized features and the second layer normalized features, for example, the feature matrix of the first layer normalized features is transposed, and the feature matrix of the transposed first layer normalized features is multiplied by the feature matrix of the second layer normalized features to obtain the correlation matrix. In this way, by performing layer normalization on the first image feature and the second image feature, the corresponding layer normalized features are obtained to obtain a correlation matrix with higher accuracy, so that the features of the second image feature with higher correlation with the first image feature can be accurately enhanced according to the correlation matrix with higher accuracy.

[0079] See also Figure 3 and Fig.11 Optionally, before superimposing the high-frequency features of the third image features and the first input image S11 to generate the target image S3, the image processing method may further include:

[0080] Step 016: Perform convolution and nonlinear activation processing on the third image feature to obtain high-frequency features of the third image feature.

[0081] Specifically, after obtaining the third image feature, a pre-trained convolution layer and a non-linear activation layer can be introduced to perform convolution and non-linear activation processing on the third image feature to obtain the high-frequency features of the third image feature and balance the high-frequency noise, so as to facilitate further enhancement of the details and facilitate the subsequent generation of a target image S3 with more complete details based on the combination of the high-frequency features of the third image feature and the first input image S11.

[0082] See also Fig.12 and Fig.13 , the training method of the embodiment of the present application includes:

[0083] Step 021: down-sample the first input image S11 and the second input image S21 respectively to obtain a first sample image P1 and a second sample image P2 respectively, the first input image S11 is generated according to the first image S1, the second input image S21 is generated according to the second image S2, the focal length of the first camera 20 for capturing the first image S1 is smaller than the focal length of the second camera 30 for capturing the second image S2, and the fields of view corresponding to the first input image S11 and the second image S2 at least partially overlap;

[0084] Step 022: extracting features from the first sample image P1 to obtain first training image features, and extracting features from the second sample image P2 to obtain second training image features;

[0085] Step 023: Generate a third training image feature according to the first training image feature and the second training image feature, and superimpose the high-frequency feature of the third training image feature and the first sample image P1 to generate a first training image P3; and

[0086] Step 024: Based on the first loss function, the first training image P3, the first input image S11 and the second input image S21, a first loss value is calculated, and model parameters of the image processing model are adjusted according to the first loss value to train the image processing model to convergence.

[0087] Specifically, the above-mentioned image processing methods are all processed by an image processing model. In order to improve the processing accuracy of the image processing model, the image processing model may be trained before being put into use.

[0088] In a dual-camera system, it is difficult to obtain high-resolution images of the real world. Therefore, the first input image S11 and the second input image S21 can be downsampled to obtain the first sample image P1 and the second sample image P2, respectively, so that the first sample image P1 and the second sample image P2 can be used as input samples of the image processing model, and the first input image S11 and the second input image S21 can be used as true values ​​for supervised learning.

[0089] During training, the first camera 20 with a smaller focal length can be controlled to shoot to obtain the first image S1, and the second camera 30 with a larger focal length can be controlled to shoot to obtain the second image S2. Then, the first input image S11 is generated according to the first image S1, and the second input image S21 is generated according to the second image S2. Then, the first input image S11 and the second input image S21 are downsampled respectively to obtain the first sample image P1 and the second sample image P2 respectively. After obtaining the first sample image P1 and the second sample image P2, feature extraction can be performed on the first sample image P1 and the second sample image P2 respectively to obtain the first training image feature and the second training image feature respectively, so as to improve the imaging effect of the first training image P3 generated last according to the first training image feature and the second training image feature. After obtaining the first training image feature and the second training image feature, the third training image feature can be generated according to the first training image feature and the second training image feature, so that the third training image feature contains the details of the first training image feature and the details of the second training image feature. Then, the third training feature image is processed to obtain the high-frequency features of the third training image feature. Then, after superimposing the high-frequency features of the third training image features and the first sample image P1, a first training image P3 with relatively complete details can be generated.

[0090] Then, it is necessary to evaluate the image processing model based on the first training image P3. According to the generated first training image P3, the first loss function, the first input image S11 and the second input image S21, the first loss value can be calculated. For example, the first loss function is in is the first input image S11, is the first training image P3. When the first loss value is less than a certain threshold, it can be confirmed that the image processing model has converged. Then, the model parameters of the image processing model can be adjusted according to the first loss value to train the image processing model until convergence, thereby ensuring the image processing accuracy of the converged image processing model.

[0091] Furthermore, please combine Figure 5When generating the first input image S11, the first image S1 also needs to be cropped and upsampled. In order to ensure the training effect, in the training stage, only the target image area S12 corresponding to the second input image S21 in the first input image S11 is used for block cropping training to ensure that the trained image processing model has a better processing effect on the target image area S12.

[0092] See also Fig.14 and Fig.15 Optionally, the training method further includes:

[0093] Step 025: Input the first input image S11 and the second input image S21 to the image processing model to output a second training image P4;

[0094] Step 026: Based on the second loss function, the second training image P4, the low-frequency features of the first input image S11 and the high-frequency features of the second input image S21, a second loss value is calculated, and the model parameters are adjusted again according to the second loss value to train the image processing model to convergence.

[0095] Specifically, models trained on downsampled images show significant performance degradation on real images. Therefore, a self-supervised adaptation scheme is also needed to fine-tune the trained image processing models.

[0096] At this time, the first input image S11 and the second input image S21 are input into the image processing model as training sets to output the second training image P4. Similar to the above-mentioned processing method of the first sample image P1 and the second image S2 samples, after acquiring the first input image S11 and the second input image S21, the image processing model will perform the same processing as the above-mentioned processing method of the first sample image P1 and the second sample image P2 according to the first input image S11 and the second input image S21, and then output the processed second training image P4 with more complete details. Then, it is necessary to evaluate the image processing model based on the second training image P4, that is, the second loss value can be calculated based on the generated second training image P4, the second loss function, the low-frequency features of the first input image S11 and the high-frequency features of the second input image S21. For example, the second loss function is in is the first input image S11, I2 is the second input image S21, is the second training image P4, blur() represents Gaussian blur (3*3) on the image, δ i (X,Y) represents the distance between pixel x and its neighboring pixel y, c iRepresents confidence, and λ is the weight of high-frequency features. The distance between pixel x and neighboring pixel y is the cosine distance between the two, which in the formula represents the cosine distance between the second input image S21 and the second training image P4. The confidence is confirmed based on the cosine distance between each pixel. At this time, the matching degree (i.e. cx) of each point of the second input image S21 to each point of the second training image P4 can be calculated according to the formulas: relative distance = cosine distance / (minimum cosine distance+1e-5), w = exp((1-relative distance) / 0.5) and cx = w / sum(w). The confidence c i is the maximum value of cx, that is, the matching degree of the most matched point. At the same time, it can be seen in the second loss function that the pixel with higher confidence will be given a greater weight. According to the formula index = argmax (cx), the position of the point with the highest matching degree of the second training image relative to the second input image can be determined. i (X, Y) is the value of the point at this position calculated according to the formula exp((1-cosine distance) / 0.5). Then, after the second loss value is calculated according to the above formula, the model parameters of the image processing model can be adjusted according to the second loss value to train the image processing model to convergence, thereby ensuring the image processing accuracy of the converged image processing model.

[0097] Furthermore, please combine Figure 5 When generating the second training sample, the second input image S21 may be randomly occluded to train the image processing model for the enhancement effect on the image area outside the target image area S12 in the first input image S11, thereby combining the training effect on the target image area S12 in the previous step, so that the enhancement effect of the image processing model on all image areas of the first input image S11 can be improved, and the details of the obtained target image S3 are more complete, thereby improving the imaging effect of the target image S3.

[0098] In order to better implement the image processing method of the embodiment of the present application, the embodiment of the present application also provides an image processing device 10. Figure 2 , Figure 3 and Fig.16 , Fig.16 The following is a schematic diagram of the structure of an image processing device 10 provided in an embodiment of the present application. The image processing device 10 may include:

[0099] An acquisition module 11 is used to acquire a first input image S11 and a second input image S21, wherein the first input image S11 is generated according to the first image S1, and the second input image S21 is generated according to the second image S2, the focal length of the first camera 20 for acquiring the first image S1 is smaller than the focal length of the second camera 30 for acquiring the second image S2, and the fields of view corresponding to the first input image S11 and the second image S2 at least partially overlap;

[0100] The feature extraction module 12 is used to perform feature extraction on the first input image S11 to obtain first image features, and to perform feature extraction on the second input image S21 to obtain second image features;

[0101] The feature processing module 13 is used to generate a third image feature according to the first image feature and the second image feature, and to superimpose the high-frequency feature of the third image feature and the first input image S11 to generate a target image S3.

[0102] Please combine Figure 5 The image processing device 10 may further include a first generating module 14 and a second generating module 15. The first generating module 14 is used to crop the first image S1 according to the current zoom ratio and upsample the first image S1 to obtain a first input image S11. The resolution of the first input image S11 is the same as that of the first image S1. The second generating module 15 is used to determine a target image area S12 in the first input image S11 corresponding to the second image S2, and adjust the target image area S12 according to the second image S2 to generate a second input image S21.

[0103] The second generation module 15 is further specifically used to determine the target image area S12 corresponding to the second image S2 in the first input image S11 according to the field of view range and the relative position of the first camera 20 and the second camera 30; based on a preset histogram matching algorithm, obtain the histogram statistical information of the target image area S12, and process the second image S2 according to the histogram statistical information to generate a first mapping image; align the first mapping image and the first input image S11 to adjust the target image area S12; downsample the first mapping image to obtain a second mapping image that matches the adjusted target image area S12; and replace the adjusted target image area S12 with the second mapping image to generate a second input image S21.

[0104] The feature processing module 13 is also specifically used to determine the correlation matrix between the first image feature and the second image feature; calculate the weights of each channel of the second image feature according to the correlation matrix and a preset activation function; generate a fourth image feature according to the weights of each channel and the second image feature; and superimpose the first image feature and the fourth image feature to generate a third image feature.

[0105] The feature processing module 13 is specifically used to perform layer normalization on the first image feature to obtain a first layer normalized feature, and to perform layer normalization on the second image feature to obtain a second layer normalized feature; and to determine a correlation matrix based on the first layer normalized feature and the second layer normalized feature.

[0106] Please combine Fig.13, the image processing device 10 may further include:

[0107] The feature enhancement module 16 is used to perform convolution and nonlinear activation processing on the third image feature to obtain high-frequency features of the third image feature.

[0108] The acquisition module 11 is specifically used to downsample the first input image S11 and the second input image S21 respectively to obtain a first sample image P1 and a second sample image P2 respectively. The first input image S11 is generated according to the first image S1, and the second input image S21 is generated according to the second image S2. The focal length of the first camera 20 that captures the first image S1 is smaller than the focal length of the second camera 30 that captures the second image S2, and the field of view corresponding to the first input image S11 and the second image S2 at least partially overlaps.

[0109] The feature extraction module 12 is specifically used to perform feature extraction on the first sample image P1 to obtain first training image features, and to perform feature extraction on the second sample image P2 to obtain second training image features.

[0110] The feature processing module 13 is specifically used to generate a third training image feature according to the first training image feature and the second training image feature, and to superimpose the high-frequency feature of the third training image feature and the first sample image P1 to generate a first training image P3.

[0111] Please combine Fig.15 , the image processing device 10 may further include:

[0112] The adjustment module 17 is used to calculate a first loss value based on the first loss function, the first training image P3, the first input image S11 and the second input image S21, and adjust the model parameters of the image processing model according to the first loss value to train the image processing model to convergence.

[0113] The image processing device 10 may further include:

[0114] The input module 18 is used to input the first input image S11 and the second input image S21 to the image processing model to output the second training image P4.

[0115] The adjustment module 17 can also be used to calculate a second loss value based on a second loss function, a second training image P4, low-frequency features of the first input image S11 and high-frequency features of the second input image S21, and adjust the model parameters again according to the second loss value to train the image processing model to convergence.

[0116] Each module in the above-mentioned image processing device 10 can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of the processor 40 in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor 40 can call and execute the operations corresponding to each module.

[0117] See also Figure 2 The embodiment of the present application also provides an electronic device 100, which includes a processor 40. The processor 40 is used to execute the image processing method of any one of the above embodiments and the training method of any one of the above embodiments. For the sake of brevity, they are not described here.

[0118] The electronic device 100 includes but is not limited to a structure with a camera function such as a mobile phone, a tablet computer, a camera, a video camera, a personal digital assistant, a wearable device, an intelligent robot, and an intelligent vehicle. Among them, the wearable device includes a smart bracelet, a smart watch, and smart glasses.

[0119] See also Fig.17 The embodiment of the present application also provides a non-volatile computer-readable storage medium 300, on which a computer program 310 is stored. When the computer program 310 is executed by the processor 40, the steps of the image processing method of any one of the above-mentioned embodiments and the steps of the training method of any one of the above-mentioned embodiments are implemented. For the sake of brevity, they are not repeated here.

[0120] It can be understood that the computer program 310 includes computer program code. The computer program code can be in source code form, object code form, executable file or some intermediate form, etc. Computer readable storage media can include: any entity or device capable of carrying computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), and software distribution media, etc.

[0121] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiments or examples are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.

[0122] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0123] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. An image processing method, characterized in that: include: Acquire a first input image and a second input image, wherein the first input image is generated according to the first image, and the second input image is generated according to the second image, a focal length of a first camera that acquires the first image is smaller than a focal length of a second camera that acquires the second image, and a field of view corresponding to the first input image and the second image at least partially overlaps; performing feature extraction on the first input image to obtain first image features, and performing feature extraction on the second input image to obtain second image features; and A third image feature is generated according to the first image feature and the second image feature, and a high-frequency feature of the third image feature and the first input image are superimposed to generate a target image.

2. The image processing method according to claim 1, characterized in that: Also includes: Cropping the first image according to the current zoom ratio, and upsampling the first image to obtain the first input image, where the resolution of the first input image is the same as the resolution of the first image; and A target image region corresponding to the second image in the first input image is determined, and the target image region is adjusted according to the second image to generate the second input image.

3. The image processing method according to claim 2, characterized in that: The determining a target image region in the first input image corresponding to the second image, and adjusting the target image region according to the second image to generate the second input image includes: Determine, according to the field of view ranges and relative positions of the first camera and the second camera, the target image region corresponding to the second image in the first input image; Based on a preset histogram matching algorithm, obtaining histogram statistical information of the target image area, and processing the second image according to the histogram statistical information to generate a first mapping image; aligning the first mapped image and the first input image to adjust the target image area; downsampling the first mapped image to obtain a second mapped image matching the adjusted target image area; and The second mapped image is used to replace the adjusted target image region to generate the second input image.

4. The image processing method according to claim 1, characterized in that: The step of generating a third image feature according to the first image feature and the second image feature includes: determining a correlation matrix of the first image feature and the second image feature; Calculating the weights of each channel of the second image feature according to the correlation matrix and a preset activation function; generating a fourth image feature according to the weights of each channel and the second image feature; and The first image feature and the fourth image feature are superimposed to generate the third image feature.

5. The image processing method according to claim 4, characterized in that: The determining of the correlation matrix of the first image feature and the second image feature comprises: performing layer normalization on the first image features to obtain first layer normalized features, and performing layer normalization on the second image features to obtain second layer normalized features; and The correlation matrix is ​​determined according to the first-layer normalized features and the second-layer normalized features.

6. The image processing method according to claim 1, characterized in that: Before superimposing the high-frequency features of the third image features and the first input image to generate a target image, the method further includes: The third image feature is subjected to convolution and nonlinear activation processing to obtain high-frequency features of the third image feature.

7. A training method, characterized in that: include: Downsampling a first input image and a second input image respectively to obtain a first sample image and a second sample image respectively, wherein the first input image is generated according to the first image, and the second input image is generated according to the second image, a focal length of a first camera that acquires the first image is smaller than a focal length of a second camera that acquires the second image, and fields of view corresponding to the first input image and the second image at least partially overlap; Performing feature extraction on the first sample image to obtain first training image features, and performing feature extraction on the second sample image to obtain second training image features; Generate a third training image feature according to the first training image feature and the second training image feature, and superimpose high-frequency features of the third training image feature and the first sample image to generate a first training image; and Based on a first loss function, the first training image, the first input image, and the second input image, a first loss value is calculated, and model parameters of the image processing model are adjusted according to the first loss value to train the image processing model to convergence.

8. The training method according to claim 7, characterized in that: Also includes: Inputting the first input image and the second input image into the image processing model to output a second training image; and Based on a second loss function, the second training image, low-frequency features of the first input image and high-frequency features of the second input image, a second loss value is calculated, and the model parameters are adjusted again according to the second loss value to train the image processing model to convergence.

9. An image processing device, characterized in that: include: an acquisition module, configured to acquire a first input image and a second input image, wherein the first input image is generated according to a first image, and the second input image is generated according to a second image, a focal length of a first camera for acquiring the first image is smaller than a focal length of a second camera for acquiring the second image, and a field of view corresponding to the first input image and the second image at least partially overlaps; a feature extraction module, configured to perform feature extraction on the first input image to obtain first image features, and perform feature extraction on the second input image to obtain second image features; and The feature processing module is used to generate a third image feature according to the first image feature and the second image feature, and to superimpose the high-frequency feature of the third image feature and the first input image to generate a target image.

10. An electronic device, characterized in that: It includes a processor, and the processor is used to execute the image processing method described in any one of claims 1 to 6 and the training method described in any one of claims 7 to 8.

11. A non-volatile computer-readable storage medium comprising a computer program, wherein when the computer program is executed by a processor, the processor executes the image processing method according to any one of claims 1 to 6 and the training method according to any one of claims 7 to 8.