Image processing method and electronic equipment

Abstract

The invention provides an image processing method and device, a storage medium and a program product, and the method comprises the steps: determining a blank region in a second image under the condition that a first image in a first display mode is converted into the second image in a second display mode; one of the first display mode and the second display mode is a horizontal screen mode, and the other display mode is a vertical screen mode; and processing the target detection result of the second image by adopting a generative adversarial network of a context attention mechanism to generate an image of the blank area.

Description

Technical Field

[0001] This application belongs to the technical field of Artificial Intelligence (AI), and relates to, but is not limited to, an image processing method and an electronic device. Background Technology

[0002] In related technologies, when converting a landscape image to a portrait image, or vice versa, blank areas such as black screens or useless parts may appear on the screen. There is no effective solution to the problem of how to accurately fill the blank areas. Summary of the Invention

[0003] In order to solve the technical problems existing in the related technologies, this application proposes an image processing method and an electronic device.

[0004] This application provides an image processing method, the method comprising: When converting a first image in a first display mode to a second image in a second display mode, a blank area in the second image is determined; one of the first display mode and the second display mode is a landscape mode, and the other display mode is a portrait mode; By employing Generative Adversarial Networks (GANs) with a contextual attention mechanism, the target detection results of the second image are processed to generate the image of the blank area.

[0005] This application also proposes an electronic device, which includes a processor and a memory for storing a computer program capable of running on the processor; wherein the processor is used to run the computer program to perform any of the above-described image processing methods.

[0006] This application also provides a computer storage medium storing a computer program thereon, which, when executed by a processor, implements any of the above-described image processing methods.

[0007] This application also provides a computer program product, including a computer program that, when executed by a processor, implements any of the above-described image processing methods.

[0008] As can be seen, the embodiments of this application can process the target detection results of the second image by using a generative adversarial network with a contextual attention mechanism. The target detection results of the second image are existing image information in the second image. Therefore, the embodiments of this application can generate images of blank areas more accurately by effectively utilizing the existing image information in the second image. Attached Figure Description

[0009] Figure 1 This is a flowchart of an image processing method according to an embodiment of this application; Figure 2 An architecture diagram of a generative adversarial network provided for an embodiment of this application; Figure 3 This is the second image in the embodiments of this application where the blank areas are not filled; Figure 4 This is the second image in the embodiments of this application, where the blank areas have been filled. Figure 5 This is a flowchart of another image processing method according to an embodiment of this application; Figure 6 This is a schematic diagram of the structure of the image processing apparatus according to an embodiment of this application; Figure 7 This is a schematic diagram of the composition structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0010] When converting an image from landscape mode to portrait mode, or vice versa, blank areas such as black screens or unused parts can easily appear on the screen. To address this issue, one technical solution involves sending a data request carrying a video identifier to the server; receiving the coordinates of the valid video content corresponding to the video identifier returned by the server; then, scaling the valid content based on the size of the valid content indicated by the coordinates and the screen size of the terminal device running the client; finally, rendering the scaled valid content on the screen for video playback. This solution determines the scaling ratio of the valid content based on the size of the video frame within the valid content and the screen size. However, this solution requires coordinate calculations during image scaling and can only remove blank areas through image scaling; it cannot accurately fill these blank areas with new image content.

[0011] To address the technical problems in related technologies, this application proposes technical solutions based on its embodiments. The embodiments of this application will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the embodiments provided herein are merely illustrative of the present application and are not intended to limit the scope of the present application. Furthermore, the embodiments provided below are only some embodiments for implementing this application, and not all embodiments for implementing this application. Unless otherwise specified, the technical solutions described in the embodiments of this application can be implemented in any combination. It should be noted that, in the embodiments of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a method or apparatus that includes a list of elements includes not only the elements expressly described, but also other elements not expressly listed, or elements inherent to implementing the method or apparatus. Without further limitations, an element defined by the phrase "comprising a..." does not exclude the presence of other related elements (e.g., steps in the method or units in the apparatus, such as portions of circuitry, processors, programs, or software, etc.) in the method or apparatus that includes that element. The image processing method provided in this application includes a series of steps, but the image processing method provided in this application is not limited to the steps described. Similarly, the image processing apparatus provided in this application includes a series of modules, but the apparatus provided in this application is not limited to the modules explicitly described, and may also include modules that need to be set up for obtaining relevant information or processing based on information.

[0012] This application provides an image processing method that can be applied to electronic devices with a display screen. The image display mode of the electronic device can be a landscape mode or a portrait mode.

[0013] Figure 1 This is a flowchart of an image processing method according to an embodiment of this application, such as... Figure 1 As shown, the process includes: Step 101: When converting the first image of the first display mode to the second image of the second display mode, determine the blank area in the second image; one of the first display mode and the second display mode is landscape mode and the other is portrait mode.

[0014] In this embodiment, the first image can be any frame from a pre-acquired video. In practical applications, the video can be obtained from the network or from local storage. Of course, the first image can also be obtained in other ways, and this embodiment does not limit the method of obtaining the first image.

[0015] After acquiring the first image, the display mode corresponding to the first image can be identified. For example, AI image recognition technology can be used to analyze the main content and layout of the first image to determine the display mode corresponding to the first image. The display mode corresponding to the first image is either a landscape mode or a portrait mode.

[0016] After determining the display mode corresponding to the first image, the canvas size of the first image can be adjusted according to the target display mode (landscape or portrait mode) to generate a second image in the target display mode. For example, when the display mode corresponding to the first image is landscape mode and the target display mode is portrait mode, the canvas size of the first image can be adjusted to convert the landscape mode first image into a portrait mode second image. Conversely, when the display mode corresponding to the first image is portrait mode and the target display mode is landscape mode, the canvas size of the first image can be adjusted to convert the portrait mode first image into a landscape mode second image. In some embodiments, adjusting the canvas size can be achieved in vector graphics software such as Adobe Illustrator by adjusting the canvas size or creating a new canvas that conforms to the target orientation.

[0017] When converting a first image in a first display mode to a second image in a second display mode, blank areas that need to be filled may appear due to the change in canvas size. In this embodiment, by identifying the blank areas in the second image, it is easier to accurately fill the blank areas subsequently.

[0018] In this embodiment, blank areas in the second image can be identified using edge detection technology. In some embodiments, the second image can be converted into a grayscale image; the outline of a target object in the second image can be identified based on the grayscale image; and blank areas in the second image can be identified based on the outline of the target object in the second image.

[0019] For example, when the second image is a color image such as a red-green-blue (RGB) image, it can be converted into a grayscale image to simplify the subsequent contour recognition process. After converting the second image to grayscale, edge detection methods or other methods can be used to identify the contours of target objects in the second image. Here, the target object can be various objects such as buildings, vehicles, human bodies, and animals.

[0020] For example, after converting the second image to grayscale, Otsu's thresholding method can be used to convert the grayscale image to a binary image, highlighting blank areas.

[0021] Understandably, recognizing contour information in grayscale images is simpler and easier than recognizing contour information in color images. Therefore, after converting the second image to grayscale, the contour of the target object can be more easily identified in the grayscale image, thus enabling convenient and quick identification of blank areas in the second image based on the contour of the target object.

[0022] In some embodiments, the Canny edge detection algorithm can be used to extract edge information from the image to identify the outline of the target object based on the grayscale image.

[0023] In some embodiments, the grayscale image can be converted into a binary image to identify the outline of the target object in the second image based on the grayscale image; the outline of the target object in the second image can then be identified based on the binary image.

[0024] In a binary image, each pixel has only two possible values; for example, the two possible values ​​correspond to black pixels and white pixels, respectively. For instance, after converting a grayscale image to a binary image, OpenCV's findContours method can be used to detect contours in the binary image, thereby identifying blank areas.

[0025] Understandably, binary images have the characteristic of highlighting the contours in an image. Therefore, after converting a grayscale image into a binary image, the contours of the target object in the second image can be more easily identified based on the binary image.

[0026] In some embodiments, to identify blank areas in a second image based on the outline of a target object, the area and position of the target object's outline can be calculated. Based on the area and position of the target object's outline, larger blank areas can be selected in the second image to determine the blank areas that need to be filled.

[0027] After identifying the blank areas in the second image, a mask can be created to facilitate subsequent processing of the blank areas. The mask is an image used to mark the blank areas.

[0028] Step 102: The target detection results of the second image are processed by a generative adversarial network using a contextual attention mechanism to generate an image of the blank area.

[0029] Attention mechanisms are a data processing method in machine learning, widely used in various machine learning tasks such as natural language processing, image recognition, and speech recognition. Essentially, attention mechanisms are similar to how humans observe the external world. Generally, when observing things, people first focus on certain important local information, and then combine information from different areas to form an overall impression of the observed object. Attention mechanisms were first applied in natural language processing, primarily to improve text encoding methods, enabling the learning of better sequence information through encoding-decoding.

[0030] In this embodiment of the application, a generative adversarial network employing a contextual attention mechanism can be pre-trained. After object detection is performed on the second image, the trained generative adversarial network can be used to process the object detection results of the second image to generate an image of the blank area.

[0031] In practical applications, steps 101 to 102 can be implemented based on a processor, which can be at least one of the following: Application Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Digital Signal Processing Device (DSPD), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), Central Processing Unit (CPU), Controller, Microcontroller, and Microprocessor.

[0032] As can be seen, the embodiments of this application can process the target detection results of the second image by using a generative adversarial network with a contextual attention mechanism. The target detection results of the second image are existing image information in the second image. Therefore, the embodiments of this application can generate images of blank areas more accurately by effectively utilizing the existing image information in the second image.

[0033] In this embodiment of the application, when converting the first image into a second image, an object detection algorithm can be used to perform object detection on the second image to obtain object detection results. For example, the object detection results include the location and category of the target object in the second image.

[0034] Here, the location of a target object can be represented by its bounding box. For example, for the second image, the YOLO (You Only Look Once) algorithm can be used for object detection. The YOLO algorithm detects and identifies key elements, treating object detection as a single regression problem, thus achieving fast and efficient detection. YOLOv8 can load pre-trained models and use them to perform object detection on the second image. The object detection results typically include the category of each detected target object and the location of its bounding box. In practical applications, the object detection results can be accessed through the result object. In some embodiments, the bounding boxes and labels of the object detection results can be drawn in the second image to visualize the results; here, the labels indicate the category of the corresponding target object.

[0035] It can be seen that, in determining the location and category of the target object in the second image, the blank areas can be filled with image content more accurately by combining the location and category of the target object in the second image and using a generative adversarial network with a contextual attention mechanism.

[0036] In some embodiments, step 102 can be implemented by generating an image associated with the target detection result in a blank area based on a contextual attention mechanism in the generator of a generative adversarial network.

[0037] In this embodiment, by combining a contextual attention mechanism with a generative adversarial network, the accuracy of filling in blank areas in an image can be significantly improved. Based on the bounding boxes and labels in the target detection results, contextual information of the target object can be extracted. Then, based on this extracted contextual information, combined with the contextual attention mechanism and by focusing on utilizing information from known regions in the image, blank areas can be filled more intelligently, thereby generating a more natural and coherent image.

[0038] The core idea of ​​contextual attention mechanisms is to calculate the correlation between different regions in an image. Based on the existing content in the image and the calculated correlations, similar or related image elements are generated to fill in the blank areas. For example, when a key element labeled "landscape" is detected in an image, natural elements such as trees and clouds can be generated. When the background expands to a new canvas size, the spatial information of the bounding box positions of the key elements is used to determine the location of the generated filling content, thus maintaining both the continuity of the key elements and the coherence of the original image. Unlike traditional convolution operations, attention mechanisms can "focus" on related parts of the image, thereby generating content for missing parts more accurately. By providing spatial location information of the target object through bounding boxes and category information through labels, combining bounding boxes and labels allows the generator of a generative adversarial network to have a more comprehensive understanding of the image content when filling in blank areas.

[0039] As can be seen, the embodiments of this application can more accurately determine the image associated with the target detection result based on the context attention mechanism, thereby accurately generating the image associated with the target detection result in the blank area.

[0040] In some embodiments, the generator may generate an attention map representing the similarity between the blank area and the known area in the second image, and generate a feature map corresponding to the second image based on the attention map. The feature map may be processed to generate an image associated with the target detection result in the blank area.

[0041] Here, the known region refers to the non-blank region in the second image, which is the region used for object detection.

[0042] As can be seen, the image generated in the blank area is determined based on the feature map corresponding to the second image, and the feature map corresponding to the second image is generated based on the attention map. The attention map represents the similarity between the blank area in the second image and the known area. Therefore, in this embodiment, the image associated with the known area can be filled in the blank area more accurately based on the similarity between the blank area in the second image and the known area.

[0043] Generative Adversarial Networks (GANs) can include a generator and a discriminator. For example, a GAN can be built using Pix2Pix. In a GAN, the generator is responsible for generating missing parts from an image. Its advantage lies in its suitability for image-to-image translation tasks; for example, the generator architecture could be U-Net or other architectures. The discriminator is used to determine whether the generated image is realistic. Its input is a concatenation of the generated image and the real image. Figure 2 An architecture diagram of a generative adversarial network provided in this application embodiment is shown below. Figure 2 The generator G can process random noise z to obtain the generated image G(z). The discriminator D is input to G(z) and the concatenation of real data x. The discriminator D is used to determine the authenticity of the generated image G(z).

[0044] In this embodiment, the generator may include an input layer, an encoder, and a decoder, with the encoder including a context attention module. In some embodiments, the input layer may receive a second image and a mask corresponding to the second image, the mask being used to mark the blank areas in the second image; in the encoder, an attention map is generated by the context attention module, and the feature map is generated based on the attention map; then, the decoder processes the feature map to generate an image associated with the target detection result in the blank areas.

[0045] An encoder encodes an input image into a set of feature representations. Encoders typically consist of convolutional layers and progressively extract local and global features from the image. At certain layers of the encoder, a contextual attention mechanism can be introduced to leverage information from known regions of the image to enhance feature extraction. For example, a contextual attention module can be inserted into an intermediate layer of the encoder. This module calculates the similarity between blank areas and known regions through convolution, generating an attention map. Based on this attention map, the features of the known regions can be weighted and applied to the blank areas. This allows information from other parts of the image to be used to fill in the blank areas.

[0046] The decoder receives feature maps enhanced by an attention mechanism, decodes them step by step back to the original image size, and finally outputs the repaired image. In the last few layers of the decoder, deconvolution can be used to generate high-resolution output images, for example, output images with pixel values ​​in the range of [-1, 1].

[0047] As can be seen, the embodiments of this application can generate an attention map representing the similarity between the blank area and the known area in the second image through the context attention module in the encoder. Thus, when the feature map is generated based on the attention map, the image associated with the known area can be filled in the blank area more accurately according to the feature map.

[0048] The training process of a Generative Adversarial Network (GAN) is illustrated below. As described above, during the training of a GAN, the generator outputs a generated image, and the discriminator's input is a concatenation of the generated and real images. For example, a loss function for the GAN can be constructed based on the generated and real images. This loss function can be the L1 loss function (also known as the absolute error loss function) or other loss functions used to represent the difference between the generated and real images. Here, the L1 loss function represents the absolute difference between each pixel of the generated and real images. Using the L1 loss function, the difference between the generated image and the original complete image can be measured, ensuring that the generated image is consistent with the real scene. In practical applications, tensors of the generated and real images can be obtained, and the absolute difference between the two tensors can be calculated pixel by pixel. The average of the absolute differences of all pixels is then used to obtain the final value of the L1 loss function. That is, the value of the L1 loss function can be calculated using formula (1). .

[0049] (1) in, This represents the total number of pixels in the image. Indicates the first element in the generated image The pixel value of each pixel. Represents the first in the real image The pixel value of each pixel.

[0050] In training the Generative Adversarial Network (GAN), a standard GAN training strategy is adopted. Through adversarial training between the generator and the discriminator, the generator gradually improves its image generation capabilities. During training, emphasis is placed on the generator's learning, enabling it to effectively fill in blank areas using a contextual attention mechanism.

[0051] After training and generating an adversarial network using a contextual attention mechanism, the target detection results of the second image can be processed by the generative adversarial network using a contextual attention mechanism, thereby filling in the blank areas. Figure 3 This is the second image in the embodiments of this application where the blank areas are not filled. Figure 4 This is the second image in the embodiments of this application with the blank areas already filled, by comparison. Figure 3 and Figure 4 It can be seen that the image elements or extended background generated by the generative adversarial network can be effectively integrated with the existing images in the second image, ensuring a natural transition without obvious seams.

[0052] Figure 5 A flowchart of another image processing method according to an embodiment of this application is shown below. Figure 5 As shown, the process includes: Step 501: Identify key elements in any frame of the video.

[0053] Here, key elements may include the object detection results of the image.

[0054] Step 502: Fill blank areas more intelligently based on key elements in the image.

[0055] In practical applications, generative adversarial networks with contextual attention mechanisms can be used to process key elements in the image and generate images with blank areas. This allows for more intelligent filling of blank areas, resulting in more natural and coherent images.

[0056] The implementation methods of steps 501 to 502 have been explained in the aforementioned content and will not be repeated here.

[0057] By performing steps 501 to 502, a low-resolution image can be reconstructed into a high-resolution image, enhancing image details and quality, and improving image clarity.

[0058] In practical applications, steps 501 to 502 can be implemented based on a processor, which can be at least one of ASIC, DSP, DSPD, PLD, FPGA, CPU, controller, microcontroller, and microprocessor.

[0059] In summary, this application provides a method for effective content recognition and blank area filling in images. It can determine the display mode of an image, adjust the image canvas size according to the target display mode, and detect the blank areas that need to be filled. It identifies key elements in the image and then uses a combination of generative adversarial networks (GANs) and contextual attention mechanisms to fill the blank areas, effectively integrating the generated image elements with existing image regions. Specifically, a contextual attention mechanism can be introduced into the generator of the GAN to enhance its ability to utilize existing image information when filling blank areas, thereby more accurately generating the missing content and achieving a more natural and realistic effect when the blank areas are filled.

[0060] The technical solutions of this application embodiment have at least the following beneficial effects: 1) Compared with the related technologies that only obtain coordinate information of valid content, the embodiments of this application can improve the accuracy of key information recognition based on the target classification results and target localization results of the image. 2) Compared with the image processing scheme of resetting coordinates, the embodiments of this application combine the contextual attention mechanism with the generative adversarial network, which significantly improves the realism and accuracy of filling blank areas in the image, and eliminates the need to perform coordinate calculation steps during image reconstruction, simplifying the process and improving image processing efficiency. 3) When different video platforms adopt different image display modes, the solution of this application embodiment can convert the image display mode, thereby adapting to the image display of more video platforms.

[0061] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0062] This application also proposes an image processing apparatus located in an electronic device. Figure 6 This is a schematic diagram of the structure of the image processing apparatus according to an embodiment of this application, such as... Figure 6 As shown, the device includes: The first processing module 601 is used to determine a blank area in the second image when converting the first image of the first display mode to the second image of the second display mode; one of the first display mode and the second display mode is a landscape mode and the other display mode is a portrait mode. The second processing module 602 is used to process the target detection result of the second image by employing a generative adversarial network with a context attention mechanism to generate the image of the blank area.

[0063] In some embodiments, the target detection result includes the location and category of the target object in the second image.

[0064] In some embodiments, the second processing module 602 is specifically used in the generator of the generative adversarial network to generate an image associated with the target detection result in the blank area based on the context attention mechanism.

[0065] In some embodiments, the second processing module 602 is specifically used in the generator to generate an attention map representing the similarity between the blank area and the known area in the second image, and to generate a feature map corresponding to the second image based on the attention map, and to process the feature map to generate an image associated with the target detection result in the blank area.

[0066] In some embodiments, the generator includes an input layer, an encoder, and a decoder, wherein the encoder includes a context attention module; The second processing module 602 is specifically used to receive the second image and a mask corresponding to the second image through the input layer, the mask being used to mark the blank area in the second image; in the encoder, the attention map is generated through the context attention module, and the feature map is generated based on the attention map; the feature map is processed by the decoder to generate an image associated with the target detection result in the blank area.

[0067] In some embodiments, the first processing module 601 is specifically used to convert the second image into a grayscale image; identify the outline of a target object in the second image based on the grayscale image; and identify blank areas in the second image based on the outline of the target object.

[0068] In some embodiments, the first processing module 601 is specifically used to convert the grayscale image into a binary image; and to identify the contour of a target object in the second image based on the binary image.

[0069] In practical applications, the first processing module 601 and the second processing module 602 can be implemented based on a processor.

[0070] It should be noted that the description of the above device embodiments is similar to the description of the above method embodiments, and has similar beneficial effects. For technical details not disclosed in the device embodiments of this application, please refer to the description of the method embodiments of this application for understanding.

[0071] It should be noted that, in the embodiments of this application, if the above-described methods are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of this application, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a terminal, server, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of this application are not limited to any specific hardware and software combination.

[0072] Correspondingly, this application embodiment further provides a computer program product, the computer program product including computer executable instructions, which are used to implement any of the image processing methods provided in this application embodiment.

[0073] Accordingly, this application embodiment further provides a computer storage medium storing computer-executable instructions, which are used to implement any of the image processing methods provided in the above embodiments.

[0074] This application also provides an electronic device. Figure 7 This is a schematic diagram of the composition structure of an electronic device provided in an embodiment of this application, such as... Figure 7 As shown, the electronic device 70 may include: Memory 701 is used to store executable instructions; The processor 702 is used to implement any of the above-described image processing methods when executing executable instructions stored in the memory 701.

[0075] The processor 702 mentioned above can be at least one of ASIC, DSP, DSPD, PLD, FPGA, CPU, controller, microcontroller, and microprocessor.

[0076] The aforementioned computer-readable storage medium and memory 701 may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM), etc.; it may also be various terminals that include one or any combination of the above-mentioned memories, such as mobile phones, computers, tablet devices, personal digital assistants, etc.

[0077] In some embodiments, the functions or modules of the apparatus provided in this application can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0078] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.

[0079] The methods disclosed in the various method embodiments provided in this application can be arbitrarily combined to obtain new method embodiments without conflict.

[0080] The features disclosed in the various product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0081] The features disclosed in the various method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method or device embodiments.

[0082] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0083] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims. All of these forms are within the protection scope of this application.

Claims

1. An image processing method, characterized in that, The method includes: When converting a first image in a first display mode to a second image in a second display mode, a blank area in the second image is determined; one of the first display mode and the second display mode is a landscape mode, and the other display mode is a portrait mode; By employing a generative adversarial network with a contextual attention mechanism, the target detection results of the second image are processed to generate the image of the blank area.

2. The method according to claim 1, characterized in that, The target detection result includes the location and category of the target object in the second image.

3. The method according to claim 1 or 2, characterized in that, The step of processing the target detection results of the second image using a generative adversarial network employing a context attention mechanism to generate the image of the blank area includes: In the generator of the generative adversarial network, an image associated with the target detection result is generated in the blank area based on the context attention mechanism.

4. The method according to claim 3, characterized in that, In the generator of the generative adversarial network, generating an image associated with the target detection result in the blank area based on the context attention mechanism includes: In the generator, an attention map representing the similarity between the blank area and the known area in the second image is generated, and a feature map corresponding to the second image is generated based on the attention map. The feature map is processed to generate an image associated with the target detection result in the blank area.

5. The method according to claim 4, characterized in that, The generator includes an input layer, an encoder, and a decoder, and the encoder includes a context attention module; The generator generates an attention map representing the similarity between blank areas and known areas in the second image, and generates a feature map corresponding to the second image based on the attention map, including: The second image and its corresponding mask are received through the input layer, the mask being used to mark the blank areas in the second image; in the encoder, the attention map is generated through the context attention module, and the feature map is generated based on the attention map; The step of processing the feature map to generate an image associated with the target detection result in the blank area includes: processing the feature map through the decoder to generate an image associated with the target detection result in the blank area.

6. The method according to claim 1, characterized in that, Determining the blank areas in the second image includes: Convert the second image to a grayscale image; Based on the grayscale image, the outline of the target object in the second image is identified; Based on the outline of the target object in the second image, identify blank areas in the second image.

7. The method according to claim 6, characterized in that, The step of identifying the contour of the target object in the second image based on the grayscale image includes: Convert the grayscale image into a binary image; The outline of the target object in the second image is identified based on the binary image.

8. An electronic device, characterized in that, The electronic device includes a processor and a memory for storing computer programs that can run on the processor, wherein, The processor is used to run the computer program to perform the image processing method according to any one of claims 1 to 7.

9. A computer storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the image processing method according to any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the image processing method according to any one of claims 1 to 7.

Images