Image processing method, device and equipment and computer readable storage medium

Abstract

The invention relates to the technical field of terminals, and provides an image processing method, device and equipment and a computer readable storage medium, and the method comprises the steps: determining the semantic information of a main image in a to-be-processed image; based on the semantic information, determining a target background image having semantic association with the main body image; and fusing the main body image and the target background image to obtain a target image. Based on the technical scheme of the invention, the fusion of the target image can be improved, and the user experience is improved.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to an image processing method, apparatus, device and computer-readable storage medium. Background Art

[0002] At present, some image editing applications (Application, APP) support extracting portraits from the image to be processed, and fusing the portraits with a preset target background image to obtain a target image. However, since the style of the preset target background image may be inconsistent with the style of the portrait, the generated target image has poor fusion. Summary of the invention

[0003] The present application provides an image processing method, apparatus, device and computer-readable storage medium, which can improve the integration of target images and enhance user experience.

[0004] In a first aspect, an image processing method is provided, the method comprising: determining semantic information of a main image in an image to be processed; based on the semantic information, determining a target background image having a semantic association with the main image; and fusing the main image and the target background image to obtain a target image.

[0005] In an embodiment of the present application, semantic information represents the image semantics of the main image, that is, the meaning of the image content of the main image; based on the semantic information, a target background image having a semantic association with the main image is determined, so that the image content of the main image and the image content of the target background image are associated, and then the main image and the target background image are fused to obtain the target image, thereby improving the fusion of the target image and thus improving the user experience.

[0006] It should be understood that the image processing method of the present application is applied to an electronic device, which may be a server or a terminal. The semantic information may be text or keywords that characterize the image semantics of the subject image. The target background image may be generated based on the semantic information; the target background image may also be obtained by searching based on the semantic information.

[0007] In a possible implementation manner, determining the semantic information of the main image in the image to be processed may be achieved in the following manner: performing a first semantic analysis on the main image to obtain initial semantic information, and determining the semantic information based on the initial semantic information.

[0008] The initial semantic information includes a first keyword representing the image semantics of the main image and / or a text representing the image semantics of the main image.

[0009] In the embodiment of the present application, the initial semantic information can be used as the semantic information, and the semantic information can also be determined from the subject image based on the initial semantic information. In this way, the semantic information can be flexibly determined based on the initial semantic information, which improves the flexibility of determining the semantic information.

[0010] In one possible implementation, a first semantic analysis is performed on the main image to obtain initial semantic information, and semantic information is determined based on the initial semantic information. This is achieved by: performing a first semantic analysis on the main image to obtain a first keyword representing the image semantics of the main image; and determining the semantic information of the main image based on the first keyword.

[0011] In a possible implementation, a first semantic analysis is performed on the subject image to obtain a first keyword representing the image semantics of the subject image, which is achieved by using a first image description model to perform a first semantic analysis on the subject image to obtain the first keyword.

[0012] In one possible implementation, semantic information is determined based on a first keyword, and the step is implemented as follows: determining the first keyword as semantic information of a main image; determining a second keyword used to describe the first keyword, and performing a second semantic analysis on the main image based on the second keyword to obtain the semantic information of the main image; determining a second image description model corresponding to the first keyword, and performing a second semantic analysis on the main image using the second image description model to obtain semantic information, wherein the first image description model and the second image description model are different.

[0013] In a possible implementation, determining a second image description model corresponding to a first keyword, classifying the first keyword, and obtaining a category to which the first keyword belongs; and determining a vertical image description model corresponding to the category from multiple vertical image description models to obtain a second image description model may be implemented in the following manner:

[0014] The vertical domain image description model is an image description model that focuses on a specific person or field. The vertical domain image description model can be called an image description model in a vertical field. The number of the first keywords can be multiple.

[0015] In an embodiment of the present application, a vertical model classifier can be used to classify the first keyword to obtain the category to which the first keyword belongs and determine the model identification information based on the category. Then, the vertical image description model corresponding to the model identification information is determined from multiple vertical image description models to obtain the second image description model.

[0016] It should be noted that the number of the second image description model may be at least one. The vertical domain model classifier is used to output model identification information corresponding to the category to which the first keyword belongs according to the first keyword.

[0017] In a possible implementation, a plurality of first keywords may be classified by a vertical domain model classifier, a category to which the plurality of first keywords belong and model identification information corresponding to the category may be determined, and then a vertical domain image description model may be determined from a plurality of vertical domain image description models based on the model identification information to obtain a second image description model. The number of the second image description model is one.

[0018] In a possible implementation, a plurality of first keywords can be classified by a vertical domain model classifier to obtain a plurality of categories of first keywords, and a model identification information corresponding to each category is determined, and a vertical domain image description model corresponding to each model identification information is determined from a plurality of vertical domain image description models to obtain a second image description model. The number of the second image description models is multiple. The granularity of the second image description model is higher than the granularity of the first image description model.

[0019] Based on the above solution, a second semantic analysis can be performed on the subject image through at least one second image description model to obtain semantic information, thereby improving the accuracy of the obtained semantic information.

[0020] In combination with the first aspect, in certain implementations of the first aspect, determining a target background image having a semantic association with a main image based on semantic information can be achieved in the following ways: determining background description information based on the semantic information; generating a target background image based on the background description information; and / or determining an image matching the background description information from multiple background images in a background image library to obtain the target background image.

[0021] In an embodiment of the present application, the background description information may be text or background keywords. The background description information may be generated based on semantic information, and / or, based on semantic information, the background description information may be searched locally or on the Internet. The background image library may be a local image library having multiple background images, or a network image library having multiple background images.

[0022] Based on the above scheme, background description information that has semantic association with the main image can be determined based on semantic information, and then the target background image can be determined based on the background description information, so that the image content of the target background image and the image content of the main image are associated, thereby improving the fusion of the target image obtained after the subsequent fusion of the target background image and the main image.

[0023] In a possible implementation, determining the background description information based on the semantic information can be achieved in some ways: using a background keyword generator to generate background keywords based on the semantic information.

[0024] The background description information includes background keywords.

[0025] Specifically, the semantic information may be input into the background keyword generator so that the background keyword generator may output background keywords that have a semantic association with the subject image.

[0026] In a possible implementation, generating a target background image based on the background description information can be achieved in the following manner: using a background image generation model to generate the target background image based on the background description information.

[0027] Specifically, only the background description information can be input into the background image generation model, so that the background image generation model outputs the target background image. In this way, the process of generating the target background image is simplified, the complexity of generating the target background image is reduced, the time used for generating the target background image is reduced, and the efficiency of generating the target background image is improved.

[0028] In addition, both the background description information and the semantic information of the main image can be input into the background image generation model, so that the background image generation model can combine the semantic information of the main image and the background description information to generate the target background image, thereby improving the correlation between the image semantics of the generated target background image and the image semantics of the main image.

[0029] In a possible implementation, the electronic device is deployed with a Generative Pre-Trained Transformer (GPT), referred to as a GPT large model, and the semantic information of the main image in the image to be processed is determined by the GPT large model; the target background image having a semantic association with the main image is determined by the GPT large model based on the semantic information; the main image and the target background image are fused by the GPT large model to obtain the target image. In this way, a target background image having a semantic association with the main image is determined based on the semantic information of the main image in the image to be processed by a GPT large model, and the main image and the target background image are fused to obtain the target image, which reduces the complexity of obtaining the target image and reduces the complexity of deploying software on the electronic device.

[0030] In one possible implementation, the subject image and the target background image are fused to obtain the target image, which can be achieved in the following manner: the subject image is fused with each target background image to obtain a target image corresponding to each target background image; wherein each target background image has a different source.

[0031] Specifically, multiple methods can be used to determine the target background image, obtain the target background image corresponding to each method, and generate the target image corresponding to each method according to the subject image and the target background image determined in each method. Each method corresponds to a source.

[0032] Among them, the multiple methods include at least three methods, for example, the first method is to search for the target background image from the background image library on the electronic device based on the background description information; the second method is to search for the target background image from the background image library on the network based on the background description information; the third method is to generate the target background image through a background image generation model, at least based on the background description information.

[0033] Based on the above scheme, a target background image can be generated in multiple ways, and the target background image generated in each way is fused with the main image to obtain a target image, and at least one target image is displayed for user selection, thereby improving user experience.

[0034] In a possible implementation manner, the method further includes: in response to a switching operation on a target image, switching between a plurality of target images and displaying them.

[0035] Based on the above solution, the target image displayed on the display interface can be switched according to the switching operation, so that the user can select the desired image from multiple target images according to the actual application, thereby improving the user experience.

[0036] In one possible implementation, the image to be processed is obtained by at least one of the following methods: obtaining the image to be processed in the process of acquiring images by an image acquisition component; obtaining the image to be processed in the process of the electronic device outputting the display of the object within the framing range to the target device; obtaining the image to be processed when a browsing operation for an image in a gallery is detected; obtaining the image to be processed when a setting operation for the current background image of the display device is detected; obtaining the image to be processed when the image is edited by a target application.

[0037] Based on the above solution, at least one of the above methods can be used to obtain the image to be processed, which improves the flexibility of obtaining the image to be processed and enhances the user experience.

[0038] In a second aspect, an image processing device is provided, which includes: a determination unit, used to determine the semantic information of a main image in an image to be processed; the determination unit, also used to determine a target background image having a semantic association with the main image based on the semantic information; and a processing unit, used to fuse the main image and the target background image to obtain a target image.

[0039] In a possible implementation, the determination unit is specifically configured to perform the following steps: perform a first semantic analysis on the subject image to obtain a first keyword representing the image semantics of the subject image; and determine semantic information based on the first keyword.

[0040] In a possible implementation manner, the determination unit is specifically configured to execute the following steps: using a first image description model to perform a first semantic analysis on the subject image to obtain a first keyword.

[0041] In a possible implementation, the determination unit is specifically used to perform the following steps: determine that the first keyword is semantic information; determine a second keyword used to describe the first keyword, and perform a second semantic analysis on the subject image based on the second keyword to obtain semantic information; determine a second image description model corresponding to the first keyword, and use the second image description model to perform a second semantic analysis on the subject image to obtain semantic information. The first image description model and the second image description model are different.

[0042] In a possible implementation, the determination unit is specifically used to perform the following steps: classify the first keyword to obtain the category to which the first keyword belongs; determine the vertical image description model corresponding to the category from multiple vertical image description models to obtain the second image description model.

[0043] In one possible implementation, the determination unit is specifically used to perform the following steps: determine background description information based on semantic information; generate a target background image based on the background description information; and / or determine an image that matches the background description information from multiple background images in a background image library to obtain a target background image.

[0044] In a possible implementation manner, the determination unit is specifically configured to perform the following steps: using a background keyword generator to generate background keywords based on semantic information; wherein the background description information includes background keywords.

[0045] In a possible implementation manner, the determination unit is specifically configured to perform the following steps: using a background image generation model to generate a target background image based on the background description information.

[0046] In a possible implementation, the processing unit is specifically used to perform the following steps: fusing the subject image with each target background image to obtain a target image corresponding to each target background image; wherein each target background image has a different source;

[0047] In a possible implementation, the processing unit is further configured to perform the following steps: in response to a switching operation on a target image, switching between a plurality of target images and displaying them.

[0048] In a possible implementation, the determination unit is specifically used to perform at least one of the following steps: acquiring the image to be processed in the process of acquiring images through the image acquisition component; acquiring the image to be processed in the process of the electronic device outputting the display of the object within the framing range to the target device; acquiring the image to be processed when a browsing operation for an image in a gallery is detected; acquiring the image to be processed when a setting operation for the current background image of the display device is detected; and acquiring the image to be processed when the image is edited through the target application.

[0049] According to a third aspect, an electronic device is provided, comprising one or more processors and a memory, wherein the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code comprises computer instructions, and the one or more processors call the computer instructions to enable the electronic device to execute: determining semantic information of a subject image in an image to be processed; based on the semantic information, determining a target background image having a semantic association with the subject image; and fusing the subject image and the target background image to obtain a target image.

[0050] In a possible implementation, one or more processors call computer instructions to enable the electronic device to execute: performing a first semantic analysis on the subject image to obtain a first keyword representing the image semantics of the subject image; and determining semantic information based on the first keyword.

[0051] In a possible implementation, one or more processors call computer instructions to enable the electronic device to execute: using a first image description model, performing a first semantic analysis on the subject image, and obtaining a first keyword.

[0052] In one possible implementation, one or more processors call computer instructions to cause an electronic device to perform one of the following: determine a first keyword as semantic information; determine a second keyword used to describe the first keyword, and perform a second semantic analysis on a subject image based on the second keyword to obtain semantic information; determine a second image description model corresponding to the first keyword, and perform a second semantic analysis on the subject image using the second image description model to obtain semantic information, wherein the first image description model and the second image description model are different.

[0053] In one possible implementation, one or more processors call computer instructions to cause the electronic device to execute: classify the first keyword to obtain the category to which the first keyword belongs; determine the image description model corresponding to the category from multiple vertical domain image description models to obtain a second image description model.

[0054] In one possible implementation, one or more processors call computer instructions to cause the electronic device to execute: determining background description information based on semantic information; generating a target background image based on the background description information; and / or determining an image that matches the background description information from multiple background images in a background image library to obtain a target background image.

[0055] In a possible implementation, one or more processors invoke computer instructions to enable the electronic device to execute: using a background keyword generator to generate background keywords based on semantic information; wherein the background description information includes background keywords.

[0056] In a possible implementation, one or more processors invoke computer instructions to enable the electronic device to execute: using a background image generation model to generate a target background image based on background description information.

[0057] In one possible implementation, one or more processors call computer instructions to enable the electronic device to execute: fusing the subject image with each target background image to obtain a target image corresponding to each target background image; wherein each target background image has a different source.

[0058] In a possible implementation, one or more processors invoke computer instructions to cause the electronic device to execute: in response to a switching operation on a target image, switching between and displaying a plurality of target images.

[0059] In one possible implementation, one or more processors call computer instructions to cause the electronic device to perform at least one of the following steps: acquiring an image to be processed during the process of acquiring an image through an image acquisition component; acquiring an image to be processed during the process of the electronic device displaying an object within a viewing range and outputting it to a target device; acquiring an image to be processed when a browsing operation on an image in a gallery is detected; acquiring an image to be processed when a setting operation on a current background image of a display device is detected; and acquiring an image to be processed when an image is edited through a target application.

[0060] In a fourth aspect, a chip is provided. The chip is applied to an electronic device. The chip includes one or more processors. The processor is used to call computer instructions so that the electronic device executes the first aspect or any one of the image processing methods in the first aspect.

[0061] In a fifth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program code. When the computer program code is executed by an electronic device, the electronic device executes the first aspect or any one of the image processing methods in the first aspect.

[0062] In a sixth aspect, a computer program product is provided, the computer program product comprising: a computer program code, when the computer program code is executed by an electronic device, the electronic device executes the first aspect or any one of the image processing methods in the first aspect.

[0063] In an embodiment of the present application, the semantic information of the main image is taken into consideration, and a target background image having a semantic association with the main image is determined based on the semantic information of the main image, so that the image content of the main image and the image content of the target background image are correlated. The main image and the target background image are then fused to obtain a target image, thereby improving the fusion of the target image and solving the problem of poor fusion of the target image generated in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 is a structural schematic diagram of an electronic device provided in an embodiment of the present application;

[0065] Figure 2 is a software structure block diagram of the electronic device 100 according to an embodiment of the present application;

[0066] Figure 3 A schematic diagram of a display interface of an image editing APP provided in an embodiment of the present application;

[0067] Figure 4 A schematic diagram of a display interface of another image editing APP provided in an embodiment of the present application;

[0068] Figure 5 It is a flowchart of an image processing method provided in an embodiment of the present application;

[0069] Figure 6 is a flowchart of another image processing method provided in an embodiment of the present application;

[0070] Figure 7 It is a flowchart of another image processing method provided in an embodiment of the present application;

[0071] Figure 8 is a working schematic diagram of an intelligent analysis system provided by another embodiment of the present application;

[0072] Fig. 9 is a working schematic diagram of another intelligent analysis system provided by another embodiment of the present application;

[0073] Fig.10 is a flowchart of an image processing method provided by another embodiment of the present application;

[0074] Fig.11 is a schematic diagram of an image to be processed in an image processing method provided in an embodiment of the present application;

[0075] Fig.12 is a schematic diagram of a target background image in an image processing method provided in an embodiment of the present application;

[0076] Fig.13 is a schematic diagram of a target image generated in an image processing method provided in an embodiment of the present application;

[0077] Fig.14 is a structural schematic diagram of an image processing device provided by an embodiment of the present application;

[0078] Fig.15 It is a structural schematic diagram of an electronic device provided in another embodiment of the present application. DETAILED DESCRIPTION

[0079] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0080] In the description of the embodiments of the present application, unless otherwise specified, " / " means or; for example, A / B can mean A or B; "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships; for example, A and / or B can mean: A exists alone, or A and B exist at the same time, or B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0081] In the following, the terms "first", "second", and "third" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first", "second", and "third" may explicitly or implicitly include one or more of the features.

[0082] An embodiment of the present application provides an electronic device, which is used to execute the image processing method provided by the present application. In some embodiments of the present application, the electronic device can be a mobile phone, a tablet computer, a wearable device, a vehicle-mounted device, an augmented reality (AR) / virtual reality (VR) device, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), etc., or can be other devices or apparatuses capable of performing image processing. The embodiments of the present application do not impose any restrictions on the specific type of the electronic device.

[0083] For example, Figure 1It is a structural schematic diagram of an electronic device provided in an embodiment of the present application.

[0084] like Figure 1 As shown, the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0085] It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0086] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0087] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0088] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0089] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0090] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple groups of I2C buses. The processor 110 may be coupled to the touch sensor 180K, the charger, the flash, the camera 193, etc. through different I2C bus interfaces. For example: the processor 110 may be coupled to the touch sensor 180K through the I2C interface, so that the processor 110 communicates with the touch sensor 180K through the I2C bus interface, thereby realizing the touch function of the electronic device 100.

[0091] The I2S interface can be used for audio communication. In some embodiments, the processor 110 can include multiple I2S buses. The processor 110 can be coupled to the audio module 170 via the I2S bus to achieve communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 can transmit an audio signal to the wireless communication module 160 via the I2S interface to achieve the function of answering a call through a Bluetooth headset.

[0092] The PCM interface can also be used for audio communication, sampling, quantizing and encoding analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 can be coupled via a PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 via the PCM interface to implement the function of answering calls via a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

[0093] The UART interface is a universal serial data bus for asynchronous communication. The bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, the UART interface is generally used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 through the UART interface to implement the Bluetooth function.

[0094] In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 through the UART interface to achieve the function of playing music through Bluetooth headphones.

[0095] The MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a camera serial interface (CSI), a display serial interface (DSI), etc. In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to implement the shooting function of the electronic device 100. The processor 110 and the display screen 194 communicate via the DSI interface to implement the display function of the electronic device 100.

[0096] The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 with the camera 193, the display 194, the wireless communication module 160, the audio module 170, the sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0097] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present application is only a schematic illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0098] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0099] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Figure 1 The structures of antenna 1 and antenna 2 in the electronic device 100 are only examples. Each antenna in the electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of the antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0100] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0101] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0102] The electronic device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0103] ISP is used to process the data fed back by camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, ISP can be set in camera 193.

[0104] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0105] The digital signal processor is used to process digital signals, and can process not only digital image signals but also other digital signals. For example, when the electronic device 100 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0106] Video codecs are used to compress or decompress digital videos. The electronic device 100 may support one or more video codecs. Thus, the electronic device 100 may play or record videos in a variety of coding formats, such as moving picture experts group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0107] NPU is a neural network computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. Through NPU, applications such as intelligent cognition of the electronic device 100 can be realized, such as: image recognition, face recognition, voice recognition, text understanding, etc.

[0108] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function, such as storing music, video and other files in the external memory card.

[0109] The pressure sensor 180A is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A can be set on the display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. The capacitive pressure sensor can be a parallel plate including at least two conductive materials. When a force acts on the pressure sensor 180A, the capacitance between the electrodes changes. The electronic device 100 determines the intensity of the pressure according to the change in capacitance. When a touch operation acts on the display screen 194, the electronic device 100 detects the touch operation intensity according to the pressure sensor 180A. The electronic device 100 can also calculate the touch position according to the detection signal of the pressure sensor 180A. In some embodiments, touch operations acting on the same touch position but with different touch operation intensities can correspond to different operation instructions. For example: when a touch operation with a touch operation intensity less than the first pressure threshold acts on the short message application icon, an instruction to view the short message is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure threshold acts on the short message application icon, an instruction to create a new short message is executed.

[0110] The proximity light sensor 180G may include, for example, a light emitting diode (LED) and a light detector, such as a photodiode. The light emitting diode may be an infrared light emitting diode. The electronic device 100 emits infrared light outward through the light emitting diode. The electronic device 100 uses a photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the electronic device 100. When insufficient reflected light is detected, the electronic device 100 can determine that there is no object near the electronic device 100. The electronic device 100 can use the proximity light sensor 180G to detect that the user holds the electronic device 100 close to the ear to talk, so as to automatically turn off the screen to save power. The proximity light sensor 180G can also be used in leather case mode and pocket mode to automatically unlock and lock the screen.

[0111] The ambient light sensor 180L is used to sense the ambient light brightness. The electronic device 100 can adaptively adjust the brightness of the display screen 194 according to the perceived ambient light brightness. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the electronic device 100 is in a pocket to prevent accidental touches.

[0112] The fingerprint sensor 180H is used to collect fingerprints. The electronic device 100 can use the collected fingerprint characteristics to implement fingerprint unlocking, access application locks, fingerprint photography, fingerprint call answering, etc.

[0113] The touch sensor 180K is also called a "touch panel". The touch sensor 180K can be set on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, also called a "touch screen". The touch sensor 180K is used to detect touch operations acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In other embodiments, the touch sensor 180K can also be set on the surface of the electronic device 100, which is different from the position of the display screen 194.

[0114] The key 190 includes a power key, a volume key, etc. The key 190 may be a mechanical key or a touch key. The electronic device 100 may receive key input and generate key signal input related to user settings and function control of the electronic device 100.

[0115] The software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. The embodiment of the present application takes the Android system of the layered architecture as an example to exemplify the software structure of the electronic device 100.

[0116] Figure 2 It is a software structure block diagram of the electronic device 100 according to an embodiment of the present application.

[0117] It should be understood that the layered architecture can divide the software into several layers, each layer has a clear role and division of labor; the layers can communicate with each other through software interfaces.

[0118] like Figure 2 As shown, the Android system can be divided into four layers: from top to bottom, they are application layer, application framework layer, Android runtime and system library, and kernel layer. The application layer can include a series of application packages.

[0119] like Figure 2 As shown, the application package may include camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message and other applications.

[0120] The application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0121] like Figure 2 As shown, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, and the like.

[0122] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0123] Content providers are used to store and retrieve data and make it accessible to applications. The data can include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0124] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.

[0125] The phone manager is used to provide communication functions of the electronic device 100, such as management of call status (including connecting, hanging up, etc.).

[0126] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0127] The notification manager enables applications to display notification information in the status bar. It can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as notifications of applications running in the background, or a notification that appears on the screen in the form of a dialog window. For example, a text message is displayed in the status bar, a prompt sound is emitted, the terminal device vibrates, the indicator light flashes, etc.

[0128] Android runtime includes core libraries and virtual machines. Android runtime is responsible for scheduling and management of the Android system.

[0129] The core library consists of two parts: one part is the function that needs to be called by the Java language, and the other part is the Android core library.

[0130] The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0131] The system library may include multiple functional modules, such as: surface manager, media library, 3D graphics processing library (such as open graphics library for embedded systems (OpenGL ES)) and 2D graphics engine (such as skia graphics library (SGL)).

[0132] The surface manager is used to manage the display subsystem and provide the fusion of 2D and 3D layers for multiple applications.

[0133] The media library supports playback and recording of multiple audio formats, playback and recording of multiple video formats, and still image files. The media library can support multiple audio and video coding formats, such as: MPEG4, H.264, moving picture experts group audio layer III (MP3), advanced audio coding (AAC), adaptive multi-rate (AMR), joint photographic experts group (JPG) and portable network graphics (PNG).

[0134] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0135] A 2D graphics engine is a drawing engine for 2D drawings.

[0136] The kernel layer is the layer between hardware and software. The kernel layer contains at least display driver, camera driver, audio driver, and sensor driver.

[0137] For ease of understanding, the following examples of this application will be Figure 1 and Figure 2 Taking the electronic device shown as an example, the image processing method provided in the embodiment of the present application is specifically described in combination with the accompanying drawings and application scenarios.

[0138] At present, some image editing apps support using a target background image to replace the background image other than the portrait in the image to be processed to obtain the target image. The target background image can be a preset background template of the image editing app by default; of course, the target background image can also be a background template selected by user A from multiple preset background templates corresponding to the image editing app; the target background image can also be an image selected from multiple images stored locally by user A. The image editing app supports changing the color of the target background image within the app.

[0139] For example, Figure 3 A schematic diagram of a display interface of an image editing APP provided in an embodiment of the present application.

[0140] like Figure 3 As shown, an image 3 to be processed is displayed in the display interface of the image editing APP. User A can click the cutout button on the display interface, and then the image editing APP can use the default preset background template to replace the image except the portrait in the image to be processed, and obtain the following Figure 4 The target image 4 is displayed on the display interface of the image editing APP. Obviously, there is no connection between the image content of the default preset background template and the image content of the portrait, resulting in the problem of poor fusion of the target image generated based on the default preset background template and the portrait.

[0141] In view of this, the present application provides an image processing method, in which an electronic device can extract an image of an object with a human face from an acquired image to be processed to obtain a main image; determine the semantic information of the main image, and determine a target background image that has a semantic association with the main image based on the semantic information; fuse the main image and the target background image to obtain a target image; wherein the semantic information represents the meaning of the image content of the image to be processed. In this way, based on the determined semantic information of the main image, a target background image that has a semantic association with the main image is determined, so that the content of the main image and the content of the target background image are correlated, and then the main image and the target background image are fused to obtain the target image, thereby improving the fusion of the target image and solving the problem of poor fusion of the target image generated in the prior art.

[0142] Exemplarily, the electronic device may be a terminal or a server on the cloud side.

[0143] The following is a combination of application scenarios and Figure 5 to Figure 7 The image processing method provided in the embodiment of the present application is described in detail.

[0144] The image processing method provided in the embodiments of the present application is applied to at least one of the following scenarios: a photo shooting scenario, a video call scenario, a gallery browsing image scenario, a scene of setting the current background image of a display screen, or a scene of editing an image using an image editing APP. It should be understood that the image processing method provided in the present application can also be applied to any other reasonable scenario.

[0145] In a photo-taking scenario, user A can trigger a photo button on the image acquisition interface of the terminal. The terminal responds to the triggering operation of the photo button by user A, acquires the image to be processed within the framing range through the image acquisition component, and displays the acquired image to be processed on the image acquisition interface. Afterwards, user A can trigger the background change button on the image acquisition interface to trigger the process of determining the target background image based on the semantic information of the main image in the image to be processed, and fusing the main image and the target background image to generate the target image.

[0146] Among them, the electronic device can be a terminal, which can respond to the trigger operation of user A on the background change button on the image acquisition interface, extract the main image from the image to be processed based on the trigger operation, and determine the target background image based on the semantic information of the main image, and then fuse the main image and the target background image to obtain the target image, and display the target image.

[0147] For example, the terminal may be a mobile phone of user A. User A triggers the photo button on the photo interface (i.e., image acquisition interface) of the mobile phone camera APP. In response to the triggering operation of the photo button by user A, the mobile phone can collect the image of user A trying on a wedding dress (i.e., the image to be processed) through the camera (i.e., the image acquisition component), and display the image of user A trying on a wedding dress. User A can click the background change button on the photo interface. In response to the triggering operation of the background change button by user A, the mobile phone extracts the image of the area where user A is located (i.e., the main image) from the image of user A trying on a wedding dress, and determines the semantic information of the image of the area where user A is located (e.g., wedding dress, woman), and then determines the target background image that has a semantic association with the image of the area where user A is located based on the semantic information. For example, the target background image may be an image of a wedding scene, and the image of the area where user A is located and the image of the wedding scene may be fused to obtain the target image. In this way, the image content of the image of the area where user A is located in the image of user A trying on a wedding dress is associated with the image content of the image of the wedding scene, thereby improving the fusion of the generated target image.

[0148] When the electronic device is a server on the cloud side, the server can receive a processing instruction sent by the terminal based on the background change button on the image acquisition interface triggered by user A. In response to the processing instruction, the server extracts the subject image from the image to be processed; determines the semantic information of the subject image, and determines the target background image based on the semantic information of the subject image; fuses the subject image and the target background image to obtain the target image; and sends the target image to the terminal of user A, so that the terminal of user A displays the target image. The processing instruction is a processing instruction for instructing the server to replace the background of the image to be processed; and the processing instruction carries the image to be processed.

[0149] In a video call scenario, the terminal can transmit the captured video stream of user A to the target device of user B in real time; wherein, a background change button is displayed on the video call interface of the terminal. When user A triggers the background change button of the video call interface, the terminal can respond to user A's triggering operation on the background change button on the video call interface to trigger the process of changing the background of each image to be processed in user A's video stream.

[0150] Among them, the electronic device can be a terminal, which can respond to the user's triggering operation of the background change button on the video call interface, and for each image to be processed in the video stream of user A, extract the main image from the image to be processed, determine the semantic information of the main image, determine the target background image based on the semantic information, fuse the main image and the target background image to obtain the target image, and output the target image to the target device.

[0151] For example, the terminal may be a mobile phone of user A, and the video call scene may be a video conference scene. When user A is having a video conference with user B, the mobile phone may transmit the collected video stream of user A to the target device of user B in real time. The target device may be a laptop used by user B. If user A does not want to expose the privacy of the physical space where he is located (such as a study), he may trigger the background change button of the video conference interface. In response to the user's triggering operation of the background change button of the video conference interface, the mobile phone extracts the image of the area where user A is located from each frame of the collected video stream of user A (i.e., the image to be processed), and performs semantic analysis on the image of the area where user A is located to obtain semantic information (for example, it may be office, work), and determines a target background image that has a semantic association with the image based on the semantic information. For example, the target background image may be an image of an office space, and the image of the office space and the image of the area where user A is located are fused to obtain the target image, and then the target image is used to replace the image to be processed corresponding to the target image in the video stream, and the video stream replaced with the image to be processed is output to the target device of user B. In this way, the spatial privacy of the physical space where user A is located during the video conference is protected, and the integration of each frame of the video stream output to the target device is guaranteed, thereby improving the user experience.

[0152] The electronic device may also be a server, which may receive a processing instruction sent by the terminal based on the background change button of the video conference interface triggered by user A. In response to the processing instruction, the server determines the main image from each frame image (i.e., the image to be processed) in the video stream of user A, performs semantic analysis on the main image, obtains semantic information, determines the target background image based on the semantic information, fuses the main image and the target background image to generate the target image, replaces the corresponding image to be processed in the video stream with the target image, and sends the video stream replaced with the image to be processed to the terminal, so that the terminal can output the video stream replaced with the image to be processed to the target device of user B. The processing instruction is a processing instruction for instructing the server to replace the background of each frame image in the video stream of user A; the processing instruction carries the video stream of user A.

[0153] In the scenario of browsing images in the gallery, there is a background change button on the image browsing interface of the terminal. User A can trigger the background change button on the image browsing interface. The terminal responds to the triggering operation of the background change button on the image browsing interface by user A to trigger the process of determining the target background image based on the semantic information of the main image in the image currently displayed on the image browsing interface (i.e., the image to be processed), and fusing the main image and the target background image to generate the target image.

[0154] Among them, the electronic device can be a terminal, which can respond to the triggering operation of user A on the background change button on the image browsing interface, extract the main image from the image currently displayed on the image browsing interface, perform semantic analysis on the main image to obtain semantic information, determine the target background image that has a semantic association with the main image based on the semantic information, fuse the main image and the target background image to obtain the target image, and display the target image.

[0155] For example, the electronic device is a mobile phone of user A. When user A browses photos in an album (gallery) on the mobile phone, there is a background change button on the image browsing interface. User A can select a photo from the local gallery. For example, the photo can be a photo of user A watering flowers (i.e., the image to be processed). User A can trigger the background change button on the image browsing interface at this time. The electronic device responds to user A's triggering operation of the background change button on the image browsing interface, extracts a main image from the photo of user A watering flowers, and performs semantic analysis on the main image to obtain semantic information. The semantic information can be, for example, "woman", "kettle", or "flower". The electronic device can determine a target background image that has a semantic association with the main image based on the semantic information. For example, the target background image can be an image of a garden. The main image and the image of the garden are then merged to obtain a target image. The target image can be an image of a woman watering flowers in a garden.

[0156] The electronic device may also be a server, which may receive a processing instruction sent by the terminal based on the background change button on the image browsing interface triggered by user A. The server responds to the processing instruction, determines the target background image based on the semantic information of the main image in the image to be processed, and fuses the main image and the target background image to generate the target image, and sends the target image to the terminal of user A, so that the terminal of user A can display the target image. The processing instruction is a processing instruction for instructing the server to replace the background of the image to be processed; the processing instruction carries the image to be processed.

[0157] In the scenario of setting the background image of the display screen, the background setting interface of the terminal has a background change button, user A can trigger the background change button on the background setting interface, and the terminal can respond to the triggering operation of user A on the background change button on the background setting interface to trigger the process of setting the background image of the display screen. Among them, the process of setting the background image of the display screen is: determine the target background image based on the semantic information of the main image in the current background image (i.e., the image to be processed) on the display screen, or determine the target background image based on the semantic information of the main image in the default preset background template (the image to be processed), or determine the target background image based on the semantic information of the main image in the image to be processed input by user A; fuse the main image and the determined target background image to obtain the target image, so as to replace the current background image on the display screen with the target image.

[0158] Among them, the electronic device can be a terminal, which can respond to the triggering operation of user A on the background change button on the background setting interface, extract the main image from the obtained image to be processed, perform semantic analysis on the main image to obtain semantic information, determine the target background image based on the semantic information, and fuse the main image with the target background image to obtain the target image, and use the target image to replace the image to be processed.

[0159] For example, the terminal can be a laptop computer of user A. User A can select an image from the gallery of the laptop computer, such as an image of user A setting off fireworks (i.e., the image to be processed), extract the main image from the image of user A setting off fireworks, perform semantic analysis on the main image to obtain semantic information, such as the semantic information may be a woman and fireworks, and then determine a target background image associated with the main image based on the semantic information, such as the target background image may be an image of the night sky at the seaside, and fuse the main image and the target background image to obtain the target image, which may be an image of a woman setting off fireworks at the seaside at night.

[0160] The electronic device may also be a server, which may receive a processing instruction sent by the terminal based on the background change button on the background setting interface triggered by user A. The server responds to the processing instruction, determines the target background image based on the semantic information of the main image in the image to be processed, and fuses the main image and the target background image to generate the target image, and sends the target image to the terminal of user A, and the terminal of user A may display the target image. The processing instruction is a processing instruction for instructing the server to replace the background of the image to be processed; the processing instruction carries the image to be processed.

[0161] In a scenario where an image is edited using an image editing APP, the image editing interface of the terminal's image editing APP has a background change button, and user A can trigger the background change button on the background editing interface. The terminal responds to user A's triggering operation on the background change button to trigger a process of determining a target background image based on semantic information of a main image in an image to be processed currently displayed on the image editing interface, and fusing the main image and the target background image to generate a target image.

[0162] Among them, the electronic device can be a terminal, which can respond to the triggering operation of user A on the background change button of the image editing interface of the image editing APP, extract the main image from the image to be processed, determine the target background image based on the determined semantic information of the main image, and fuse the main image with the target background image to obtain the target image, and use the target image to replace the image to be processed.

[0163] For example, the electronic device may be a mobile phone of user A. User A may edit a selected image on an image editing app running on the mobile phone. For example, the image may be an image of a child doing homework. The main image may then be extracted from the image of the child doing homework, and semantic analysis may be performed on the main image to obtain semantic information. For example, the semantic information may be a child, a desk, or homework. The target background image determined based on the semantic information may be an image of a classroom. The main image and the image of the classroom may be fused to obtain a target image. The target image may be an image of a child doing homework in a classroom.

[0164] The electronic device may also be a server, which may receive a processing instruction sent by the terminal of user A based on the trigger operation of the background replacement button of the image editing interface of the image editing APP. The server responds to the processing instruction, determines the main image from the image to be processed, performs semantic analysis on the main image to obtain semantic information, determines the target background image based on the semantic information, fuses the main image and the target background image to obtain the target image, and sends the target image to the terminal of user A, so that the terminal of user A can display the target image. The processing instruction carries the image to be processed. The processing instruction is a processing instruction for instructing the server to replace the background of the image to be processed.

[0165] For example, Figure 5 This is an image processing method provided by the present application, which is applied to a terminal or a server. The method can be applied to any of the above multiple scenarios. The method may include S501 to S508. Each step is described below. It should be understood that Figure 5 In the example, the image to be processed is an image of user A trying on a wedding dress, but the present application is not limited to the image to be processed being an image of user A trying on a wedding dress. The image to be processed in the present application can be flexibly acquired according to the application scenario, and the embodiments of the present application do not specifically limit the image to be processed.

[0166] S501. Obtain an image of user A trying on a wedding dress. The image of user A trying on a wedding dress is the image to be processed.

[0167] In an embodiment of the present application, when the electronic device is a terminal, the image of user A trying on a wedding dress may be captured by an image capture component on the terminal; the image of user A trying on a wedding dress may also be sent to the terminal by other devices; of course, the image of user A trying on a wedding dress may also be determined in response to user A's selection operation on multiple images stored on the terminal.

[0168] When the electronic device is a server, the image of user A trying on a wedding dress may be sent to the server by a terminal or other device; of course, the image of user A trying on a wedding dress may also be downloaded by the server through the network.

[0169] S502: extracting a subject image from the image to be processed.

[0170] In the embodiment of the present application, the electronic device can perform a cutout process on the image of user A trying on a wedding dress to obtain an image of the area where user A is located. The main image is the image of the area where user A is located.

[0171] Exemplarily, the algorithm used for the cutout processing may be the U2Net algorithm, and of course may be other algorithms. The specific algorithm used for the cutout processing is not limited in the embodiments of the present application.

[0172] S503: Perform semantic analysis on the main image to obtain semantic information of the main image.

[0173] The semantic information represents the image semantics of the subject image, that is, the meaning of the image content of the subject image. The semantic information can be text representing the meaning of the image content of the subject image, such as a sentence; the semantic information can also be a keyword representing the meaning of the image content of the subject image.

[0174] In the embodiment of the present application, an image description model may be used to perform semantic analysis on the subject image to obtain semantic information of the subject image, wherein the image description model may be pre-trained based on sample subject images and sample semantic information.

[0175] Exemplarily, the image description model can be an Image Caption model, and the subject image can be input into the Image Caption model. The Image Caption model performs semantic analysis on the subject image and outputs a sentence representing the meaning of the image content of the subject image, for example, it can be "woman wearing a wedding dress", or the Image Caption model outputs keywords representing the meaning of the image content of the subject image, such as "woman" and "clothing", or "woman" and "trying on clothes".

[0176] S504: Determine background description information associated with the semantics of the subject image based on the semantic information.

[0177] The background description information may be text associated with the semantics of the main image; the background description information may also be background keywords associated with the semantics of the main image.

[0178] In the embodiment of the present application, the background description information having a semantic association with the main image can be searched from the network based on the semantic information; the background description information having a semantic association with the main image can also be determined from the preset background description information stored in the electronic device based on the semantic information; and the background description information can also be generated based on the semantic information. In this way, the flexibility of determining the background description information based on the semantic information is improved.

[0179] Exemplarily, the semantic information may be keywords, such as "young", "female", "black hair", "wedding dress", "elegant", "romantic", "princess", "white", "train", "high shoulder", "crystal", "sequins", "bow" and "backless"; the background description information determined based on the semantic information may be background keywords, such as the background keywords may be "wedding", "white wedding dress", "train wedding dress", "bow", "romance" and "diamond".

[0180] It should be noted that at least one of steps S505, S506 and S507 may be executed after S504.

[0181] S505: Search for an image matching the background description information from a plurality of background images stored in a background image library on the electronic device to obtain a target background image.

[0182] For example, the matching degree between the background description information and the preset background description information corresponding to each background image can be calculated, and the background image corresponding to the maximum matching degree can be determined from multiple matching degrees as the target background image; of course, at least one target matching degree greater than a preset matching degree threshold can also be determined from multiple matching degrees, and the background image corresponding to the target matching degree can be used as the target background image. In this way, based on the background description information, the target background image is searched from multiple background images stored in the background image library on the electronic device, which saves the time of obtaining the target background image and improves the efficiency of obtaining the target background image.

[0183] It should be noted that S508 may be executed after S505.

[0184] S506: Search for an image matching the background description information from a plurality of background images stored in a background image library on the network to obtain a target background image.

[0185] It should be noted that searching for a target background image from a plurality of background images stored in a background image library on the network expands the search range, increases the number of target background images obtained, and improves the accuracy of the target background images obtained.

[0186] It should be noted that S508 can be executed after S506.

[0187] S507: Using a background image generation model, generate a target background image based on the background description information.

[0188] Among them, the background image generation model can be a preset text image model trained by a generative artificial intelligence (Artificial Intelligence Generated Content, AIGC) algorithm.

[0189] In an embodiment of the present application, the background description information may be input into a background image generation model, so that the background image generation model generates a target background image according to the background description information.

[0190] Of course, both the background description information and the semantic information of the subject image may be input into the background image generation model, so that the background image generation model generates a target background image according to the background description information and the semantic information of the subject image.

[0191] It should be noted that the background image generation model is used to generate the target background image based on the background description information, which improves the correlation between the image content of the target background image and the image content of the main image, and further improves the fusion of the target image obtained by subsequently fusing the target background image and the main image.

[0192] It should be noted that S508 can be executed after S507.

[0193] S508: Fusing the subject image and the target background image to obtain a target image.

[0194] In an embodiment of the present application, the attribute information of the background objects in the target background image and the position information of the background objects in the target background image can be determined, and the size of the main image can be adjusted based on the attribute information and the position information of the background objects in the target background image; the target position information of the main image in the target background image can be determined based on the attribute information and the position information of the background objects in the target background image; and based on the target position information, the resized main image can be integrated into the target background image to obtain the target image.

[0195] The attribute information includes at least one of parameters such as the shape of the background object, the size of the background object in the target background image, and the color.

[0196] Based on the above scheme, the attribute information of the background objects in the target background image and the position information of the background objects in the target background image are considered, and the main image and the target background image are fused to obtain the target image, thereby improving the fusion of the target image.

[0197] Exemplarily, the main image is an image of the area where user A is located in an image of user A trying on a wedding dress, and the target background image may be an image of a wedding scene. Based on the size and position of the stage, the size and position of the hall, the size and position of the dining chairs, and the size and position of the ceiling in the image of the wedding scene, the size of the main image may be adjusted and the target position information of the main image in the target background image may be determined. Then, based on the determined target position information of the main image in the target background image, the main image may be integrated into the target background image to obtain the target image.

[0198] It should be noted that when the electronic device is a terminal, the terminal can output the target image and display it, or send the target image to other devices so that the other devices display the target image. When the electronic device is a server, after obtaining the target image, the target image can be sent to the terminal, and the terminal can display the target image.

[0199] An image processing method provided in an embodiment of the present application takes into account the semantic information of a main image, determines a target background image having a semantic association with the main image based on the semantic information of the main image, makes the image content of the main image and the image content of the target background image have correlation, and then fuses the main image and the target background image to obtain a target image, thereby improving the fusion of the target image and solving the problem of poor fusion of the target image generated in the prior art.

[0200] For example, Figure 6 This is an image processing method provided by the present application, which is applied to a terminal or a server. The method can be applied to any of the above multiple scenarios. The method may include S601 to S608. Each step is described below. It should be understood that Figure 6 In the example, the image to be processed is an image of user A trying on a wedding dress, but the present application is not limited to the image to be processed being an image of user A trying on a wedding dress. The image to be processed in the present application can be flexibly acquired according to the application scenario, and the embodiments of the present application do not specifically limit the image to be processed.

[0201] S601. Acquire an image of user A trying on a wedding dress. The image of user A trying on a wedding dress is the image to be processed.

[0202] It should be noted that the implementation process of S601 is the same as the implementation process of S501. The specific implementation process of S601 can be implemented by referring to the specific process of S501, and the embodiments of the present application will not be repeated here.

[0203] S602: extracting a subject image from the image to be processed.

[0204] It should be noted that the implementation process of S602 is the same as the implementation process of S502. The specific implementation process of S602 can be implemented by referring to the specific process of S502, and the embodiments of the present application will not be repeated here.

[0205] It should be noted that S503 in the above embodiment can be implemented by S603:

[0206] S603: Use the first image description model to perform a first semantic analysis on the main image to obtain a first keyword representing the image semantics of the main image.

[0207] The semantic analysis at least includes a first semantic analysis. The semantic information includes a first keyword. The image description model includes a first image description model, and the first image description model can be called a coarse-grained general image caption model.

[0208] In the embodiment of the present application, the subject image can be input into the coarse-grained general image caption model, so that the coarse-grained general image caption model performs a first semantic analysis on the subject image and outputs a first keyword representing the image semantics of the subject image; for example, ("woman" and "clothing"). The first keyword can also be, for example, "woman" and "fitting clothes".

[0209] It should be noted that S504 in the above embodiment can be implemented by the following S604:

[0210] S604: Process the first keyword to obtain a background keyword through a background keyword generator.

[0211] In the embodiment of the present application, the first keyword can be input into the background keyword generator, and the background keyword generator can analyze the first keyword to generate the background keyword. In this way, the background keyword generator analyzes the first keyword representing the image semantics of the subject image to generate the background keyword, thereby improving the correlation between the generated background keyword and the image semantics of the subject image, and further improving the accuracy of the target background image subsequently determined based on the background keyword.

[0212] The background keyword generator is obtained by training the sample keywords and the sample background keywords corresponding to the sample keywords. The background keyword generator includes an AIGC background keyword generator; the AIGC background keyword generator can be obtained by training the sample keywords and the sample background keywords corresponding to the sample keywords using the AIGC algorithm.

[0213] Exemplarily, the first keywords may be "women" and "clothing", and "women" and "clothing" may be input into the background keyword generator, which may analyze "women" and "clothing" and output background keywords associated with the semantics of the main image, such as "indoor" and "fashion".

[0214] It should be noted that at least one of steps S605, S606 and S607 may be executed after S604.

[0215] S605: Search for an image matching the background keyword from a plurality of background images stored in a background image library on the electronic device to obtain a target background image.

[0216] In the embodiment of the present application, a local searcher is deployed on the electronic device, and the local searcher determines the background image matching the background keywords ("indoor" and "fashion") from the multiple background images stored in the background image library on the electronic device to obtain the target background image. In this way, the target background image can be determined from the multiple background images stored in the electronic device, which saves the time required to obtain the target background image and improves the efficiency of obtaining the target background image.

[0217] Exemplarily, the terminal stores preset background keywords corresponding to each background image, wherein the preset background keywords represent the meaning of the image content of the background image. The matching degree of "indoor" and "fashion" with the preset background keywords corresponding to each background image can be calculated, and then the background image corresponding to the maximum matching degree is determined from multiple matching degrees to obtain the target background image.

[0218] It should be noted that S608 can be executed after S605.

[0219] S606: Search for an image matching the background keyword from a plurality of background images stored in a background image library on the network to obtain a target background image.

[0220] In the embodiment of the present application, a browser (searcher) is deployed on the electronic device, and the browser (searcher) can be used to search for background images that match background keywords ("indoor" and "fashion") from multiple background images on the Internet to obtain a target background image. In this way, the target background image can be obtained by searching the Internet for background images that match the background keywords, which expands the search scope, increases the number of target background images obtained, and improves the accuracy of the target background images obtained.

[0221] It should be noted that S608 may be executed after S606.

[0222] It should be noted that S507 in the above embodiment can be implemented by the following S607:

[0223] S607: Using a background image generation model, a target background image is generated based on background keywords and semantic information of the subject image.

[0224] The background description information includes background keywords.

[0225] The background image generation model is obtained by training sample background keywords and sample semantic information of sample subject images and sample background images. The background image generation model can also be called a cultural image model. The background image generation model includes an AIGC background image generation model; the AIGC background image generation model can be obtained by training sample background keywords and sample semantic information of sample subject images and sample background images using an AIGC algorithm.

[0226] Exemplarily, the background keywords may be "indoor" and "fashion", and the semantic information of the main image may be "woman" and "clothing"; a background image generation model is deployed on the electronic device, and "woman", "clothing", "indoor" and "fashion" may be input into the background image generation model, and the background image generation model may generate and output a target background image based on "woman", "clothing", "indoor" and "fashion".

[0227] It should be noted that a background image generation model is used to analyze the semantic information of background keywords and the main image to obtain a target background image, which improves the content relevance between the target background image and the main image, and further improves the fusion of the target image obtained by subsequently fusing the target background image and the main image.

[0228] It should be noted that S608 can be executed after S607.

[0229] S608: Fusing the subject image and the target background image to obtain a target image.

[0230] It should be noted that the implementation process of S608 is the same as the implementation process of S508. The specific implementation process of S608 can be implemented by referring to the specific process of S508, and the embodiments of the present application will not be repeated here.

[0231] It should be noted that when the electronic device is a terminal, the terminal can output the target image and display it, or send the target image to other devices so that the other devices can display the target image. When the electronic device is a server, after obtaining the target image, the target image can be sent to the terminal, and the terminal can display the target image.

[0232] An image processing method provided in an embodiment of the present application takes into account the semantic information of a main image, determines a target background image having a semantic association with the main image based on the semantic information of the main image, makes the content of the main image and the content of the target background image relevant, and then fuses the main image and the target background image to obtain a target image, thereby improving the fusion of the target image and solving the problem of poor fusion of the target image generated in the prior art.

[0233] For example, Figure 7 This is an image processing method provided by the present application, which is applied to a terminal or a server. The method can be applied to any of the above multiple scenarios. The method may include S701 to S710. Each step is described below. It should be understood that Figure 7 In the example, the image to be processed is an image of user A trying on a wedding dress, but the present application is not limited to the image to be processed being an image of user A trying on a wedding dress. The image to be processed in the present application can be flexibly acquired according to the application scenario, and the embodiments of the present application do not specifically limit the image to be processed.

[0234] S701. Obtain an image of user A trying on a wedding dress. The image of user A trying on a wedding dress is the image to be processed.

[0235] It should be noted that the implementation process of S701 is the same as the implementation process of S501. The specific implementation process of S701 can be implemented by referring to the specific process of S501, and the embodiments of the present application will not be repeated here.

[0236] S702: extracting a subject image from the image to be processed.

[0237] It should be noted that the implementation process of S702 is the same as the implementation process of S502. The specific implementation process of S702 can be implemented by referring to the specific process of S502, and the embodiments of the present application will not be repeated here.

[0238] It should be noted that S503 in the above embodiment can be implemented by S703; S503 can also be implemented by S703 and S704; S503 in the above embodiment can also be implemented by S703 and S705:

[0239] S703: Using a first image description model, perform a first semantic analysis on the subject image to obtain a first keyword representing the image semantics of the subject image.

[0240] Among them, the semantic information may be the first keyword.

[0241] It should be noted that the implementation process of S703 is the same as the implementation process of S603. The specific implementation process of S703 can be implemented by referring to the specific process of S603, and the embodiments of the present application will not be repeated here.

[0242] It should be noted that any one of steps S704, S705 and S706 may be executed after S703.

[0243] S704: Determine a second keyword used to describe the first keyword, and perform a second semantic analysis on the main image based on the second keyword to obtain semantic information.

[0244] The second keyword may be a keyword that has a semantic association with the first keyword.

[0245] In an embodiment of the present application, a preset keyword set corresponding to the first keyword can be determined from multiple preset keyword sets, and the keyword in the preset keyword set corresponding to the first keyword is used as the second keyword. The first keyword may be, for example, "woman" and "clothing", then the second keyword corresponding to "woman" may be, for example, "hair color" and "skin color", and the second keyword corresponding to "clothing" may be, for example, "style", "color" and "material". A second semantic analysis may be performed on the subject image based on "hair color", "skin color", "style", "color" and "material" to obtain semantic information, and the semantic information may be a target keyword that characterizes the image semantics of the subject image, such as "black hair", "yellow", "elegant" and "crystal and sequins".

[0246] It should be noted that, based on the second keyword, a second semantic analysis is performed on the main image, thereby improving the precision and accuracy of the obtained semantic information.

[0247] It should be noted that S706 can be executed after S704.

[0248] S705: Determine a second image description model corresponding to the first keyword, and use the second image description model to perform a second semantic analysis on the main image to obtain semantic information.

[0249] The semantic analysis may include a first semantic analysis and a second semantic analysis; and the image description model in the aforementioned embodiment may include a first image description model and a second image description model.

[0250] In an embodiment of the present application, a vertical domain model classifier is deployed on the electronic device. The vertical domain model classifier can be used to classify the first keyword to obtain the category to which the first keyword belongs and determine the model identification information corresponding to the first keyword based on the category. After that, the vertical domain image description model corresponding to the model identification information can be determined from multiple vertical domain image description models to obtain a second image description model. The second image description model is used to perform a second semantic analysis on the main image to obtain semantic information.

[0251] Among them, the vertical domain model classifier can be called a fine-grained vertical domain model classifier; the first keyword can be input into the fine-grained vertical domain model classifier, and the fine-grained vertical domain model classifier can output model identification information corresponding to the category to which the first keyword belongs. The second image description model can be called a fine-grained vertical domain Image Caption model. The second image description model can be trained using sample image description keywords and sample fine-grained description keywords.

[0252] Exemplarily, the first keyword includes "woman" and "clothing", and then "woman" and "clothing" are input into the fine-grained vertical model classifier, so that the fine-grained vertical model classifier can analyze "woman" and "clothing" to determine the category to which "woman" and "clothing" belong, and the fine-grained vertical model classifier can output the model identification information corresponding to the category. Afterwards, the fine-grained vertical Image Caption model corresponding to the model identification information can be called, and the fine-grained vertical Image Caption model can be used to perform a second semantic analysis on the subject image to obtain semantic information; wherein, the semantic information can be a target keyword that characterizes the image semantics of the subject image. Among them, the model identification information can be a model number or a model name, and the model identification information is used to uniquely refer to a vertical image description model. The model identification information can be called model index information.

[0253] It should be noted that S706 can be executed after S705.

[0254] S706: Use a background keyword generator to analyze the semantic information to obtain background keywords.

[0255] In the embodiment of the present application, the semantic information may be input into the background keyword generator, so that the background keyword generator analyzes the semantic information to obtain background keywords.

[0256] Exemplarily, the background keyword generator is an AIGC background keyword generator, and semantic information may be, for example, "black hair", "yellow skin", "elegant", "white", "crystal and sequins". "Black hair", "yellow skin", "elegant", "white", "crystal and sequins" may be input into the AIGC background keyword generator, and the AIGC background keyword generator may output background keywords.

[0257] It should be noted that at least one of steps S707, S708 and S709 may be executed after S706.

[0258] S707: Search for an image matching the background keyword from a plurality of background images stored in a background image library on the electronic device to obtain a target background image.

[0259] It should be noted that the implementation process of S707 is the same as the implementation process of S605. The specific implementation process of S707 can be implemented by referring to the specific process of S605, and the embodiments of the present application will not be repeated here.

[0260] It should be noted that S710 may be executed after S707.

[0261] S708: Search for an image matching the background keyword from a plurality of background images stored in a background image library on the network to obtain a target background image.

[0262] It should be noted that the implementation process of S708 is the same as the implementation process of S606. The specific implementation process of S708 can be implemented by referring to the specific process of S606, and the embodiments of the present application will not be repeated here.

[0263] It should be noted that S710 may be executed after S708.

[0264] S709: Using a background image generation model, a target background image is generated based on background keywords and semantic information of the subject image.

[0265] In an embodiment of the present application, the background description information and the semantic information of the subject image can be directly input into the background image generation model, so that the background image generation model analyzes the background keywords and semantic information to generate a target background image; it is also possible to generate a target processing instruction based on the semantic information and the background description information, and transmit the target processing instruction to the background image generation model, so that the background image generation model responds to the target processing instruction and generates a target background image; wherein the target processing instruction is an instruction for instructing the background image generation model to generate a target background image, and the target processing instruction carries background keywords and target keywords. The background image generation model is an AIGC background image generation model; it is also possible to generate a prompt text based on the background description information and the semantic information, and input the prompt text into the background image generation model, so that the background image generation model outputs the target background image. In this way, the flexibility of generating the target background image is improved. wherein the prompt text is a text that characterizes the meaning of the background description information and the meaning of the semantic information.

[0266] S710: Fusing the target background image with the subject image to obtain a target image.

[0267] It should be noted that when the electronic device is a terminal, the terminal can output and display the target image, or send the target image to other devices so that the other devices can display the target image. When the electronic device is a server, after obtaining the target image, the target image can be sent to the terminal, and the terminal can display the target image.

[0268] It should be noted that the implementation process of S710 is the same as the implementation process of S508. The specific implementation process of S710 can be implemented by referring to the specific process of S508, and the embodiments of the present application will not be repeated here.

[0269] An image processing method provided in an embodiment of the present application takes into account the semantic information of a main image, determines a target background image having a semantic association with the main image based on the semantic information of the main image, makes the content of the main image and the content of the target background image relevant, and then fuses the main image and the target background image to obtain a target image, thereby improving the fusion of the target image and solving the problem of poor fusion of the target image generated in the prior art.

[0270] Based on the foregoing embodiments, in other embodiments of the present application, the electronic device may be a server on the cloud side, and a large GPT model is deployed on the server.

[0271] Exemplarily, the server can perform cutout processing on the obtained image to be processed through the GPT large model to obtain the main image, and perform semantic analysis on the main image to obtain semantic information. The GPT large model can output background description information associated with the semantics of the main image based on the semantic information; the server can generate and output a target background image based on the background description information through the GPT large model; or, the server can search for an image that matches the background description information from multiple stored images based on the semantic information to obtain the target background image, or, based on the background description information, search for an image that matches the background description information through the network to obtain the target background image, or, an AIGC background image generation model is also deployed on the server, and the AIGC background image generation model can be used to analyze the background image description information and the semantic information of the main image to generate the target background image.

[0272] It should be noted that, in the aforementioned multiple embodiments, multiple methods can be used to determine the target background image to obtain the target background image corresponding to each method. The target image corresponding to each method is generated based on the main image and the target background image determined by each method. The multiple generated target images can be switched and displayed in response to the switching operation of user A.

[0273] Among them, the multiple methods include at least three methods, for example, the first method is to search for the target background image from the background image library on the electronic device based on the background description information; the second method is to search for the target background image from the background image library on the Internet based on the background description information; the third method is to generate the target background image based on at least the background description information through the AIGC background image generation model.

[0274] For example, the display interface of the electronic device includes a button for displaying a target image generated according to a first method, a button for displaying a target image generated according to a second method, and a button for displaying a target image generated according to a third method; the display interface can display the target image generated according to the first method by default, and when user A clicks the button for displaying the target image generated according to the second method (i.e., a switching operation), the electronic device can display the target image generated according to the second method in response to the switching operation.

[0275] Based on the above scheme, the target image generated according to the main image and the target background image determined from different sources (different methods) displayed on the display interface can be switched according to the switching operation of user A, so that user A can select the required image from multiple target images according to actual application, thereby improving the user experience.

[0276] Based on the above embodiments, the present application also provides an intelligent analysis system. The above image processing method can be applied to the intelligent analysis system. Figure 8 As shown, the intelligent analysis system 8 includes a terminal 81 and a server 82 on the cloud side; wherein, a GPT large model is deployed on the server 82, and a coarse-grained general Image Caption model is deployed on the terminal 81. The following takes the image to be processed as an example of user A trying on a wedding dress to explain the working process of the intelligent analysis system 8 in detail.

[0277] For example, Figure 8 is a schematic diagram of a workflow of an intelligent analysis system provided by the present application. The method can be applied to any of the above multiple scenarios. The method may include S801 to S807. Figure 8 As shown, the step includes:

[0278] S801, the terminal 81 and the server 82 can both obtain an image of user A trying on a wedding dress.

[0279] S802. The server 82 may send a first processing instruction for describing the image content of the main image in the image of the user A trying on the wedding dress to the terminal 81 through the GPT large model.

[0280] S803, terminal 81 can respond to the first processing instruction, call the coarse-grained general Image Caption model to perform a first semantic analysis on the main image, obtain the first keyword (for example, women / clothing), use the first keyword as semantic information, and then input the semantic information into the background keyword generator to obtain background keywords.

[0281] It should be noted that at least one of steps S804, S805, and S806 may be executed after S803.

[0282] S804. The terminal 81 may search for a target background image from multiple background images on the electronic device based on background keywords through a local searcher.

[0283] It should be noted that S807 can be executed in S804.

[0284] S805 . The terminal 81 searches for a target background image from multiple background images on the Internet based on background keywords through a browser.

[0285] It should be noted that S807 can be executed in S805.

[0286] S806. The terminal 81 generates a target background image based on at least the background keywords by using a background image generation model.

[0287] It should be noted that S807 can be executed in S806.

[0288] S807. Terminal 81 fuses the target background image with the main image to obtain a target image.

[0289] It should be noted that, for the description of the same steps and the same contents in the embodiment of the present application as in other embodiments, reference can be made to the description in other embodiments and will not be repeated here.

[0290] It should be noted that the server 82 sends a first processing instruction to the terminal 81. After receiving the first processing instruction sent by the server 82, the terminal 81 calls the coarse-grained general Image Caption model in response to the first processing instruction to perform a first semantic analysis on the main image, and obtains the first keyword, so as to execute the subsequent process of generating the target image based on the first keyword.

[0291] Of course, exemplarily, after receiving the first processing instruction sent by the server 82, the terminal 81 responds to the first processing instruction, uses the keyword obtained by performing a first semantic analysis on the main image as the first initial keyword, and sends the first initial keyword to the server 82. When the server 82 determines that the first initial keyword meets the target condition through the GPT large model, a prompt message indicating that the first initial keyword meets the target condition is sent to the terminal 81, so that the terminal 81 can use the first initial keyword as the first keyword to execute the subsequent process of generating the target image.

[0292] When the server 82 determines that the first initial keyword does not meet the target condition through the GPT large model, a second processing instruction for instructing the terminal 81 to perform a first semantic analysis on the main image can be generated based on the first initial keyword, and the second processing instruction is sent to the terminal 81, so that the terminal 81 performs a first semantic analysis on the main image based on the second processing instruction to obtain the second initial keyword, and sends the second initial keyword to the server 82. The server 82 determines whether the second initial keyword meets the target condition through the GPT large model. When it is determined that the target condition is met, a prompt message that the second initial keyword meets the target condition is sent to the terminal 81, so that the terminal 81 can use the second initial keyword as the first keyword to execute the subsequent process of generating the target image. If the second initial keyword does not meet the target condition, a third processing instruction is generated based on the second initial keyword until the server 82 determines that the Nth initial keyword sent by the terminal 81 meets the target condition, and sends a prompt message that the Nth initial keyword meets the target condition to the terminal, so that the terminal 81 can use the Nth initial keyword as the first keyword to execute the subsequent process of generating the target image. Wherein, N is a positive integer greater than 1. Exemplarily, the target condition may be that the number of keywords is greater than the first target number threshold.

[0293] It should be noted that after obtaining the first keyword, the terminal 81 can also call the fine-grained vertical domain model classifier deployed on the terminal 81 to classify the first keyword to obtain the category to which the first keyword belongs and determine the model identification information corresponding to the first keyword based on the category, and then determine the vertical domain image description model corresponding to the model identification information from multiple vertical domain image description models to obtain the second image description model, and perform a second semantic analysis on the subject image through the second image description model to obtain semantic information. The semantic information can be a target keyword that represents the image semantics of the subject image.

[0294] For example, Fig. 9 It is a schematic diagram of the workflow of another intelligent analysis system provided in the present application. The method can be applied to any of the above-mentioned multiple scenarios. The method may include S901 to S913.

[0295] S901 , the terminal 81 and the server 82 may both obtain an image of user A trying on a wedding dress.

[0296] S902. The server 82 may send a first processing instruction for describing the image content of the main image in the image of the user A trying on the wedding dress to the terminal 81 through the GPT large model.

[0297] S903. The terminal 81 may respond to the first processing instruction by calling the coarse-grained general Image Caption model to perform a first semantic analysis on the subject image to obtain a first keyword (eg, woman / clothing).

[0298] S904 . After obtaining the first keyword, the terminal 81 may send the first keyword to the server 82 .

[0299] S905. After receiving the first keyword, the server 82 sends a second processing instruction for performing a second semantic analysis on the main image to the terminal 81 when determining through the GPT large model that the number of the first keywords is greater than the first target number threshold.

[0300] S906. The terminal 81 may call the fine-grained vertical domain model classifier based on the second processing instruction to determine the model identification information corresponding to the first keyword, and then call the second image description model corresponding to the model identification information to perform a second semantic analysis on the main image to obtain semantic information.

[0301] S907 . The terminal 81 sends semantic information to the server 82 .

[0302] S908. When the server 82 determines through the GPT large model that the number of keywords in the semantic information is greater than the second target number threshold, the server 82 sends a third processing instruction to the terminal 81 for instructing to generate background keywords based on the semantic information.

[0303] S909, terminal 81 can respond to the third processing instruction and call the background keyword generator deployed on terminal 81 to generate background keywords based on semantic information, or call a local searcher to search for background keywords based on semantic information, or call a browser to search for background keywords from the Internet based on semantic information.

[0304] It should be noted that at least one of steps S910, S911, and S912 may be executed after S909.

[0305] S910. The terminal 81 may search for a target background image based on background keywords through a local searcher.

[0306] S911. The terminal 81 searches for a target background image on the Internet based on background keywords through a browser.

[0307] S912. The terminal 81 generates a target background image based on background keywords using a background image generation model.

[0308] S913. The terminal 81 merges the target background image with the main image to obtain a target image.

[0309] It should be noted that after the terminal 81 obtains the background keywords, the background keywords can also be sent to the server 82. When the server 82 determines through the GPT large model that the number of background keywords is greater than the third target number threshold, it generates a prompt text based on the semantic information and the background keywords, and sends the prompt text to the terminal 81. After receiving the prompt text, the terminal 81 calls the deployed AIGC background image generation model to process the prompt text and obtain the target background image.

[0310] It should be noted that, for the description of the same steps and the same contents in the embodiment of the present application as in other embodiments, reference can be made to the description in other embodiments and will not be repeated here.

[0311] The intelligent analysis system provided by the embodiment of the present application determines the semantic information of the main image in the image to be processed; based on the semantic information, determines the target background image that has a semantic association with the main image; and fuses the main image and the target background image to obtain the target image. In this way, the target background image that has a semantic association with the main image is determined based on the determined semantic information of the main image, so that the content of the main image and the content of the target background image are correlated, and then the main image and the target background image are fused to obtain the target image, thereby improving the fusion of the generated target image and solving the problem of poor fusion of the target image generated in the prior art.

[0312] Fig.10 is a schematic diagram of the process of the image processing method provided in the embodiment of the present application, Fig.10 The method includes S1001 to S1003, and S1001 to S1003 are described in detail below.

[0313] S1001: Determine semantic information of a main image in an image to be processed.

[0314] In the embodiment of the present application, the image to be processed may be obtained in response to the input operation of user A. The image to be processed may also be sent to the electronic device by other devices; the image to be processed may also be downloaded by the electronic device through the network. In this way, different methods may be used to obtain the image to be processed, which improves the flexibility of obtaining the image to be processed.

[0315] The subject in the image to be processed may be an object with a human face; of course, it may also be an object without a human face, such as an animal or a plant.

[0316] It should be noted that, when the subject in the image to be processed is an object with a face, the extracted subject image includes clothing information of the object with a face.

[0317] It should be noted that the image to be processed in S1001 can be obtained by at least one of the following methods:

[0318] The first way is to obtain the image to be processed during the process of collecting images through the image collection component.

[0319] The image acquisition component may be a camera on an electronic device.

[0320] In an embodiment of the present application, in the process of acquiring images through the image acquisition component, at least one image acquired by the image acquisition component can be obtained to obtain an image to be processed; or in the process of acquiring images by the image acquisition component, a selection operation of user A for multiple acquired images can be received, and the image to be processed can be determined from the multiple images based on the selection operation.

[0321] For example, user A can click a photo button on the image acquisition interface of the electronic device to trigger the image acquisition component to acquire multiple images within the viewing range, and then select the multiple images to obtain images to be processed.

[0322] The second way is to obtain the image to be processed when the electronic device outputs the display of the object within the viewing range to the target device.

[0323] The target device and the electronic device may be of the same type; of course, the target device and the electronic device may also be of different types. Both the target device and the electronic device have image acquisition capabilities.

[0324] In an embodiment of the present application, in the process of the electronic device outputting the display of the object within the viewing range to the target device, a video stream for the object can be collected in real time, and the image to be processed can be obtained from the video stream; wherein the object can be user A; the image to be processed can be each frame image in the video stream; of course, the image to be processed can be at least one image selected from multiple frames of images in the video stream in response to the selection operation of user A.

[0325] For example, in a video call scenario or a live broadcast scenario, when the electronic device displays an object within a viewing range and outputs it to a target device, at least one image to be processed can be acquired from the captured video stream in real time.

[0326] The third way is to obtain the image to be processed when a browsing operation on the image in the gallery is detected.

[0327] In the embodiment of the present application, the image to be processed may be an image corresponding to the browsing operation; the image to be processed may also be any image in the gallery; the image to be processed may also be determined from multiple images browsed based on the browsing time determined according to the browsing operation for each image browsed;

[0328] For example, when a browsing operation of user A on an image in an album is detected, the browsing time of each image in the multiple images in the album browsed by user A can be determined based on the browsing operation, and the image corresponding to the maximum browsing time can be determined from the multiple images to obtain the image to be processed. The gallery includes the album.

[0329] Of course, the image to be processed may also be determined from a plurality of images in the gallery in response to a selection operation of user A when a browsing operation on an image in the gallery is detected.

[0330] A fourth manner is to obtain an image to be processed when a setting operation for a current background image of a display device is detected.

[0331] The display device may be a display screen.

[0332] Exemplarily, the display device may be a display screen of the electronic device itself, or may be a display screen external to the electronic device;

[0333] In the embodiment of the present application, the setting operation may be an update operation for the background image of the display device; the image to be processed may also be the image currently displayed on the display device; of course, the image to be processed may also be the default image corresponding to the display device; the image to be processed may also be an image that matches the style of the background image determined from the gallery based on the background images used in the history of the display device. The image to be processed may also be an image to be processed for a target object that is collected in real time by the image collection component when a setting operation for the background image of the display device is detected; wherein the target object is an object with a human face.

[0334] For example, when user A needs to set the desktop wallpaper of the display screen on an electronic device, he can trigger a button for setting the background image of the display screen. The electronic device can respond to the setting operation by displaying multiple images in the gallery to user A. User A can perform a selection operation on the displayed multiple images to obtain an image to be processed.

[0335] A fifth method is to obtain the image to be processed when the image is edited by a target application.

[0336] Among them, the target application is an image editing APP on an electronic device.

[0337] In an embodiment of the present application, when it is detected that the target application is editing an image, the image in the current editing window of the target application can be acquired to obtain the image to be processed; of course, it is also possible to determine the image to be processed from the multiple images displayed in the current editing window in response to the selection operation of user A.

[0338] In this way, the acquisition of the image to be processed can be triggered by at least one of the above five methods, so as to achieve the subsequent determination of the target background image semantically related to the main image in the image to be processed, thereby improving the flexibility of acquiring the image to be processed.

[0339] The subject image refers to an image of the area where the subject is located in the image to be processed. For example, the subject image may be a foreground image. The subject may be an object with a face; or the subject may be an object without a face.

[0340] In the embodiment of the present application, the image to be processed may be subjected to a cutout process to obtain a main image.

[0341] In the embodiment of the present application, semantic analysis can be performed on the subject image to obtain semantic information, wherein the semantic information represents the meaning of the image content of the subject image, wherein the semantic information can be text or keywords representing the image semantics of the subject image.

[0342] Exemplarily, the image to be processed may be an image captured by an image acquisition component of an electronic device when user A tries on a wedding dress; the subject image may be an image of the area where user A is located extracted from the image to be processed.

[0343] The semantic information obtained after semantic analysis of the main image may be a sentence, such as "a woman wearing a wedding dress", or may be multiple keywords, such as "a woman", "clothing", and "trying on clothes".

[0344] For example, Fig.11 A schematic diagram of an image to be processed in the image processing method provided in an embodiment of the present application. The image to be processed may be an image captured by an image acquisition component of an electronic device when a dog is indoors; the main image may be an image of the area where the dog is located extracted from the image to be processed.

[0345] The semantic information obtained after semantic analysis of the main image may be a sentence, such as a dog is squatting. Of course, the semantic information may also be keywords, such as "dog" and "squatting".

[0346] It should be noted that the semantic information of the subject image in S1001 can also be realized in the following ways:

[0347] Optionally, a first semantic analysis is performed on the main image to obtain a first keyword representing the image semantics of the main image.

[0348] In an embodiment of the present application, an image semantic analysis algorithm may be used to perform a first semantic analysis on the subject image to obtain a first keyword; of course, a pre-trained image description model may also be used to perform a first semantic analysis on the subject image to obtain a first keyword.

[0349] Exemplarily, the subject image may be an image of user A extracted from the image to be processed, wherein the first keyword may be “woman” and “clothing”; the first keyword may also be “woman” and “trying on clothes”.

[0350] Exemplarily, the image description model may be a coarse-grained image description model.

[0351] Optionally, semantic information is determined based on the first keyword.

[0352] In an embodiment of the present application, the first keyword can be used as semantic information; of course, a second keyword used to describe the first keyword can also be determined, and the semantic information can be determined based on the second keyword; of course, a second semantic analysis of the main image can also be performed based on the first keyword to obtain semantic information.

[0353] It should be noted that, according to the actual application scenario, the semantic information of the main image can be flexibly determined according to the first keyword, which improves the flexibility of determining the semantic information.

[0354] It should be noted that the determination of semantic information based on the first keyword can be achieved in the following way:

[0355] Optionally, the first keyword is determined to be semantic information.

[0356] Optionally, a second keyword used to describe the first keyword is determined, and a second semantic analysis is performed on the main image based on the second keyword to obtain semantic information.

[0357] Among them, semantic information includes target keywords.

[0358] Optionally, a second image description model corresponding to the first keyword is determined, and the second image description model is used to perform a second semantic analysis on the subject image to obtain semantic information.

[0359] S1002: Determine a target background image that has a semantic association with the main image based on the semantic information.

[0360] In an embodiment of the present application, the background description information can be determined based on the semantic information; based on the background description information, an image matching the background description information can be determined from multiple background images in a background image library to obtain a target background image; or, based on the background description information, a target background image can be generated. The background image library can be an image library with multiple background images on an electronic device or an image library with multiple background images on a network.

[0361] For example, the semantic information may be a plurality of keywords, such as "dog" and "squatting", and the background description information may be determined based on "dog" and "squatting", and the background description information may be background description keywords, such as "lawn" and "outdoors", and the target background image may be determined based on "lawn" and "outdoors", such as Fig.13 As shown, the target background image may be an image of an outdoor lawn.

[0362] It should be noted that, based on the semantic information, determining the background description information can be achieved in the following ways:

[0363] Optionally, background description information is generated based on the semantic information.

[0364] The background description information includes background keywords.

[0365] In an embodiment of the present application, a background keyword generator may be used to analyze the background description information, so that the background keyword generator may generate background keywords based on the background description information and output them.

[0366] By way of example, the above-mentioned background keyword generator may be an AIGC background keyword generator.

[0367] Optionally, based on the semantic information, a background keyword matching the semantic information is determined from a plurality of preset background keywords stored in the electronic device.

[0368] Optionally, based on the semantic information, background keywords matching the semantic information are searched on the Internet.

[0369] It should be noted that the above-mentioned generation of the target background image based on the background description information can be achieved in the following ways:

[0370] Specifically, semantic information and background description information can be input into the background image generation model, and the background image generation model can generate and output the target background image based on the semantic information and background description information; or only background description information can be input into the background image generation model, and the background image generation model can generate and output the target background image based on the background description information.

[0371] For example, the above-mentioned background image generation model may be an AIGC background image generation model.

[0372] Optionally, based on the semantic information, the background keyword is determined from a plurality of preset background keywords stored in the electronic device.

[0373] Optionally, based on semantic information, background keywords are searched from the Internet.

[0374] S1003: Fusing the subject image and the target background image to obtain a target image.

[0375] For example, the subject image may be Fig.11 The target background image can be the image of the area where the dog is located. Fig.12 An image of an outdoor lawn is shown, which can be based on Fig.12 The image of the area where the dog is located is determined by the location information and size of the lawn, the location information and size of the sky, and the location information and size of the trees. Fig.12 The target position information in the image and the image of the area where the dog is located are resized, and then the resized image of the area where the dog is located can be combined with the image of the area where the dog is located based on the target position information. Fig.12 The image of the outdoor lawn shown in FIG. Fig.13 The image shown is of a dog squatting on a lawn (ie, the target image).

[0376] It should be noted that the implementation process of S1003 is the same as the implementation process of S508. The implementation process of S1003 may refer to the implementation process of S508 for details, and the embodiment of the present application will not be repeated here.

[0377] It should be noted that S1003 can be implemented in the following ways:

[0378] The subject image and each target background image are fused to obtain a target image corresponding to each target background image; wherein each target background image has a different source.

[0379] In response to a switching operation for a target image, switching is performed between a plurality of target images and displayed.

[0380] It should be noted that, for the description of the same steps and the same contents in the embodiment of the present application as in other embodiments, reference can be made to the description in other embodiments and will not be repeated here.

[0381] The image processing method provided in this embodiment determines the semantic information of the main image in the image to be processed; based on the semantic information, determines the target background image that has a semantic association with the main image; and fuses the main image and the target background image to obtain the target image. In this way, the semantic information of the main image is considered to determine the target background image that has a semantic association with the main image, so that the content of the main image and the content of the target background image are relevant, and then the main image and the target background image are fused to obtain the target image, thereby improving the fusion of the generated target image and solving the problem of poor fusion of the target image generated in the prior art.

[0382] The following will be combined Fig.14The embodiments of the image processing device of the present application are described in detail. It should be understood that the image processing device in the embodiments of the present application can execute the various methods of the aforementioned embodiments of the present application, that is, the specific working processes of the following various products can refer to the corresponding processes in the aforementioned method embodiments.

[0383] Fig.14 1 is a schematic diagram of the structure of an image processing device 11 provided in an embodiment of the present application. The image processing device 11 includes a determining unit 1110 and a processing unit 1120.

[0384] Among them, the determination unit 1110 is used to determine the semantic information of the main image in the image to be processed; the determination unit 1110 is also used to determine the target background image that has a semantic association with the main image based on the semantic information; the processing unit 1120 is used to fuse the main image and the target background image to obtain the target image.

[0385] In a possible implementation manner, as an embodiment, the determination unit 1110 is specifically used to: perform a first semantic analysis on the main image to obtain a first keyword representing the image semantics of the main image; and determine the semantic information based on the first keyword.

[0386] In a possible implementation, the determining unit 1110 is specifically configured to: use a first image description model to perform a first semantic analysis on the subject image to obtain a first keyword.

[0387] In one possible implementation, the determination unit 1110 is specifically used to: determine that the first keyword is semantic information; determine a second keyword used to describe the first keyword, and perform a second semantic analysis on the main image based on the second keyword to obtain semantic information; determine a second image description model corresponding to the first keyword, and use the second image description model to perform a second semantic analysis on the main image to obtain semantic information, and the first image description model and the second image description model are different.

[0388] In a possible implementation, the determination unit 1110 is specifically used to: classify the first keyword to obtain the category to which the first keyword belongs; determine the image description model corresponding to the category from multiple vertical domain image description models to obtain the second image description model.

[0389] In one possible implementation, the determination unit 1110 is specifically used to: determine background description information based on semantic information; generate a target background image based on the background description information; and / or determine an image that matches the background description information from multiple background images in a background image library to obtain a target background image.

[0390] In a possible implementation manner, the determining unit 1110 is specifically configured to: use a background keyword generator to generate background keywords based on semantic information; wherein the background description information includes background keywords.

[0391] In a possible implementation manner, the determination unit 1110 is specifically configured to: use a background image generation model to generate a target background image based on the background description information.

[0392] In a possible implementation, the processing unit 1120 is specifically configured to: fuse the subject image with each target background image to obtain a target image corresponding to each target background image; wherein each target background image has a different source.

[0393] In a possible implementation, the processing unit 1120 is further configured to: in response to a switching operation on a target image, switch between and display a plurality of target images.

[0394] In one possible implementation, the determination unit 1110 is specifically used to perform at least one of the following steps: in the process of acquiring images through the image acquisition component, obtaining the image to be processed; in the process of the electronic device outputting the display of the object within the framing range to the target device, obtaining the image to be processed; in the case of detecting a browsing operation for an image in a gallery, obtaining the image to be processed; in the case of detecting a setting operation for the current background image of the display device, obtaining the image to be processed; in the case of editing the image through the target application, obtaining the image to be processed.

[0395] It should be noted that the above-mentioned image processing device 11 is embodied in the form of a functional unit. The term "unit" here can be implemented in the form of software and / or hardware, and is not specifically limited to this.

[0396] For example, a "unit" may be a software program, a hardware circuit, or a combination of the two that implements the above functions. The hardware circuit may include an application specific integrated circuit (ASIC), an electronic circuit, a processor (such as a shared processor, a dedicated processor, or a group processor, etc.) and a memory for executing one or more software or firmware programs, a combined logic circuit, and / or other suitable components that support the described functions.

[0397] Therefore, the units of each example described in the embodiments of the present application can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present application.

[0398] Fig.15 A structural schematic diagram of an electronic device provided by the present application is shown. Fig.15 The dotted line in indicates that the instruction or the program is optional; the electronic device 12 can be used to implement the image processing method described in the above method embodiment.

[0399] The electronic device 12 includes one or more processors 1210, which can support the electronic device 12 to implement the image processing method in the method embodiment. The processor 1210 can be a general-purpose processor or a special-purpose processor. For example, the processor 1210 can be a central processing unit (CPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, such as discrete gates, transistor logic devices, or discrete hardware components.

[0400] The processor 1210 may be used to control the electronic device 12, execute software programs, and process data of the software programs. The electronic device 12 may also include a communication unit 1250 to implement input (reception) and output (transmission) of signals.

[0401] For example, the electronic device 12 may be a chip, and the communication unit 1250 may be an input and / or output circuit of the chip, or the communication unit 1250 may be a communication interface of the chip, and the chip may be a component of a terminal device or other electronic devices.

[0402] For another example, the electronic device 12 may be a terminal device, and the communication unit 1250 may be a transceiver of the terminal device, or the communication unit 1250 may be a transceiver circuit of the terminal device.

[0403] The electronic device 12 may include one or more memories 1220 on which a program 1240 is stored. The program 1240 can be executed by the processor 1210 to generate instructions 1230, so that the processor 1210 executes the image processing method described in the above method embodiment according to the instructions 1230.

[0404] Optionally, data may also be stored in the memory 1220 .

[0405] Optionally, the processor 1210 may also read data stored in the memory 1220 , and the data may be stored at the same storage address as the program 1240 , or the data may be stored at a different storage address from the program 1240 .

[0406] The processor 1210 and the memory 1220 may be provided separately or integrated together, for example, integrated on a system on chip (SOC) of a terminal device.

[0407] Exemplarily, the memory 1220 can be used to store the related program 1240 of the image processing method provided in the embodiment of the present application, and the processor 1210 can be used to call the related program 1240 of the image processing method stored in the memory 1220 when processing the image, and execute the method of the embodiment of the present application; for example, determine the semantic information of the main image in the image to be processed; based on the semantic information, determine the target background image that has a semantic association with the main image; and fuse the main image and the target background image to obtain the target image.

[0408] The present application also provides a computer program product, which, when executed by the processor 1210, implements the image processing method of any method embodiment in the present application.

[0409] The computer program product may be stored in the memory 1220 , for example, a program 1240 , which is converted into an executable target file that can be executed by the processor 1210 after preprocessing, compiling, assembling, and linking.

[0410] The present application also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a computer, the image processing method of any method embodiment of the present application is implemented. The computer program can be a high-level language program or an executable target program.

[0411] The computer-readable storage medium is, for example, a memory 1220. The memory 1220 may be a volatile memory or a nonvolatile memory, or the memory 1220 may include both a volatile memory and a nonvolatile memory. Among them, the nonvolatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0412] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described devices and equipment and the technical effects produced can refer to the corresponding processes and technical effects in the aforementioned method embodiments, and will not be repeated here.

[0413] In several embodiments provided in the present application, the disclosed systems, devices and methods can be implemented in other ways. For example, some features of the method embodiments described above can be ignored or not performed. The device embodiments described above are merely schematic, and the division of units is only a logical function division. There may be other division methods in actual implementation, and multiple units or components may be combined or integrated into another system. In addition, the coupling between the units or the coupling between the components may be direct coupling or indirect coupling, and the above coupling includes electrical, mechanical or other forms of connection.

[0414] It should be understood that in the various embodiments of the present application, the size of the serial number of each process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0415] In addition, the terms "system" and "network" are often used interchangeably in this article. The term "and / or" in this article is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0416] In short, the above are only preferred embodiments of the technical solution of this application, and are not intended to limit the protection scope of this application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of this application should be included in the protection scope of this application.

[0417] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. An image processing method, characterized in that: Applied to electronic equipment, the method comprises: Determine semantic information of a subject image in the image to be processed; Based on the semantic information, determining a target background image having a semantic association with the subject image; The subject image and the target background image are fused to obtain a target image.

2. The method according to claim 1, characterized in that The determining of semantic information of a main image in the image to be processed includes: Performing a first semantic analysis on the subject image to obtain a first keyword representing the image semantics of the subject image; Based on the first keyword, the semantic information is determined.

3. The method according to claim 2, characterized in that The performing a first semantic analysis on the subject image to obtain a first keyword representing the image semantics of the subject image includes: A first image description model is used to perform a first semantic analysis on the subject image to obtain the first keyword.

4. The method according to claim 2, characterized in that: The determining the semantic information based on the first keyword includes one of the following: Determining that the first keyword is the semantic information; determining a second keyword for describing the first keyword, and performing a second semantic analysis on the subject image based on the second keyword to obtain the semantic information; A second image description model corresponding to the first keyword is determined, and the second image description model is used to perform a second semantic analysis on the subject image to obtain the semantic information, wherein the first image description model and the second image description model are different.

5. The method according to claim 4, characterized in that The determining the second image description model corresponding to the first keyword includes: Classifying the first keyword to obtain a category to which the first keyword belongs; A vertical image description model corresponding to the category is determined from a plurality of vertical image description models to obtain the second image description model.

6. The method according to any one of claims 1 to 5, characterized in that: The step of determining, based on the semantic information, a target background image having a semantic association with the main image includes: Based on the semantic information, determining background description information; Based on the background description information, the target background image is generated; and / or, an image matching the background description information is determined from a plurality of background images in a background image library to obtain the target background image.

7. The method according to claim 6, characterized in that The determining of background description information based on the semantic information includes: A background keyword generator is used to generate background keywords based on the semantic information; wherein the background description information includes the background keywords.

8. The method according to claim 6, characterized in that The step of generating the target background image based on the background description information includes: A background image generation model is used to generate the target background image based on the background description information.

9. The method according to any one of claims 1 to 8, characterized in that: The step of fusing the subject image with the target background image to obtain a target image includes: The subject image is merged with each target background image to obtain a target image corresponding to each target background image; wherein each target background image has a different source; The method further comprises: In response to a switching operation for a target image, switching is performed between a plurality of target images and displayed.

10. The method according to any one of claims 1 to 9, characterized in that: The image to be processed is obtained by at least one of the following methods: In the process of collecting images by the image collection component, the image to be processed is obtained; Acquiring an image to be processed during the process of the electronic device outputting a display of an object within a viewing range to a target device; When a browsing operation on an image in the gallery is detected, acquiring the image to be processed; In case a setting operation for a current background image of a display device is detected, obtaining the image to be processed; In case of editing an image through a target application, the image to be processed is acquired.

11. An image processing device, characterized in that: The device comprises: A determination unit, used to determine the semantic information of the main image in the image to be processed; The determining unit is further configured to determine, based on the semantic information, a target background image that has a semantic association with the subject image; The processing unit is used to fuse the subject image and the target background image to obtain a target image.

12. An electronic device, characterized in that: include: one or more processors and memory; The memory is coupled to the one or more processors, and the memory is used to store computer program codes, wherein the computer program codes include computer instructions, and the one or more processors call the computer instructions to enable the electronic device to execute the image processing method according to any one of claims 1-10.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor executes the image processing method according to any one of claims 1 to 10.

Images