Image processing method, device and electronic equipment based on artificial intelligence

Abstract

The present application provides an image processing method, device, electronic device and computer-readable storage medium based on artificial intelligence; the method includes: determining the background color in the image to be processed; determining the fusion transparency of the pixel points according to the chromaticity difference between the pixel points in the image to be processed and the background color; obtaining the background image corresponding to the image to be processed; according to the fusion transparency of the pixel points in the image to be processed, the image to be processed and the background image are subjected to image fusion processing in units of pixels to obtain a fused image. Through the present application, the edge details of the image to be processed can be retained during the image processing process, and the quality and accuracy of the obtained fused image can be improved.

Description

Technical Field

[0001] The present application relates to artificial intelligence technology, and in particular to an image processing method, device, electronic device and computer-readable storage medium based on artificial intelligence. Background Art

[0002] With the rapid development of computer technology and image technology, image keying, as a new method, has been widely used in various scenes such as film and television production. Image keying refers to removing a certain color from the image to be processed and superimposing a specific background image on the image to be processed, thereby forming a superimposed synthesis of two layers of images. For example, after shooting a person indoors to obtain the image to be processed, the indoor background in the image to be processed can be removed by image keying technology, and the person in the image to be processed can be superimposed with the specific background.

[0003] In the solutions provided by the related art, the user usually sets the background color to be cut out and the corresponding tolerance range, and controls the range of the cutout by the set tolerance range, thereby performing image cutout. However, this solution will cause the edge details (such as the hair details of the characters) to be lost when the image is cutout, that is, the image processing accuracy is low. Summary of the invention

[0004] The embodiments of the present application provide an artificial intelligence-based image processing method, device, electronic device, and computer-readable storage medium, which can retain the original edge details of the image to be processed during the image processing process and improve the quality and accuracy of the obtained fused image.

[0005] The technical solution of the embodiment of the present application is implemented as follows:

[0006] The present application provides an artificial intelligence-based image processing method, including:

[0007] Determine the background color in the image to be processed;

[0008] Determining the fusion transparency of the pixel point according to the chromaticity difference between the pixel point in the image to be processed and the background color;

[0009] Acquire a background image corresponding to the image to be processed;

[0010] According to the fusion transparency of the pixels in the image to be processed, the image to be processed and the background image are subjected to image fusion processing in units of pixels to obtain a fused image.

[0011] The present application provides an artificial intelligence-based image processing device, comprising:

[0012] A background color determination module, used to determine the background color in the image to be processed;

[0013] A transparency determination module, used for determining the fusion transparency of the pixel point according to the chromaticity difference between the pixel point in the image to be processed and the background color;

[0014] An image acquisition module, used to acquire a background image corresponding to the image to be processed;

[0015] The image fusion module is used to perform image fusion processing on the image to be processed and the background image in units of pixels according to the fusion transparency of the pixels in the image to be processed, so as to obtain a fused image.

[0016] An embodiment of the present application provides an electronic device, including:

[0017] A memory for storing executable instructions;

[0018] The processor is used to implement the artificial intelligence-based image processing method provided in the embodiment of the present application when executing the executable instructions stored in the memory.

[0019] An embodiment of the present application provides a computer-readable storage medium storing executable instructions for causing a processor to execute and implement the artificial intelligence-based image processing method provided in the embodiment of the present application.

[0020] The embodiments of the present application have the following beneficial effects:

[0021] The fusion transparency of the pixel points is determined according to the chromaticity difference between the pixel points in the image to be processed and the background color, and according to the fusion transparency of the pixel points, the image to be processed and the background image are subjected to image fusion processing in units of pixels. Compared with the method of performing image matting according to the tolerance range, the embodiment of the present application can retain the original edge details of the image to be processed through smoother image fusion processing, while improving the quality (image quality) and accuracy of the obtained fused image. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 is a schematic diagram of the architecture of an image processing system based on artificial intelligence provided in an embodiment of the present application;

[0023] Figure 2 It is a schematic diagram of the architecture of a terminal device provided in an embodiment of the present application;

[0024] Figure 3A It is a flowchart of an image processing method based on artificial intelligence provided in an embodiment of the present application;

[0025] Figure 3B It is a flowchart of an image processing method based on artificial intelligence provided in an embodiment of the present application;

[0026] Figure 3C It is a flowchart of an image processing method based on artificial intelligence provided in an embodiment of the present application;

[0027] Figure 3D It is a flowchart of an image processing method based on artificial intelligence provided in an embodiment of the present application;

[0028] Figure 3E It is a flowchart of an image processing method based on artificial intelligence provided in an embodiment of the present application;

[0029] Figure 4 is a schematic diagram of a fused image provided in an embodiment of the present application;

[0030] Figure 5 is a schematic diagram of an image to be processed and a fused image provided in an embodiment of the present application;

[0031] Figure 6 is a comparative schematic diagram of the fusion effect provided in the embodiment of the present application;

[0032] Figure 7 It is a schematic diagram of a process of determining a foreground image and a transparent channel image provided by an embodiment of the present application;

[0033] Figure 8 It is a flowchart of table lookup acceleration provided in an embodiment of the present application. DETAILED DESCRIPTION

[0034] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0035] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0036] In the following description, the terms "first\second\third" are only used to distinguish similar objects, and do not represent a specific order for the objects. It is understandable that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In the following description, the term "plurality" refers to at least two.

[0037] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0038] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0039] 1) Artificial Intelligence (AI): The theory, method, technology and application system of using digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. Computer vision technology (CV) is an important application of artificial intelligence. It refers to the use of cameras and computers to replace human eyes to identify, track and measure targets, and further perform graphic processing to make computer processing into images that are more suitable for human eye observation or transmission to instrument detection. In the embodiment of the present application, image processing can be achieved based on computer vision technology.

[0040] 2) Background color: refers to the color that needs to be removed / covered in the image to be processed. In the embodiment of the present application, the background color can be specified by the user, or can be obtained by intelligently identifying the image to be processed.

[0041] 3) Chroma: The three elements of color may include hue (tone), saturation (purity) and brightness, where color is the sensation produced by the physical reflection of light from an object onto the optic nerve of the human eye. The difference in color is determined by the difference in the frequency of light, and hue is used to reflect the situation of colors of different frequencies; saturation is used to indicate the degree of vividness or vividness of a color; brightness is used to indicate the degree of brightness of a color. In the embodiment of the present application, chroma indicates the properties of a color that are different from brightness, for example, chroma may include at least one of hue and saturation.

[0042] 4) Pixel: Also known as pixel, refers to the element in the image that cannot be further divided.

[0043] 5) Color Space: Also known as color gamut or color space, it refers to an abstract model that describes color through a set of values. The color being described is objective in itself, and different color spaces only describe the color from different perspectives. The color space includes multiple channels, and the color can be described by the channel values ​​corresponding to each channel.

[0044] Color spaces can be divided into two categories according to their basic structure: primary color space and color-brightness separation color space. The former, such as the RGB color space, includes red, green, and blue channels, that is, the primary color space does not clearly distinguish between brightness and chroma; the latter, such as the YUV color space, HSV color space, HSL color space, and LAB color space, clearly distinguish between brightness and chroma in the color-brightness separation color space. Among them, the Y channel included in the YUV color space corresponds to brightness, and the U channel and V channel both correspond to chroma; the H channel, S channel, and V channel included in the HSV color space correspond to hue, saturation, and brightness respectively; the H channel, S channel, and L channel included in the HSL color space correspond to hue, saturation, and brightness respectively; the L channel included in the LAB color space corresponds to brightness, and the A channel and B channel both correspond to chroma.

[0045] 6) Table lookup acceleration: An acceleration method that trades space (storage space) for time (running time), which can be used in the embodiments of the present application to accelerate image processing, thereby improving the efficiency of image processing and reducing the real-time computing load. For example, various results that may be involved in the image processing process can be pre-calculated and stored in a specific data structure (such as a table), so that the data structure can be directly queried during the real-time image processing process without the need for real-time computing.

[0046] For image matting, the more typical one is the Super Key algorithm (such as the Super Key algorithm in Premiere software). In this Super Key algorithm, the user needs to set the background color to be matted (such as green or blue, etc.) and the corresponding tolerance range. The set tolerance range is used to control the size of the matting range. In addition, the user can also perform various operations such as feathering and shrinking. However, when using the super key algorithm to perform image matting, there are at least the following problems: 1) The original edge details in the image to be processed (such as the details of the person's hair) will be lost during the image matting process, resulting in poor matting effect; 2) The highlight and shadow effects in the final superimposed image are poor, for example, the shadow is gray, and it is impossible to effectively simulate the original shadow effect in the image to be processed; 3) It is impossible to accurately process translucent objects (such as glass cups) in the image to be processed, and the translucent objects in the final superimposed image may be distorted, blurred, and other problems; 4) In the final superimposed image, the foreground (or target) will appear to be biased towards the background color. For example, when the background color is green, the person (i.e., the foreground) in the final superimposed image will have a green glow; 5) There are too many operations that the user needs to perform, and the learning difficulty and cost of the user are high.

[0047] The embodiments of the present application provide an image processing method, device, electronic device, and computer-readable storage medium based on artificial intelligence, which can retain the edge details of the image to be processed during the image processing process and improve the quality and accuracy of the obtained fused image. The following describes an exemplary application of the electronic device provided by the embodiments of the present application. The electronic device provided by the embodiments of the present application can be implemented as various types of terminal devices or as a server.

[0048] See also Figure 1 , Figure 1 It is a schematic diagram of the architecture of an artificial intelligence-based image processing system 100 provided in an embodiment of the present application, wherein a terminal device 400 is connected to a server 200 via a network 300, and the server 200 is connected to a database 500, wherein the network 300 may be a wide area network or a local area network, or a combination of the two.

[0049] In some embodiments, taking the electronic device as a terminal device as an example, the image processing method based on artificial intelligence provided in the embodiment of the present application can be implemented by the terminal device. For example, the terminal device 400 obtains the image to be processed, wherein the image to be processed can be pre-stored locally in the terminal device 400, can be taken by the terminal device 400 in real time, or can be obtained by the terminal device 400 from the outside world (such as the server 200, the database 500, and the blockchain, etc.). For the acquired image to be processed, the terminal device 400 determines the background color in the image to be processed. For example, the terminal device 400 can respond to the background color setting operation and obtain the background color set by the user through the background color setting operation. For example, the terminal device 400 can perform intelligent recognition on the image to be processed to automatically identify the background color. Then, the terminal device 400 determines the fusion transparency of the pixel point according to the chromaticity difference between the pixel point and the background color in the image to be processed, and according to the fusion transparency, the image to be processed and the acquired background image are subjected to image fusion processing in units of pixels to obtain a fused image. Similarly, the background image can also be pre-stored, taken in real time, or obtained from the outside world by the terminal device 400.

[0050] In some embodiments, taking the electronic device as a server as an example, the image processing method based on artificial intelligence provided in the embodiment of the present application can also be implemented by the server. For example, the server 200 (such as the background server of the image application) can obtain the image to be processed from the database 500. Of course, the image to be processed can also be obtained by the server 200 from other locations (such as the terminal device 400 and the blockchain, etc.). For the image to be processed obtained, the server 200 determines the background color in the image to be processed. For example, the server 200 can receive the background color set by the user sent by the terminal device 400. For example, the server 200 can intelligently identify the image to be processed to automatically identify the background color. Then, the server 200 determines the fusion transparency of the pixel point according to the chromaticity difference between the pixel point and the background color in the image to be processed, and according to the fusion transparency, the image to be processed and the obtained background image are subjected to image fusion processing in units of pixels to obtain a fused image. Similarly, the background image can also be obtained by the server 200 from the database 500 or other locations.

[0051] In some embodiments, the artificial intelligence-based image processing method provided in the embodiments of the present application can also be implemented by the terminal device and the server in collaboration. Figure 1 As shown, the terminal device 400 sends the image to be processed, the background color and the background image to the server 200 in response to the user operation, and the image processing is performed by the server 200. After the server 200 obtains the fused image through image processing, the server 200 sends the fused image to the terminal device 400.

[0052] In some embodiments, various results involved in the image processing process (such as the image to be processed, the background color, the background image, the fused image, and various tables for table lookup and acceleration) can be stored in the blockchain. Since the blockchain has the characteristic of being tamper-proof, the accuracy of the data in the blockchain can be guaranteed. An electronic device (such as a terminal device 400 or a server 200) can send a query request to the blockchain to query the data stored in the blockchain.

[0053] In some embodiments, the terminal device 400 or the server 200 can implement the artificial intelligence-based image processing method provided in the embodiments of the present application by running a computer program. For example, the computer program can be a native program or software module in the operating system; it can be a native application (APP, Application), that is, a program that needs to be installed in the operating system to run, such as an image application (such as Figure 1The client 410 shown in the figure may be, for example, an APP for image editing / video editing); it may also be a small program, that is, a program that can be run by downloading it to a browser environment; it may also be a small program that can be embedded in any APP, such as a small program component embedded in an image application, wherein the small program component can be controlled to run or closed by the user. In short, the above-mentioned computer program may be an application, module or plug-in in any form.

[0054] In some embodiments, the server 200 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms, among which the cloud service may be an image processing service for the terminal device 400 to call. The terminal device 400 may be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart TV, a smart watch, etc., but is not limited thereto. The terminal device and the server may be directly or indirectly connected via wired or wireless communication, which is not limited in the embodiments of the present application.

[0055] In the embodiment of the present application, the database can provide data support for the server. For example, the server can connect to the database, obtain the data in the database for corresponding processing, and store the obtained processing results in the database. Figure 1 As shown, the database 500 can be used to store data related to image processing (including but not limited to the image to be processed, the background color and the background image), and the server 200 can obtain the data in the database 500 to perform image processing.

[0056] In some embodiments, the database and the server can be independently arranged. In some embodiments, the database and the server can also be integrated, that is, the database can be regarded as existing inside the server, integrated with the server, and the server can provide the data management function of the database.

[0057] Taking the electronic device provided in the embodiment of the present application as a terminal device as an example, it can be understood that in the case where the electronic device is a server, Figure 2 Some of the structures shown in FIG. 1 (such as the user interface, the presentation module, and the input processing module) may be omitted. Figure 2 , Figure 2 is a schematic diagram of the structure of a terminal device 400 provided in an embodiment of the present application, Figure 2The terminal device 400 shown includes: at least one processor 410, a memory 450, at least one network interface 420 and a user interface 430. The various components in the terminal device 400 are coupled together via a bus system 440. It is understood that the bus system 440 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 440 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, the bus system 440 is not shown in FIG. Figure 2 Various buses are labeled as bus system 440 .

[0058] Processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., where the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0059] The user interface 430 includes one or more output devices 431 that enable presentation of media content, including one or more speakers and / or one or more visual display screens. The user interface 430 also includes one or more input devices 432, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0060] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drives, optical disk drives, etc. The memory 450 may optionally include one or more storage devices that are physically remote from the processor 410.

[0061] The memory 450 includes a volatile memory or a non-volatile memory, and may also include both volatile and non-volatile memories. The non-volatile memory may be a read-only memory (ROM), and the volatile memory may be a random access memory (RAM). The memory 450 described in the embodiments of the present application is intended to include any suitable type of memory.

[0062] In some embodiments, memory 450 can store data to support various operations, examples of which include programs, modules, and data structures, or a subset or superset thereof, as exemplarily described below.

[0063] Operating system 451, including system programs for processing various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks;

[0064] A network communication module 452, used to reach other electronic devices (such as other terminal devices or other servers) via one or more (wired or wireless) network interfaces 420, exemplary network interfaces 420 include: Bluetooth, wireless compatibility certification (WiFi), and Universal Serial Bus (USB), etc.;

[0065] a presentation module 453 for enabling presentation of information via one or more output devices 431 (e.g., display screen, speaker, etc.) associated with the user interface 430 (e.g., a user interface for operating peripherals and displaying content and information);

[0066] The input processing module 454 is used to detect one or more user inputs or interactions from one of the one or more input devices 432 and translate the detected inputs or interactions.

[0067] In some embodiments, the artificial intelligence-based image processing device provided in the embodiments of the present application can be implemented in software. Figure 2 An artificial intelligence-based image processing device 455 stored in a memory 450 is shown, which may be software in the form of a program or a plug-in, including the following software modules: a background color determination module 4551, a transparency determination module 4552, an image acquisition module 4553, and an image fusion module 4554. These modules are logical, and thus may be arbitrarily combined or further split according to the functions implemented. The functions of each module will be described below.

[0068] The artificial intelligence-based image processing method provided in the embodiments of the present application will be explained in combination with the exemplary application and implementation of the electronic device provided in the embodiments of the present application.

[0069] See also Figure 3A , Figure 3A is a flowchart of an image processing method based on artificial intelligence provided by an embodiment of the present application, which will be combined with Figure 3A The steps shown are explained.

[0070] In step 101, the background color in the image to be processed is determined.

[0071] Here, the image to be processed is obtained, and the background color to be cut out / covered in the image to be processed is determined. The embodiment of the present application does not limit the number of bits of the image to be processed, for example, it can be 8 bits (Bites), 16 bits or 32 bits, etc.; the content of the image to be processed is also not limited, for example, it can be a real person, a cat or a dog, etc.; the size of the image to be processed is also not limited.

[0072] It is worth noting that the embodiment of the present application does not limit the execution order between the step of obtaining the image to be processed and the step of determining the background color. For example, they can be executed one after the other (such as determining the background color before obtaining the image to be processed, or determining the background color after obtaining the image to be processed), or they can be executed simultaneously. The determined background color can be applicable to a specific image to be processed, or it can be applicable to different images to be processed at the same time.

[0073] In some embodiments, the above-mentioned determination of the background color in the image to be processed can be achieved by any of the following methods: obtaining the background color set for the image to be processed, and the set background color is the background color in the image to be processed; performing target recognition processing on the image to be processed to obtain the target area, and taking the color that appears most frequently in the background area of ​​the image to be processed as the background color; wherein the background area is an area in the image to be processed that is different from the target area.

[0074] The embodiment of the present application provides two methods for determining the background color. The first method is to obtain the background color set for the image to be processed, for example, in response to the background color setting operation, obtain the background color set by the user through the background color setting operation, and use the set background color as the background color in the image to be processed. This method can meet the user's needs to the greatest extent, but requires the user to participate in the setting.

[0075] The second way is to perform target recognition processing on the processed image to obtain the target area (also known as the foreground area), use the area in the processed image that is different from the target area as the background area, and use the color that appears most frequently in the background area as the background color. For example, the target recognition processing can be performed by a target recognition model, and the target recognition model can be a machine learning model built based on the principle of machine learning (Machine Learning, ML), such as a one-time recognition (You Only Look Once, YOLO) model or a region-based convolutional neural network (Region-Convolutional Neural Networks, R-CNN) model, etc., wherein machine learning is the core of artificial intelligence, and specializes in how computers simulate or implement human learning behavior to acquire new knowledge or skills, and reorganize existing knowledge structures to continuously improve their own performance. The embodiment of the present application does not limit the type of target to be identified, for example, it can include people, cats and dogs, etc.

[0076] After determining the background area, the pixel point can be used as a unit to determine the color corresponding to each pixel point in the background area, wherein the color can be represented by the channel value of the pixel point in the color space channel, and the color space is not limited here, for example, it can be a primary color space or a color brightness separation color space. Generally speaking, the background color occupies a large proportion in the background area, so the color with the highest frequency (or the most occurrences) in the background area can be used as the background color. For example, the background area includes a total of 100 pixels, of which 60 pixels correspond to color A, then color A has the highest frequency of occurrence in the background area, that is, 60 / 100, so color A is used as the determined background color. The second method can effectively reduce user operations and save labor costs (especially when the number of images to be processed is large) by automatically determining the background color. At the same time, it can also ensure the accuracy of the determined background color to a certain extent.

[0077] In step 102, the fusion transparency of the pixel is determined according to the chromaticity difference between the pixel in the image to be processed and the background color.

[0078] Here, for each pixel in the image to be processed, the chromaticity difference (also called chromaticity distance) between the chromaticity of the pixel and the chromaticity of the background color is determined, and the fusion transparency of the pixel is determined based on the chromaticity difference. For example, the chromaticity difference can be directly used as the fusion transparency of the pixel, or the chromaticity difference can be further processed to obtain the fusion transparency of the pixel.

[0079] The embodiments of the present application do not limit the method for determining the chromaticity difference. For example, the chromaticity difference can be obtained by subtracting the chromaticity of the background color from the chromaticity of the pixel point; the chromaticity difference can be obtained by subtracting the chromaticity of the background color from the chromaticity of the pixel point, and then performing an absolute value calculation on the result; or the chromaticity difference can be obtained by subtracting the chromaticity of the background color from the chromaticity of the pixel point, and then squaring the result.

[0080] In the embodiment of the present application, chromaticity includes at least one of hue and saturation. Therefore, in step 102, the fusion transparency of the pixel can be determined based on the hue difference between the hue of the pixel and the hue of the background color; the fusion transparency of the pixel can be determined based on the saturation difference between the saturation of the pixel and the saturation of the background color; or the hue difference and the saturation difference can be calculated at the same time, and the fusion transparency of the pixel can be determined based on the hue difference and the saturation difference.

[0081] In some embodiments, the above-mentioned determination of the fusion transparency of pixels based on the chromaticity difference between the pixels in the image to be processed and the background color can be achieved in the following manner: the image to be processed and the background color are converted to the same color space; wherein the color space includes multiple chromaticity channels; for each chromaticity channel, the difference between the channel value of the chromaticity channel corresponding to the pixel in the image to be processed and the channel value of the chromaticity channel corresponding to the background color is determined, and the determined difference is used as the chromaticity difference corresponding to the chromaticity channel; the chromaticity differences corresponding to the multiple chromaticity channels are subjected to difference fusion processing to obtain the fusion transparency of the pixels.

[0082] In order to ensure the accuracy of the obtained fusion transparency, the image to be processed and the background color can be converted to the same color space, where the color space can be a color-brightness separation color space, that is, it includes multiple chromaticity channels. It is worth noting that each chromaticity channel can be used only to represent one property of color, such as the H channel (used to represent hue) and S channel (used to represent saturation) in the HSV color space and the HSL color space; each chromaticity channel can also represent hue and saturation at the same time, such as the U channel and V channel in the YUV color space, and the A channel and B channel in the LAB color space.

[0083] For each pixel in the image to be processed, first, for each chromaticity channel, determine the difference between the channel value of the chromaticity channel corresponding to the pixel (i.e., the chromaticity in the chromaticity channel) and the channel value of the chromaticity channel corresponding to the background color, and use the determined difference as the chromaticity difference corresponding to the chromaticity channel. Then, perform difference fusion processing on the chromaticity differences corresponding to all chromaticity channels to obtain the fusion transparency of the pixel. Among them, there is no limitation on the method of difference fusion processing, for example, it can be a summation process or a weighted summation process. The above method determines the fusion transparency by integrating multiple chromaticity channels in the same color space, which can improve the accuracy and comprehensiveness of the obtained fusion transparency.

[0084] In step 103, a background image corresponding to the image to be processed is obtained.

[0085] Here, the background image is different from the image to be processed. The embodiment of the present application also does not limit the image bit number, content, and size of the background image, and the background image can be determined according to the actual application scenario.

[0086] In step 104, according to the fusion transparency of the pixels in the image to be processed, the image to be processed and the background image are subjected to image fusion processing in units of pixels to obtain a fused image.

[0087] Compared with the method of matting according to the tolerance range, in the embodiment of the present application, the image to be processed and the background image are subjected to image fusion processing in units of pixels according to the fusion transparency of each pixel in the image to be processed to obtain a fused image. Since the image fusion processing method is smoother, the edge details of the target in the image to be processed (such as the details of the hair of a person) can be retained. The embodiment of the present application does not limit the method of image fusion processing, for example, it can be weighted processing, etc.

[0088] Among them, the greater the fusion transparency of the pixel point, the more different the color corresponding to the pixel point is from the background color, and the higher the degree of retention of the color corresponding to the pixel point in the fused image, that is, the degree of retention of the color corresponding to the pixel point in the processed image in the fused image is positively correlated with the fusion transparency of the pixel point.

[0089] In some embodiments, the above-mentioned image fusion processing of the image to be processed and the background image in units of pixels can be achieved in the following manner: the image to be processed and the background image are converted to the same color space; wherein the color space includes multiple channels; the following processing is performed for each channel: the first channel value of the channel corresponding to the first pixel in the image to be processed and the second channel value of the channel corresponding to the second pixel in the background image are determined; according to the fusion transparency of the first pixel, the first channel value and the second channel value are weighted to obtain the third channel value of the channel corresponding to the third pixel in the fused image; wherein there is a mapping relationship between the pixel positions corresponding to the first pixel, the second pixel and the third pixel, respectively.

[0090] In order to ensure the accuracy of image fusion processing, the image to be processed and the background image can be converted into the same color space, where the color space can be a base color space or a color-brightness separation color space, wherein the color space includes multiple channels.

[0091] For each pixel in the image to be processed (named as the first pixel for ease of understanding), for each channel in the color space, determine the channel value of the channel corresponding to the first pixel (named as the first channel value for ease of understanding, and the same applies below), and the second channel value of the channel corresponding to the second pixel in the background image, and perform weighted processing on the first channel value and the second channel value according to the fusion transparency of the first pixel, and obtain the third channel value of the channel corresponding to the third pixel in the fused image. Among them, there is a mapping relationship between the pixel position of the first pixel in the image to be processed, the pixel position of the second pixel in the background image, and the pixel position of the third pixel in the fused image. For example, the image to be processed and the background image can be scaled to the same size, and the existence of a mapping relationship can refer to the same pixel position. Of course, in the embodiment of the present application, it is not necessary to scale the image to be processed and the background image to the same size. As long as the mapping relationship between the pixel positions corresponding to the first pixel, the second pixel, and the third pixel are set, image fusion processing can be performed when the size of the image to be processed is different from that of the background image.

[0092] In addition, weighted processing of the first channel value and the second channel value according to the fusion transparency of the first pixel may refer to: using the fusion transparency of the first pixel as the weight of the first channel value, using the complementary fusion transparency corresponding to the fusion transparency of the first pixel as the weight of the second channel value, and performing weighted summation processing on the first channel value and the second channel value. The sum of the fusion transparency of the first pixel and the corresponding complementary fusion transparency is the maximum transparency in the transparency range, and the transparency range is the value range of the fusion transparency. In the above manner, the accuracy of the image fusion processing can be guaranteed, and the obtained fused image can be guaranteed to have high quality and precision.

[0093] like Figure 3A As shown, the embodiment of the present application determines the fusion transparency and performs image fusion processing based on the fusion transparency. Compared with the method of performing matting according to the tolerance range, the image processing solution provided by the embodiment of the present application is smoother and can retain more edge details in the image to be processed, thereby improving the precision of image processing.

[0094] In some embodiments, see Figure 3B , Figure 3B is a flowchart of an image processing method based on artificial intelligence provided in an embodiment of the present application, based on Figure 3A After step 102, in step 201, the image to be processed and the replacement color may be fused in units of pixels according to the fusion transparency of the pixels in the image to be processed to obtain a foreground image.

[0095] In the embodiment of the present application, color optimization can also be performed based on the background color. First, the image to be processed and the replacement color can be fused in units of pixels according to the fusion transparency of the pixels in the image to be processed to obtain a foreground image.

[0096] For example, the image to be processed and the replacement color can be converted to the same color space, where the color space can be a base color space or a color-brightness separation color space, wherein the color space includes multiple channels. For each pixel in the image to be processed (named as the first pixel for ease of understanding), for each channel in the color space, according to the fusion transparency of the first pixel, the channel value of the channel corresponding to the first pixel and the channel value of the replacement color corresponding to the channel are weighted to obtain the channel value of the fourth pixel in the foreground image corresponding to the channel. The pixel position of the first pixel in the image to be processed is the same as the pixel position of the fourth pixel in the foreground image.

[0097] It is worth noting that the replacement color is used to remove the background color in the image to be processed. The replacement color can be set according to the actual application scenario, such as being set to gray, that is, the channel value of each channel in the RGB color space is 128.

[0098] In step 202, the background color cast of the pixel is determined based on the chromaticity difference between the pixel in the foreground image and the background color.

[0099] For example, for each pixel in the foreground image, the chromaticity difference between the chromaticity of the pixel and the chromaticity of the background color is determined, and the background color deviation of the pixel is determined based on the chromaticity difference. For example, the chromaticity difference can be directly used as the background color deviation of the pixel, or the chromaticity difference can be further processed to obtain the background color deviation of the pixel (such as when there are multiple chromaticity channels).

[0100] It is worth noting that since color deviation can be effectively reflected based on hue difference, the hue difference between the hue of the pixel and the background color can be determined, and the background color deviation of the pixel can be determined based on the hue difference. In this way, the accuracy of color optimization can be guaranteed while reducing the amount of calculation.

[0101] In step 203, the chromaticity of the pixel is updated according to the background color deviation of the pixel.

[0102] Since the background color deviation of a pixel reflects the degree of deviation (or degree of distinction or difference) between the chromaticity of the pixel and the chromaticity of the background color, the chromaticity of the pixel can be updated according to the background color deviation of the pixel to avoid the appearance of colors that are more biased towards the background color in the foreground image.

[0103] In some embodiments, the above-mentioned updating of the chromaticity of the pixel point according to the background color deviation of the pixel point can be achieved in the following manner: when the background color deviation of the pixel point reaches a set color deviation degree, the chromaticity of the pixel point is kept unchanged; when the background color deviation of the pixel point does not reach the set color deviation degree, the chromaticity of the background color is deviated to obtain a deviated chromaticity, and the chromaticity of the pixel point is updated to the deviated chromaticity.

[0104] Here, when the background color deviation of a pixel reaches the set color deviation degree, it proves that the difference between the color of the pixel and the background color is large, so the chromaticity of the pixel can be kept unchanged; when the background color deviation of a pixel does not reach the set color deviation degree, it proves that the difference between the color of the pixel and the background color is small, so the chromaticity of the background color can be deviated to obtain the deviated chromaticity, and the chromaticity of the pixel can be updated to the deviated chromaticity. It is worth noting that deviating the chromaticity of the background color does not mean updating the chromaticity of the background color. In fact, the deviation processing step is only to update the chromaticity of the pixel, and the chromaticity of the background color remains unchanged.

[0105] For ease of understanding, an example is given. First, for each pixel in the foreground image, the hue difference between the hue of the pixel and the hue of the background color can be determined as the background color deviation of the pixel. For example, the hue of the background color can be subtracted from the hue of the pixel to obtain the background color deviation of the pixel. For ease of calculation, the background color deviation can also be updated. For example, when the background color deviation is greater than π (i.e., 180 degrees), the background color deviation is subtracted by 2π to update the background color deviation; when the background color deviation is less than -π (i.e., -180 degrees), the background color deviation is added by 2π to update the background color deviation; when the background color deviation is within the color deviation range such as [-π, π], the background color deviation is kept unchanged.

[0106] Then, it can be determined whether the background color deviation of the pixel reaches the set color deviation degree according to the set color deviation threshold T, wherein T is greater than zero and less than π, and can be set according to the actual application scenario, such as being set to For example, when the background color deviation of a pixel is ≤-T, or the background color deviation of a pixel is ≥T, it is determined that the background color deviation of the pixel has reached the set color deviation degree, and thus the hue of the pixel is kept unchanged; when the background color deviation of a pixel is <0 and greater than -T, it is determined that the background color deviation of the pixel has not reached the set color deviation degree, and the hue of the background color is deviated according to T, such as subtracting T / 2 from the hue of the background color to obtain the deviated hue, and the hue of the pixel is updated to the deviated hue; when the background color deviation of a pixel is >0 and less than T, it is determined that the background color deviation of the pixel has not reached the set color deviation degree, and the hue of the background color is deviated according to T, such as adding T / 2 to the hue of the background color to obtain the deviated hue, and the hue of the pixel is updated to the deviated hue.

[0107] It is worth noting that when the background color deviation of a pixel point = 0, the hue of the background color can be subtracted by T / 2 to obtain the deviated hue, and the hue of the pixel point can be updated to the deviated hue; or the hue of the background color can be added with T / 2 to obtain the deviated hue, and the hue of the pixel point can be updated to the deviated hue.

[0108] Through the above method, the effect of updating the chromaticity of the pixel points can be improved, and the appearance of a color that is closer to the background color in the foreground image can be effectively avoided.

[0109] exist Figure 3B middle, Figure 3A The illustrated step 104 may be updated to step 204. In step 204, according to the fusion transparency of the pixels in the foreground image, the foreground image and the background image are subjected to image fusion processing in units of pixels to obtain a fused image.

[0110] Here, the foreground image and the background image may be fused in units of pixels according to the fusion transparency of each pixel in the foreground image to obtain a fused image. The image fusion processing method here is similar to step 104.

[0111] like Figure 3B As shown, the embodiment of the present application performs color optimization on the image to be processed to obtain a foreground image, and performs image fusion processing on the foreground image and the background image, which can further improve the accuracy of image processing.

[0112] In some embodiments, see Figure 3C , Figure 3C is a flowchart of an image processing method based on artificial intelligence provided in an embodiment of the present application, based on Figure 3A After step 102, in step 301, the brightness difference between the pixel points in the image to be processed and the background color may be determined.

[0113] In an embodiment of the present application, shadow optimization (or highlight optimization) can be performed for the image to be processed. For example, the image to be processed and the background color can be converted to the same color space, where the color space can be a color-brightness separation color space, that is, including a brightness channel. For each pixel in the image to be processed, the difference between the channel value of the brightness channel corresponding to the pixel (that is, the brightness in the brightness channel) and the channel value of the brightness channel corresponding to the background color is determined, and the difference determined here is used as the brightness difference.

[0114] In step 302, the brightness difference is weighted according to the shadow compensation parameter to obtain the shadow compensation intensity of the pixel point.

[0115] Here, the brightness difference of the pixel point is weighted according to the set shadow compensation parameter to obtain the shadow compensation intensity of the pixel point, wherein the shadow compensation parameter is positively correlated with the shadow compensation intensity, and the shadow compensation parameter can be set according to the needs of the actual application scenario.

[0116] In some embodiments, after step 302, the method further includes: constraining the shadow compensation intensity of the pixel points according to the intensity range.

[0117] Since the shadow compensation intensity is obtained by weighting the brightness difference according to the shadow compensation parameters, the shadow compensation intensity may exceed the intensity range, where the intensity range refers to the normal value range of the shadow compensation intensity. Therefore, in the embodiment of the present application, the shadow compensation intensity of each pixel in the image to be processed can be constrained according to the intensity range to ensure that the shadow compensation intensity after the constraint processing is within the normal range.

[0118] In some embodiments, the intensity range includes a minimum intensity and a maximum intensity; the above-mentioned constraint processing of the shadow compensation intensity of the pixel point according to the intensity range can be achieved in the following way: when the shadow compensation intensity is less than the minimum intensity, the shadow compensation intensity is updated to the minimum intensity; when the shadow compensation intensity is within the intensity range, the shadow compensation intensity is kept unchanged; when the shadow compensation intensity is greater than the maximum intensity, the shadow compensation intensity is updated to the maximum intensity.

[0119] Here, the intensity range can be the interval of [minimum intensity, maximum intensity]. When the shadow compensation intensity is less than the minimum intensity, the shadow compensation intensity is updated to the minimum intensity; when the shadow compensation intensity is within the interval of [minimum intensity, maximum intensity], the shadow compensation intensity is kept unchanged; when the shadow compensation intensity is greater than the maximum intensity, the shadow compensation intensity is updated to the maximum intensity. In this way, it can be effectively ensured that the shadow compensation intensity after constraint processing is within the intensity range.

[0120] In step 303, shadow compensation processing is performed on the brightness and fusion transparency of the pixel points according to the shadow compensation intensity of the pixel points.

[0121] Here, for each pixel in the image to be processed, shadow compensation processing is performed on the brightness and fusion transparency of the pixel according to the shadow compensation intensity of the pixel. The shadow compensation processing may be an amplification processing, and the shadow compensation intensity is positively correlated with the amplitude of the amplification processing, so that the shadow in the image to be processed can be simulated more accurately in the final fused image.

[0122] In some embodiments, the above-mentioned shadow compensation processing of the brightness and fusion transparency of the pixel points according to the shadow compensation intensity of the pixel points can be achieved in the following manner: the brightness of the pixel points is amplified according to the shadow compensation intensity, brightness and fusion transparency of the pixel points, and the brightness obtained by the brightness amplification processing is constrained according to the brightness range; the fusion transparency of the pixel points is amplified according to the shadow compensation intensity and fusion transparency of the pixel points, and the fusion transparency obtained by the fusion transparency amplification processing is constrained according to the transparency range.

[0123] For the brightness of a pixel, brightness amplification processing can be performed on the pixel according to the shadow compensation intensity, brightness and fusion transparency of the pixel, wherein the shadow compensation intensity of the pixel is positively correlated with the amplification intensity of the brightness amplification processing. The embodiment of the present application does not limit the brightness amplification processing method. For example, when the brightness range (i.e., the brightness value range) is [0, 1], the brightness obtained by the brightness amplification processing can be expressed as Among them, Y f represents the brightness of the pixel before the brightness amplification process, α represents the fusion transparency of the pixel (the fusion transparency before the fusion transparency amplification process), Y c Indicates the shadow compensation intensity of the pixel. In addition, e is a parameter set to avoid numerical overflow, and can be set to 0.01, for example. The brightness obtained by brightness amplification may exceed the brightness range. Therefore, the brightness obtained by brightness amplification can also be constrained according to the brightness range, and the brightness obtained after the constraint processing is used as the new brightness of the pixel. The process of constraining the brightness is similar to the process of constraining the shadow compensation intensity mentioned above.

[0124] For the fusion transparency of the pixel point, the fusion transparency amplification processing can be performed on the pixel point according to the shadow compensation strength and the fusion transparency of the pixel point, wherein the shadow compensation strength of the pixel point is positively correlated with the amplification strength of the fusion transparency amplification processing. The embodiment of the present application does not limit the method of fusion transparency amplification processing. For example, when the transparency range (i.e., the value range of fusion transparency) is [0, 1], the fusion transparency obtained by the fusion transparency amplification processing can be expressed as (1-α)·|Y c |+α, where Δ represents the fusion transparency of the pixel before the fusion transparency increase process, Y c Indicates the shadow compensation intensity of the pixel, |Y c | refers to Y cThe absolute value operation is performed. The fused transparency obtained by the fused transparency amplification process may exceed the transparency range. Therefore, the fused transparency obtained by the fused transparency amplification process can also be constrained according to the transparency range, and the fused transparency obtained after the constraint processing is used as the new fused transparency of the pixel point, wherein the process of constraining the fused transparency is similar to the process of constraining the shadow compensation intensity mentioned above.

[0125] Through the above-mentioned manner, accurate and effective shadow compensation processing can be achieved, so that in the final fused image, the shadows in the image to be processed can be further accurately simulated.

[0126] In some embodiments, shadow optimization may be performed on the foreground image, that is, the image to be processed in step 301 may be replaced by the foreground image.

[0127] In some embodiments, shadow optimization can be performed on the fused image, that is, the image to be processed in step 301 can be replaced by the fused image. Since there is no need to update the fusion transparency for the fused image, in this case, step 303 can be updated as follows: shadow compensation is performed on the brightness of the pixel according to the shadow compensation intensity of the pixel.

[0128] like Figure 3C As shown, the embodiment of the present application determines the shadow compensation intensity according to the brightness difference, and performs shadow compensation processing according to the shadow compensation intensity, which can achieve effective shadow optimization, so that in the final fused image, the shadow / highlight in the image to be processed can be accurately simulated.

[0129] In some embodiments, see Figure 3D , Figure 3D is a flowchart of an image processing method based on artificial intelligence provided in an embodiment of the present application, based on Figure 3A After step 102, in step 401, a transparency range may be obtained; wherein the transparency range includes a minimum transparency and a maximum transparency.

[0130] Here, the transparency range is the value range of the fusion transparency, which can be expressed as the interval of [minimum transparency, maximum transparency].

[0131] In step 402, the fusion transparency of the pixels is stretched according to the transparency range to update the fusion transparency of at least part of the pixels in the image to be processed to the minimum transparency, and update the fusion transparency of at least part of the remaining pixels to the maximum transparency.

[0132] Here, the fusion transparency of each pixel in the image to be processed is stretched according to the transparency range, so that after the stretching process, the distribution of the fusion transparency of each pixel in the image to be processed is smoother and more uniform, thereby improving the effect of subsequent image fusion processing. During the stretching process, the fusion transparency of at least some of the pixels in the image to be processed is updated to the minimum transparency, and among the remaining pixels, the fusion transparency of at least some of the pixels is updated to the maximum transparency.

[0133] In some embodiments, the above-mentioned stretching of the fusion transparency of pixels according to the transparency range can be achieved in the following manner: when the fusion transparency is less than the first transparency threshold, the fusion transparency is updated to the minimum transparency; when the fusion transparency is greater than or equal to the first transparency threshold and less than the second transparency threshold, the intermediate transparency is determined according to the fusion transparency, the first transparency threshold and the second transparency threshold, and the fusion transparency is updated to the intermediate transparency; when the fusion transparency is greater than or equal to the second transparency threshold, the fusion transparency is updated to the maximum transparency; wherein the first transparency threshold is less than the second transparency threshold; the first transparency threshold, the second transparency threshold and the intermediate transparency are all within the transparency range.

[0134] In an embodiment of the present application, the fusion transparency of the pixel points can be stretched according to the set first transparency threshold and the second transparency threshold, wherein the first transparency threshold and the second transparency threshold are both within the transparency range and can be set according to the actual application scenario. For example, when the transparency range is [0, 1], the first transparency threshold can be set to 0.001, and the second transparency threshold can be set to 0.01.

[0135] For each pixel in the image to be processed, when the fused transparency of the pixel is less than the first transparency threshold, the fused transparency of the pixel is updated to the minimum transparency; when the fused transparency of the pixel is greater than or equal to the first transparency threshold and less than the second transparency threshold, the intermediate transparency is determined according to the fused transparency of the pixel, the first transparency threshold and the second transparency threshold, and the fused transparency of the pixel is updated to the intermediate transparency; when the fused transparency of the pixel is greater than or equal to the second transparency threshold, the fused transparency of the pixel is updated to the maximum transparency. Among them, the intermediate transparency corresponding to the pixel is also within the transparency range and is positively correlated with the fused transparency of the pixel before the update. The embodiment of the present application does not limit the method of determining the intermediate transparency. For example, the intermediate transparency can be , where α represents the fusion transparency of the pixel before updating, T1 represents the first transparency threshold, and T2 represents the second transparency threshold. The embodiment of the present application provides the stretching processing method shown above, which can improve the effectiveness of the stretching processing.

[0136] like Figure 3D As shown, the embodiment of the present application can make the distribution of fusion transparency smoother and more uniform by stretching the fusion transparency of each pixel in the image to be processed, thereby improving the effect of subsequent image fusion processing.

[0137] In some embodiments, see Figure 3E , Figure 3E is a flowchart of an image processing method based on artificial intelligence provided in an embodiment of the present application, based on Figure 3A Before step 102, enumeration processing can also be performed according to the chromaticity range in step 501.

[0138] In the embodiment of the present application, the efficiency of image processing can be improved by accelerating the table lookup. For example, enumeration processing can be performed according to the chromaticity range (ie, the range of chromaticity values) to obtain all possible chromaticities.

[0139] In step 502, the fusion transparency corresponding to any chromaticity is determined according to the chromaticity difference between any chromaticity obtained by enumeration and the chromaticity of the background color.

[0140] For each chromaticity obtained by enumeration, the fusion transparency corresponding to the chromaticity is determined according to the chromaticity difference between the chromaticity and the chromaticity of the background color. In some embodiments, the fusion transparency corresponding to the chromaticity can also be stretched according to the transparency range.

[0141] In step 503, a fusion transparency table is established according to the multiple chromaticities obtained by enumeration processing and the fusion transparency corresponding to the multiple chromaticities.

[0142] Here, a fusion transparency table can be established according to all chromaticities obtained by enumeration and the fusion transparency corresponding to each chromaticity. The fusion transparency table stores the correspondence between chromaticity and fusion transparency, that is, chromaticity can be used as an index of fusion transparency.

[0143] It is worth noting that steps 501 to 503 may be performed before acquiring the image to be processed, thereby reducing adverse effects on the real-time image processing process (here, the process of performing image processing on the acquired image to be processed).

[0144] exist Figure 3E middle, Figure 3A The illustrated step 102 may be updated to step 504. In step 504, a query is performed in the fusion transparency table according to the chromaticity of the pixel in the image to be processed, and the queried fusion transparency is used as the fusion transparency of the pixel.

[0145] After obtaining the image to be processed, for each pixel in the image to be processed, a query can be performed in the fusion transparency table according to the chromaticity of the pixel, and the fusion transparency corresponding to the queried chromaticity can be used as the fusion transparency of the pixel. In this way, the efficiency of determining the fusion transparency of the pixel can be greatly improved.

[0146] In some embodiments, in addition to establishing a fused transparency table, other tables may be established to facilitate table lookup acceleration. For example, for the color optimization process, enumeration processing may be performed based on the transparency range and the chromaticity range to obtain all possible fused transparency-chromaticity combinations. For each fused transparency-chromaticity combination, the chromaticity is updated in a manner similar to steps 201 to 203. Then, an updated chromaticity table is established based on all fused transparency-chromaticity combinations and the updated chromaticity corresponding to each fused transparency-chromaticity combination. In the updated chromaticity table, the fused transparency-chromaticity combination may be used as an index of the updated chromaticity.

[0147] For another example, for the shadow optimization process, enumeration processing can be performed based on the transparency range and the brightness range to obtain all possible fused transparency-brightness combinations. For each fused transparency-brightness combination, shadow compensation processing is performed on the fused transparency and brightness in a manner similar to steps 301 to 303. Then, a compensated fused transparency table is established based on all fused transparency-brightness combinations and the fused transparency after shadow compensation processing corresponding to each fused transparency-brightness combination. In the compensated fused transparency table, the fused transparency-brightness combination can be used as an index of the fused transparency after shadow compensation processing; a compensated brightness table is established based on all fused transparency-brightness combinations and the brightness after shadow compensation processing corresponding to each fused transparency-brightness combination. In the compensated brightness table, the fused transparency-brightness combination can be used as an index of the brightness after shadow compensation processing. In the above manner, the comprehensiveness of table lookup acceleration can be improved.

[0148] like Figure 3E As shown, the embodiment of the present application can greatly improve the efficiency of image processing by accelerating the table lookup, while reducing the computational load of real-time image processing. In this way, it is convenient to deploy the image processing solution provided by the embodiment of the present application in various types of electronic devices (such as mobile phones).

[0149] The following will describe an exemplary application of the embodiment of the present application in a practical application scenario. The embodiment of the present application can be applied to a variety of scenarios related to image matting, which are described below with examples.

[0150] 1) In live broadcast / celebrity-watching scenarios, the real person in the image to be processed can be fused (superimposed) with a specific background to enhance the image perception. Figure 4 As shown, the fused image 41 includes the original character 42 in the image to be processed, and also includes the game virtual background and barrage in the background image, wherein the game virtual background and barrage are both presented in an area outside the area where the character 42 is located in the fused image 41.

[0151] 2) In virtual production, real people and virtual scenes can be mixed to achieve cool and flexible special effects, bringing a good special effects experience to variety shows, live broadcasts and on-demand programs. Figure 5 As shown, the image to be processed 51 includes a person 511 and a curtain 512, that is, the person 511 stands in front of the curtain 512, and the curtain 512 is a green curtain or a blue curtain, etc. After the image to be processed 51 is processed by the solution provided in the embodiment of the present application, a fused image 52 can be obtained, in which the curtain 512 is replaced by a virtual background 521, wherein the background color in the image processing process is the color of the curtain 512, and the virtual background 521 can be defined according to the requirements of the actual application scenario.

[0152] 3) The image processing solution provided in the embodiment of the present application can be deployed as an online service or an offline service, and thus applied to various types of image editing / video editing.

[0153] Compared with the solutions provided by related technologies (such as the super key algorithm), the embodiments of the present application can at least achieve the following technical effects: 1) It can retain the edge details in the image to be processed, achieve hair-level matting, and improve the fineness of image processing; 2) It can retain the shadows / highlights in the background area of ​​the image to be processed, ensuring that when used in combination with other special effect filters (such as virtual background), the effect is more realistic; 3) It can perform high-quality matting for translucent objects (such as glass cups) to ensure that the translucent objects in the fused image can be accurately presented; 4) It can alleviate the situation where the foreground is biased towards the background color (such as when using a green curtain, the characters in the final image are green), thereby improving the visual effect.

[0154] As evidence, the following are provided: Figure 6 As shown in the comparison diagram, after the image 61 to be processed is processed by the solution provided in the embodiment of the present application, a fused image 62 is obtained, and after the image 61 to be processed is processed by the super key algorithm provided by the related technology, an image 63 is obtained, wherein the virtual backgrounds in the fused image 62 and the image 63 are the same. Figure 6It can be determined that, for the hair of the person, compared with image 63, the fused image 62 retains the hair details in the image to be processed 61 more finely; for the translucent object, compared with image 63, the fused image 62 can present the translucent object more accurately without distortion or blurring; for the skin color, the fused image 62 can restore the skin color in the image to be processed 61 more, while the skin color in image 63 will have a reddish problem; for the shadow, compared with image 63, the fused image 62 can simulate the shadow in the image to be processed 61 more accurately. In addition, the person in the fused image 62 will not have a green edge (for example, the background in the image to be processed 61 is a green curtain, and the background color in the image processing process is green).

[0155] Next, the image processing solution provided by the embodiment of the present application is described from the perspective of the underlying implementation. The embodiment of the present application may include two steps, the first step is to calculate the foreground image and the transparent channel image according to the received image to be matted (corresponding to the image to be processed above), and the second step is to superimpose the foreground image on the background image, wherein, according to actual needs, the table lookup can be accelerated for step one, which will be explained later.

[0156] The present application embodiment provides Figure 7 The flowchart of calculating the foreground image and the transparent channel image is shown in FIG. 1 . The above step 1 can be implemented by steps 1 to 7. Figure 7 Provide explanation.

[0157] Step 1: Set the keying color C k (Corresponding to the background color above). The matte color can be set by the user according to the actual application scenario. For example, the matte color can be consistent with the color of the curtain in the image to be matted. For example, if the curtain in the image to be matted is a green curtain, the matte color can be set to the corresponding green.

[0158] Step 2: Set the image to be keyed I and keying color C k Convert to the same color space, where the color space at least includes the chrominance channel, for example, the color space can be YUV, LAB, HSL and HSV, etc., without limitation. For the sake of convenience, the color space here is a floating-point YUV color space as an example. In this way, when the image to be keyed I and the keyed color C k When converting to the YUV color space, the channel values ​​corresponding to the Y channel, the U channel, and the V channel are normalized to the range of [0, 1].

[0159] Step 3: Calculate the transparent channel image α based on hue and saturation. Step 3 can be implemented through steps ① and ②.

[0160] Step ①: Since the chroma channels of the YUV color space include the U channel and the V channel, the fusion transparency of each pixel in the image to be matted I is determined based on the U channel and the V channel. Taking the i-th pixel in the image to be matted I as an example, the fusion transparency of the i-th pixel can be determined by the following formula:

[0161] α i =(U i -U k ) 2 +(V i -V k ) 2

[0162] Among them, α i Indicates the fusion transparency of the i-th pixel, U i Indicates the channel value of the U channel corresponding to the i-th pixel, U k Indicates key color C k Corresponding to the channel value of U channel, V i Indicates the channel value of the V channel corresponding to the i-th pixel, V k Indicates key color C k Corresponds to the channel value of the V channel. In addition, (U i -U k ) 2 Indicates the chromaticity difference corresponding to the U channel, (V i -V k ) 2 Indicates the chromaticity difference corresponding to the V channel.

[0163] It is worth noting that the embodiment of the present application does not limit the order of counting pixels in the image to be matted I. For example, the counting can be performed in order from top to bottom and from left to right. For example, the first pixel may refer to the pixel located at the upper left corner of the image to be matted I, and the second pixel may refer to the adjacent pixel located to the right of the first pixel.

[0164] Step ②: Stretch the fusion transparency of each pixel according to the transparency range (i.e., the range of [0, 1]) to update the fusion transparency. i For example, the stretching function is described as follows:

[0165]

[0166] Wherein, T1 corresponds to the first transparency threshold mentioned above; T2 corresponds to the second transparency threshold mentioned above, and T1 is less than T2, for example, the value of T1 is 0.001, and the value of T2 is 0.01; Corresponds to the intermediate transparency above.

[0167] After the stretching process, a transparent channel image α can be obtained, and each fused transparency in the transparent channel image α corresponds to a pixel point in the image to be matted I.

[0168] Step 4: Copy the image I to be matted to obtain the foreground image I f .

[0169] Step 5: This step is an optional color optimization step and can be achieved through steps ① and ②.

[0170] Step 1: Remove the foreground image I f The keying color in to update the foreground image I f Among them, the foreground image I can be removed according to the set replacement color f The keying color and replacement color can be set according to the actual application scenario. Here, gray C gray Take this as an example to illustrate that the C gray In the RGB color space, the channel value of each channel is 128. f ) describes the process of de-keying color, the formula is as follows:

[0171] S(I f )=(1-α)·C gray +α·I f

[0172] Step 2: Remove foreground color, that is, remove the foreground image I f The foreground edge of the image has a color cast that is biased toward the keying color. Step ② can be implemented through steps a and b.

[0173] Step a: Foreground image I f and key color C k The conversion to the same color space is performed by taking the case where the color space is the HSV color space or the HSL color space as an example for the convenience of explanation. Of course, other color spaces, such as the YUV color space, may also be applied.

[0174] For the foreground image I f For each pixel in , determine the channel value H of the hue channel (i.e., H channel) corresponding to the pixel f At the same time, determine the keying color C k Corresponding to the channel value H of the H channel k Then, calculate H according to the following formula f With H k The error between (corresponding to the background color deviation above):

[0175]

[0176] Among them, d fIndicates H f With H k The error between them, 180 refers to 180 degrees in the color circle (color wheel), and 360 refers to 360 degrees.

[0177] Step b: According to the following function G(H f ) Update H f Values:

[0178]

[0179] Among them, "d f ≤-T3 or d f ≥T3” corresponds to the situation where the background color deviation reaches the set color deviation degree, so the hue of the pixel point is kept unchanged; “-T3<d f ≤0" and "0<d f The case of "<T3" corresponds to the case where the background color deviation does not reach the set color deviation degree, so the hue of the pixel is updated. In addition, T3 is the color deviation threshold, and the value of T3 is less than π. For example, T3 can be set to

[0180] Step 6: This step is an optional step for optimizing shadow details, which can be achieved through steps ① and ②.

[0181] Step 1: Determine the shadow compensation intensity (also called shadow and highlight compensation intensity) based on brightness. f and key color C k Convert to the same color space. Here we take the YUV color space as an example. Of course, other color spaces can also be used. f For each pixel in , determine the channel value Y of the brightness channel (i.e., Y channel) corresponding to the pixel f At the same time, determine the keying color C k Corresponding to the channel value Y of the Y channel k Then, the shadow compensation intensity of the pixel can be calculated according to the following formula:

[0182]

[0183] Among them, Y c Indicates the shadow compensation intensity of the pixel, Y f -Y k Corresponding to the brightness difference above, [-1, 1] corresponds to the intensity range above. m corresponds to the shadow compensation parameter above. The larger the value, the greater the intensity of shadow compensation. For example, it can be set to 1.0. When m is set to 0.0, it is equivalent to not performing shadow compensation processing.

[0184] Step ②: Perform shadow compensation on the brightness and fusion transparency of the pixel points according to the shadow compensation intensity of the pixel points. The formula is as follows:

[0185]

[0186] B(α)=clip((1-α)·|Y c |+α)

[0187]

[0188] Among them, A(Y f ) represents the brightness Y used to update the pixel f function, B(α) represents the function used to update the fusion transparency α of the pixel point, clip(x) represents the stage function, and the x in clip(x) has no actual meaning and is only used for reference. In addition, e is a parameter set to avoid numerical overflow, for example, it can be set to 0.01. In the above formula, the brightness range and transparency range are both [0, 1].

[0189] Step 7: Output foreground image I f And the transparent channel image α.

[0190] After completing step 1, we get the foreground image I f After obtaining the transparent channel image α, step 2, i.e., image fusion processing, can be performed. For example, the fused image (also called the synthesized result image) can be obtained according to the following formula:

[0191] I o =(1-α)·I b +α·I f

[0192] Among them, I b Represents the background image, which can be set according to the needs of the actual application scenario. o Denotes a fused image. When performing image fusion processing according to the above formula, weighted summation processing can be performed in units of pixels. It is worth noting that step 6 for shadow optimization above can also be performed after obtaining the fused image.

[0193] In the embodiment of the present application, the above step 1 can also be accelerated by table lookup to improve the efficiency of image processing. As an example, the embodiment of the present application provides the following Figure 8 The table lookup acceleration diagram shown in the figure will be in the form of steps, combined with Figure 8 Provide explanation.

[0194] Step 1: Set the keying color C k .

[0195] Step 2: Create a two-dimensional table t1(U f , V f ), used to look up the table to obtain the fusion transparency, the two-dimensional table t1(U f , V f ) corresponds to the above fusion transparency table. f The definition of U is the same as above. i Same, V f Similarly, U f and V f The value range of is an integer [0, P], where P is the result of raising 2 to the power of the number of image bits and then subtracting 1. The number of image bits refers to the number of bits of the image to be matted I. For example, when the image to be matted I is an 8-bit image, the value of P is 2 8 -1=255.

[0196] According to U f and V f The value range of U f and V f By enumerating, we can get 256×256=65536 combinations, each of which includes a U f And a V f For each combination obtained through enumeration, the fusion transparency is determined according to the following formula and stored in the two-dimensional table t1(U f , V f )middle:

[0197]

[0198]

[0199] The calculation process in the above formula can be floating-point calculation.

[0200] It is worth noting that for each combination obtained by enumeration processing, the U f and V f Determine the sequence number of the combination, for example, you can f As a high position, V f As the low bit, the sequence number q of the combination is obtained by splicing. After calculating the fusion transparency corresponding to the combination with the sequence number q, the fusion transparency can be stored in the two-dimensional table t1 (U f , V f ) is at the position of q in the subscript, so that the corresponding fusion transparency can be queried according to the subscript later, that is, the dimension of the table refers to the dimension of the subscript, and the subscript is equivalent to the index in the above text. The following table creation process is similar.

[0201] Step 3: Create a three-dimensional table u (α,U f, V f ) is used to look up the table to obtain the foreground image I f The pixel points in the corresponding channel values ​​of the U channel are used to establish a three-dimensional table t v (α,U f , V f ) is used to look up the table to obtain the foreground image I f The pixel points in the V channel correspond to the channel values, and the three-dimensional table t u (α,U f , V f ) and t v (α,U f , V f ) corresponds to the updated chromaticity table above. For each combination obtained by enumeration (the combination here includes α, U f and V f , where α represents the fusion transparency of a pixel), execute steps ① to ⑤.

[0202] Step 1: Update U f and V f The value of S(I f ) is similar to the following:

[0203]

[0204] Among them, I gray Corresponding to C above gray , for example, it can be set to gray, that is, the channel value of each channel in the RGB color space is 128. The x in S(x) is the U in the combination f and V f The pixel represented.

[0205] Step ②: Convert pixel point x to HSV or HSL color space and determine the channel value H of the H channel corresponding to pixel point x. x At the same time, the key color C k Convert to HSV or HSL color space and determine the key color C k Corresponding to the channel value H of the H channel k .

[0206] Step ③: Determine H according to the following formula x With H k The error between x :

[0207]

[0208] Step ④: Update H according to the following formula x Values:

[0209]

[0210] Among them, T3 is greater than 0 and less than π.

[0211] Step 5: Convert pixel x to YUV color space, and store the channel value of the U channel corresponding to pixel x in the three-dimensional table t u (α,U f , V f ), and at the same time store the channel value of the V channel corresponding to the pixel point x into the three-dimensional table t v (α,U f , V f )middle.

[0212] Step 4: Create a two-dimensional table t y (α, Y f ) is used to look up the table to obtain the foreground image I f The pixel points in the Y channel correspond to the channel values, and a two-dimensional table t is established. α (α, Y f ) is used to look up the table to obtain the foreground image I f The fusion transparency of the pixels in the two-dimensional table t y (α, Y f ) corresponds to the compensation brightness table above, the two-dimensional table t α (α, Y f ) corresponds to the above compensation fusion transparency table. For each combination obtained by enumeration processing (the combination here includes α and Y f , where α represents the fusion transparency of a pixel), execute steps ① to ③.

[0213] Step 1: Determine the shadow compensation intensity according to the following formula:

[0214]

[0215] Step 2: Brightness Y f And the fusion transparency α is used for shadow compensation to achieve the update. The formula is as follows:

[0216]

[0217]

[0218]

[0219] Step 3: Update the Y f Store in two-dimensional table t y (α, Y f ), and store the updated fusion transparency α in the two-dimensional table t α (α, Y f )middle.

[0220] Step 5: Process the image I to be matted, for example, by steps ① to ④.

[0221] Step ①: Convert the image I to be matted into the YUV color space to obtain the foreground image I f .

[0222] Step 2: Foreground image I f The following processing is performed for each pixel in the pixel: a subscript is determined according to the channel value of the U channel corresponding to the pixel and the channel value of the V channel corresponding to the pixel, and a subscript is added to the two-dimensional table t1(U f , V f ) to obtain the fusion transparency α of the pixel.

[0223] Step 3: Foreground image I f The following processing is performed for each pixel in: a subscript is determined according to the fusion transparency α of the pixel, the channel value of the corresponding U channel, and the channel value of the corresponding V channel, and a subscript is generated in the three-dimensional table t according to the subscript. u (α,U f , V f ) and updates the channel value of the U channel corresponding to the pixel point to the value in the three-dimensional table t u (α,U f , V f ) in the three-dimensional table t v (α,U f , V f ) and updates the channel value of the V channel corresponding to the pixel point to the channel value in the three-dimensional table t v (α,U f , V f )

[0224] Step ④: Foreground image I f The following processing is performed for each pixel in: a subscript is determined based on the fusion transparency α of the pixel and the channel value of the corresponding Y channel, and a subscript is generated in the two-dimensional table t according to the subscript. y (α, Y f ) and updates the channel value of the Y channel corresponding to the pixel point to the channel value in the two-dimensional table t y (α, Y f ) in the two-dimensional table t α (α, Y f ) and updates the fusion transparency α of the pixel point to the value in the two-dimensional table t α (α, Y f )

[0225] Step 6: Output foreground image I f and transparent channel images.

[0226] In the above-mentioned table lookup acceleration method, the corresponding table is pre-calculated and established, and when the matte image needs to be processed, it can be directly queried in the table, without the need to frequently convert the color space, which can effectively improve the efficiency of image processing and reduce the real-time computing load. It has been experimentally demonstrated that by applying the table lookup acceleration method, only one core of the central processing unit (CPU), such as one core of Intel E5 v4, is required for real-time image processing. At the same time, the memory occupancy is also low, only at the level of tens of megabytes (MB). In this way, the image processing solution provided in the embodiment of the present application can be deployed not only to the server side and the computer side, but also to the mobile side (such as a mobile terminal device).

[0227] The following is a description of an exemplary structure of an artificial intelligence-based image processing device 455 provided in an embodiment of the present application implemented as a software module. In some embodiments, Figure 2 As shown, the software modules in the artificial intelligence-based image processing device 455 stored in the memory 450 may include: a background color determination module 4551, used to determine the background color in the image to be processed; a transparency determination module 4552, used to determine the fusion transparency of the pixel points according to the chromaticity difference between the pixel points in the image to be processed and the background color; an image acquisition module 4553, used to acquire the background image corresponding to the image to be processed; an image fusion module 4554, used to perform image fusion processing on the image to be processed and the background image in units of pixels according to the fusion transparency of the pixel points in the image to be processed, so as to obtain a fused image.

[0228] In some embodiments, the artificial intelligence-based image processing device 455 also includes a color optimization module, which is used to: fuse the image to be processed and the replacement color in units of pixels according to the fusion transparency of the pixels in the image to be processed to obtain a foreground image; the image fusion module 4554 is also used to fuse the foreground image and the background image in units of pixels according to the fusion transparency of the pixels in the foreground image.

[0229] In some embodiments, the color optimization module is also used to: determine the background color deviation of the pixel point based on the chromaticity difference between the pixel point and the background color in the foreground image; when the background color deviation of the pixel point reaches a set color deviation degree, keep the chromaticity of the pixel point unchanged; when the background color deviation of the pixel point does not reach the set color deviation degree, deviate the chromaticity of the background color to obtain a deviated chromaticity, and update the chromaticity of the pixel point to the deviated chromaticity.

[0230] In some embodiments, the artificial intelligence-based image processing device 455 also includes a shadow optimization module, which is used to: determine the brightness difference between the pixel points in the image to be processed and the background color; weight the brightness difference according to the shadow compensation parameter to obtain the shadow compensation intensity of the pixel point; and perform shadow compensation processing on the brightness and fusion transparency of the pixel point according to the shadow compensation intensity of the pixel point.

[0231] In some embodiments, the shadow optimization module is also used to: constrain the shadow compensation intensity of the pixel points according to the intensity range; perform brightness amplification processing on the pixel points according to the shadow compensation intensity, brightness and fusion transparency of the pixel points, and constrain the brightness obtained by the brightness amplification processing according to the brightness range; perform fusion transparency amplification processing on the pixel points according to the shadow compensation intensity and fusion transparency of the pixel points, and constrain the fusion transparency obtained by the fusion transparency amplification processing according to the transparency range.

[0232] In some embodiments, the intensity range includes a minimum intensity and a maximum intensity; the shadow optimization module is further used to: when the shadow compensation intensity is less than the minimum intensity, update the shadow compensation intensity to the minimum intensity; when the shadow compensation intensity is within the intensity range, keep the shadow compensation intensity unchanged; when the shadow compensation intensity is greater than the maximum intensity, update the shadow compensation intensity to the maximum intensity.

[0233] In some embodiments, the artificial intelligence-based image processing device 455 also includes a stretching module, which is used to: obtain a transparency range; wherein the transparency range includes a minimum transparency and a maximum transparency; and stretch the fused transparency of the pixel points according to the transparency range to update the fused transparency of at least some of the pixel points in the image to be processed to the minimum transparency, and update the fused transparency of at least some of the remaining pixel points to the maximum transparency.

[0234] In some embodiments, the stretching module is also used to: when the fusion transparency is less than the first transparency threshold, update the fusion transparency to the minimum transparency; when the fusion transparency is greater than or equal to the first transparency threshold and less than the second transparency threshold, determine the intermediate transparency according to the fusion transparency, the first transparency threshold and the second transparency threshold, and update the fusion transparency to the intermediate transparency; when the fusion transparency is greater than or equal to the second transparency threshold, update the fusion transparency to the maximum transparency; wherein the first transparency threshold is less than the second transparency threshold; the first transparency threshold, the second transparency threshold and the intermediate transparency are all within the transparency range.

[0235] In some embodiments, the transparency determination module 4552 is also used to: convert the image to be processed and the background color to the same color space; wherein the color space includes multiple chromaticity channels; for each chromaticity channel, determine the difference between the channel value of the chromaticity channel corresponding to the pixel point in the image to be processed and the channel value of the chromaticity channel corresponding to the background color, and use the determined difference as the chromaticity difference corresponding to the chromaticity channel; perform difference fusion processing on the chromaticity differences corresponding to multiple chromaticity channels to obtain the fused transparency of the pixel point.

[0236] In some embodiments, the image fusion module 4554 is also used to: convert the image to be processed and the background image to the same color space; wherein the color space includes multiple channels; and perform the following processing for each channel: determine the first channel value of the channel corresponding to the first pixel in the image to be processed, and the second channel value of the channel corresponding to the second pixel in the background image; according to the fusion transparency of the first pixel, perform weighted processing on the first channel value and the second channel value to obtain the third channel value of the channel corresponding to the third pixel in the fused image; wherein there is a mapping relationship between the pixel positions corresponding to the first pixel, the second pixel, and the third pixel, respectively.

[0237] In some embodiments, the artificial intelligence-based image processing device 455 also includes a table building module, which is used to: perform enumeration processing according to the chromaticity range; determine the fusion transparency corresponding to any chromaticity based on the chromaticity difference between any chromaticity obtained by the enumeration processing and the chromaticity of the background color; establish a fusion transparency table based on multiple chromaticities obtained by the enumeration processing and the fusion transparencies corresponding to the multiple chromaticities; the transparency determination module 4552 is also used to: perform query processing in the fusion transparency table according to the chromaticity of the pixel point in the image to be processed, and use the queried fusion transparency as the fusion transparency of the pixel point.

[0238] In some embodiments, the background color determination module 4551 is also used to: determine the background color in the image to be processed by any of the following methods: obtain the background color set for the image to be processed, and the set background color is the background color in the image to be processed; perform target recognition processing on the image to be processed to obtain the target area, and use the color that appears most frequently in the background area of ​​the image to be processed as the background color; wherein the background area is an area in the image to be processed that is different from the target area.

[0239] The artificial intelligence-based image processing device provided in the embodiment of the present application can at least achieve the following technical effects: 1) During the image processing process, the edge details in the image to be processed can be retained, hair-level matting can be achieved, and the fineness of image processing can be improved; 2) The shadows / highlights in the background area of ​​the image to be processed can be retained, ensuring that after image fusion processing with the background image, the shadows / highlights in the fused image obtained are more realistic; 3) High-quality matting can be performed on translucent objects (such as glass cups) to ensure that the translucent objects in the fused image can be accurately presented; 4) The situation in which the foreground in the fused image is biased towards the background color (such as the character having a green glow) can be effectively avoided, which can improve the visual effect.

[0240] The embodiment of the present application provides a computer program product or a computer program, which includes computer instructions (i.e., executable instructions), which are stored in a computer-readable storage medium. The processor of the electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the electronic device performs the above-mentioned artificial intelligence-based image processing method in the embodiment of the present application.

[0241] The present application embodiment provides a computer-readable storage medium storing executable instructions, wherein the executable instructions are stored. When the executable instructions are executed by a processor, the processor will be caused to execute the method provided by the present application embodiment, for example, Figure 3A , Figure 3B , Figure 3C , Figure 3D and Figure 3E The image processing method based on artificial intelligence is shown.

[0242] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface storage, optical disk, or CD-ROM; or it may be various devices including one or any combination of the above memories.

[0243] In some embodiments, executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0244] As an example, executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file that stores other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions).

[0245] As an example, the executable instructions may be deployed to be executed on one electronic device, or on multiple electronic devices located at one site, or on multiple electronic devices distributed at multiple sites and interconnected by a communication network.

[0246] The above are only embodiments of the present application and are not intended to limit the protection scope of the present application. Any modifications, equivalent substitutions and improvements made within the spirit and scope of the present application are included in the protection scope of the present application.

Claims

1. An image processing method based on artificial intelligence, characterized in that: The method comprises: Determine the background color in the image to be processed; Converting the image to be processed and the background color to the same color space; wherein the color space includes multiple chromaticity channels; For each of the chromaticity channels, determining the difference between the channel value of the pixel point in the image to be processed corresponding to the chromaticity channel and the channel value of the background color corresponding to the chromaticity channel, and using the determined difference as the chromaticity difference corresponding to the chromaticity channel; Performing difference fusion processing on the chromaticity differences corresponding to the multiple chromaticity channels to obtain fusion transparency of the pixel points in the image to be processed; the difference fusion processing includes summation or weighted summation; Acquire a background image corresponding to the image to be processed; According to the fusion transparency of the pixels in the image to be processed, the image to be processed and the background image are subjected to image fusion processing in units of pixels to obtain a fused image.

2. The method according to claim 1, characterized in that After performing difference fusion processing on the chromaticity differences respectively corresponding to the multiple chromaticity channels to obtain the fusion transparency of the pixel points in the image to be processed, the method further includes: Converting the image to be processed and the replacement color to the same color space; For each of the chromaticity channels in the color space, according to the fusion transparency of each pixel in the image to be processed, a channel value of each pixel in the image to be processed corresponding to the chromaticity channel and a channel value of the replacement color corresponding to the chromaticity channel are weighted to obtain a foreground image; Wherein, the replacement color is used to remove the background color in the image to be processed; The step of performing image fusion processing on the image to be processed and the background image in units of pixels according to the fusion transparency of the pixels in the image to be processed comprises: According to the fusion transparency of the pixels in the foreground image, the foreground image and the background image are subjected to image fusion processing in units of pixels.

3. The method according to claim 2, characterized in that After performing weighted processing on the channel value of the chromaticity channel corresponding to each pixel point in the image to be processed and the channel value of the chromaticity channel corresponding to the replacement color to obtain the foreground image, the method further includes: Determining a background color cast of a pixel in the foreground image according to a chromaticity difference between the pixel in the foreground image and the background color; When the background color deviation of a pixel point in the foreground image reaches a set color deviation degree, the chromaticity of the pixel point in the foreground image is kept unchanged; When the background color deviation of the pixel in the foreground image does not reach the set color deviation degree, the chromaticity of the background color is deviated to obtain deviated chromaticity, and the chromaticity of the pixel in the foreground image is updated to the deviated chromaticity.

4. The method according to claim 1, characterized in that: After performing difference fusion processing on the chromaticity differences respectively corresponding to the multiple chromaticity channels to obtain the fusion transparency of the pixel points in the image to be processed, the method further includes: Determining the brightness difference between a pixel in the image to be processed and the background color; Performing weighted processing on the brightness difference according to the shadow compensation parameter to obtain the shadow compensation intensity of the pixel points in the image to be processed; According to the shadow compensation intensity of the pixel points in the image to be processed, shadow compensation processing is performed on the brightness and fusion transparency of the pixel points in the image to be processed.

5. The method according to claim 4, characterized in that After performing weighted processing on the brightness difference according to the shadow compensation parameter to obtain the shadow compensation intensity of the pixel points in the image to be processed, the method further includes: Performing constraint processing on the shadow compensation intensity of the pixel points in the image to be processed according to the intensity range; The shadow compensation processing is performed on the brightness and fusion transparency of the pixel points in the image to be processed according to the shadow compensation intensity of the pixel points in the image to be processed, including: Performing brightness amplification processing on the pixels in the image to be processed according to the shadow compensation intensity, brightness and fusion transparency of the pixels in the image to be processed, and performing constraint processing on the brightness obtained by the brightness amplification processing according to the brightness range, wherein the amplification intensity of the brightness amplification processing is positively correlated with the shadow compensation intensity of the pixels in the image to be processed; According to the shadow compensation intensity and fusion transparency of the pixels in the image to be processed, fusion transparency amplification processing is performed on the pixels in the image to be processed, and the fusion transparency obtained by the fusion transparency amplification processing is constrained according to the transparency range, and the amplification intensity of the fusion transparency amplification processing is positively correlated with the shadow compensation intensity of the pixels in the image to be processed.

6. The method according to claim 5, characterized in that The intensity range includes a minimum intensity and a maximum intensity; and the constraining process of the shadow compensation intensity of the pixel points in the image to be processed according to the intensity range includes: When the shadow compensation intensity is less than the minimum intensity, updating the shadow compensation intensity to the minimum intensity; When the shadow compensation intensity is within the intensity range, keeping the shadow compensation intensity unchanged; When the shadow compensation intensity is greater than the maximum intensity, the shadow compensation intensity is updated to the maximum intensity.

7. The method according to any one of claims 1 to 6, characterized in that: After performing difference fusion processing on the chromaticity differences respectively corresponding to the multiple chromaticity channels to obtain the fusion transparency of the pixel points in the image to be processed, the method further includes: Obtaining a transparency range; wherein the transparency range includes a minimum transparency and a maximum transparency; When the fusion transparency is less than the first transparency threshold, updating the fusion transparency to the minimum transparency; When the fusion transparency is greater than or equal to the first transparency threshold and less than the second transparency threshold, determining an intermediate transparency according to the fusion transparency, the first transparency threshold and the second transparency threshold, and updating the fusion transparency to the intermediate transparency; When the fusion transparency is greater than or equal to the second transparency threshold, the fusion transparency is updated to the maximum transparency; wherein the first transparency threshold is less than the second transparency threshold; the first transparency threshold, the second transparency threshold and the intermediate transparency are all within the transparency range.

8. The method according to any one of claims 1 to 6, characterized in that: The step of performing image fusion processing on the image to be processed and the background image in units of pixels according to the fusion transparency of the pixels in the image to be processed to obtain a fused image comprises: Converting the image to be processed and the background image to the same color space; The following processing is performed for each of the chroma channels: Determine that a first pixel in the image to be processed corresponds to a first channel value of the chromaticity channel, and a second pixel in the background image corresponds to a second channel value of the chromaticity channel; According to the fusion transparency of the first pixel point, weighted processing is performed on the first channel value and the second channel value to obtain a third channel value of the chromaticity channel corresponding to the third pixel point in the fused image; There is a mapping relationship between the pixel positions corresponding to the first pixel point, the second pixel point and the third pixel point respectively.

9. The method according to any one of claims 1 to 6, characterized in that: Before performing difference fusion processing on the chromaticity differences respectively corresponding to the multiple chromaticity channels to obtain the fusion transparency of the pixel points in the image to be processed, the method further includes: Enumerate according to the chromaticity range; Determining a fusion transparency corresponding to any one chromaticity according to a chromaticity difference between any one chromaticity obtained by the enumeration process and the chromaticity of the background color; Establishing a fusion transparency table according to the multiple chromaticities obtained by the enumeration process and the fusion transparency corresponding to the multiple chromaticities respectively; The step of determining the fusion transparency of the pixel points in the image to be processed according to the chromaticity difference between the pixel points in the image to be processed and the background color comprises: A query process is performed in the fusion transparency table according to the chromaticity of the pixel points in the image to be processed, and the queried fusion transparency is used as the fusion transparency of the pixel points in the image to be processed.

10. The method according to any one of claims 1 to 6, characterized in that: The step of determining the background color in the image to be processed includes: Determine the background color in the image to be processed by any of the following methods: Acquire a background color set for the image to be processed, wherein the set background color is the background color in the image to be processed; Performing target recognition processing on the image to be processed to obtain a target area, and taking the color that appears most frequently in the background area of ​​the image to be processed as the background color; The background area is an area in the image to be processed that is different from the target area.

11. An image processing device based on artificial intelligence, characterized in that: The device comprises: A background color determination module, used to determine the background color in the image to be processed; A transparency determination module, used for converting the image to be processed and the background color to the same color space; wherein the color space includes a plurality of chromaticity channels; for each of the chromaticity channels, determining the difference between the channel value of the pixel point in the image to be processed corresponding to the chromaticity channel and the channel value of the background color corresponding to the chromaticity channel, and using the determined difference as the chromaticity difference corresponding to the chromaticity channel; performing difference fusion processing on the chromaticity differences corresponding to the plurality of chromaticity channels respectively, to obtain the fusion transparency of the pixel point in the image to be processed; the difference fusion processing includes summation or weighted summation; an image acquisition module, used for acquiring the background image corresponding to the image to be processed; The image fusion module is used to perform image fusion processing on the image to be processed and the background image in units of pixels according to the fusion transparency of the pixels in the image to be processed, so as to obtain a fused image.

12. An electronic device, characterized in that: include: A memory for storing executable instructions; A processor, used to implement the artificial intelligence-based image processing method described in any one of claims 1 to 10 when executing the executable instructions stored in the memory.

13. A computer-readable storage medium, characterized in that: Executable instructions are stored, which are used to implement the artificial intelligence-based image processing method described in any one of claims 1 to 10 when executed by a processor.

14. A computer program product comprising computer executable instructions or a computer program, characterized in that When the computer executable instructions or computer program are executed by a processor, the artificial intelligence-based image processing method according to any one of claims 1 to 10 is implemented.

Images