Image processing method and related equipment

Abstract

The invention provides an image processing method and related equipment. The image processing method comprises the steps that a first image with a first target object is acquired, and at least part of the first target object is covered by a shielding object; extracting a shielding object image from the first image; obtaining a second image with an unshielded second target object, wherein the first target object and the second target object are objects of the same type; performing image fusion on the shielding object image and the second image to obtain a target image; and training an initial model by using the target image to obtain an image processing model.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to an image processing method and related equipment. Background Art

[0002] In the related art, when deforming an image, the shape of the target object is usually changed. However, the inventors of the present disclosure have found that if there is an obstruction on the target object, the obstruction will also be deformed during the deformation process, and the image processing effect is not good. Summary of the invention

[0003] The present disclosure proposes an image processing method and related devices to solve or partially solve the above problems.

[0004] In a first aspect, the present disclosure provides an image processing method, comprising:

[0005] Acquire a first image with a first target object, wherein the first target object is at least partially covered by an obstruction;

[0006] Extracting an occlusion image from the first image;

[0007] Acquire a second image with an unobstructed second target object, wherein the first target object and the second target object are objects of the same type;

[0008] Performing image fusion on the occlusion image and the second image to obtain a target image;

[0009] The target image is used to train an initial model to obtain an image processing model.

[0010] In a second aspect of the present disclosure, there is provided an image processing method, comprising:

[0011] Get the image to be processed;

[0012] The image to be processed is processed using the image processing model obtained by the method described in the first aspect to obtain a processed image.

[0013] According to a third aspect of the present disclosure, there is provided an image processing device, comprising:

[0014] A first acquisition module is configured to: acquire a first image with a first target object, where the first target object is at least partially covered by an obstruction;

[0015] An extraction module is configured to: extract an occlusion image from the first image;

[0016] A second acquisition module is configured to: acquire a second image with an unobstructed second target object, wherein the first target object and the second target object are objects of the same type;

[0017] A fusion module is configured to: fuse the occlusion image with the second image to obtain a target image;

[0018] The training module is configured to: use the target image to train the initial model to obtain an image processing model.

[0019] In a fourth aspect of the present disclosure, there is provided an image processing device, comprising:

[0020] The acquisition module is configured to: acquire the image to be processed;

[0021] The processing module is configured to: process the image to be processed using the image processing model obtained by the device described in the third aspect to obtain a processed image.

[0022] In a fifth aspect of the present disclosure, a computer device is provided, comprising one or more processors, a memory; and one or more programs, wherein the one or more programs are stored in the memory and executed by the one or more processors, and the program includes instructions for executing the method according to the first aspect or the second aspect.

[0023] According to a sixth aspect of the present disclosure, a non-volatile computer-readable storage medium containing a computer program is provided. When the computer program is executed by one or more processors, the processors execute the method described in the first aspect or the second aspect.

[0024] In a seventh aspect of the present disclosure, a computer program product is provided, comprising computer program instructions, which, when executed on a computer, cause the computer to execute the method described in the first aspect or the second aspect.

[0025] The image processing method and related equipment provided by the embodiments of the present disclosure allow the model to learn the shape of the occluder in the image with the occluder, so that when the target object is deformed, the shape of the occluder can be kept as unaffected as possible, thereby obtaining a better deformation effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions in the present disclosure or related technologies, the drawings required for use in the embodiments or related technical descriptions are briefly introduced below. Obviously, the drawings described below are only embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0027] Figure 1 A schematic diagram showing an exemplary system provided by an embodiment of the present disclosure is shown.

[0028] Figure 2A A flowchart of an exemplary method provided by an embodiment of the present disclosure is shown.

[0029] Figure 2B A schematic flow chart of an exemplary method for extracting an occlusion image according to an embodiment of the present disclosure is shown.

[0030] Figure 2C A flowchart of an exemplary method for fusing images according to an embodiment of the present disclosure is shown.

[0031] Figure 3A A schematic diagram showing an exemplary first image according to an embodiment of the present disclosure is shown.

[0032] Figure 3B A schematic diagram of an exemplary occluder image according to an embodiment of the present disclosure is shown.

[0033] Figure 3C A schematic diagram showing another exemplary first image according to an embodiment of the present disclosure is shown.

[0034] Figure 3D A schematic diagram showing an exemplary second image according to an embodiment of the present disclosure is shown.

[0035] Figure 3E A schematic diagram showing another exemplary second image according to an embodiment of the present disclosure is shown.

[0036] Figure 3F A schematic diagram showing another exemplary occluder image according to an embodiment of the present disclosure is shown.

[0037] Figure 3G A schematic diagram showing an exemplary target image according to an embodiment of the present disclosure is shown.

[0038] Figure 4 A schematic diagram showing another exemplary method provided by an embodiment of the present disclosure is shown.

[0039] Figure 5 A schematic diagram of the hardware structure of an exemplary computer device provided in an embodiment of the present disclosure is shown.

[0040] Figure 6 A schematic diagram of an exemplary device provided by an embodiment of the present disclosure is shown.

[0041] Figure 7 A schematic diagram showing another exemplary device provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0042] In order to make the objectives, technical solutions and advantages of the present disclosure more clearly understood, the present disclosure is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0043] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present disclosure should be understood by people with ordinary skills in the field to which the present disclosure belongs. The "first", "second" and similar words used in the embodiments of the present disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. "Including" or "comprising" and similar words mean that the elements or objects appearing before the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0044] It is understandable that before using the technical solutions of each embodiment of the present disclosure, the type, scope of use, usage scenarios, etc. of the personal information involved will be informed to the user in an appropriate manner, and the user's authorization will be obtained.

[0045] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly remind the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can independently choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0046] As an optional but non-limiting implementation, in response to receiving the user's active request, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0047] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0048] Figure 1 A schematic diagram of an exemplary system 100 provided by an embodiment of the present disclosure is shown.

[0049] like Figure 1As shown, the system 100 may include a computer device 102, a terminal device 104, a server 106, and a database server 108. A medium (e.g., a network) providing a communication link may be included between the computer device 102 and the terminal device 104 and the server 106 and the database server 108. The network may include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.

[0050] In an exemplary model training and image processing scenario, user 110 can use computer device 102 to edit a training data set, and then upload the training data set to server 106 so that server 106 can use the training data set to train an initial model to obtain an image processing model. Various applications (APPs) that call the image processing model to process images can be downloaded and installed by terminal device 104, and then the images can be processed in terminal device 104.

[0051] Various applications (APPs) may be installed on the terminal device 104, such as video conferencing applications, reading applications, video applications, social applications, payment applications, web browsers, and instant messaging tools, etc. These applications may be used to display the delivered information.

[0052] The computer device 102 and the terminal device 104 here can be hardware or software. When the computer device 102 and the terminal device 104 are hardware, they can be various electronic devices with display screens, including but not limited to smart phones, tablet computers, e-book readers, MP3 players, laptop computers (Laptops) and desktop computers (PCs), etc. When the computer device 102 and the terminal device 104 are software, they can be installed in the electronic devices listed above. It can be implemented as multiple software or software modules (for example, to provide distributed services), or it can be implemented as a single software or software module. No specific limitation is made here.

[0053] The server 106 may be a server that provides various services, such as a background server that provides support for various applications displayed on the terminal device 104. The database server 108 may also be a database server that provides various services. It is understood that if the server 106 can implement the relevant functions of the database server 108, the database server 108 may not be set in the system 100.

[0054] Here, the server 106 and the database server 108 can also be hardware or software. When they are hardware, they can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. When they are software, they can be implemented as multiple software or software modules (for example, to provide distributed services), or as a single software or software module. No specific limitation is made here.

[0055] It should be noted that the method for generating a page provided in the embodiment of the present disclosure may be executed by the computer device 102 and / or the terminal device 104. It should be understood that Figure 1 The number of computer devices, terminal devices, users, servers and database servers in the embodiment is only for illustration. According to the implementation requirements, there can be any number of computer devices, terminal devices, users, servers and database servers.

[0056] As described in the background art, in the related art, when deforming an image, the shape of the target object is usually changed. However, this deformation process will also deform the occluders covering the target object, resulting in poor processing effect.

[0057] For example, in the task of exaggerating facial deformation stylization, the shape of the face and facial features can be controlled by allowing the model to learn a deformation region to obtain the desired exaggerated shape, but this often has a negative impact (e.g., deformation, distortion, etc.) on the occluders (or decorations) that appear around the face, such as the distortion of fingers and the deformation of glasses.

[0058] In view of this, an embodiment of the present disclosure provides an image processing method, which allows the model to learn the shape of the occluder in the image with the occluder, so that when the target object is deformed, the shape of the occluder can be kept as unaffected as possible, thereby obtaining a better deformation effect.

[0059] Figure 2A FIG. 2 is a flow chart of an exemplary method 200 provided in an embodiment of the present disclosure. The method 200 may be used to train an image processing model. Optionally, the method 200 may be Figure 1 The computer device 102 and the terminal device 104 can also be implemented by Figure 1 The method 200 is implemented by the server 106. The following is described using the server 106 implementing the method 200.

[0060] like Figure 2A As shown, the method 200 may further include the following steps.

[0061] In step 202, a first image with a first target object may be acquired, where the first target object is at least partially covered by an occluder.

[0062] Figure 3A A schematic diagram of an exemplary first image 300 according to an embodiment of the present disclosure is shown.

[0063] like Figure 3A As shown, the first image 300 may include a first target object 302 (eg, a face), and a portion of the first target object 302 is covered by an occluder 304 (eg, a hand).

[0064] As mentioned above, in the related art, when the target object is blocked by an occluder, the occluder will be deformed together when the target object is deformed, resulting in poor image processing effect. Therefore, in this step, a large amount of image data of occluders with rich shapes can be collected for subsequent model learning, so that the original appearance of the occluder can be retained during the deformation process, thereby improving the image processing effect.

[0065] In some embodiments, the number of the first images may be multiple. In order to provide better quality occluder images for subsequent model training, the multiple first images may include occluders of at least two postures, occluders of at least two lighting effects, and / or occluders of at least two image proportions.

[0066] As an optional embodiment, the high quality requirements of the first image with the occlusion may include the following aspects:

[0067] 1. The shapes of occluders in the image are rich, especially the data of hands. Try to include hand postures corresponding to more common photo postures; this ensures that the data used to train the model can reflect common occluder scenes;

[0068] 2. Rich lighting effects, such as dark light, colored light, side light and back light;

[0069] 3. The first target object (e.g., face) has various sizes. Since the collected targets are the occluders of the target object, the first target object is required to have a diverse proportion in the first image to ensure that the occluder body extracted in the subsequent steps has diverse resolutions.

[0070] Then, in some embodiments, if the occluders in the images in the original training data set have been deformed, the original training data set may be cleaned, for example, all data with occluders may be removed.

[0071] In step 204, an occluder image is extracted from the first image.

[0072] In some embodiments, extracting the occluder image from the first image may further include: using an image segmentation algorithm to segment the first image to obtain the occluder image, such as Figure 3B In this way, a better occlusion image can be extracted, which is beneficial to model training and has good training effect.

[0073] In some embodiments, Figure 2B As shown, using an image segmentation algorithm to segment the first image to obtain the occlusion image may further include:

[0074] In step 2042, the first target object 302 is detected in the first image 300. For example, an object detection algorithm may be used to detect the first target object 302 from the first image 300.

[0075] In step 2044, the first image is cropped to a target size (e.g., 512×512) according to the detected first target object. For example, the region where the first target object 302 is located is determined according to a detection frame drawn in the first image 300 by a target detection algorithm, and then the first image 300 is cropped according to the target size based on the detection frame, thereby obtaining a cropped first image.

[0076] In step 2046, the first target object in the cropped first image is aligned (for example, the first target object 302 is moved to the center of the cropped first image) to obtain an aligned first image 310, as shown in FIG. Figure 3C shown.

[0077] In step 2048, the image segmentation algorithm is used to segment the aligned first image to obtain the occluder image. Optionally, the aligned first image 310 may be processed with a segmentation algorithm to obtain a segmentation mask of the occluder, and then the mask is multiplied by the aligned first image 310 to obtain the occluder image 304, as shown in FIG. Figure 3B shown.

[0078] In this way, after preprocessing the first image 300 and obtaining the preprocessed first image 310, the image segmentation algorithm is used to segment the occlusion image 304 from the aligned first image 310, so that a better and more complete occlusion image 304 can be obtained, which is beneficial to model training and has a good training effect.

[0079] In step 206, a second image with an unobstructed second target object is acquired, where the first target object and the second target object are objects of the same type.

[0080] Figure 3D A schematic diagram of an exemplary second image 320 according to an embodiment of the present disclosure is shown.

[0081] like Figure 3D As shown, the second image 320 may include a second target object 322 (eg, a face), and the second target object 322 is completely not covered by any occluder.

[0082] In step 208, the occluder image is fused with the second image to obtain a target image. In this way, the occluder image is fused with the second image to obtain a target image, and then the target image is used to train the model, which can improve the robustness of the model, making the model more applicable and more effective.

[0083] In some embodiments, after extracting the occluder image from the first image, the method further includes: determining a target position, the target position being used to indicate the placement position of the occluder image on the second image; associating the target position with the occluder image to obtain the occluder image associated with the target position. The goal of this step is to construct paired data (paired data of occluder and target position) without occluder deformation, thereby increasing the diversity of data and simulating common positions of occluders.

[0084] In some embodiments, the target position is determined based on the key point position of the second target object in the second image, so as to better simulate the common position of the occluder. As an optional embodiment, when the second target object 322 is a face, the key point can be the key point of the face, for example, the key point of the corner of the mouth, the key point of the nose, the key point of the chin, the key point of the mandible, etc. When placing, the target position can be randomly selected, so as to improve the robustness of the model obtained by subsequent training.

[0085] In some embodiments, Figure 2C As shown, the step 208 of fusing the occluder image with the second image to obtain the target image may further include the following steps:

[0086] In step 2082, the occluder image is placed at the target position of the second image. Here, the target position is used as an anchor point, and the center of the occluder image can be placed at the position of the anchor point, thereby simulating the posture of a real scene.

[0087] Since the occluder image has a variety of colors and lighting effects, it would be awkward to directly paste it onto the second image. Figure 3E As shown, the color of the occluder image 304 is too different from the overall color of the second image 320, which will cause a sense of disharmony and is not conducive to training the model. Therefore, the color of the occluder image can be transferred to improve the sense of reality.

[0088] Therefore, in step 2084, the color migration process may be performed on the occluder image 304 according to the second image 320 on which the occluder image is placed.

[0089] The basic idea of ​​color migration is to determine a linear transformation based on the statistical analysis of the colored image so that the target image and the source image have the same mean and variance in the Lab color space.

[0090] Therefore, it is necessary to calculate the mean and standard deviation of the two images. Assume that l, a, b are the original data of the source image lαβ channel, L, A, B are the values ​​of the new source image lαβ channel after transformation, ml, ma, mb and ml', ma', mb' are the means of the three color channels of the source image and the colored image, and nl, na, nb and nl', na', nb' represent their standard deviations.

[0091] First, the original data of the source image (e.g., l, a, b) is subtracted from the mean value of the source image (e.g., ml, ma, mb);

[0092] Then the new data is scaled proportionally, and the scaling factor is the ratio of the standard deviations of the two images (for example, nl' / nl, na' / na, nb' / nb);

[0093] The new data obtained by scaling are added to the mean values ​​of the three channels of the target image (eg, ml', ma', mb') to obtain the final data.

[0094] In fact, according to the above processing steps, the final result is a linear equation, with the ratio of the standard deviations of the two images as the slope and the mean of each channel of the two images as a point. This simple linear transformation ensures that the target image and the colored image have the same mean and variance in the Lab color space, and the values ​​of the transformed lαβ are used as the values ​​of the three channels of the new image, and then the three channels are converted to RGB values ​​for display when displayed.

[0095] In view of this, in some embodiments, performing color migration processing on the obstruction image according to the second image on which the obstruction image is placed includes:

[0096] Converting the second image (target image) and the second image with the occluder image placed therein (source image) to a first color space (for example, Lab color space);

[0097] Respectively calculating a first mean and a first variance, a second mean and a second variance corresponding to the second image and the second image with the occlusion image placed thereon;

[0098] Performing color migration processing on the occluder image converted to the first color space according to the first mean, the first variance, the second mean, the second variance, and the second image to obtain the occluder image in the first color space;

[0099] The occluder image in the first color space is converted into a second color space (eg, RGB color space) to obtain the occluder image after color migration processing, such as Figure 3F shown.

[0100] In step 2086, the occluder image after color migration is fused with the second image to obtain the target image 330, such as Figure 3G shown.

[0101] Optionally, the mask corresponding to the occluder image is used to perform Laplacian pyramid fusion on the occluder image placed at the target position and the second image to obtain the expected result, such as Figure 3G shown.

[0102] In step 210, the target image is used to train the initial model to obtain an image processing model, so that the image processing model can learn images with occluders, thereby ensuring that the occluders are not affected as much as possible during deformation processing, thereby improving the image processing effect. Optionally, the initial model can be any machine learning model, such as a neural network model, a convolutional neural network model, a Transformer model, and the like.

[0103] In some embodiments, the processed training data set (a set of target images 330) can be mixed into the cleaned, occlusion-free data set described above in a certain proportion and used as the final training data set to train the initial model, thereby obtaining an image processing model.

[0104] It can be seen from the above embodiments that the image processing method provided by the embodiments of the present disclosure, by collecting a large amount of image data of occluders with rich shapes, uses the segmented occluder images to produce a large amount of image data with undeformed occluders, and then uses these data to train a stylized model with exaggerated facial feature deformation, so that external occluders that appear around the target object, especially decorations such as hands and glasses, are not affected by the exaggerated deformation of the face and facial features.

[0105] It should be noted that the method of the embodiment of the present disclosure can be performed by a single device, such as a computer or a server. The method of the present embodiment can also be applied in a distributed scenario and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only perform one or more steps in the method of the embodiment of the present disclosure, and the multiple devices will interact with each other to complete the described method.

[0106] It should be noted that the above describes some embodiments of the present disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the above embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0107] Figure 4 FIG. 4 is a flow chart of an exemplary method 400 provided in an embodiment of the present disclosure. The method 400 may be used to process an image. Optionally, the method 400 may be Figure 1 The computer device 102 and the terminal device 104 can also be implemented by Figure 1 The method 400 is implemented by the server 106. The following description is made by taking the terminal device 104 or the server 106 as implementing the method 400.

[0108] like Figure 4 As shown, the method 400 may further include the following steps.

[0109] In step 402, an image to be processed is obtained.

[0110] In step 404, the image to be processed is processed using the image processing model trained by method 200 to obtain a processed image. External obstructions around the target object, especially decorations such as hands and glasses, are not affected by the exaggerated deformation of the target object (for example, face and facial features), and the image processing effect is good.

[0111] The embodiment of the present disclosure further provides a computer device for implementing the above method 200 or 400. Figure 5 FIG. 5 is a schematic diagram showing the hardware structure of an exemplary computer device 500 provided in an embodiment of the present disclosure. The computer device 500 can be used to implement Figure 1 The server 106 can also be used to implement Figure 1 The computer device 102 and the terminal device 104. In some scenarios, the computer device 500 can also be used to implement Figure 1 The database server 108.

[0112] like Figure 5 As shown, the computer device 500 may include: a processor 502, a memory 504, a network module 506, a peripheral interface 508 and a bus 510. The processor 502, the memory 504, the network module 506 and the peripheral interface 508 are connected to each other in communication within the computer device 500 through the bus 510.

[0113] Processor 502 may be a central processing unit (CPU), an image processor, a neural network processor (NPU), a microcontroller (MCU), a programmable logic device, a digital signal processor (DSP), an application specific integrated circuit (ASIC), or one or more integrated circuits. Processor 502 may be used to perform functions related to the technology described in this disclosure. In some embodiments, processor 502 may also include multiple processors integrated into a single logical component. For example, Figure 5 As shown, processor 502 may include a plurality of processors 502a, 502b, and 502c.

[0114] The memory 504 may be configured to store data (eg, instructions, computer code, etc.). Figure 5 As shown, the data stored in the memory 504 may include program instructions (e.g., program instructions for implementing the method 200 or 400 of the embodiment of the present disclosure) and data to be processed (e.g., the memory may store configuration files of other modules, etc.). The processor 502 may also access the program instructions and data stored in the memory 504, and execute the program instructions to operate on the data to be processed. The memory 504 may include a volatile storage device or a non-volatile storage device. In some embodiments, the memory 504 may include a random access memory (RAM), a read-only memory (ROM), an optical disk, a magnetic disk, a hard disk, a solid-state drive (SSD), a flash memory, a memory stick, etc.

[0115] The network interface 506 can be configured to provide the computer device 500 with communication with other external devices via a network. The network can be any wired or wireless network capable of transmitting and receiving data. For example, the network can be a wired network, a local wireless network (e.g., Bluetooth, WiFi, near field communication (NFC)), a cellular network, the Internet, or a combination thereof. It is understood that the type of network is not limited to the above specific examples.

[0116] The peripheral interface 508 can be configured to connect the computer device 500 to one or more peripheral devices to achieve information input and output. For example, the peripheral devices can include input devices such as a keyboard, a mouse, a touch pad, a touch screen, a microphone, and various sensors, and output devices such as a display, a speaker, a vibrator, and an indicator light.

[0117] The bus 510 may be configured to transmit information between various components of the computer device 500 (e.g., the processor 502, the memory 504, the network interface 506, and the peripheral interface 508), such as an internal bus (e.g., a processor-memory bus), an external bus (USB port, PCI-E bus), etc.

[0118] It should be noted that, although the architecture of the above-mentioned computer device 500 only shows the processor 502, the memory 504, the network interface 506, the peripheral interface 508 and the bus 510, in the specific implementation process, the architecture of the computer device 500 may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the architecture of the above-mentioned computer device 500 may also only include the components necessary for implementing the embodiments of the present disclosure, and does not necessarily include all the components shown in the figure.

[0119] The embodiment of the present disclosure also provides an image processing device. Figure 6 FIG. 6 is a schematic diagram of an exemplary device 600 provided by an embodiment of the present disclosure. Figure 6 As shown, the device 600 can be used to implement the method 200, and can further include the following modules.

[0120] The first acquisition module 602 is configured to: acquire a first image with a first target object, where the first target object is at least partially covered by an obstruction;

[0121] The extraction module 604 is configured to: extract an occlusion image from the first image;

[0122] The second acquisition module 606 is configured to: acquire a second image with an unobstructed second target object, wherein the first target object and the second target object are objects of the same type;

[0123] The fusion module 608 is configured to: perform image fusion on the occlusion image and the second image to obtain a target image;

[0124] The training module 610 is configured to: use the target image to train the initial model to obtain an image processing model.

[0125] In some embodiments, the extraction module 604 is configured to: segment the first image to obtain the occlusion image using an image segmentation algorithm.

[0126] In some embodiments, the extraction module 604 is configured to:

[0127] detecting the first target object in the first image;

[0128] According to the detected first target object, cropping the first image to a target size;

[0129] Performing alignment processing on the first target object in the cropped first image to obtain an aligned first image;

[0130] The occluder image is obtained by segmenting the aligned first image using the image segmentation algorithm.

[0131] In some embodiments, the fusion module 608 is configured to:

[0132] Determine a target position, where the target position is used to indicate a placement position of the occlusion image on the second image;

[0133] The target position is associated with the obstruction image to obtain the obstruction image associated with the target position.

[0134] In some embodiments, the target position is determined based on key point positions of the second target object in the second image.

[0135] In some embodiments, the fusion module 608 is configured to:

[0136] Placing the occluder image at the target position of the second image;

[0137] According to the second image on which the obstruction image is placed, performing color migration processing on the obstruction image;

[0138] The occluder image after color migration processing is fused with the second image to obtain the target image.

[0139] In some embodiments, the fusion module 608 is configured to:

[0140] Converting the second image and the second image with the occluder image placed therein into a first color space;

[0141] Respectively calculating a first mean and a first variance, a second mean and a second variance corresponding to the second image and the second image with the occlusion image placed thereon;

[0142] Performing color migration processing on the occluder image converted to the first color space according to the first mean, the first variance, the second mean, the second variance, and the second image to obtain the occluder image in the first color space;

[0143] The occluder image in the first color space is converted into the second color space to obtain the occluder image after color migration processing.

[0144] In some embodiments, the first image includes multiple images, and the multiple first images include occluders of at least two postures, occluders of at least two lighting effects, and / or occluders of at least two image proportions.

[0145] For the convenience of description, the above device is described by dividing it into various modules according to its functions. Of course, when implementing the present disclosure, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0146] The device of the above embodiment is used to implement the corresponding method 200 in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0147] The embodiment of the present disclosure also provides an image processing device. Figure 7 FIG. 7 is a schematic diagram of an exemplary device 700 provided by an embodiment of the present disclosure. Figure 7 As shown, the device 700 can be used to implement the method 400, and can further include the following modules.

[0148] The acquisition module 702 is configured to: acquire an image to be processed;

[0149] The processing module 704 is configured to: process the image to be processed using the image processing model trained by the apparatus 600 to obtain a processed image.

[0150] For the convenience of description, the above device is described by dividing it into various modules according to its functions. Of course, when implementing the present disclosure, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0151] The device of the above embodiment is used to implement the corresponding method 400 in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0152] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present disclosure also provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute method 200 or 400 described in any of the above embodiments.

[0153] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0154] The computer instructions stored in the storage medium of the above embodiments are used to enable the computer to execute method 200 or 400 as described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0155] Based on the same inventive concept, corresponding to the method 200 or 400 in any of the above embodiments, the present disclosure further provides a computer program product, which includes a computer program. In some embodiments, the computer program is executable by one or more processors so that the processors execute the method 200 or 400. Corresponding to the execution subject corresponding to each step in each embodiment of the method 200 or 400, the processor that executes the corresponding step may belong to the corresponding execution subject.

[0156] The computer program product of the above embodiment is used to enable the processor to execute method 200 or 400 as described in any of the above embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be described in detail here.

[0157] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present disclosure (including the claims) is limited to these examples. Based on the concept of the present disclosure, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present disclosure as described above, which are not provided in detail for the sake of simplicity.

[0158] In addition, to simplify the description and discussion, and in order not to make the embodiments of the present disclosure difficult to understand, the known power / ground connections to the integrated circuit (IC) chips and other components may or may not be shown in the provided figures. In addition, the device can be shown in the form of a block diagram to avoid making the embodiments of the present disclosure difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform on which the embodiments of the present disclosure will be implemented (that is, these details should be fully within the scope of understanding of those skilled in the art). Where specific details (e.g., circuits) are set forth to describe exemplary embodiments of the present disclosure, it is apparent to those skilled in the art that the embodiments of the present disclosure can be implemented without these specific details or with changes in these specific details. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0159] Although the present disclosure has been described in conjunction with specific embodiments of the present disclosure, many replacements, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.

[0160] The embodiments of the present disclosure are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present disclosure should be included in the scope of protection of the present disclosure.

Claims

1. An image processing method, comprising: Acquire a first image with a first target object, wherein the first target object is at least partially covered by an obstruction; Extracting an occlusion image from the first image; Acquire a second image with an unobstructed second target object, wherein the first target object and the second target object are objects of the same type; Performing image fusion on the occlusion image and the second image to obtain a target image; The target image is used to train an initial model to obtain an image processing model.

2. The method of claim 1, wherein: Extracting an occlusion image from the first image includes: The occlusion image is obtained by segmenting the first image using an image segmentation algorithm.

3. The method of claim 2, wherein: Segmenting the first image to obtain the occlusion image using an image segmentation algorithm includes: detecting the first target object in the first image; According to the detected first target object, cropping the first image to a target size; Performing alignment processing on the first target object in the cropped first image to obtain an aligned first image; The occluder image is obtained by segmenting the aligned first image using the image segmentation algorithm.

4. The method of claim 1, wherein: After extracting the occluder image from the first image, the method further includes: Determine a target position, where the target position is used to indicate a placement position of the occlusion image on the second image; The target position is associated with the obstruction image to obtain the obstruction image associated with the target position.

5. The method of claim 4, wherein: The target position is determined according to the key point positions of the second target object in the second image.

6. The method of claim 4, wherein: The method of fusing the occluder image with the second image to obtain a target image includes: Placing the occluder image at the target position of the second image; According to the second image on which the obstruction image is placed, performing color migration processing on the obstruction image; The occluder image after color migration processing is fused with the second image to obtain the target image.

7. The method of claim 6, wherein: According to the second image on which the obstruction image is placed, color migration processing is performed on the obstruction image, including: Converting the second image and the second image with the occluder image placed therein into a first color space; Respectively calculating a first mean and a first variance, a second mean and a second variance corresponding to the second image and the second image with the occlusion image placed thereon; Performing color migration processing on the occluder image converted to the first color space according to the first mean, the first variance, the second mean, the second variance, and the second image to obtain the occluder image in the first color space; The occluder image in the first color space is converted into the second color space to obtain the occluder image after color migration processing.

8. The method of claim 1, wherein: The first images include multiple images, and the multiple first images include occluders of at least two postures, occluders of at least two lighting effects, and / or occluders of at least two image proportions.

9. An image processing method, comprising: Get the image to be processed; The image to be processed is processed using the image processing model obtained by the method according to any one of claims 1 to 8 to obtain a processed image.

10. An image processing device, comprising: A first acquisition module is configured to: acquire a first image with a first target object, where the first target object is at least partially covered by an obstruction; An extraction module is configured to: extract an occlusion image from the first image; A second acquisition module is configured to: acquire a second image with an unobstructed second target object, wherein the first target object and the second target object are objects of the same type; A fusion module is configured to: fuse the occlusion image with the second image to obtain a target image; The training module is configured to: use the target image to train the initial model to obtain an image processing model.

11. An image processing device, comprising: The acquisition module is configured to: acquire the image to be processed; The processing module is configured to: process the image to be processed using the image processing model obtained by the device according to claim 10 to obtain a processed image.

12. A computer device comprising one or more processors, a memory; and one or more programs, wherein the one or more programs are stored in the memory and executed by the one or more processors, and the programs include instructions for executing the method according to any one of claims 1 to 8 or the method according to claim 9.

13. A non-volatile computer-readable storage medium containing a computer program, which, when executed by one or more processors, causes the processors to perform the method according to any one of claims 1 to 8 or the method according to claim 9.

14. A computer program product, comprising computer program instructions, which, when executed on a computer, enable the computer to execute the method according to any one of claims 1 to 8 or the method according to claim 9.

Images