Image processing method, device and storage medium
By occluding the image and training the animation image conversion model, identifying and processing the occlusion areas in the image, the shadow or bright spot area recognition problems caused by lighting are solved, and the animation quality and processing speed of the image are improved.
Patent Information
- Application Number
- CN202111045756.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-09-07
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2041-09-07
AI Technical Summary
In the process of animation of images, shadows or bright spots caused by lighting cannot be recognized, which reduces the quality of animation images.
By acquiring the image to be trained and its corresponding animation images for occlusion processing, the animation image conversion model is trained, and the occlusion area in the image to be processed is identified and processed, thereby improving the quality of the animation image.
Effectively identify and process the occluded areas in the image, improve the quality of the anime image when the image is anime, and improve the processing speed and the effect of background anime.
Smart Images

Figure CN113688937B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image processing technology, and in particular to an image processing method and device, and a storage medium. Background Art
[0002] With the continuous development of the animation industry, users are increasingly inclined to use artificial intelligence to transform portrait images into corresponding animation images.
[0003] In the prior art, the facial image of the target object in the real image is detected and the facial image is segmented, and then the segmented facial image is animated, and then the background in the real image is virtualized to obtain an animated image corresponding to the real image. Due to lighting reasons, the facial image may have shadows or bright spots, and the shadow or bright spot area cannot be recognized, so the shadow or bright spot area cannot be converted into an animated area, which reduces the quality of the animated image when the image is animated. Summary of the invention
[0004] In order to solve the above technical problems, the embodiments of the present application hope to provide an image processing method and device, and a storage medium, which can improve the quality of animated images when the images are animated.
[0005] The technical solution of this application is implemented as follows:
[0006] The present application provides an image processing method, which includes:
[0007] Acquire an image to be trained and an animation image to be trained corresponding to the image to be trained;
[0008] Performing occlusion processing on the image to be trained to obtain the occluded image to be trained;
[0009] Using the image to be trained, the masked image to be trained and the animation image to be trained to train an initial animation image conversion model to obtain an animation image conversion model;
[0010] When the image to be processed is received, the image to be processed is input into the animation image conversion model to obtain the animation image corresponding to the image to be processed.
[0011] The present application provides an image processing device, the device comprising:
[0012] An acquisition unit, used for acquiring an image to be trained and an animation image to be trained corresponding to the image to be trained;
[0013] A processing unit, used for performing occlusion processing on the image to be trained to obtain the occluded image to be trained;
[0014] A training unit, used for training an initial cartoon image conversion model using the image to be trained, the masked image to be trained and the cartoon image to be trained to obtain the cartoon image conversion model;
[0015] The input unit is used to input the image to be processed into the animation image conversion model when receiving the image to be processed, so as to obtain the animation image corresponding to the image to be processed.
[0016] The present application provides an image processing device, the device comprising:
[0017] A memory, a processor and a communication bus, wherein the memory communicates with the processor via the communication bus, and the memory stores an image processing program executable by the processor. When the image processing program is executed, the image processing method described above is executed by the processor.
[0018] An embodiment of the present application provides a storage medium having a computer program stored thereon, which is applied to an image processing device, and is characterized in that the computer program implements the above-mentioned image processing method when executed by a processor.
[0019] The embodiment of the present application provides an image processing method, device, and storage medium. The image processing method includes: obtaining an image to be trained and an animation image to be trained corresponding to the image to be trained; performing occlusion processing on the image to be trained to obtain an image to be trained after occlusion; training an initial animation image conversion model using the image to be trained, the image to be trained after occlusion, and the animation image to be trained to obtain an animation image conversion model; when receiving the image to be processed, inputting the image to be processed into the animation image conversion model to obtain an animation image corresponding to the image to be processed. By adopting the above method implementation scheme, the image processing device performs occlusion processing on the acquired image to be trained to obtain an image to be trained after occlusion, and trains the initial animation image conversion model using the image to be trained, the image to be trained after occlusion, and the animation image to be trained to obtain an animation image conversion model, so that the image processing device can use the animation image conversion model to identify the occlusion area in the occluded image to be processed, obtain an unoccluded image to be processed, and convert the unoccluded image to be processed into an animation image, thereby improving the quality of the animated image when the image is animated. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] Figure 1 A flow chart of an image processing method provided in an embodiment of the present application;
[0021] Figure 2 An exemplary image processing diagram provided in the embodiment of the present application Figure 1 ;
[0022] Figure 3An exemplary image processing diagram provided in the embodiment of the present application Figure 2 ;
[0023] Figure 4 An exemplary image processing diagram provided in the embodiment of the present application Figure 3 ;
[0024] Figure 5 An exemplary image processing diagram provided in the embodiment of the present application Figure 4 ;
[0025] Figure 6 A schematic diagram of the structure of an image processing device provided in an embodiment of the present application Figure 1 ;
[0026] Figure 7 A schematic diagram of the structure of an image processing device provided in an embodiment of the present application Figure 2 . DETAILED DESCRIPTION
[0027] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. It should be understood that the specific embodiments described here are only used to explain the present application and are not used to limit the present application.
[0028] Embodiment 1
[0029] The present application embodiment provides an image processing method, which is applied to an image processing device. Figure 1 A flowchart of an image processing method provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, the image processing method may include:
[0030] S101, obtaining an image to be trained and an animation image to be trained corresponding to the image to be trained.
[0031] An image processing method provided in an embodiment of the present application is suitable for training an animation image conversion model using an image to be trained, an animation image to be trained, and an image to be trained after occlusion, so as to use the animation image conversion model to convert the image to be processed into an animation image.
[0032] In the embodiments of the present application, the image processing device can be implemented in various forms. For example, the image processing device described in the present application can include devices such as mobile phones, cameras, tablet computers, laptop computers, PDAs, portable media players (PMPs), navigation devices, wearable devices, smart bracelets, pedometers, and devices such as digital TVs, desktop computers, and servers.
[0033] In the embodiments of the present application, the images to be trained and the animation images to be trained may be images transmitted to the image processing device from other devices; the images to be trained and the animation images to be trained may also be images transmitted by the client and received by the image processing device; the images to be trained and the animation images to be trained may also be images stored in the image processing device; the images to be trained and the animation images to be trained may also be images acquired by the image processing device in other ways; the specific way in which the image processing device acquires the images to be trained and the animation images to be trained may be determined according to actual conditions, and the embodiments of the present application do not limit this.
[0034] In the embodiment of the present application, the number of images to be trained is the same as the number of animation images to be trained.
[0035] It should be noted that the number of images to be trained can be 1,000; the number of images to be trained can also be 6,000; the number of images to be trained can also be 10,000; the specific number of images to be trained can be determined according to actual conditions, and the embodiments of the present application do not limit this.
[0036] In the embodiment of the present application, the image to be trained can be a real image, such as a photo image. Specifically, the image to be trained can be a portrait photo; the image to be trained can also be an animal photo; the image to be trained can also be a landscape photo; the image to be trained can also be other photo images; the specific image to be trained can be determined according to the actual situation, and the embodiment of the present application does not limit this.
[0037] It should be noted that the animation image to be trained is an animation image obtained based on the image to be trained and has a high similarity to the image to be trained. Specifically, the animation image to be trained can be an animation image manually drawn based on the image to be trained; the animation image to be trained can also be an animation image drawn based on the image to be trained using an animation image drawing device; the animation image to be trained can also be an animation image obtained by converting the image to be trained into an animation using an animation image training model; the specific animation image to be trained can be determined according to actual conditions, and the embodiments of the present application do not limit this.
[0038] In an embodiment of the present application, the process of an image processing device acquiring an image to be trained and an animation image to be trained corresponding to the image to be trained includes: the image processing device acquires a first initial image to be trained, a first initial animation image to be trained corresponding to the first initial image to be trained, and a second initial image to be trained; the image processing device uses the first initial image to be trained and the first initial animation image to be trained to train an initial animation image training model to obtain an animation image training model; the image processing device inputs the second initial image to be trained into the animation image training model to obtain a second initial animation image to be trained corresponding to the two initial images to be trained; the image processing device uses the first initial animation image to be trained and the second initial animation image to be trained as animation images to be trained; and uses the first initial image to be trained and the second initial image to be trained as images to be trained.
[0039] It should be noted that the first initial animation image to be trained is an animation image drawn based on the first initial animation image to be trained. Specifically, the first initial animation image to be trained is an animation image manually drawn based on the first initial animation image to be trained and has a high similarity to the first initial animation image to be trained.
[0040] For example, Figure 2 As shown: Figure 2 Figure (a) is a real image, i.e., the first initial image to be trained. Figure 2 Figure (b) in FIG. 1 is the first initial animation image to be trained corresponding to the first initial animation image to be trained.
[0041] In the embodiment of the present application, the first initial image to be trained is a partial image in the image to be trained, and the second initial image to be trained is an image in the image to be trained except the first initial image to be trained.
[0042] It should be noted that the number of the first initial images to be trained can be 1,000; the number of the first initial images to be trained can also be 5,000; the number of the first initial images to be trained can also be other numbers; the specific number of the first initial images to be trained can be determined according to actual conditions, and the embodiments of the present application do not limit this.
[0043] It should be noted that the number of the second initial images to be trained can be 5,000; the number of the second initial images to be trained can also be 8,000; the number of the second initial images to be trained can also be other numbers; the specific number of the second initial images to be trained can be determined according to actual conditions, and the embodiments of the present application do not limit this.
[0044] In the embodiment of the present application, the image processing device includes an initial cartoon image training model. Specifically, the initial cartoon image training model can be an existing Generative Adversarial Networks (GAN) pix2pix; the initial cartoon image training model can also be other network models; the specific initial cartoon image training model can be determined according to actual conditions, and the embodiment of the present application does not limit this.
[0045] In an embodiment of the present application, an information processing device trains an initial animation image training model by using a first initial image to be trained and a first initial animation image to be trained to obtain an animation image training model with high animation quality and similar to the initial image to be trained, and uses the animation image training model to perform animation conversion on an input second initial image to be trained to obtain a second initial animation image to be trained with high animation quality and similar to the second initial image to be trained.
[0046] For example, Figure 3 As shown: Figure 3 Figure (a) is a real image, i.e., the second initial image to be trained. Figure 3 Figure (b) in the figure is a second initial animation image to be trained obtained by animating the second initial animation image to be trained using the animation image training model.
[0047] It can be understood that since the animation image to be trained for training the initial animation image conversion model is the animation image corresponding to the image to be trained, that is, the image to be trained and the animation image to be trained have a one-to-one correspondence; the image processing device uses the image to be trained and the animation image to be trained with a one-to-one correspondence to train the initial animation image conversion model to obtain the animation image conversion model, so that the animation image conversion model can convert the image to be processed into an animation image with a high similarity with the image to be processed, thereby improving the accuracy of image animation.
[0048] It can also be understood that since the animation images to be trained and the images to be trained for training the initial animation image conversion model are both images with backgrounds, the obtained animation image conversion model can not only perform animation conversion on the face of the image to be processed, but also directly perform animation conversion on the background of the image to be processed. When the image processing device uses the animation image conversion model to process the image to be processed, there is no need to perform face detection, segmentation, etc. on the image to be processed, and the animation effects of the face and background can be completed at the same time, which improves the processing speed of the image processing device when using the animation image conversion model to process the image to be processed, and also improves the quality of the animated image when the background of the image to be processed is animated.
[0049] S102, performing occlusion processing on the image to be trained to obtain the occluded image to be trained.
[0050] In the embodiment of the present application, after the image processing device obtains the image to be trained and the animation image to be trained corresponding to the image to be trained, the image processing device can perform occlusion processing on the image to be trained to obtain the occluded image to be trained.
[0051] In an embodiment of the present application, the way in which the image processing device processes the occlusion of the training image includes the image processing device performing light spot occlusion processing on the training image; it may also be the image processing device performing shadow occlusion processing on the training image; or the image processing device performs light spot occlusion processing on the first position of the training image and performs shadow occlusion processing on the second position of the training image; the specific way in which the image processing device processes the occlusion of the training image can be determined according to actual conditions, and the embodiment of the present application is not limited to this.
[0052] In an embodiment of the present application, the image processing device performs occlusion processing on the training image to obtain the occluded image to be trained, including: the image processing device performs key point detection on the training image to obtain the key point coordinates; the image processing device determines the target area and / or target contour according to the key point coordinates; the image processing device determines the target position in the target area and / or target contour; the image processing device generates an occluded image at the target position to obtain the occluded image to be trained.
[0053] In an embodiment of the present application, the image processing device performs key point detection on the training image, and the method for obtaining the key point coordinates can be: the image processing device can perform 68 key point detection on the training image to obtain the coordinates of the 68 key points.
[0054] In an embodiment of the present application, if the image to be trained is a portrait image, a method of performing key point detection on the training image to obtain key point coordinates may be to perform face key point detection on the training image to obtain key point coordinates.
[0055] In an embodiment of the present application, if the image to be trained is a portrait image, the target area may be the area where the human eyes are located; or the target area may be the area where the mouth is located; the target area may also be other position areas, and the specific target area may be determined based on actual conditions, and the embodiment of the present application does not limit this.
[0056] In an embodiment of the present application, if the image to be trained is a portrait image, the target contour may be a face contour; the target contour may also be a portrait contour; the target contour may also be other contours; the specific target contour may be determined according to actual conditions, and the embodiment of the present application does not limit this.
[0057] In an embodiment of the present application, the manner in which the image processing device determines the target position in the target area and / or target contour may be that the image processing device determines the position of the eyes in the target area and / or target contour and uses the position as the target position; or the image processing device randomly selects a position in the target area and / or target contour to obtain the target position; or the image processing device determines the position of the face contour in the target area and / or target contour and uses the face contour position as the target position; the specific manner in which the image processing device determines the target position in the target area and / or target contour may be determined based on actual conditions, and the embodiment of the present application is not limited to this.
[0058] In an embodiment of the present application, an image processing device generates an occluded image at a target position to obtain an occluded image to be trained, comprising: the image processing device generates an occluded area at the target position according to an area generation rule; the image processing device adjusts the brightness of the occluded area according to a preset brightness adjustment method to obtain the occluded image to be trained.
[0059] In an embodiment of the present application, the region generation rule may be a rule configured in the image processing device; the region generation rule may also be a rule received by the image processing device before the image processing device generates an occlusion region at the target position according to the region generation rule; the region generation rule may also be a rule obtained by the image processing device in other ways. The specific way in which the image processing device obtains the region generation rule can be determined based on actual conditions, and the embodiment of the present application does not limit this.
[0060] In an embodiment of the present application, the region generation rule may be a rule for generating a smooth polygonal contour; the region generation rule may also be a rule for generating a smooth circular contour; the region generation rule may also be a rule for generating a smooth contour of an irregular shape; the specific region generation rule may be determined based on actual conditions, and the embodiment of the present application does not limit this.
[0061] In the embodiment of the present application, if the key point coordinates are the coordinates of the 68 key points, the occluded area may be an area including multiple key points among the 68 key points.
[0062] In an embodiment of the present application, the image processing device adjusts the brightness of the occluded area according to a preset brightness adjustment method. The image processing device can first obtain the V channel information of the HSV channel corresponding to the image to be trained, and adjust the brightness information of the V channel corresponding to the occluded area in the image to be trained according to the preset brightness adjustment method.
[0063] In an embodiment of the present application, the preset brightness adjustment method may be a brightness adjustment method configured in the image processing device; the preset brightness adjustment method may also be a brightness adjustment method transmitted by other devices and received by the image processing device before the image processing device adjusts the brightness of the occluded area according to the preset brightness adjustment method to obtain the occluded image to be trained; the preset brightness adjustment method may also be a brightness adjustment method obtained by the image processing device in other ways. The specific image processing device may determine the preset brightness adjustment method based on actual conditions, and the embodiment of the present application does not limit this.
[0064] It should be noted that the preset brightness adjustment method includes a preset bright spot brightness adjustment method and a preset shadow brightness adjustment method. For example, the preset bright spot brightness adjustment method can be a brightness adjustment method that increases the brightness of the blocked area by +70; the preset shadow brightness adjustment method can be a brightness adjustment method that decreases the brightness of the blocked area by -50.
[0065] In the embodiment of the present application, the preset bright spot brightness adjustment method can also be a method of increasing the brightness of the blocked area by another magnitude, and the specific preset bright spot brightness adjustment method can be determined according to actual conditions, and the embodiment of the present application does not limit this. The preset shadow brightness adjustment method can also be a method of reducing the brightness of the blocked area by another magnitude, and the specific preset shadow brightness adjustment method can be determined according to actual conditions, and the embodiment of the present application does not limit this.
[0066] For example, Figure 4 As shown, Figure 4 Figure (a) is the real image, that is, the image to be trained. Figure 4 Figure (b) shows the shadow-occluded image of the eye. Figure 4 Figure (c) shows the image with bright spots occluded by the chin contour. Figure 4 Figure (d) is the animation image to be trained.
[0067] S103, training an initial animation image conversion model using the image to be trained, the masked image to be trained, and the animation image to be trained to obtain an animation image conversion model.
[0068] In an embodiment of the present application, after the image processing device performs occlusion processing on the training image to obtain the occluded training image, the image processing device can use the training image, the occluded training image and the animation image to be trained to train the initial animation image conversion model to obtain the animation image conversion model.
[0069] In an embodiment of the present application, an image processing device uses an image to be trained, an image to be trained after occlusion, and an animation image to be trained to train an initial animation image conversion model to obtain a process of the animation image conversion model, including: the image processing device prunes the initial animation image conversion model to obtain the pruned initial animation image conversion model; the image processing device uses the image to be trained, the image to be trained after occlusion, and the animation image to be trained to train the pruned initial animation image conversion model to obtain the animation image conversion model.
[0070] In an embodiment of the present application, the initial animation image conversion model may be a model built using a GAN-like network; the initial animation image conversion model may also be other network models; the specific initial animation image conversion model may be determined based on actual conditions, and the embodiment of the present application does not limit this.
[0071] In an embodiment of the present application, the image processing device may first prune the initial cartoon image conversion model to obtain a pruned initial cartoon image conversion model; then the image processing device trains the pruned initial cartoon image conversion model using the image to be trained, the image to be trained after occlusion, and the cartoon image to be trained to obtain the cartoon image conversion model. The image processing device may also first train the initial cartoon image conversion model using the image to be trained, the image to be trained after occlusion, and the cartoon image to be trained to obtain a trained initial cartoon image conversion model; then the image processing device prunes the trained initial cartoon image conversion model to obtain the cartoon image conversion model. The specific way in which the image processing device obtains the cartoon image conversion model can be determined according to actual conditions, and the embodiment of the present application does not limit this.
[0072] In an embodiment of the present application, the image processing device prunes the initial cartoon image conversion model to obtain a pruned initial cartoon image conversion model. After the image processing device prunes the initial cartoon image conversion model, it determines the inference speed of the pruned model. When the inference speed of the pruned model meets the preset inference speed, the image processing device determines that the pruned model is the pruned initial cartoon image conversion model. Otherwise, the image processing device continues to prune the pruned model until the inference speed of the pruned model meets the preset inference speed.
[0073] It should be noted that, when the input image to be trained has a pixel value of 256×256, the preset inference speed may be up to 40+FPS, or the preset inference speed may be up to 70+FPS, or the preset inference speed may be other speed values, which may be determined according to actual conditions, and the embodiments of the present application do not limit this.
[0074] It can be understood that the image processing device prunes the initial animation image conversion model, that is, prunes the convolutional layers in the initial animation image conversion model that have little impact on the obtained animation image and / or reduces the number of convolutional kernels in each convolutional layer in the initial animation image conversion model, thereby improving the speed at which the initial animation image conversion model processes the image to be processed, and improving the speed at which the image processing device converts the image to be processed into the animation image.
[0075] S104: When receiving the image to be processed, input the image to be processed into a cartoon image conversion model to obtain a cartoon image corresponding to the image to be processed.
[0076] In an embodiment of the present application, the image processing device uses the image to be trained, the image to be trained after occlusion, and the animation image to be trained to train an initial animation image conversion model. After obtaining the animation image conversion model, when the image processing device receives the image to be processed, the image processing device can input the image to be processed into the animation image conversion model to obtain the animation image corresponding to the image to be processed.
[0077] For example, Figure 5 As shown, Figure 5 Figure (a) is the real image, i.e. the image to be processed. The right eye of the image to be processed is shadowed due to the brim of the hat. Figure 5 Figure (b) is an animated image obtained by using the animation image conversion model to convert the processed image into an animation image. A shadow is also generated corresponding to the right eye of the animation image, and the right eye of the animation image still maintains a big-eyed animation image.
[0078] It can be understood that the image processing device performs occlusion processing on the acquired image to be trained to obtain the occluded image to be trained, and uses the image to be trained, the occluded image to be trained and the animation image to be trained to train the initial animation image conversion model to obtain the animation image conversion model, so that the image processing device can use the animation image conversion model to identify the occluded area in the occluded image to be processed, obtain the unoccluded image to be processed, and convert the unoccluded image to be processed into the animation image, thereby improving the quality of the animated image when the image is animated.
[0079] Embodiment 2
[0080] Based on the same inventive concept of the first embodiment, the embodiment of the present application provides an image processing device 1, corresponding to an image processing method; Figure 6 A schematic diagram of the structure of an image processing device provided in an embodiment of the present application Figure 1 , the image processing device 1 may include:
[0081] An acquisition unit 11 is used to acquire an image to be trained and an animation image to be trained corresponding to the image to be trained;
[0082] A processing unit 12 is used to perform occlusion processing on the image to be trained to obtain the occluded image to be trained;
[0083] A training unit 13 is used to train an initial cartoon image conversion model using the image to be trained, the masked image to be trained and the cartoon image to be trained to obtain the cartoon image conversion model;
[0084] The input unit 14 is used to input the image to be processed into the animation image conversion model when receiving the image to be processed, so as to obtain the animation image corresponding to the image to be processed.
[0085] In some embodiments of the present application, the apparatus further comprises a detection unit, a determination unit, and a generation unit;
[0086] The detection unit is used to perform key point detection on the image to be trained to obtain key point coordinates;
[0087] The determination unit is used to determine the target area and / or target outline according to the key point coordinates; determine the target position in the target area and / or target outline;
[0088] The generating unit is used to generate a blocked image at the target position to obtain the blocked image to be trained.
[0089] In some embodiments of the present application, the device further comprises an adjustment unit;
[0090] The generating unit is used to generate a blocking area at the target position according to an area generating rule;
[0091] The adjustment unit is used to adjust the brightness of the blocked area according to a preset brightness adjustment method to obtain the blocked image to be trained.
[0092] In some embodiments of the present application, the acquisition unit 11 is used to acquire a first initial image to be trained, a first initial animation image to be trained corresponding to the first initial image to be trained, and a second initial image to be trained; the first initial animation image to be trained is an animation image drawn according to the first initial image to be trained;
[0093] The training unit 13 is used to train an initial animation image training model using the first initial image to be trained and the first initial animation image to be trained to obtain an animation image training model;
[0094] The input unit 14 is used to input the second initial image to be trained into the animation image training model to obtain a second initial animation image to be trained corresponding to the two initial images to be trained; use the first initial animation image to be trained and the second initial animation image to be trained as the animation images to be trained; use the first initial image to be trained and the second initial image to be trained as the images to be trained.
[0095] In some embodiments of the present application, the apparatus further comprises a pruning unit;
[0096] The pruning unit is used to prune the initial cartoon image conversion model to obtain a pruned initial cartoon image conversion model;
[0097] The training unit 13 is used to train the pruned initial cartoon image conversion model using the image to be trained, the occluded image to be trained and the cartoon image to be trained to obtain the cartoon image conversion model.
[0098] It should be noted that, in actual applications, the above-mentioned acquisition unit 11, processing unit 12, training unit 13 and input unit 14 can be implemented by a processor 15 on the image processing device 1, specifically a CPU (Central Processing Unit), an MPU (Microprocessor Unit), a DSP (Digital Signal Processing) or a field programmable gate array (FPGA); the above-mentioned data storage can be implemented by a memory 16 on the image processing device 1.
[0099] The present application also provides an image processing device 1, such as Figure 7 As shown, the image processing device 1 includes: a processor 15, a memory 16 and a communication bus 17. The memory 16 communicates with the processor 15 via the communication bus 17. The memory 16 stores a program executable by the processor 15. When the program is executed, the image processing method described above is executed by the processor 15.
[0100] In practical applications, the memory 16 may be a volatile memory, such as a random access memory (RAM); or a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk (HDD) or a solid-state drive (SSD); or a combination of the above types of memory, and provide instructions and data to the processor 15.
[0101] An embodiment of the present application provides a computer-readable storage medium having a computer program thereon, and when the program is executed by the processor 15, the image processing method as described above is implemented.
[0102] It can be understood that the image processing device performs occlusion processing on the acquired image to be trained to obtain the occluded image to be trained, and uses the image to be trained, the occluded image to be trained and the animation image to be trained to train the initial animation image conversion model to obtain the animation image conversion model, so that the image processing device can use the animation image conversion model to identify the occluded area in the occluded image to be processed, obtain the unoccluded image to be processed, and convert the unoccluded image to be processed into the animation image, thereby improving the quality of the animated image when the image is animated.
[0103] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of hardware embodiments, software embodiments, or embodiments in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) that contain computer-usable program code.
[0104] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0105] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0106] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0107] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application.
Claims
1. An image processing method, characterized in that: The method comprises: Obtaining an image to be trained and an animation image to be trained corresponding to the image to be trained; Performing occlusion processing on the image to be trained to obtain the occluded image to be trained; Using the image to be trained, the masked image to be trained, and the animation image to be trained, an initial animation image conversion model is trained to obtain an animation image conversion model; When an image to be processed is received, the image to be processed is input into the animation image conversion model to obtain an animation image corresponding to the image to be processed.
2. The method according to claim 1, characterized in that The occlusion processing of the image to be trained to obtain the occluded image to be trained includes: Performing key point detection on the image to be trained to obtain key point coordinates; Determine the target area and / or target contour according to the key point coordinates; determining a target position in the target area and / or target outline; An occluded image is generated at the target position to obtain the occluded image to be trained.
3. The method according to claim 2, characterized in that Generating an occluded image at the target position to obtain the occluded image to be trained includes: Generate an occlusion area at the target location according to an area generation rule; The brightness of the blocked area is adjusted according to a preset brightness adjustment method to obtain the blocked image to be trained.
4. The method according to claim 1, wherein The obtaining of the image to be trained and the animation image to be trained corresponding to the image to be trained includes: Acquire a first initial image to be trained, a first initial animation image to be trained corresponding to the first initial image to be trained, and a second initial image to be trained; the first initial animation image to be trained is an animation image drawn based on the first initial image to be trained; Using the first initial image to be trained and the first initial cartoon image to be trained to train an initial cartoon image training model to obtain a cartoon image training model; Inputting the second initial image to be trained into the animation image training model to obtain a second initial animation image to be trained corresponding to the second initial image to be trained; The first initial to-be-trained animated image and the second initial to-be-trained animated image are used as the to-be-trained animated images; and the first initial to-be-trained image and the second initial to-be-trained image are used as the to-be-trained images.
5. The method according to claim 1, characterized in that The method of training an initial cartoon image conversion model using the image to be trained, the masked image to be trained, and the cartoon image to be trained to obtain the cartoon image conversion model includes: Pruning the initial cartoon image conversion model to obtain a pruned initial cartoon image conversion model; The pruned initial animation image conversion model is trained using the image to be trained, the occluded image to be trained, and the animation image to be trained to obtain the animation image conversion model.
6. An image processing device, characterized in that The device comprises: An acquisition unit, configured to acquire an image to be trained and an animation image to be trained corresponding to the image to be trained; a processing unit, configured to perform occlusion processing on the image to be trained to obtain the occluded image to be trained; A training unit, configured to train an initial cartoon image conversion model using the image to be trained, the masked image to be trained, and the cartoon image to be trained, to obtain a cartoon image conversion model; The input unit is used to input the image to be processed into the animation image conversion model when receiving the image to be processed, so as to obtain the animation image corresponding to the image to be processed.
7. The device according to claim 6, characterized in that The apparatus further comprises a detection unit, a determination unit, and a generation unit; The detection unit is used to perform key point detection on the image to be trained to obtain key point coordinates; The determining unit is configured to determine a target area and / or a target contour according to the key point coordinates; determining a target position in the target area and / or target outline; The generating unit is configured to generate an occluded image at the target position to obtain the occluded image to be trained.
8. The device according to claim 7, characterized in that The device further comprises an adjustment unit; The generating unit is configured to generate an occlusion area at the target position according to an area generating rule; The adjustment unit is configured to adjust the brightness of the blocked area according to a preset brightness adjustment method to obtain the blocked image to be trained.
9. An image processing device, characterized in that: The device comprises: A memory, a processor and a communication bus, wherein the memory communicates with the processor via the communication bus, the memory stores an image processing program executable by the processor, and when the image processing program is executed, the method according to any one of claims 1 to 5 is performed by the processor.
10. A storage medium having a computer program stored thereon, applied to an image processing device, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.
