Methods for drawing prominent object outlines, methods for animate scene images, and related devices

Abstract

This application provides a method for drawing salient object contours, a method for animating scene images, and related apparatus, relating to the field of image processing technology. This application uses a salientity detection model to detect salient objects in a target scene image requiring animation processing, obtaining a salient image. Then, it calls fine-grained edge detection and coarse-grained edge detection models to perform image edge detection on the target scene image, obtaining fine-grained edge images and coarse-grained edge images respectively. Next, based on the fine-grained and coarse-grained edge images, it performs contour edge calibration on the salient object parts of the salient image, obtaining a salient object contour image of the target scene image. Finally, it draws contour lines on the target scene image based on the salient object contour image, obtaining a scene image to be animated that includes the salient object contour lines. This ensures that the final scene animation image can effectively delineate the contour lines of salient objects and achieve the desired flat-painted animation effect.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and more specifically, to a method for drawing the outline of a salient object, a method for animateting scene images, and related apparatus. Background Technology

[0002] With the rapid development of artificial intelligence technology and image- or video-based social media technology, image animation technology has been widely used in video communication, live streaming and other fields to improve user experience because it can convert real photos into animated images with the same semantic content and specific animation style effects. Scene image animation is an important research direction of image animation technology.

[0003] Currently, for flat-painted anime-style scene animation images, the boundaries between different color blocks are obvious, the color blocks are smooth, and the outlines of prominent objects (e.g., prominent characters or objects) in the scene animation image have clear lines, while the textures of non-prominent objects (e.g., buildings, trees, mountains and rivers) are more coarse.

[0004] However, it is worth noting that existing flat-color anime style scene image animation solutions often fail to clearly outline the contour lines of prominent objects in the scene animation images during the actual implementation process. They also fail to perform differentiated conversion processing on the image textures of prominent and non-prominent object parts in real anime images, resulting in the final scene animation images failing to achieve the expected flat-color anime effect. Summary of the Invention

[0005] In view of this, the purpose of this application is to provide a method and apparatus for drawing the outline of a salient object, a method and apparatus for animate scene images, a computer device and a readable storage medium, which can effectively draw the outline lines of salient objects in real scene images that need to be animate, so that the scene animation image obtained by subsequent image animation processing can effectively outline the outline lines of salient objects, and ensure that the scene animation image achieves the expected flat-color animation effect.

[0006] To achieve the above objectives, the technical solutions adopted in the embodiments of this application are as follows:

[0007] In a first aspect, this application provides a method for drawing the outline of a salient object, the method comprising:

[0008] The pre-stored saliency detection model is invoked to perform salient object detection on the target scene image that needs to be animated, thereby obtaining the saliency image of the target scene image;

[0009] The pre-stored fine-grained edge detection model and coarse-grained edge detection model are called to perform image edge detection on the target scene image, respectively, to obtain the fine-grained edge image and coarse-grained edge image of the target scene image;

[0010] Based on the fine-grained edge image and the coarse-grained edge image, the salient object region of the salient image is calibrated to obtain the salient object contour image of the target scene image;

[0011] Based on the salient object contour image, the target scene image is drawn with contour lines to obtain a scene image to be animated, which includes the salient object contour lines.

[0012] In an optional implementation, the step of calling a pre-stored saliency detection model to perform salient object detection on the target scene image requiring image animation processing, and obtaining a salient image of the target scene image, includes:

[0013] The target scene image is input into the saliency detection model to identify salient objects, thereby obtaining the corresponding initial saliency map;

[0014] The initial saliency map is dilated by calling the first preset convolution kernel to obtain the saliency image of the target scene image.

[0015] In an optional implementation, the step of calling a pre-stored coarse-grained edge detection model to perform image edge detection on the target scene image to obtain a coarse-grained edge image of the target scene image includes:

[0016] The target scene image is input into the coarse-grained edge detection model to perform image edge recognition, and the corresponding coarse-grained edge grayscale image is obtained.

[0017] A preset sliding window is invoked to perform sliding adaptive threshold binarization processing on the coarse-grained edge grayscale image to obtain the corresponding coarse-grained edge binarized image.

[0018] The second preset convolution kernel is invoked to perform image dilation processing on the coarse-grained edge binarization image to obtain the coarse-grained edge image of the target scene image.

[0019] In an optional implementation, the fine-grained edge image, the coarse-grained edge image, and the salient image are all binarized images. The step of performing contour edge calibration on the salient object regions of the salient image based on the fine-grained edge image and the coarse-grained edge image to obtain the salient object contour image of the target scene image includes:

[0020] Perform an image pixel intersection operation on the fine-grained edge image and the coarse-grained edge image to obtain the corresponding contour edge intersection image;

[0021] The saliency image is binarized based on the intersection image of the contour edges to obtain the target calibration image;

[0022] The intersection of the contour edges and the target calibration image is performed to obtain the salient object contour image of the target scene image.

[0023] In an optional implementation, the step of drawing contour lines on the target scene image based on the salient object contour image to obtain an animation scene image including the salient object contour lines includes:

[0024] All contour pixels of the salient object contour image are mapped to the corresponding pixel positions in the target scene image for pixel replacement to obtain the scene image to be animated.

[0025] Secondly, this application provides a method for animateting scene images, the method comprising:

[0026] Obtain the target scene image that needs to be animated;

[0027] The salient object contour drawing method described in any of the foregoing embodiments is used to draw salient object contour lines on the target scene image to obtain a corresponding scene image to be animated, which includes salient object contour lines.

[0028] The pre-stored image animation model is used to perform flat-color animation on the scene image to be animated, resulting in the target scene animation image.

[0029] Thirdly, this application provides a salient object outline drawing apparatus, the apparatus comprising:

[0030] The salient object detection module is used to call a pre-stored salientity detection model to perform salient object detection on the target scene image that needs to be animated, and obtain the salient image of the target scene image;

[0031] The image edge detection module is used to call the pre-stored fine-grained edge detection model and coarse-grained edge detection model to perform image edge detection on the target scene image, respectively, to obtain the fine-grained edge image and coarse-grained edge image of the target scene image;

[0032] A salient contour calibration module is used to perform contour edge calibration on the salient object parts of the salient image based on the fine-grained edge image and the coarse-grained edge image, so as to obtain the salient object contour image of the target scene image;

[0033] The salient outline drawing module is used to draw outline lines on the target scene image based on the salient object outline image, so as to obtain a scene image to be animated that includes the salient object outline lines.

[0034] Fourthly, this application provides a scene image animation device, the device comprising:

[0035] The scene image acquisition module is used to acquire the target scene image that needs to be animated.

[0036] The prominent line drawing module is used to call the prominent object contour drawing device described in the aforementioned embodiments to draw prominent object contour lines on the target scene image, thereby obtaining a corresponding scene image to be animated that includes prominent object contour lines.

[0037] The scene image animation module is used to call a pre-stored image animation model to perform flat-color animation on the scene image to be animated, so as to obtain the target scene animation image.

[0038] Fifthly, this application provides a computer device including a processor and a memory, wherein the memory stores a computer program executable by the processor, and the processor can execute the computer program to implement the salient object outline drawing method described in any of the foregoing embodiments, or to implement the scene image animation method described in the foregoing embodiments.

[0039] Sixthly, this application provides a readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, it implements the salient object outline drawing method described in any of the foregoing embodiments, or the scene image animation method described in the foregoing embodiments.

[0040] In this case, the beneficial effects of the embodiments of this application may include the following:

[0041] This application uses a saliency detection model to detect salient objects in the target scene image that needs to be animated, obtaining the corresponding saliency image. Then, it calls fine-grained edge detection and coarse-grained edge detection models to perform image edge detection on the target scene image, obtaining corresponding fine-grained edge images and coarse-grained edge images. Next, based on the fine-grained and coarse-grained edge images, it performs contour edge calibration on the salient object parts of the salient image, obtaining the salient object contour image of the target scene image. Finally, it draws contour lines on the target scene image based on the salient object contour image, obtaining an animated scene image that includes the salient object contour lines. This effectively draws the salient object contour lines in the real scene image that needs to be animated, so that the animated scene image obtained after subsequent image animation processing can effectively outline the contour lines of the salient objects, ensuring that the animated scene image achieves the expected flat-painted animation effect.

[0042] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0043] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 This is a schematic diagram of the device composition of a computer device provided in an embodiment of this application;

[0045] Figure 2 A flowchart illustrating the method for drawing salient object outlines provided in this application example;

[0046] Figure 3 for Figure 2 The flowchart of the sub-steps included in step S330 is shown below;

[0047] Figure 4 This is a schematic diagram illustrating the drawing of salient object outlines from a real-world scene image, provided in an embodiment of this application.

[0048] Figure 5 A flowchart illustrating the scene image animation method provided in this application embodiment;

[0049] Figure 6 A schematic diagram of the composition of the salient object outline drawing device provided in the embodiments of this application;

[0050] Figure 7This is a schematic diagram of the composition of the scene image animation device provided in the embodiments of this application.

[0051] Icons: 10-Computer equipment; 11-Memory; 12-Processor; 13-Communication unit; 100-Salience object contour drawing device; 200-Scene image animation device; 110-Salience object detection module; 120-Image edge detection module; 130-Salience contour calibration module; 140-Salience contour drawing module; 210-Scene image acquisition module; 220-Salience line drawing module; 230-Scene image animation module. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0053] Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0054] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0055] In the description of this application, it should be understood that relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. Those skilled in the art will understand the specific meaning of the above terms in this application based on the specific circumstances.

[0056] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0057] Please refer to Figure 1 , Figure 1 This is a schematic diagram of the device composition of the computer device 10 provided in this application embodiment. In this application embodiment, the computer device 10 can effectively draw the outline lines of significant objects in real scene images that require flat-painted anime-style image animation processing, and / or perform flat-painted anime-style image animation processing on the basis of obtaining the outline lines of significant objects in real scene images, ensuring that the final anime-processed scene animation image can effectively outline the outline lines of significant objects, while the image textures of significant and non-significant object parts in the scene animation image can be clearly distinguished, so that the scene animation image achieves the expected flat-painted anime effect.

[0058] In this embodiment, the computer device 10 may include a memory 11, a processor 12, and a communication unit 13. The memory 11, the processor 12, and the communication unit 13 are electrically connected to each other directly or indirectly to achieve data transmission or interaction. For example, these components can be electrically connected to each other via one or more communication buses or signal lines.

[0059] In this embodiment, the memory 11 may be, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc. The memory 11 is used to store computer programs, and the processor 12 can execute the computer programs accordingly after receiving execution instructions.

[0060] In this embodiment, the processor 12 can be an integrated circuit chip with signal processing capabilities. The processor 12 can be a general-purpose processor, including at least one of a Central Processing Unit (CPU), Graphics Processing Unit (GPU), Network Processor (NP), Digital Signal Processor (DSP), Application-Specific Integrated Circuit (ASIC), Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The general-purpose processor can be a microprocessor or any conventional processor, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application.

[0061] In this embodiment, the communication unit 13 is used to establish a communication connection between the computer device 10 and other devices via a network, and to send and receive data through the network, wherein the network includes wired communication networks and wireless communication networks. For example, the computer device 10 can communicate with a user's handheld terminal device through the communication unit 13 to obtain real scene images taken by the user for a target scene; the computer device 10 can send real scene images with prominent object outlines drawn on them, and / or scene animation images obtained by image animation processing of the real scene images with prominent object outlines drawn on them, to the terminal device through the communication unit 13.

[0062] Optionally, in this embodiment, the computer device 10 may further include a salient object contour drawing device 100. The salient object contour drawing device 100 may include at least one software function module that can be stored in the memory 11 in the form of software or firmware or embedded in the operating system of the computer device 10. The processor 12 can be used to execute executable modules stored in the memory 11, such as the software function modules and computer programs included in the salient object contour drawing device 100. The computer device 10 can effectively draw the contour lines of salient objects in real scene images that require image animation using the salient object contour drawing device 100, so that the scene animation image obtained through subsequent image animation processing can effectively outline the contour lines of salient objects, ensuring that the scene animation image achieves the expected flat-painted animation effect.

[0063] Optionally, in this embodiment, the computer device 10 may further include a scene image animation device 200. The scene image animation device 200 may include at least one software function module that can be stored in the memory 11 in the form of software or firmware, or embedded in the operating system of the computer device 10. The processor 12 can be used to execute executable modules stored in the memory 11, such as the software function modules and computer programs included in the scene image animation device 200. The computer device 10 can use the scene image animation device 200 to perform image animation processing on a real scene image with drawn outlines of prominent objects, ensuring that the final animated scene image can effectively outline the contours of prominent objects, while the image textures of prominent and non-prominent object parts in the scene animation image can be clearly distinguished, so that the scene animation image achieves the expected flat-painted animation effect.

[0064] Understandable, Figure 1 The block diagram shown is only a schematic diagram of one configuration of the computer device 10. The computer device 10 may also include components such as... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown. Figure 1 The components shown can be implemented using hardware, software, or a combination thereof.

[0065] In this application, to ensure that the computer device 10 can effectively draw the outlines of salient objects in real-world scene images requiring image animation, this application provides a method for drawing salient object outlines to achieve the aforementioned objective. The method for drawing salient object outlines provided in this application will be described in detail below.

[0066] Please refer to Figure 2 , Figure 2 This is a flowchart illustrating a method for drawing the outline of a salient object provided in this application. In this embodiment, the method for drawing the outline of a salient object may include steps S310 to S340.

[0067] Step S310: Call the pre-stored saliency detection model to perform salient object detection on the target scene image that needs to be animated, and obtain the saliency image of the target scene image.

[0068] In this embodiment, for real-world scene images, salient objects generally refer to objects with clear outlines that are in the foreground, the main subject of the image, or near people, animals, or still life. Buildings, trees, and landscapes, even if located in the center of the image, are generally not treated as salient objects but rather as background content. The saliency detection model pre-stored in the computer device 10 can effectively identify salient objects in the input scene image and obtain the corresponding binarized saliency map. In this embodiment, the target scene image is a real-world scene image that needs to be animated in a flat-painting anime style. The saliency detection model can be trained using a deep learning-based saliency detection algorithm. The saliency detection model can be, but is not limited to, models with good saliency detection performance such as BASNet, F3Net, and PoolNet.

[0069] In the binarized saliency map obtained after processing by the saliency detection model, image pixels with a grayscale value of 255 can be used to represent salient objects, and image pixels with a grayscale value of 0 can be used to represent insignificant objects. In this case, the salient object areas in the binarized saliency map are white areas, and the insignificant object areas are black areas. Alternatively, the binarized saliency map obtained after processing by the saliency detection model can also use image pixels with a grayscale value of 0 to represent salient objects and image pixels with a grayscale value of 255 to represent insignificant objects. In this case, the salient object areas in the binarized saliency map are black areas, and the insignificant object areas are white areas. In one embodiment of this example, image pixels with a grayscale value of 255 are used to represent salient objects in the binarized saliency map, and image pixels with a grayscale value of 0 are used to represent insignificant objects.

[0070] In this embodiment, to ensure that the computer device 10 can ensure through the saliency detection model that the detected salient object region can fully cover the corresponding salient object edge, step S310 may include:

[0071] The target scene image is input into the saliency detection model to identify salient objects, thereby obtaining the corresponding initial saliency map;

[0072] The initial saliency map is dilated by calling the first preset convolution kernel to obtain the saliency image of the target scene image.

[0073] The kernel size of the first preset convolution kernel can be set to 3*3 to perform image processing on the binarized initial saliency map through image dilation, so as to ensure that the salient object region in the salient image of the target scene image can fully cover the corresponding salient object edge.

[0074] Step S320: Call the pre-stored fine-grained edge detection model and coarse-grained edge detection model to perform image edge detection on the target scene image, and obtain the fine-grained edge image and coarse-grained edge image of the target scene image.

[0075] In this embodiment, the computer device 10 can perform image edge detection on the target scene image using a pre-stored fine-grained edge detection model to obtain a fine-grained edge image of the target scene image. The fine-grained edge image is a binary image, where the grayscale value of image pixels representing fine-grained contour edge lines is 0, while the grayscale value of image pixels representing non-fine-grained contour edge regions is 255. In this case, the fine-grained contour edge lines are black, and the non-fine-grained contour edge regions are white. In this embodiment, the fine-grained edge detection model can be implemented using an algorithm based on traditional image filtering principles, such as the Canny edge detection algorithm.

[0076] It is worth noting that the edge lines in the fine-grained edge images processed by the above fine-grained edge detection model are often very fine and fragmented, and a lot of redundant and broken edge lines will be detected in some rough image areas.

[0077] In this embodiment, the computer device 10 can perform image edge detection on the target scene image using a pre-stored coarse-grained edge detection model to obtain a coarse-grained edge image of the target scene image. The coarse-grained edge view detected directly from the real scene image by the coarse-grained edge detection model is a grayscale image. The detected edge lines are often quite thick, greatly exceeding the expected thickness of the contour lines. Therefore, it is necessary to binarize the coarse-grained edge view so that the corresponding binarized coarse-grained edge image can be combined with the aforementioned fine-grained edge image to determine the contour edge lines of certain objects that meet the expected line thickness. In this embodiment, the coarse-grained edge detection model can be trained using a deep learning-based edge detection algorithm. The coarse-grained edge detection model can be, but is not limited to, the DexiNed edge detection model, the BDCN edge detection model, etc.

[0078] Therefore, the step S320, which involves calling a pre-stored coarse-grained edge detection model to perform image edge detection on the target scene image to obtain a coarse-grained edge image of the target scene image, may include:

[0079] The target scene image is input into the coarse-grained edge detection model to perform image edge recognition, and the corresponding coarse-grained edge grayscale image is obtained.

[0080] A preset sliding window is invoked to perform sliding adaptive threshold binarization processing on the coarse-grained edge grayscale image to obtain the corresponding coarse-grained edge binarized image.

[0081] The second preset convolution kernel is invoked to perform image dilation processing on the coarse-grained edge binarization image to obtain the coarse-grained edge image of the target scene image.

[0082] This can be achieved by sliding a preset sliding window across the coarse-grained edge grayscale image. Each time the preset sliding window is slid, the grayscale values ​​of each pixel within the image region corresponding to the preset sliding window are averaged. Then, an addition or subtraction operation is performed between the average grayscale value of the image region and a preset deviation value to obtain a grayscale threshold for the image region. Finally, based on this grayscale threshold, each pixel within the image region is binarized. After all window sliding operations are performed, a corresponding coarse-grained edge binarized image is obtained. In this image, the grayscale value of the pixel representing the coarse-grained contour edge line is 0, while the grayscale value of the pixel representing the non-coarse-grained contour edge region is 255. At this point, the coarse-grained contour edge line in the corresponding coarse-grained edge binarized image is black, and the non-coarse-grained contour edge region is white. In one embodiment, the preset sliding window has a window size of 15*15, the preset deviation value is 2, and the grayscale threshold for the corresponding image region is calculated based on a subtraction operation.

[0083] In addition, the kernel size of the second preset convolution kernel can be set to 3*3 to perform image processing on the coarse-grained edge binarized image through image dilation, ensuring that the coarse-grained contour edge lines in the coarse-grained edge image can fully cover the edges of objects.

[0084] Step S330: Based on the fine-grained edge image and the coarse-grained edge image, perform contour edge calibration on the salient object parts of the salient image to obtain the salient object contour image of the target scene image.

[0085] In this embodiment, the computer device 10 can effectively combine the fine-grained edge image and the coarse-grained edge image of the target scene image to extract the contour edge lines of each object in the target scene image that meet the desired line thickness. Then, based on the positional distribution of the salient object parts recorded in the salient image, the salient object contour lines of the target scene image are determined from the extracted contour edge lines that meet the line thickness, thus obtaining the salient object contour image of the target scene image.

[0086] Alternatively, please refer to Figure 3 , Figure 3yes Figure 2 The flowchart shown illustrates the sub-steps included in step S330. In this embodiment, step S330 may include sub-steps S331 to S333.

[0087] Sub-step S331 involves performing an intersection operation on the fine-grained edge image and the coarse-grained edge image to obtain the corresponding contour edge intersection image.

[0088] In this embodiment, since both the fine-grained edge image and the coarse-grained edge image are binarized images and have the same image size as the target scene image, the gray values ​​of two image pixels with the same pixel position in the fine-grained edge image and the coarse-grained edge image can be compared. If the gray values ​​of the two image pixels are the same, then the gray value of the pixel position corresponding to the two image pixels at the intersection of the contour edges is the same as the gray value of the two image pixels. If the gray values ​​of the two image pixels are different, then the gray value of the pixel position corresponding to the two image pixels at the intersection of the contour edges is 255. Therefore, the image pixels represented by black (i.e., gray value of 0) in the intersection of the contour edges belong to the pixels on the contour edge lines of each object in the target scene image that meet the desired line thickness.

[0089] Sub-step S332: Perform binary unification processing on the saliency image according to the intersection image of the contour edges to obtain the target calibration image.

[0090] In this embodiment, since step S330 is used to extract the salient object contour image of the target scene image, the current contour edge intersection image uses black (i.e., grayscale value of 0) to represent the contour edge lines of each object in the target scene image that meet the desired line thickness. At this time, in order to facilitate the subsequent intersection processing to obtain the salient object contour image, the binary meanings of the salient image and the contour edge intersection image can be unified, so that the salient image uses black (i.e., grayscale value of 0) to represent the salient object parts in the binary unified target calibration image.

[0091] In one embodiment of this example, if the salient object region in the salient image is a white area and the non-salient object region is a black area, the binary representation of the salient image can be inverted so that the salient object region in the corresponding target calibration image is a black area and the non-salient object region is a white area, thus ensuring that the binary meanings of the target calibration image and the contour edge intersection image are consistent with each other.

[0092] Sub-step S333: Perform an intersection operation on the contour edge intersection image and the target calibration image to obtain the salient object contour image of the target scene image.

[0093] In this embodiment, the gray values ​​of two image pixels with consistent pixel positions in the intersection image of the contour edges and the target calibration image can be compared. If the gray values ​​of the two image pixels are consistent, the gray value of the pixel position corresponding to the two image pixels in the contour image of the salient object is consistent with the gray value of the two image pixels. If the gray values ​​of the two image pixels are inconsistent, the gray value of the pixel position corresponding to the two image pixels in the contour image of the salient object is 255. Therefore, the image pixels in the intersection image of the contour edges that are black (i.e., have a gray value of 0) belong to the pixels on the contour lines of the salient object in the target scene image that meet the desired line thickness.

[0094] by Figure 4 The following illustration uses a diagram of a salient object outline drawn from a real-world scene image as an example: Figure 4 (a) is the real-world scene image that needs to be animated, which can be analyzed based on a saliency detection model. Figure 4 (a) Perform salient object detection and obtain Figure 4 (b) shows the saliency image; it can be used based on a fine-grained edge detection model. Figure 4 (a) Perform image edge detection to obtain Figure 4 (c) shows a fine-grained edge image; a coarse-grained edge detection model can be used to... Figure 4 (a) Perform image edge detection to obtain Figure 4 (d) shows a coarse-grained edge image; at this point, it can be obtained by... Figure 4 (c) and Figure 4 (d) Perform an intersection operation on the image pixels to obtain... Figure 4 (e) shows the contour edge intersection image; then, by following... Figure 4 (e) Figure 4 (b) Perform binary unification processing and then... Figure 4 (e) Perform an intersection operation on the image pixels of the binarized image obtained through binarization to obtain... Figure 4 (f) shows the salient object outline image.

[0095] Therefore, by executing the above sub-steps S331 to S333, this application can construct a binarized salient object contour image with salient object contour lines that meet the desired line thickness for the target scene image.

[0096] Step S340: Draw contour lines on the target scene image based on the salient object contour image to obtain a scene image to be animated that includes the salient object contour lines.

[0097] In this embodiment, pixel replacement can be performed by mapping all contour pixels of the salient object contour image to the corresponding pixel positions in the target scene image. This mapping of the salient object contour lines in the salient object contour image to the target scene image completes the salient object contour line drawing process of the target scene image, resulting in a scene image to be animated that includes the salient object contour lines. The contour pixels of the salient object contour image are the image pixels in the salient object contour image that are represented by black (i.e., corresponding to a grayscale value of 0).

[0098] Therefore, by performing the above steps S310 to S340, this application can effectively draw the outline lines of significant objects in the real scene image that needs to be animated, so that the scene animation image obtained by subsequent image animation processing can effectively outline the outline lines of significant objects, ensuring that the scene animation image achieves the expected flat-color animation effect.

[0099] In this application, to ensure that the computer device 10 can perform image animation processing on a real scene image with drawn outlines of prominent objects, and to ensure that the final animated scene image can effectively outline the outlines of prominent objects, while clearly distinguishing the image textures of prominent and non-prominent object parts in the scene animation image, so that the scene animation image achieves the expected flat-painted animation effect, this application provides a scene image animation method to achieve the aforementioned objective. The scene image animation method provided in this application will be described in detail below.

[0100] Please refer to Figure 5 , Figure 5 This is a schematic flowchart of the scene image animation method provided in this application embodiment. In this application embodiment, the scene image animation method may include steps S410 to S430.

[0101] Step S410: Obtain the target scene image that needs to be animated.

[0102] Step S420: Use the salient object contour drawing method to draw salient object contour lines on the target scene image to obtain the corresponding scene image to be animated, which includes salient object contour lines.

[0103] Step S420 may be based on the above. Figure 2 The salient object outline drawing method shown is used to process the target scene image to obtain the scene image to be animated.

[0104] Step S430: Call the pre-stored image animation model to perform flat-paint animation on the image of the scene to be animated, and obtain the target scene animation image.

[0105] In this embodiment, the image animation model pre-stored in the computer device 10 can be trained based on the framework of generative adversarial networks to achieve the transformation of animation image style, such as the image semantic content module, animation color learning module, animation texture learning module, and animation smoothness learning module. This enables the image animation model to ensure that when performing flat-paint animation on a scene image to be animated, including prominent object outlines, the final animated scene image can effectively outline the prominent object outlines. At the same time, the image textures of prominent and non-prominent object parts in the scene animation image can be clearly distinguished, so that the scene animation image achieves the expected flat-paint animation effect.

[0106] Therefore, by performing the above steps S410 to S430, the real scene image with the outline of the prominent object is processed into an animation, ensuring that the final animation image can effectively outline the outline of the prominent object, and that the image textures of the prominent object parts and the non-prominent object parts in the scene animation image can be clearly distinguished, so that the scene animation image achieves the expected flat-paint animation effect.

[0107] In this application, to ensure that the computer device 10 can effectively execute the aforementioned salient object contour drawing method, the aforementioned function is achieved by dividing the salient object contour drawing device 100 stored in the computer device 10 into functional modules. The specific composition of the salient object contour drawing device 100 applied to the aforementioned computer device 10 provided in this application will be described below.

[0108] Please refer to Figure 6 , Figure 6 This is a schematic diagram of the composition of the salient object contour drawing device 100 provided in an embodiment of this application. In this embodiment, the salient object contour drawing device 100 may include a salient object detection module 110, an image edge detection module 120, a salient contour calibration module 130, and a salient contour drawing module 140.

[0109] The salient object detection module 110 is used to call a pre-stored salientity detection model to perform salient object detection on the target scene image that needs to be animated, and obtain the salient image of the target scene image.

[0110] The image edge detection module 120 is used to call the pre-stored fine-grained edge detection model and coarse-grained edge detection model to perform image edge detection on the target scene image, so as to obtain the fine-grained edge image and coarse-grained edge image of the target scene image.

[0111] The salient contour calibration module 130 is used to perform contour edge calibration on the salient object parts of the salient image based on the fine-grained edge image and the coarse-grained edge image, so as to obtain the salient object contour image of the target scene image.

[0112] The salient outline drawing module 140 is used to draw outline lines on the target scene image based on the salient object outline image, so as to obtain a scene image to be animated that includes the salient object outline lines.

[0113] It should be noted that the salient object contour drawing device 100 provided in this application embodiment has the same basic principle and technical effect as the aforementioned salient object contour drawing method. For the sake of brevity, any parts not mentioned in this embodiment can be referred to the above description of the salient object contour drawing method.

[0114] In this application, to ensure that the computer device 10 can effectively execute the above-described scene image animation method, the aforementioned function is achieved by dividing the scene image animation device 200 stored in the computer device 10 into functional modules. The specific composition of the scene image animation device 200 applied to the above-described computer device 10 provided in this application is described below.

[0115] Please refer to Figure 7 , Figure 7 This is a schematic diagram of the composition of the scene image animation device 200 provided in this application embodiment. In this application embodiment, the scene image animation device 200 may include a scene image acquisition module 210, a prominent line drawing module 220, and a scene image animation module 230.

[0116] The scene image acquisition module 210 is used to acquire the target scene image that needs to be animated.

[0117] The salient line drawing module 220 is used to call the salient object contour drawing device 100 to draw salient object contour lines on the target scene image, so as to obtain the corresponding scene image to be animated, which includes salient object contour lines.

[0118] The scene image animation module 230 is used to call the pre-stored image animation model to perform flat-color animation on the scene image to be animated, so as to obtain the target scene animation image.

[0119] It should be noted that the scene image animation device 200 provided in this embodiment has the same basic principle and technical effect as the aforementioned scene image animation method. For the sake of brevity, any parts not mentioned in this embodiment can be referred to the above description of the scene image animation method.

[0120] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0121] Furthermore, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part. If the various functions provided in this application are implemented in the form of software functional modules and sold or used as independent products, they can be stored in a storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned readable storage medium includes: USB flash drives, mobile hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, optical disks, and other media capable of storing program code.

[0122] In summary, the salient object contour drawing method and apparatus, scene image animation method and apparatus, computer equipment, and readable storage medium provided in this application embodiment detect salient objects in the target scene image that needs image animation processing using a salientity detection model to obtain the corresponding salient image. Then, a fine-grained edge detection model and a coarse-grained edge detection model are called to perform image edge detection on the target scene image, obtaining the corresponding fine-grained edge image and coarse-grained edge image. Next, based on the fine-grained edge image and the coarse-grained edge image, the salient object parts of the salient image are calibrated to obtain the salient object contour image of the target scene image. Finally, contour lines are drawn on the target scene image based on the salient object contour image to obtain a scene image to be animated that includes the salient object contour lines. This effectively draws the salient object contour lines in the real scene image that needs image animation, so that the scene animation image obtained through subsequent image animation processing can effectively outline the contour lines of the salient objects, ensuring that the scene animation image achieves the expected flat-paint animation effect.

[0123] The above descriptions are merely various embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for drawing the outline of a salient object, characterized in that, The method includes: The pre-stored saliency detection model is invoked to perform salient object detection on the target scene image that needs to be animated, thereby obtaining the saliency image of the target scene image; The pre-stored fine-grained edge detection model and coarse-grained edge detection model are called to perform image edge detection on the target scene image, respectively, to obtain the fine-grained edge image and coarse-grained edge image of the target scene image; Based on the fine-grained edge image and the coarse-grained edge image, the salient object region of the salient image is calibrated to obtain the salient object contour image of the target scene image; Based on the salient object contour image, the target scene image is drawn with contour lines to obtain a scene image to be animated, including the salient object contour lines. The step of calling a pre-stored coarse-grained edge detection model to perform image edge detection on the target scene image to obtain a coarse-grained edge image of the target scene image includes: The target scene image is input into the coarse-grained edge detection model to perform image edge recognition, and the corresponding coarse-grained edge grayscale image is obtained. A preset sliding window is invoked to perform sliding adaptive threshold binarization processing on the coarse-grained edge grayscale image to obtain the corresponding coarse-grained edge binarized image. The second preset convolution kernel is invoked to perform image dilation processing on the coarse-grained edge binarization image to obtain the coarse-grained edge image of the target scene image.

2. The method according to claim 1, characterized in that, The step of calling a pre-stored saliency detection model to perform salient object detection on the target scene image requiring image animation processing, and obtaining the salient image of the target scene image, includes: The target scene image is input into the saliency detection model to identify salient objects, thereby obtaining the corresponding initial saliency map; The initial saliency map is dilated by calling the first preset convolution kernel to obtain the saliency image of the target scene image.

3. The method according to claim 1 or 2, characterized in that, Since the fine-grained edge image, the coarse-grained edge image, and the salient image are all binary images, the step of performing contour edge calibration on the salient object regions of the salient image based on the fine-grained edge image and the coarse-grained edge image to obtain the salient object contour image of the target scene image includes: Perform an image pixel intersection operation on the fine-grained edge image and the coarse-grained edge image to obtain the corresponding contour edge intersection image; The saliency image is binarized based on the intersection image of the contour edges to obtain the target calibration image; The intersection of the contour edges and the target calibration image is performed to obtain the salient object contour image of the target scene image.

4. The method according to claim 3, characterized in that, The step of drawing contour lines on the target scene image based on the salient object contour image to obtain a scene image to be animated that includes salient object contour lines includes: All contour pixels of the salient object contour image are mapped to the corresponding pixel positions in the target scene image for pixel replacement to obtain the scene image to be animated.

5. A method for animating scene images, characterized in that, The method includes: Acquire the target scene image that needs to be animated; The salient object contour drawing method according to any one of claims 1-4 is used to draw salient object contour lines on the target scene image to obtain a corresponding scene image to be animated, including salient object contour lines. The pre-stored image animation model is used to perform flat-color animation on the scene image to be animated, resulting in the target scene animation image.

6. A device for drawing the outline of a prominent object, characterized in that, The device includes: The salient object detection module is used to call a pre-stored salientity detection model to perform salient object detection on the target scene image that needs to be animated, and obtain the salient image of the target scene image; The image edge detection module is used to call the pre-stored fine-grained edge detection model and coarse-grained edge detection model to perform image edge detection on the target scene image, respectively, to obtain the fine-grained edge image and coarse-grained edge image of the target scene image; A salient contour calibration module is used to perform contour edge calibration on the salient object parts of the salient image based on the fine-grained edge image and the coarse-grained edge image, so as to obtain the salient object contour image of the target scene image; A salient outline drawing module is used to draw outline lines on the target scene image based on the salient object outline image to obtain a scene image to be animated that includes the salient object outline lines. The method by which the image edge detection module calls a pre-stored coarse-grained edge detection model to perform image edge detection on the target scene image, thereby obtaining a coarse-grained edge image of the target scene image, includes: The target scene image is input into the coarse-grained edge detection model to perform image edge recognition, and the corresponding coarse-grained edge grayscale image is obtained. A preset sliding window is invoked to perform sliding adaptive threshold binarization processing on the coarse-grained edge grayscale image to obtain the corresponding coarse-grained edge binarized image. The second preset convolution kernel is invoked to perform image dilation processing on the coarse-grained edge binarization image to obtain the coarse-grained edge image of the target scene image.

7. A device for animatening scene images, characterized in that, The device includes: The scene image acquisition module is used to acquire the target scene image that needs to be animated. A prominent line drawing module is used to call the prominent object contour drawing device of claim 6 to draw prominent object contour lines on the target scene image, so as to obtain a corresponding scene image to be animated that includes prominent object contour lines. The scene image animation module is used to call a pre-stored image animation model to perform flat-color animation on the scene image to be animated, so as to obtain the target scene animation image.

8. A computer device, characterized in that, It includes a processor and a memory, the memory storing a computer program that can be executed by the processor, the processor being able to execute the computer program to implement the salient object outline drawing method according to any one of claims 1-4, or to implement the scene image animation method according to claim 5.

9. A readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the method for drawing the outline of a significant object as described in any one of claims 1-4, or the method for animateting a scene image as described in claim 5.

Images