Image coloring processing method and device, equipment, storage medium and computer program product

Abstract

The invention relates to an image coloring processing method and device, equipment, a storage medium and a computer program product. The method comprises the following steps: acquiring a reference color draft graph obtained by coloring a reference line draft graph according to a closed region, and a line draft graph to be colored having a plurality of key point matching pairs with the reference line draft graph; for each to-be-colored area in the to-be-colored line draft graph, determining a plurality of matched key points associated with the to-be-colored area; determining a mapping area of the to-be-colored area in the reference line draft graph based on a reference key point forming a key point matching pair with the matched key point and an association relationship between the to-be-colored area and the matched key point; determining a target filling color of a to-be-colored area corresponding to the mapping area based on a first filling color of at least one closed area shielded by the mapping area in the reference color draft graph; and coloring the to-be-colored area according to the target filling color. By adopting the method, the coloring accuracy can be improved.

Description

Technical Field

[0001] The present application relates to image processing technology, and in particular to a method, device, equipment, storage medium and computer program product for image coloring processing. Background Art

[0002] With the rapid development of digital image processing technology, a variety of drawing tools have been developed. For example, a draftsman can use professional drawing tools to draw a line drawing on an electronic canvas and can use image processing tools to assist in completing image coloring. And a large number of coloring objects may be involved in drawing production, which requires the draftsman to color each coloring object one by one, resulting in a great deal of human and time costs. Therefore, an efficient way is needed to complete the image coloring process. Currently, visual feature extraction can be performed with reference to the line drawing and the line drawing to be colored, so that color extraction is performed from the reference color drawing corresponding to the reference line drawing based on the similarity between the reference line drawing and the line drawing to be colored, and then the obtained color is filled into the area to be colored in the line drawing to be colored. Since the sizes and shapes of the areas in the line drawing are different, there may be incorrect correspondence of the similarity relationships of some blocks in the line drawing to be colored, resulting in the problem of incorrect coloring of the block, thereby reducing the coloring accuracy. Therefore, how to improve the coloring accuracy is an urgent problem to be solved. Summary of the Invention

[0003] Based on this, in view of the above technical problems, it is necessary to provide a method, device, equipment, storage medium and computer program product for image coloring processing that can improve the coloring accuracy.

[0004] In a first aspect, the present application provides a method for image coloring processing. The method includes:

[0005] Obtain a reference color drawing obtained by coloring a reference line drawing according to closed regions, and a line drawing to be colored that has multiple key point matching pairs with the reference line drawing; each closed region in the reference line drawing is associated with at least one reference key point in the reference line drawing;

[0006] For each area to be colored in the line drawing to be colored, determine a plurality of matched key points associated with the area to be colored; the matched key points are the key points that belong to the key point matching pairs in the line drawing to be colored;

[0007] Based on the reference key points that form key point matching pairs with the matched key points, and the association relationship between the area to be colored and the matched key points, determine the mapping area of the area to be colored in the reference line drawing;

[0008] Based on the first filling color of at least one closed region occluded by the mapping area in the reference color drawing, determine the target filling color of the area to be colored corresponding to the mapping area;

[0009] Color the area to be colored according to the target filling color.

[0010] In a second aspect, the present application also provides an image coloring processing device. The device includes:

[0011] An image acquisition module, configured to acquire a reference color draft image obtained by coloring a reference line drawing according to a closed area, and a line drawing to be colored having multiple key point matching pairs with the reference line drawing; each closed area in the reference line drawing is associated with at least one reference key point in the reference line drawing;

[0012] A key point association module, configured to determine, for each area to be colored in the line drawing to be colored, a plurality of matched key points associated with the area to be colored; the matched key points are the key points belonging to the key point matching pairs in the line drawing to be colored;

[0013] An area mapping module, configured to determine a mapping area of the area to be colored in the reference line drawing based on the reference key points forming key point matching pairs with the matched key points, and the association relationship between the area to be colored and the matched key points;

[0014] A target filling color determination module, configured to determine the target filling color of the area to be colored corresponding to the mapping area based on the first filling color of at least one closed area blocked by the mapping area in the reference color draft image;

[0015] A coloring processing module, configured to color the area to be colored according to the target filling color.

[0016] In a third aspect, the present application also provides a computer device. The computer device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the steps of the above methods are implemented.

[0017] In a fourth aspect, the present application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above methods are implemented.

[0018] In a fifth aspect, the present application also provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the steps of the above methods are implemented.

[0019] The above method, device, computer device, storage medium, and computer program product for image coloring processing obtain a reference color manuscript obtained by coloring a reference line drawing according to a closed area, and a line drawing to be colored having multiple key point matching pairs with the reference line drawing; each closed area in the reference line drawing is associated with at least one reference key point in the reference line drawing; for each area to be colored in the line drawing to be colored, determine multiple matched key points associated with the area to be colored; the matched key points are the key points belonging to the key point matching pairs in the line drawing to be colored; based on the reference key points that form key point matching pairs with the matched key points, and the association relationship between the area to be colored and the matched key points, determine the mapping area of the area to be colored in the reference line drawing; based on the first filling color of at least one closed area blocked by the mapping area in the reference color manuscript, determine the target filling color of the area to be colored corresponding to the mapping area; color the area to be colored according to the target filling color. In the above method for image coloring processing, key point matching is performed between the key points in the reference line drawing and the key points in the line drawing to be colored. Through the key point matching pairs formed by the matched key points and the reference key pairs, the similarity between the reference line drawing and the line drawing to be colored can be characterized in terms of key points. By describing the similarity between the line drawings and the correlation between the key points and the areas in the line drawings in a more fine-grained dimension, the area to be colored in the line drawing to be colored is mapped to the reference line drawing. At this time, the overlapping area blocked by the mapping area and the closed area in the reference line drawing can represent the area similarity between the reference line drawing and the line drawing to be colored. The target filling color determined thereby is obtained by first ensuring accuracy through fine-grained key point matching, then performing area mapping and determining the filling color. Therefore, coloring the area to be colored according to the more reliable target filling color can improve the coloring accuracy rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 FIG. is an application environment diagram of the method for image coloring processing in an embodiment;

[0021] Figure 2 FIG. is a schematic diagram of a reference line drawing and a line drawing to be colored in an embodiment;

[0022] Figure 3 FIG. is a schematic diagram of obtaining a reference color manuscript by coloring a reference line drawing in an embodiment;

[0023] Figure 4 FIG. is a schematic diagram of coloring a line drawing to be colored based on a reference line drawing and a reference color manuscript in an embodiment;

[0024] Figure 5 FIG. is a flowchart of the integrated coloring of area matching and key point matching in an embodiment;

[0025] Figure 6 Schematic diagram of the process of an image coloring method in an embodiment;

[0026] Figure 7 Schematic diagram of the closed area in the reference line drawing and the area to be colored in the line drawing to be colored in an embodiment;

[0027] Figure 8 Schematic diagram of multiple reference key points in the reference line drawing in an embodiment;

[0028] Figure 9 Schematic diagram of associating the key points that have been matched with the area to be colored in the line drawing to be colored in an embodiment;

[0029] Figure 10 Schematic diagram of the mapping area of the area to be colored in the reference line drawing in an embodiment;

[0030] Figure 11 Schematic diagram of the line drawing to be colored and the target color drawing in an embodiment;

[0031] Figure 12 Schematic diagram of the process of constructing key point matching pairs in an embodiment;

[0032] Figure 13 Schematic diagram of the initial reference key points in the reference line drawing in an embodiment;

[0033] Figure 14 Schematic diagram of the process of an image coloring method based on region matching and key point matching;

[0034] Figure 15 Schematic diagram of the process of determining the target filling color by the first filling color and the second filling color in an embodiment;

[0035] Figure 16 Schematic diagram of the complete process of an image coloring method in an embodiment;

[0036] Figure 17 Structure block diagram of an image coloring processing device in an embodiment;

[0037] Figure 18 Internal structure diagram of a computer device in an embodiment. Detailed implementation manners

[0038] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0039] With the rapid development of digital image processing technology, a variety of drawing production tools have been developed. For example, a draftsman can use professional drawing tools to draw a line drawing on an electronic canvas and can use image processing tools to assist in image coloring. However, a large number of coloring objects may be involved in drawing production, which requires the draftsman to color each coloring object one by one, resulting in a huge amount of manpower and time consumption. Therefore, an efficient way is needed to complete the image coloring process. Currently, visual feature extraction can be performed by referring to the line drawing and the line drawing to be colored, so that color extraction can be performed from the reference color drawing corresponding to the reference line drawing based on the similarity between the reference line drawing and the line drawing to be colored, and then the obtained color can be filled into the area to be colored in the line drawing to be colored. Since the sizes and shapes of the areas in the line drawing are different, there may be incorrect correspondence of the similarity relationships of some blocks in the line drawing to be colored, resulting in the problem of coloring the wrong color for that block, thus reducing the coloring accuracy. Therefore, how to improve the coloring accuracy is an urgent problem to be solved.

[0040] An embodiment of the present application provides a method for image coloring processing that can improve the efficiency of image coloring processing. The method for image coloring processing provided by the embodiment of the present application can be applied to an application environment as Figure 1 shown. Among them, the terminal 102 communicates with the server 104 through the network. Taking the terminal 102 as an example, the terminal 102 is the terminal for image coloring processing, and the data storage system can store the data that the server 104 needs to process. For example, reference color drawings, reference line drawings, and line drawings to be colored. The data storage system can be set separately, integrated on the server 104, or placed in the cloud or other servers.

[0041] Specifically, taking the application to the terminal 102 as an example for illustration, the terminal 102 obtains a reference color drawing obtained by coloring the reference line drawing according to a closed area, and a line drawing to be colored that has multiple key point matching pairs with the reference line drawing. Then, for each area to be colored in the line drawing to be colored, a plurality of matched key points associated with the area to be colored are determined. Then, based on the reference key points that form key point matching pairs with the matched key points and the association relationship between the area to be colored and the matched key points, the mapping area of the area to be colored in the reference line drawing is determined. Finally, based on the first filling color of at least one closed area blocked by the mapping area in the reference color drawing, the target filling color of the area to be colored corresponding to the mapping area is determined, so as to perform coloring processing on the area to be colored according to the target filling color. Since the determined target filling color is obtained by performing area mapping and filling color determination on the basis of ensuring accuracy through fine-grained key point matching, coloring processing is performed on the area to be colored according to the more reliable target filling color, thereby improving the coloring accuracy.

[0042] Among them, the terminal 102 can be, but is not limited to, various desktop computers, laptop computers, smart phones, tablet computers, Internet of Things devices, and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, etc. The portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server 104 can be implemented by an independent server or a server cluster composed of multiple servers, and can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, Content Delivery Network (CDN), and big data and artificial intelligence platforms. In addition, the method for image colorization processing provided in the embodiments of the present application can be applied to various scenarios of performing image colorization on a line drawing to obtain a color drawing, including but not limited to colorizing anime frames in anime and colorizing video frames in a video, etc.

[0043] Since the method for image colorization processing provided in the embodiments of the present application specifically needs to perform image colorization on a line drawing to obtain a color image, the line drawing and the color drawing are explained below:

[0044] 1. Line drawing.

[0045] A line drawing is a black contour drawing outlined by an image creator on a background color base map. The aforementioned black contour can be a black contour of an anime character, or a black contour of physical objects such as animals and plants, and the background color base map is usually a white base map. For ease of understanding, as Figure 2 shown in the black and white line drawing, the black and white line drawing can be divided into two types of line drawings. The black and white line drawing for reference colorization is a reference line drawing, that is, the black and white line drawing shown in Figure (A) in Figure 2 . The aforementioned black and white line drawing is a reference line drawing. The reference color drawing can be obtained by colorizing the reference line drawing, and then based on the reference color drawing and the reference line drawing, the black and white line drawing can be colorized, that is, the black and white line drawing shown in Figure (B) in Figure 2 . The aforementioned black and white line drawing is specifically the black and white line drawing to be colorized, that is, it can be defined as the line drawing to be colorized, or can also be called the target line drawing.

[0046] 2. Color drawing.

[0047] The color drawing can be obtained by filling the black and white objects in the black and white line drawing with colors, and the reference color drawing is obtained by colorizing the reference line drawing. As Figure 3 shown in the reference color drawing, that is, the reference color drawing 304 is obtained by colorizing the reference line drawing 302. The reference color drawing is often used for colorization reference of the line drawing to be colorized (or target line drawing).

[0048] The coloring reference is an example of coloring the line drawing to be colored as shown in FIG. 4. There is a reference color draft drawing 401 and a reference color draft drawing 402 obtained by coloring the reference line drawing 401. Then, for the line drawings to be colored 403, 404, 405, and 406 to be colored, it is necessary to perform coloring reference based on the relationship with the reference color draft drawing 401 and then use the colors in the reference color draft drawing 402. In this way, the target color draft drawing 407 obtained for the line drawing to be colored 403, the target color draft drawing 408 obtained for the line drawing to be colored 404, the target color draft drawing 409 obtained for the line drawing to be colored 405, and the target color draft drawing 410 obtained for the line drawing to be colored 406 can be obtained. It can be seen from this that the input information for image coloring processing is reference information and information to be colored. The reference information includes a reference line drawing and a reference color draft drawing obtained by coloring the reference line drawing. The information to be colored includes multiple line drawings to be colored. Then the output is the target color draft drawings obtained by performing coloring processing on the line drawings to be colored respectively.

[0049] In the embodiment of the present application, the method for image coloring processing specifically involves coloring fusion between region matching coloring and key point matching coloring. The aforementioned region matching coloring needs to obtain the corresponding relationship between the closed region of the reference line drawing and the region to be colored of the line drawing to be colored, and copy the color from the reference color draft to fill the region to be colored to complete the coloring. The key point matching coloring needs to obtain the corresponding relationship between the key points in the reference line drawing and the key points in the line drawing to be colored, and copy the color from the reference color draft to fill the region to be colored to complete the coloring. The following introduces the method of fusion coloring of region matching and key point matching. As Figure 5 shown in the flow of the fusion coloring method, it includes a reference frame selection module 502, a line drawing processing module 504, a key point matching and region matching module 506, and a color filling module 508.

[0050] Among them, the reference frame selection module 502 selects a reference line drawing from multiple line drawings to be colored. The line drawing processing module 504 needs to perform closed processing and region segmentation processing on the reference line drawing and the line drawing to be colored. The closed processing means that the manually drawn reference line drawing and the line drawing to be colored may have unclosed regions, resulting in the connection of this region with the background or other regions of different colors. Therefore, it is necessary to add new lines to fill the gaps in the broken regions. Based on this, the regions enclosed by the lines in the reference line drawing and the line drawing to be colored are regarded as an overall coloring block. The segmentation means that all closed regions in the reference line drawing and the line drawing to be colored are distinguished by different labels, that is, the smallest coloring operation units in the reference line drawing and the line drawing to be colored are divided. Based on this, at this time, the reference line drawing can be colored to obtain a reference color draft drawing.

[0051] The key point matching and region matching module 506 needs to determine the key point filling color obtained by key point matching for the area to be colored through the key point matching coloring method provided in this embodiment, and then consider the region matching coloring method to determine the region filling color obtained by region matching for the area to be colored. Then, in the color filling module 508, the target filling color for the area to be colored is determined through the key point filling color and the region filling color, and the area to be colored is colored according to the target filling color, that is, the image coloring processing method provided in this embodiment is completed.

[0052] The following details the method for image coloring processing in this embodiment, which is specifically described through the following embodiments: In one embodiment, as Figure 6 shown, a method for image coloring processing is provided. Taking the method applied to the Figure 1 terminal 102 as an example for illustration, it can be understood that this method can also be applied to the server 104, and can also be applied to a system including the terminal 102 and the server 104, and is implemented through the interaction between the terminal 102 and the server 104. In this embodiment, the method includes the following steps:

[0053] Step 602, obtain a reference color sketch obtained by coloring a reference line drawing according to a closed area, and a line drawing to be colored that has multiple key point matching pairs with the reference line drawing; each closed area in the reference line drawing is associated with at least one reference key point in the reference line drawing.

[0054] Among them, the closed area is the area formed by the closed contour line in the reference line drawing. Through the foregoing introduction, the reference color sketch is the color sketch obtained by coloring the reference line drawing according to the closed area. Similarly, the line drawing to be colored is a black and white line drawing that needs to be colored, and the line drawing to be colored can also have an area to be colored. The area to be colored is the area formed by the closed contour line in the line drawing to be colored. And the object to be colored in the line drawing to be colored is the same as the object that has been colored in the reference line drawing. By dividing the closed area of the object that has been colored in the reference line drawing, multiple closed areas can be obtained. Then, by dividing the closed area of the object to be colored in the line drawing to be colored, multiple areas to be colored can be obtained. For example, taking the coloring of anime frames in anime as an example, the object that has been colored can be anime character 1, and at this time, the object to be colored can also be anime character 1. For ease of understanding, as Figure 7 shown in the reference line drawing and the line drawing to be colored, Figure 7 Figure (A) in it is the reference line drawing, Figure 7 Figure (B) in it is the line drawing to be colored. The reference line drawing and the line drawing to be colored are both the same anime character. By dividing the closed area of the reference line drawing in Figure 7 Figure (A) above, multiple closed areas can be obtained, such asFigure 7 The closed regions 701, 702, etc. in Figure (A). Then for Figure 7 By dividing the line drawing to be colored in Figure (B) into closed regions, multiple regions to be colored can be obtained, such as Figure 7 the regions to be colored 703, 704, etc. in Figure (B). Here, the regions in Figure (A) and Figure 7 Figure (B) are not enumerated one by one. Figure 7

[0055] Based on this, the key point matching pairs include reference key points belonging to the reference line drawing and matched key points belonging to the line drawing to be colored. The reference key points are the initial reference key points obtained by identifying key points on the reference line drawing, and the matched key points are the key points to be matched obtained by identifying key points on the line drawing to be colored. The key point matching pairs are specifically determined according to the key point similarity between the key point features of the reference key points and the key point features of the matched key points, and the reference key points and the matched key points belonging to the same key point matching pair satisfy the key point matching condition, that is, the key point similarity between the reference key points and the matched key points belonging to the same key point matching pair satisfies at least: the maximum value and the value is greater than the key point similarity threshold. For example, for the reference line drawing and the line drawing to be colored, there are at least key point matching pairs A1, A2, and A3. Then the key point matching pair A1 may include the reference key point B1 belonging to the reference line drawing and the matched key point C1 belonging to the line drawing to be colored. Similarly, the key point matching pair A2 may include the reference key point B2 belonging to the reference line drawing and the matched key point C2 belonging to the line drawing to be colored. And the key point matching pair A3 may include the reference key point B3 belonging to the reference line drawing and the matched key point C3 belonging to the line drawing to be colored.

[0056] It can be seen from this that each closed region in the reference line drawing is associated with at least one reference key point in the reference line drawing. The reference key point associated with the closed region may be inside the closed region. Then, for the reference key points not inside the closed region, the reference key points not inside the closed region with a distance less than the distance threshold from the region contour line of the closed region can also be associated with the closed region. For ease of understanding, as Figure 8 shown, Figure 8 ​Shown are a reference line drawing, and multiple reference key points in the reference line drawing. For the closed region 801 in the reference line drawing, the closed region 801 is associated with the reference key point 802, the reference key point 803, and the reference key point 804. Among them, the reference key point 802 is not in the closed region 801 but the distance between it and the region contour line of the closed region 801 is relatively small, while the reference key point 803 and the reference key point 804 are in the closed region 801. And it can be understood that, for the convenience of more intuitively determining the relationship between the reference key points 802 to 804 and the closed region 801, the reference key points 802 to 804 are Figure 8 magnified in. In practical applications, the sizes of the key points recognized in the same line drawing are the same.

[0057] Specifically, when the terminal needs to perform color filling on the drawn line drawing, the terminal first obtains the initial line drawing to be color-filled. The terminal can obtain the initial line drawing by manually uploading a black and white line drawing, or can obtain the black and white line drawing stored in the server as the initial line drawing through a communication connection with the server. The method of obtaining the initial line drawing is not limited here. Based on this, considering that the black and white line drawing is usually a manually drawn line drawing, there may be unclosed regions, resulting in the connection between the unclosed regions and the background or other regions of different colors. At this time, it will affect the accuracy of regional color filling. Therefore, it is necessary to add new lines to fill in the gaps where the regions are broken, that is, the terminal can repair the unclosed regions of the obtained black and white line drawing, and then determine the black and white line drawing after repairing the unclosed regions as the initial line drawing.

[0058] Based on this, the terminal needs to classify the type of the initial line drawing, that is, classify the initial line drawing into a reference line drawing, and classify the initial line drawing that is not a reference line drawing into a line drawing to be color-filled. Then, the terminal needs to divide the reference line drawing into regions to obtain multiple closed regions of the reference line drawing, and thus color the reference line drawing according to the closed regions to obtain a reference color drawing. Specifically, it can be that a human colors the reference line drawing according to the closed regions to obtain a reference color drawing, or an artificial intelligence colors the reference line drawing according to the closed regions to obtain a color drawing to be processed, and then a human re-colors the closed regions with incorrect or missing colors in the color drawing to be processed, thereby obtaining a reference color drawing. The method of obtaining the reference color drawing is not limited here.

[0059] Based on this, the terminal needs to divide the regions of the line drawing to be color-filled to obtain multiple regions to be color-filled of the line drawing to be color-filled, specifically as Figure 7 shown in the closed region in the reference line drawing in Figure (A) in, and Figure 7The area to be colored in the line drawing to be colored shown in Figure (B). Based on this, the terminal also needs to perform key point recognition on the reference line drawing to obtain the initial reference key points in the reference line drawing, and perform key point recognition on the line drawing to be colored to obtain the key points to be matched in the line drawing to be colored. Thus, based on the initial reference key points in the reference line drawing, key point matching is performed with the key points to be matched in the line drawing to be colored. The initial reference key points with successful key point matching are used as reference key points, and the key points to be matched with successful key point matching are used as the matched key points. Therefore, key point matching pairs are constructed according to the matching relationship between the reference key points and the matched key points. Therefore, there are multiple key point matching pairs between the line drawing to be colored and the reference line drawing.

[0060] It can be understood that the foregoing steps may also be that the terminal performs preprocessing before the coloring process to obtain the line drawing to be colored, the reference line drawing, and the reference color drawing, and then locally stores the obtained drawings or uploads them to the server for storage. Then, when it is necessary to perform coloring processing on the line drawing to be colored, the line drawing to be colored, the reference line drawing, and the reference color drawing can be obtained from local storage, or the line drawing to be colored, the reference line drawing, and the reference color drawing can be obtained through a communication connection with the server. The specific acquisition method is not limited here.

[0061] The foregoing introduction mentions dividing the initial line drawing into a reference line drawing, and dividing the initial line drawing that is not the reference line drawing into the line drawing to be colored. The following will detail the method of determining the reference line drawing from the initial line drawing: In a specific embodiment, the method of obtaining the reference line drawing includes: performing region segmentation on multiple initial line drawings respectively to obtain the closed regions and the number of closed regions of each initial line drawing; and determining the initial line drawing with the largest number of matched closed regions as the reference line drawing.

[0062] Among them, the closed area is the internal area formed by the closed curves in the initial line drawing, and the number of closed areas is used to describe the total number of closed areas in the initial line drawing, that is, the total number of internal areas formed by the closed curves. Specifically, after obtaining the initial line drawing, the terminal performs region segmentation on multiple initial line drawings respectively, obtains the closed areas of each initial line drawing, and determines the number of closed areas in each initial line drawing. In this embodiment, the flood fill algorithm is used to perform region segmentation on multiple initial line drawings respectively. The flood fill algorithm is a method for segmenting the closed area of a line in computer vision. The specific implementation method when the flood fill algorithm is applied to image processing is: using any seed pixel point in the image as the starting pixel point, and then spreading from the starting pixel point to the adjacent pixel points nearby, and spreading to determine the pixel points with the same color or similar colors. These pixel points with the same color or similar colors can form a connected area. Based on this, when performing region division on the initial line drawing, since the initial line drawing is specifically a black line drawing, the terminal can use any pixel point in the initial line drawing as the starting pixel point, traverse and expand the surrounding pixel points from the starting pixel point until it encounters a black contour line, that is, encounters a pixel point with the color of black, and determines that the black pixel point belongs to the black contour line. Otherwise, the starting pixel point continues to traverse the pixel points. Thus, the closed areas can be divided in the initial line drawing respectively, and the region contour line of the closed area is the black contour line formed by the black pixel points.

[0063] Based on this, considering that there may be more image details in the initial line drawing with a larger number of closed areas, and coloring the initial line drawing with a larger number of closed areas can obtain more colored areas. In this way, the obtained reference color drawing can provide more area coloring references for the line drawing to be colored. Therefore, the terminal determines the initial line drawing matched with the largest number of closed areas as the reference line drawing, that is, the terminal determines the initial line drawing with richer regional details as the reference line drawing to ensure as much as possible that the selected reference line drawing has information that can be referenced by the line drawing to be colored, and ensure the reference reliability of the reference line drawing. Then, the obtained color drawing is more meaningful for reference. Then, the initial line drawing that is not determined as the reference line drawing is the line drawing to be colored.

[0064] For example, taking the existence of the initial line drawing D1, initial line drawing D2, initial line drawing D3, initial line drawing D3, and initial line drawing D4 as examples for introduction, if the number of closed regions of the initial line drawing D1 is 95, the number of closed regions of the initial line drawing D2 is 80, the number of closed regions of the initial line drawing D3 is 100, and the number of closed regions of the initial line drawing D4 is 88. From this, it can be known that the largest number of closed regions is 100. Then, the initial line drawing D3 can be determined as the reference line drawing. Then, the reference line drawing (i.e., the initial line drawing D3) is colored according to the method introduced in the foregoing embodiments to obtain the reference color drawing, and the initial line drawings D1, D2, and D4 that are not determined as the reference line drawing are the line drawings to be colored.

[0065] Secondly, considering that the initial line drawings can come from the same video to be colored or a video segment to be colored. Even if the same initial line drawing includes the same object to be colored, the object to be colored may have actions such as movement in the video to be colored, that is, there will be changes in angle and position. For example, taking an animation video as the video to be colored, if the object to be colored is an animation character, then the animation character has a moving motion when running, and there will be an angle change when the animation character jumps and rotates. Therefore, if a single reference line drawing is used as a reference for coloring, there may be a problem that there are fewer matches between the regions to be colored in the line drawing to be colored and the closed regions in the reference line drawing. For example, when the animation character jumps and rotates, there will be an angle change. If the reference line drawing is specifically the front view of the animation character, then the back view of the animation character cannot be used as a reference for coloring. At this time, the line drawing to be colored in the back view cannot be accurately colored. Therefore, in this embodiment, grouping of the initial line drawings is considered, that is, grouping and coloring are performed according to each group of line drawings. The following is a detailed introduction:

[0066] In a specific embodiment, the method for image coloring processing further includes: performing object recognition on each initial line drawing to obtain the object to be colored in each initial line drawing; dividing a plurality of initial line drawings whose object feature similarity between the objects to be colored meets the grouping condition into the same line drawing group; determining the reference line drawing included in each line drawing group, and determining the initial line drawing that is not the reference line drawing in the line drawing group as the line drawing to be colored that has a plurality of key point matching pairs with the reference line drawing.

[0067] Specifically, the terminal performs object recognition on each initial line drawing to obtain the objects to be colored in each initial line drawing. Then, for each initial line drawing, the object features of the objects to be colored are extracted. Next, the object feature similarity between the object features of the objects to be colored in each initial line drawing is compared. That is, the objects to be colored in each initial line drawing are subjected to object clustering processing according to the object feature similarity. The initial line drawings including the object feature similarity of the objects to be colored greater than the object feature similarity threshold are divided into the same line drawing group. At this time, the object feature similarity between the objects to be colored in the initial line drawings belonging to the same line drawing group meets the grouping conditions. For example, taking the existence of initial line drawings D1 to D10 as an example for introduction, and there is the same object to be colored in the initial line drawings D1 to D10. The object feature similarity between the object features of the object to be colored in the initial line drawing D1 and the object features of the objects to be colored in the initial line drawings D2 to D5 respectively meets the grouping conditions. The object feature similarity between the object features of the object to be colored in the initial line drawing D6 and the object features of the objects to be colored in the initial line drawings D7 to D10 respectively meets the grouping conditions. At this time, the initial line drawings D1 to D5 can be divided into the line drawing group E1, and the initial line drawings D6 to the initial line Figure 10 are divided into the line drawing group E2.

[0068] It can be understood that the division of the line drawing group is not based on the image similarity of the initial line drawings, nor on the similarity of the closed regions in the initial line drawings, but on the object feature similarity of the objects to be colored recognized in the initial line drawings. The aforementioned object features can be extracted using the Structure Similarity Index Measure (SSIM) and the Contrastive Language-Image Pre-Training (CLIP) model based on the comparison of text-image pairs. The object features are only used to represent the object information of the objects to be colored.

[0069] Based on this, for each line drawing group, the terminal determines the reference line drawing included in each line drawing group according to the method introduced in the foregoing embodiments, that is, the initial line drawing with the largest number of closed regions in the line drawing group is matched and determined as the reference line drawing under the line drawing group, and then the initial line drawings in the line drawing group that are not the reference line drawing are determined as the line drawings to be colored under the line drawing group. For example, as can be seen from the foregoing example, for the line drawing group E1, it is necessary to classify the reference line drawing and the line drawing to be colored from the initial line drawings D1 to D5. At this time, according to the foregoing example, the initial line drawing D3 can be determined as the reference line drawing under the line drawing group E1, then the initial line drawings D1, D2, D4, and D5 are the line drawings to be colored under the line drawing group E1. Similarly, if the initial line drawing D7 is determined as the reference line drawing under the line drawing group E2, then the initial line drawings D6, D8, D9, and D10 are the line drawings to be colored under the line drawing group E2.

[0070] It can be understood that in the case where the number of initial line drawings is large, a selection ratio of the reference line drawing can also be set, and then the number of groups of the line drawing group is determined through the reference line drawing. Therefore, the terminal needs to divide the multiple initial line drawings according to the number of groups. For example, there are 100 initial line drawings, and the selection ratio of the reference line drawing is 10%, that is, it can be determined that 10 (100 * 10%) initial line drawings need to be selected from 100 initial line drawings as the reference line drawings. At this time, the number of groups of the line drawing group is also 10, that is, 100 initial line drawings need to be divided into 10 line drawing groups, and then 1 initial line drawing is determined as the reference line drawing in each of the 10 line drawing groups.

[0071] Step 604, for each area to be colored in the line drawing to be colored, determine a plurality of matched key points associated with the area to be colored; the matched key points are the key points belonging to the key point matching pair in the line drawing to be colored.

[0072] Among them, the matched key points are the key points in the line drawing to be colored that belong to the key point matching pairs. It can be known from the foregoing embodiments that the key point matching pairs include the reference key points belonging to the reference line drawing and the matched key points belonging to the line drawing to be colored. Therefore, for the line drawing to be colored, there are multiple matched key points in the key point matching pairs that are associated with the area to be colored. Specifically, it can be known from the foregoing embodiments that the terminal performs region segmentation on multiple initial line drawings respectively, and the closed regions of each initial line drawing can be obtained. Then, the closed region of the initial line drawing classified as the line drawing to be colored is the region to be colored in the line drawing to be colored. The region division method has been introduced in the foregoing embodiments and will not be elaborated here. The terminal determines multiple matched key points associated with each region to be colored in the line drawing to be colored. The matched key points associated with the region to be colored can be within the region to be colored. Then, for the matched key points that are not within the region to be colored, the matched key points that are not within the region to be colored and whose distance from the region contour line of the region to be colored is less than the distance threshold are also associated with the region to be colored. The following introduces the method for determining multiple matched key points associated with the region to be colored:

[0073] In a specific embodiment, for each region to be colored in the line drawing to be colored, determining multiple matched key points associated with the region to be colored specifically includes: for each region to be colored in the line drawing to be colored, associating the region to be colored with the matched key points within the region to be colored; determining the region contour line of the region to be colored, and determining the distance between the matched key points that are not within the region to be colored and the region contour line, and associating the region to be colored with the matched key points whose distance is less than the distance threshold.

[0074] Specifically, after the terminal completes key point matching to obtain key point matching pairs, it determines the matched key points in the line drawing to be colored included in the key point matching pairs. Then, for each region to be colored in the line drawing to be colored, it determines the matched key points within the region to be colored, and then associates the region to be colored with the matched key points within the region to be colored.

[0075] Secondly, for the matched key points that are not within the area to be colored, the terminal first determines the area contour line of the area to be colored. That is, as can be known from the foregoing embodiments, the area contour line of a closed area is the black contour line formed by black pixel points. Similarly, for the area to be colored, the area contour line of the area to be colored is the black contour line formed by black pixel points. Based on this, the terminal determines the distance between the matched key points that are not within the area to be colored and the area contour line. That is, the terminal calculates the distance between the matched key points that are not within the area to be colored and the black pixel points that form the area contour line of the area to be colored. The foregoing distance can be obtained by calculating the point coordinates of the key points and the point coordinates of the black pixel points. Then, for each matched key point that is not within the area to be colored, the terminal determines whether the distance from the area contour line is less than the distance threshold. If it is less, this matched key point is associated with the area to be colored. Conversely, if it is greater, this matched key point is not associated with the area to be colored.

[0076] For ease of understanding, Figure 9 the line drawing to be colored shown in Figure 9 is used as an example for illustration. In the line drawing to be colored shown in

[0077] there are multiple matched key points obtained after key point recognition and screening. For the area to be colored 901, the matched key point 902 is a key point within the area to be colored 901. Therefore, the matched key point 902 can be associated with the area to be colored 901. Then, the remaining matched key points can calculate the distance from the area contour line 903 of the area to be colored 901. Here, only the matched key points 904, 905, 906, and 907 are exemplified. If the distances between the matched key points 904, 906, and 907 and the area contour line 903 are all less than the distance threshold, at this time, the matched key points 904, 906, and 907 can also be associated with the area to be colored 901. Then, for the area to be colored 901, it is specifically associated with the matched key points 902, 904, 906, and 907.

[0077] Step 606: Based on the reference key points that form key point matching pairs with the matched key points, and the association relationship between the area to be colored and the matched key points, determine the mapped area of the area to be colored in the reference line drawing.

[0078] Among them, the mapping region is the region obtained by performing a homography transformation on the region to be colored in the reference line drawing. The aforementioned homography transformation is the transformation relationship that maps the region to be colored on the plane where the line drawing to be colored is located to the plane where the reference line drawing is located, and the homography transformation can also be called a projective transformation. The homography transformation needs to be based on a homography matrix, and the homography matrix is used to describe the transformation relationship between the matching points on different planes. Specifically, the terminal determines the mapping region of the region to be colored in the reference line drawing based on the reference key points that form key point matching pairs with the already matched key points, and the association relationship between the region to be colored and the already matched key points. That is, for the region to be colored, the terminal performs a homography transformation on the region to be colored on the plane where the line drawing to be colored is located through the matching relationship between the already matched key points associated with the region to be colored and the reference key points that form key point matching pairs with the associated already matched key points, and maps the region to be colored to the plane where the reference line drawing is located to obtain the corresponding mapping region. The following details the specific implementation manner of performing the homography transformation based on the homography matrix:

[0079] In a specific embodiment, based on the reference key points that form key point matching pairs with the already matched key points, and the association relationship between the region to be colored and the already matched key points, determining the mapping region of the region to be colored in the reference line drawing specifically includes: constructing a homography matrix through the reference key points that form key point matching pairs with the already matched key points, and the association relationship between the region to be colored and the already matched key points; mapping the region to be colored to the reference line drawing through the homography matrix to obtain the mapping region of the region to be colored in the reference line drawing.

[0080] Specifically, the terminal constructs a homography matrix through the reference key points that form key point matching pairs with the already matched key points, and the association relationship between the region to be colored and the already matched key points. That is, for the region to be colored, the terminal constructs a homography matrix through the matching relationship between the already matched key points associated with the region to be colored and the reference key points that form key point matching pairs with the associated already matched key points. And at least 4 key point matching pairs are required to construct the homography matrix, that is, at least 4 already matched key points associated with the region to be colored, and the reference key points respectively matched with the 4 already matched key points are required to construct the homography matrix. Therefore, in this embodiment, the number of key points of the already matched key points associated with the region to be colored should be screened, and it is necessary to screen out the regions to be colored whose number of key points of the already matched key points associated satisfies the mapping quantity condition. The aforementioned mapping quantity condition is that the number of key points is greater than 4. And in this embodiment, the functions in the open-source opencv library are specifically used to calculate the homography matrix, which will not be introduced in detail here.

[0081] Based on this, the terminal maps the area to be colored to the reference line drawing through the homography matrix, and obtains the mapped area of the area to be colored in the reference line drawing. That is, the perspective of the area to be colored in the line drawing to be colored is changed through the homography matrix, that is, the perspective of the area to be colored in the line drawing to be colored is transformed into the perspective of the closed area in the reference line drawing, and the obtained area is the mapped area in the reference line drawing.

[0082] For the convenience of understanding the method of homography transformation, an example of 4 pairs of key point matching pairs is used for illustration. As Figure 10 shown, Figure 10 Figure (A) in shows the reference line drawing, and Figure 10 Figure (B) in shows the line drawing to be colored. For the closed area 1001 in the reference line drawing, there are reference key points 10011, reference key points 10012, reference key points 10013, and reference key points 10014. For the area to be colored 1002 in the line drawing to be colored, there are matched key points 10021, matched key points 10022, matched key points 10023, and matched key points 10024, and the reference key point 10011 is matched with the matched key point 10021, the reference key point 10012 is matched with the matched key point 10022, the reference key point 10013 is matched with the matched key point 10023, and the reference key point 10014 is matched with the matched key point 10024. At this time, a homography matrix is constructed according to the aforementioned 4 pairs of key point matching pairs, and then the area to be colored 1002 is projected onto the reference line drawing through the homography matrix, and the mapped area 1003 of the area to be colored in the reference line drawing shown in Figure (C) in can be obtained, and it can be seen that Figure 10 there is an overlapping area between the closed area 1001 in the reference line drawing and the mapped area 1003 in Figure (C). Figure 10 In Figure (C), there is an overlapping area between the closed area 1001 in the reference line drawing and the mapped area 1003.

[0083] Step 608: Determine the target filling color of the area to be colored corresponding to the mapped area based on the first filling color of at least one closed area blocked by the mapped area in the reference color drawing.

[0084] Among them, the target filling color of the area to be colored is used to perform the coloring process on the area to be colored, that is, the target filling color is the filling color after the coloring process of the area to be colored. Specifically, the terminal determines the target filling color of the area to be colored corresponding to the mapped area based on the first filling color of at least one closed area blocked by the mapped area in the reference color drawing. That is, at this time, the first filling color of at least one closed area in the reference color drawing can be determined as the target filling color of the area to be colored. Since in practical applications, there may be a situation where the mapped area blocks multiple closed areas, such as Figure 10There is an overlapping area between the closed area 1001 and the mapped area 1003 in the reference line drawing in figure (C). That is, the mapped area 1003 obscures multiple closed areas in the reference line drawing. Therefore, the terminal needs to consider the number of closed areas obscured by the mapped area to determine the first filling color. If the closed area obscured by the mapped area is a single one, the terminal determines the filling color of the closed area in the reference line drawing in the reference color drawing as the first filling color. At this time, the target filling color is the first filling color. For example, the mapped area G1 of the area F1 to be colored obscures the closed area H1 in the reference line drawing, and the filling color of the closed area H1 in the reference color drawing is red. Then the first filling color of the closed area H1 in the reference color drawing is red. At this time, it can be determined that the target filling color corresponding to the mapped area G1 of the area F1 to be colored is red.

[0085] If the closed areas obscured by the mapped area are multiple, at this time the terminal needs to determine the obscured area of each closed area obscured by the mapped area, and determine the filling color of the closed area with the largest numerical value of the obscured area in the reference color drawing as the first filling color. At this time, the target filling color is the first filling color. For example, if the mapped area G1 of the area F1 to be colored obscures the closed area H1, the closed area H2 and the closed area H3 in the reference line drawing, and the obscured area of the mapped area G1 obscuring the closed area H1 is 0.8, the obscured area of the mapped area G1 obscuring the closed area H2 is 0.15, and the obscured area of the mapped area G1 obscuring the closed area H3 is 0.05. At this time, it can be determined that the closed area with the largest numerical value of the obscured area is the closed area H1. If the filling color of the closed area H1 in the reference color drawing is red, then the first filling color of the closed area H1 in the reference color drawing is red. At this time, it can be determined that the target filling color corresponding to the mapped area G1 of the area F1 to be colored is red.

[0086] Alternatively, the terminal can also first determine the closed areas obscured by the mapped area, and determine the filling color of each closed area in the reference color drawing, thereby assigning the filling color of the closed area in the reference color drawing to the pixel points belonging to the closed area. Then, select the filling color with the most pixel points and determine it as the first filling color. At this time, the target filling color is the first filling color. Based on this, the terminal traverses each area to be colored in the line drawing to be colored, and in a similar manner as introduced in the foregoing embodiments, determines that the mapped area of each area to be colored can be determined in the reference line drawing, thereby determining the target filling color of the area to be colored. It can be understood that in practical applications, there may be a situation where there is no associated matched key point for the area to be colored, or a situation where the mapped area in the reference line drawing cannot be obtained due to insufficient number of matched key points. At this time, the target filling color of the area to be colored is determined as the background filling color, and the foregoing background filling color is usually white.

[0087] Step 610, perform coloring processing on the area to be colored according to the target filling color.

[0088] Specifically, the terminal performs coloring processing on the area to be colored according to the target filling color, that is, the terminal fills the target filling color into the area to be colored in the line drawing to be colored, so as to obtain the colored area corresponding to the area to be colored. From the introduction of the foregoing embodiments, it can be known that the terminal traverses each area to be colored in the line drawing to be colored, and can determine the target filling color of each area to be colored. Then, filling the target filling color of each area to be colored into the area to be colored can obtain the colored area corresponding to each area to be colored respectively, so as to obtain the target color drawing after coloring the line drawing to be colored. And in practical applications, when the target filling color is the background filling color, the terminal will perform a blanking process on the area to be colored, that is, do not perform coloring processing on the area to be colored in the case of the background filling color. For ease of understanding, as Figure 11 shown, Figure 11 in (A) shows the line drawing to be colored, while Figure 11 in (B) shows the target color drawing corresponding to the line drawing to be colored.

[0089] It can be understood that the foregoing examples are all used to understand the present solution, and should not be construed as specific limitations on the present solution.

[0090] In the above method for image coloring processing, key point matching is performed between the key points in the reference line drawing and the key points in the line drawing to be colored. Through the key point matching pairs composed of the matched key points and the reference key pairs, the similarity between the reference line drawing and the line drawing to be colored can be characterized in terms of key points. And by describing the similarity between the line drawings and the correlation between the key points and the regions in the line drawings in a more fine-grained dimension, mapping the areas to be colored in the line drawing to be colored to the reference line drawing. At this time, the occluded overlapping area between the mapped area and the closed area in the reference line drawing can characterize the regional similarity between the reference line drawing and the line drawing to be colored. Thus, the determined target filling color is obtained by mapping the area filling color on the basis of ensuring the accuracy through fine-grained key point matching. Then, performing coloring processing on the area to be colored according to the target filling color with higher reliability can improve the coloring accuracy.

[0091] In one embodiment, as Figure 12 shown, the construction method of the key point matching pair includes:

[0092] Step 1202, respectively perform key point recognition on the reference line drawing and the line drawing to be colored, and determine the initial reference key points in the reference line drawing and the key points to be matched in the line drawing to be colored.

[0093] Among them, the initial reference key points are all the key points obtained by key point recognition in the reference line drawing, and the key points to be matched are all the key points obtained by key point recognition in the line drawing to be colored. Specifically, the terminal respectively performs key point recognition on the reference line drawing and the line drawing to be colored, and determines the initial reference key points in the reference line drawing and the key points to be matched in the line drawing to be colored. That is, the terminal performs key point recognition on the reference line drawing through a key point extraction algorithm to obtain the initial reference key points, and performs key point recognition on the line drawing to be colored through the key point extraction algorithm to obtain the key points to be matched. The aforementioned key point extraction algorithm can be the Super Point algorithm and the sparse key point matching model, etc., which are not limited here. For the sake of understanding, as Figure 13 shown in the example diagram of the initial reference key points in the reference line drawing.

[0094] Based on this, each key point has corresponding key point information, and the key point information at least includes key point feature information and key point coordinate information. Therefore, the initial reference key points have key point feature information and key point coordinate information in the reference line drawing, while the key points to be matched have key point feature information and key point coordinate information in the line drawing to be colored.

[0095] Step 1204, perform key point matching based on the first key point feature of each initial reference key point and the second key point feature of each key point to be matched.

[0096] Specifically, the terminal performs key point matching based on the first key point feature of each initial reference key point and the second key point feature of each key point to be matched. That is, the terminal needs to first extract the corresponding first key point feature for each initial reference key point and the corresponding second key point feature for each key point to be matched. Through the aforementioned introduction, it can be known that the key point information at least includes key point feature information and key point coordinate information, that is, the key feature can at least characterize the key point feature information and key point coordinate information. The following introduces how to obtain the key point feature:

[0097] First, introduce the method for obtaining the first key point feature. In a specific embodiment, the method for obtaining the first key point feature includes: for each initial reference key point, determine the coordinate coding feature of the initial reference key point in the reference line drawing, and extract the key point coding feature of the initial reference key point; add the coordinate coding feature and the key point coding feature to construct the first key point feature.

[0098] Specifically, for each initial reference key point, the terminal determines the coordinate encoding feature of the initial reference key point in the reference line drawing, that is, the terminal performs feature encoding on the key point coordinate information of the initial reference key point to obtain the coordinate encoding feature of the initial reference key point in the reference line drawing. At this time, the terminal specifically uses a Multilayer Perceptron (MLP) to convert the key point coordinate information into a coordinate encoding feature. For example, for the initial reference key point Ki, the key point coordinate information of the initial reference key point Ki is (x, y), and the MLP is used to convert the key point coordinate information (x, y) into a coordinate encoding feature Pi. The feature dimension d of the foregoing coordinate encoding feature Pi can be 128. Therefore, the coordinate encoding feature Pi can be [1.38, 2.11, …, 0.85], specifically including 128 values. Based on this, the terminal extracts the key point encoding feature of the initial reference key point, that is, the terminal performs feature encoding on the key point feature information of the initial reference key point to obtain the key point encoding feature of the initial reference key point in the reference line drawing. It can be understood that if the Super Point algorithm is used to extract the key points in the reference line drawing, the obtained key point feature information at this time is the key point encoding feature Fi, and the feature dimension d of the key point encoding feature Fi can be 128. That is, the feature dimensions of the encoding features obtained by feature encoding different dimension information in the same line drawing are the same.

[0099] Further, the terminal adds the coordinate encoding feature of the initial reference key point and the key point encoding feature of the initial reference key point to construct the first key point feature. That is, considering that the feature dimensions of the encoding features obtained by feature encoding different dimension information in the same reference line drawing are the same. That is, the feature dimension of the coordinate encoding feature is the same as the feature dimension of the key point encoding feature. At this time, the coordinate encoding feature and the key point encoding feature can be directly added, that is, the terminal adds them in sequence according to the order of each feature value in the feature dimension to obtain the first key point feature with the same feature dimension.

[0100] It can be understood that considering the correlation between the key points belonging to the same reference line drawing, therefore, the correlation between the key points can be considered to interact and enhance the key point features. That is, the terminal adds the coordinate encoding feature and the key point encoding feature to obtain the first feature to be processed, and then performs feature interaction and feature enhancement processing on the first feature to be processed of each initial reference key point respectively, so as to obtain the first key point feature of each initial reference key point respectively. The foregoing feature interaction and feature enhancement processing can be specifically performed through a Transformer structure.

[0101] Optionally, the method for obtaining the second key-point feature includes: for each key-point to be matched, determining the coordinate encoding feature of the key-point to be matched in the line drawing to be colored, and extracting the key-point encoding feature of the key-point to be matched; adding the coordinate encoding feature and the key-point encoding feature to construct the second key-point feature.

[0102] Specifically, for each key-point to be matched, the terminal determines the coordinate encoding feature of the key-point to be matched in the line drawing to be colored, that is, the terminal performs feature encoding on the key-point coordinate information of the key-point to be matched to obtain the coordinate encoding feature of the key-point to be matched in the line drawing to be colored. At this time, the terminal specifically uses an MLP to convert the key-point coordinate information into a coordinate encoding feature. Based on this, the terminal extracts the key-point encoding feature of the key-point to be matched, that is, the terminal performs feature encoding on the key-point feature information of the key-point to be matched to obtain the key-point encoding feature of the key-point to be matched in the line drawing to be colored.

[0103] Further, the terminal adds the coordinate encoding feature of the key-point to be matched and the key-point encoding feature of the key-point to be matched to construct the second key-point feature. That is, considering that the feature dimensions of the encoding features obtained by performing feature encoding on different dimensional information in the same line drawing to be colored are the same. That is, the feature dimension of the coordinate encoding feature is the same as the feature dimension of the key-point encoding feature. At this time, the coordinate encoding feature and the key-point encoding feature can be directly added, that is, the terminal adds them in sequence according to the order of each feature value in the feature dimension to obtain the second key-point feature with the same feature dimension.

[0104] It can be understood that considering the correlation between the key-points belonging to the same line drawing to be colored, therefore, the correlation between the key-points can be considered to interact and enhance the key-point features. That is, the terminal adds the coordinate encoding feature and the key-point encoding feature to obtain the second feature to be processed, and then performs feature interaction and feature enhancement processing on the second feature to be processed of each key-point to be matched, so as to obtain the second key-point feature of each key-point to be matched. The aforementioned feature interaction and feature enhancement processing can be specifically performed through a Transformer structure.

[0105] Further, when the terminal obtains the first key-point feature and the second key-point feature, that is, based on the first key-point feature of each initial reference key-point and the second key-point feature of each key-point to be matched, key-point matching is performed, that is, the key-point feature similarity between the first key-point feature and the second key-point feature is calculated, and the key-point feature similarity is used to characterize the matching degree between the initial reference key-point and the key-point to be matched, and the key-point feature similarity is positively correlated with the matching degree between the key-points, that is, the larger the value of the key-point feature similarity, the higher the matching degree between the key-points.

[0106] Considering that the number of initial reference key points and the key points to be matched is relatively large, the key point feature similarity can be described in the form of a matrix. In a specific embodiment, key point matching is performed based on the first key point features of each initial reference key point and the second key point features of each key point to be matched, including: comparing the key point feature similarity between the first key point features of each initial reference key point and the second key point features of each key point to be matched, to obtain a key point similarity matrix.

[0107] Specifically, the terminal compares the key point feature similarity between the first key point features of each initial reference key point and the second key point features of each key point to be matched, to construct a key point similarity matrix (correlative matrix). That is, the terminal can construct a first key point matrix based on the first key point features of each initial reference key point for the reference line drawing. Similarly, the terminal constructs a second key point matrix based on the second key point features of each key point to be matched for the line drawing to be colored. Then, to compare the key point feature similarity between the first key point features and the second key point features to construct a key point similarity matrix, the first key point matrix and the second key point matrix can be multiplied to obtain the key point similarity matrix.

[0108] For example, if M initial reference key points are detected in the reference line drawing, and the feature dimension of the first key point feature of each initial reference key point is d, then through the first key point features of M initial reference key points, a first key point matrix M*d can be obtained. Similarly, if N key points to be matched are detected in the line drawing to be colored, and the feature dimension of the second key point feature of each key point to be matched is d, then through the second key point features of N key points to be matched, a second key point matrix N*d can be obtained. Then, multiplying the first key point matrix M*d by the second key point matrix N*d can obtain the key point similarity matrix N x M.

[0109] Step 1206, using the initial reference key points with successful key point matching as reference key points, and using the key points to be matched with successful key point matching as the matched key points.

[0110] Specifically, the key point matching result obtained by the terminal through step 1204 can be the key point similarity matrix introduced in the foregoing embodiments. Based on this, the terminal determines the initial reference key points with successful key point matching and the key points to be matched with successful key point matching through the key point matching result, and uses the initial reference key points with successful key point matching as the reference key points, and uses the key points to be matched with successful key point matching as the matched key points. That is, the terminal determines whether the key point feature similarity meets the key point screening condition through the key point feature similarity between the first key point feature and the second key point feature represented by the key point matching result. If it meets, that is, the initial reference key point of the first key point feature and the key point to be matched of the second key point feature are successfully matched. The foregoing key point screening condition is that the numerical value of the key point feature similarity for the same key point is the largest.

[0111] For example, there are an initial reference key point I1, an initial reference key point I2, and an initial reference key point I3, and a key point J1 to be matched. If the key point feature similarity between the first key point feature of the initial reference key point I1 and the second key point feature of the key point J1 to be matched is 90%, the key point feature similarity between the first key point feature of the initial reference key point I2 and the second key point feature of the key point J1 to be matched is 60%, and the key point feature similarity between the first key point feature of the initial reference key point I3 and the second key point feature of the key point J1 to be matched is 20%, it can be determined at this time that the initial reference key point I1 and the key point J1 to be matched are successfully matched. At this time, the initial reference key point I1 is used as the reference key point, and the key point J1 to be matched is used as the matched key point.

[0112] Secondly, as can be seen from the foregoing embodiments, a key point similarity matrix can be constructed based on the key point feature similarity, and the key point similarity matrix is only a structure obtained by multiplying feature matrices. At this time, it is necessary to perform normalization processing on the key point similarity matrix. The following details how to perform normalization processing on the key point similarity matrix:

[0113] In a specific embodiment, using the initial reference key points with successful key point matching as the reference key points and the key points to be matched with successful key point matching as the matched key points includes: performing normalization processing on the key point similarity matrix to obtain a first normalized matrix; screening out target normalized values that meet the numerical screening conditions from the first normalized matrix; and respectively determining the key points to be matched and the initial reference key points represented by the normalized values as the matched key points and the reference key points.

[0114] Among them, the first normalization value in the first normalization matrix represents the key-point similarity between the key-point to be matched and the initial reference key-point. Secondly, the numerical screening condition at least includes that the normalization value is the largest value in the matrix row, and considering the situation that the key-point similarity between the key-point to be matched and each initial reference key-point may be relatively small, in this case, the numerical screening condition may further include that the normalization value is greater than the normalization threshold. The aforementioned normalization threshold may be a value such as 0.6 or 0.5, which is not limited here. Specifically, the terminal performs normalization processing on the key-point similarity matrix to obtain the first normalization matrix. At this time, each matrix value in the first normalization matrix is between 0 and 1, so each matrix value in the first normalization matrix is specifically a probability. For example, taking the key-point similarity matrix N x M as an example, that is, the terminal uses the softmax function to perform row normalization on the key-point similarity matrix N x M, so that the sum of all the first normalization values in the matrix row is the probability 1. The specific implementation method of row normalization can be: for any row in the key-point similarity matrix N x M, there are M matrix values, and the M matrix values are specifically as follows: the values M_i1, M_i2, …, M_im. Then, exponential operations are performed on the M matrix values to obtain the exponential operation results of the M matrix values respectively. For example, the exponential operation result corresponding to M_i1 is exp(M_i1), and the exponential operation result corresponding to M_i2 is exp(M_i2). Then, the sum of the exponential operation results of the M matrix values is calculated as "exp(M_i1) + exp(M_i2) + …, +exp(M_im)". At this time, each matrix value is divided by the sum result, and the first normalization value corresponding to the matrix value can be obtained. At this time, the first normalization value is a probability value between 0 and 1.

[0115] Based on this, the terminal screens out the target normalization value that is the largest value in the matrix row and greater than the normalization threshold from the first normalization matrix. Since the first normalization value represents the key-point similarity between the key-point to be matched and the initial reference key-point, the key-point to be matched and the initial reference key-point represented by the target normalization value can be determined at this time. At this time, the key-point to be matched and the initial reference key-point represented by the normalization value are respectively determined as the matched key-point and the reference key-point.

[0116] The above describes only the process of considering row normalization. Row normalization takes the key-point to be matched as the matching basis and finds the reference key-point that matches the key-point to be matched from the initial reference key-points. However, such processing will ignore the relationship of finding the key-point to be matched that matches the initial reference key-point from the key-points to be matched with the initial reference key-point as the matching basis. Therefore, in practical applications, column normalization processing can also be performed on the key-point similarity matrix, so as to jointly determine the first normalization matrix based on the results of row normalization and column normalization. The following will introduce this in detail:

[0117] In an alternative embodiment, normalizing the key-point similarity matrix to obtain a first normalized matrix includes: performing row normalization on the key-point similarity matrix to obtain a row-normalized matrix, and performing column normalization on the key-point similarity matrix to obtain a column-normalized matrix; adding the row-normalized values in the row-normalized matrix and the column-normalized values in the column-normalized matrix to form the first normalized matrix.

[0118] Specifically, the terminal performs row normalization on the key-point similarity matrix to obtain a row-normalized matrix, and the specific implementation is similar to the foregoing introduction and will not be elaborated here. The terminal then performs column normalization on the key-point similarity matrix to obtain a column-normalized matrix. For ease of understanding, taking the key-point similarity matrix N x M as an example again, that is, the terminal uses the softmax function to perform column normalization on the key-point similarity matrix N x M, so that the sum of all matrix values in the matrix column is 1. The specific implementation of column normalization can be: for any column in the key-point similarity matrix N x M, there are N matrix values, and the N matrix values are as follows: values N_i1, N_i2,..., N_in. Then, perform column exponential operations on the N matrix values to obtain the exponential operation results of the N matrix values respectively. For example, the exponential operation result corresponding to N_i1 is exp(N_i1), and the exponential operation result corresponding to N_i2 is exp(N_i2). Then, sum the exponential operation results of the N matrix values "exp(N_i1) + exp(N_i2) +..., +exp(N_in)". At this time, divide each exponential operation result of the matrix value by the sum result to obtain the column-normalized value corresponding to the matrix value. At this time, the column-normalized value is a probability value between 0 and 1.

[0119] Based on this, the terminal adds the row-normalized values in the row-normalized matrix and the column-normalized values in the column-normalized matrix to form the first normalized matrix. That is, the obtained row-normalized matrix and column-normalized matrix have the same matrix size. At this time, directly add the row-normalized value and the column-normalized value on the same matrix element to obtain the first normalized matrix with the same matrix size. For example, the row-normalized value of the matrix element 1 in the row-normalized matrix is 0.1, and the column-normalized value of the matrix element 1 in the column-normalized matrix is 0.2. Then, for the matrix element 1, add the row-normalized value and the column-normalized value to obtain the first normalized value 0.3 (0.1 + 0.2 = 0.3) of the first normalized matrix on the matrix element 1.

[0120] Step 1208, construct key-point matching pairs according to the matching relationship between the reference key points and the matched key points.

[0121] Specifically, the terminal constructs key-point matching pairs according to the matching relationship between the reference key points and the matched key points. Since the terminal uses the initially matched reference key points as the reference key points and the to-be-matched key points with successful key-point matching as the matched key points, and the terminal specifically completes the foregoing steps based on the determination of successful matching between the initial reference key points and the to-be-matched key points. That is, after the terminal actually determines the successful matching between the initial reference key points and the to-be-matched key points, it uses the initially matched reference key points with successful key-point matching as the reference key points, and the to-be-matched key points with successful key-point matching as the matched key points. Then, according to the foregoing key-point matching relationship, the reference key points and the matched key points with successful matching can be used to construct key-point matching pairs. For example, taking the determination of successful key-point matching between the initial reference key point I1 and the to-be-matched key point J1 introduced in the foregoing example as an example, if the initial reference key point I1 is used as the reference key point B1 and the to-be-matched key point J1 is used as the matched key point C1, then the key-point matching pair A1 can be formed by the reference key point B1 and the matched key point C1.

[0122] It can be understood that the foregoing examples are all used to understand the present solution and should not be construed as specific limitations on the present solution.

[0123] In this embodiment, key points with high similarity are screened through the similarity of key-point features to ensure the accuracy of the matching relationship between the obtained matched key points and the reference key points, thereby improving the accuracy of the obtained key-point matching pairs. Secondly, in the process of numerical screening through the key-point similarity matrix, through row normalization and column normalization processing, it is ensured that the normalization process can consider the two dimensions of the key-point matching of the to-be-matched key points to the initial reference key points and the key-point matching of the initial reference key points to the to-be-matched key points, and ensure that the determined matching relationship is the optimal matching relationship of mutual matching, further improving the accuracy of the obtained key-point matching pairs. Thus, when performing region mapping through key-point matching pairs with relatively high accuracy, the accuracy of the mapped region can be ensured, and further the accuracy of the determined target filling color can be improved to improve the accuracy of region coloring.

[0124] In one embodiment, as Figure 14 shown, the method for image coloring processing further includes:

[0125] Step 1402, for each region to be colored, determine a target closed region that matches the region to be colored from each closed region.

[0126] Among them, the target closed region is a closed region that matches the completed region of the region to be colored, that is, the target closed region is the closed region with the largest region similarity to the region to be colored. Specifically, for each region to be colored, the terminal determines the region similarity between the region to be colored and each closed region in the reference line drawing. The aforementioned region similarity is used to describe the degree of region similarity between the region to be colored and the closed region, and the aforementioned region similarity can be characterized by the region feature similarity. Specifically, the region feature similarity is determined by the first region feature of the region to be colored and the second region features of each closed region. Based on this, the target closed region that matches the region to be colored is determined from each closed region through the region similarity. The target closed region that matches successfully can be: the one with the largest region similarity to the region to be colored. The following details the method for determining the target closed region:

[0127] In a specific embodiment, for each region to be colored, determining the target closed region that matches the region to be colored from each closed region includes: extracting the first region feature of each region to be colored and the second region feature of each closed region; comparing the region feature similarity between each first region feature and each second region feature to obtain a region similarity matrix; performing normalization processing on the region similarity matrix to obtain a second normalized matrix; for each region to be colored, screening out the target normalized values that meet the numerical screening conditions from the second normalized matrix; and taking the closed region represented by the target normalized value as the target closed region that matches the region to be colored.

[0128] Among them, the region feature similarity is used to characterize the region similarity between the region to be colored and the closed region. Therefore, the region feature is used to characterize the image feature information and image coordinate information of the region in the image where it is located. It can be seen from this that the first region feature is used to characterize the first image feature information and the first image coordinate information of the region to be colored in the line drawing to be colored, and the second region feature is used to characterize the second image feature information and the second image coordinate information of the closed region in the reference line drawing. Secondly, the matrix value of each matrix element in the region similarity matrix represents the region feature similarity between the region to be colored and the closed region. And the second normalized value of each matrix element in the second normalized matrix represents: the region similarity between the region to be colored and the closed region. And the numerical screening conditions at least include that the second normalized value is the largest value in the matrix row, and considering the situation where the region similarity between the region to be colored and each closed region may be relatively small, the numerical screening conditions can also include that the second normalized value is greater than the normalization threshold. The aforementioned normalization threshold can be a value such as 0.6 or 0.5, and no limitation is made here.

[0129] Specifically, the terminal extracts the first region feature of each region to be colored and the second region feature of each closed region. That is, the terminal needs to separately extract the first image feature information and the first image coordinate information of each region to be colored, perform vector conversion on the first image feature information to obtain the first image region feature, and perform vector conversion on the first image coordinate information to obtain the first region coordinate feature, and then determine the first region feature based on the first image region feature and the first region coordinate feature. Similarly, the terminal separately extracts the second image feature information and the first image coordinate information of each closed region, performs vector conversion on the second image feature information to obtain the second image region feature, and performs vector conversion on the second image coordinate information to obtain the second region coordinate feature, and then determines the second region feature based on the second image region feature and the second region coordinate feature.

[0130] The following first introduces the method for extracting the first image region feature and the second image region feature: The terminal first inputs the reference line drawing and the line drawing to be colored into the feature extraction module (Feature extraction module) respectively, and obtains the reference feature image of the reference line drawing and the feature image to be colored of the line drawing to be colored. The aforementioned feature extraction module is specifically a U-Net network, that is, the resolution size of the input image is the same as the resolution size of the output feature map, that is, the resolution of the reference feature image is consistent with the resolution of the reference line drawing, and the resolution of the feature image to be colored is consistent with the resolution of the line drawing to be colored. For example, if the height of the line drawing to be colored is H, the width is W, and the number of channels C is 3, then the height of the feature image to be colored of the line drawing to be colored output is H, the width is W, and the number of channels C is 128. Therefore, the dimension information of the image features extracted for the line drawing to be colored is represented by C x H x W in the feature image to be colored of the line drawing to be colored.

[0131] Furthermore, it can be seen from the foregoing embodiments that the terminal will perform region division on the reference line drawing and the line drawing to be colored, and obtain multiple closed regions in the reference line drawing and multiple regions to be colored in the line drawing to be colored, specifically as Figure 7The reference line drawing and the line drawing to be colored shown above will not be elaborated here. Therefore, for the line drawing to be colored, the obtained feature image to be colored can be subjected to average pooling according to the divided regions to be colored, so as to obtain the first image region feature of each region to be colored. And for the reference line drawing, the obtained reference feature image can be subjected to average pooling according to the divided closed regions, so as to obtain the second image region feature of each closed region. The terminal needs to first extract the image feature images to be colored of each region to be colored in the feature image to be colored, and extract the closed region feature images of each closed region in the reference feature image, that is, it is necessary to convert the planar image features with width and height into serialized features without width and height. For example, taking the entire line drawing to be colored, and the feature image to be colored of the line drawing to be colored is C x H x W as an example, convert the feature image to be colored C x H x W into the image feature to be colored M x d, where the aforementioned d is the feature dimension, usually 128. Then, the terminal can perform vector conversion on the image feature image to be colored to obtain the first image region feature of each region to be colored, and perform vector conversion on the reference image region feature image to obtain the second image region feature of each closed region.

[0132] The following specifically introduces the methods for extracting the image feature images to be colored and the closed region feature images: The terminal uses each region to be colored obtained according to the region division as the feature extraction range for the feature image to be colored, that is, for each region to be colored, the terminal first determines the region resolution of the region to be colored and determines the region width and height in the region resolution. The region width and height specifically include the width of the region resolution of the region to be colored and the height of the region resolution of the region to be colored. Then the terminal determines the first image region feature image with the same region width and height as the region to be colored from the feature image to be colored, and performing vector conversion on the first image region feature image can obtain the first image region feature of the region to be colored. For example, taking the feature image to be colored of the line drawing to be colored as C x H x W as an example, first correspond the region to be colored Ri to the respective image feature images to be colored Ci x Hi x W with the same region width and height in the feature image to be colored, determine the region width and height for the region to be colored Ri. At this time, it can be determined that the number of pixel points included in the region to be colored Ri is n. At this time, extract the pixel point features corresponding to these n pixel points from C x H x W in the feature image to be colored, and convert the pixel point features into pixel feature vectors with a dimension d of 128 dimensions. Take the average of the pixel point features of each pixel point included in the region to be colored Ri, and 1 feature vector Vi with a dimension d of 128 dimensions can be obtained. At this time, the feature vector Vi can be used to represent the first image region feature image of the region to be colored Ri. The feature vector Vi is specifically represented by a series of decimals, such as [1.02, 0.95,..., 1.58].

[0133] Therefore, according to the method introduced in the foregoing embodiments, the terminal can determine the first image region feature of each closed region for each closed region. Similarly, for the reference region, that is, the closed region obtained by the terminal according to the region division in a similar manner, each closed region is used as the feature extraction range for the reference feature image, that is, for each closed region, the terminal first determines the region resolution of the closed region and determines the region width and height in the region resolution. The region width and height specifically include the width of the region resolution of the closed region and the height of the region resolution of the closed region. Then, the terminal determines a second image region feature image with the same region width and height as the closed region from the reference feature image, and a vector transformation is performed on the second image region feature image to obtain the second image region feature of the closed region.

[0134] The methods for extracting image feature information and image region features are introduced above. The method for extracting image coordinate features will be introduced below. That is, considering that after converting the regions divided in the image into vector features (i.e., image region features), there is no information on height and width, that is, there is no spatial position relationship between the image region features. Therefore, for each region to be colored, the terminal determines the circumscribed rectangle of the region to be colored, then extracts the vertex coordinates of the rectangle vertices of the circumscribed rectangle of the region to be colored in the line drawing to be colored, and performs feature encoding on the vertex coordinates of the rectangle vertices to obtain the first region coordinate feature of the region to be colored. The aforementioned rectangle vertices are at least two, that is, the rectangle vertices can be the upper left corner vertex and the lower right corner vertex of the circumscribed rectangle of the region. The specific rectangle vertices are not limited here. For ease of understanding, the region to be colored Ri is further introduced. If there are the vertex coordinates (x0, y0) of the upper left corner vertex of the circumscribed rectangle of the region to be colored Ri and the vertex coordinates (x1, y1) of the lower right corner vertex of the circumscribed rectangle of the region to be colored Ri, a quadruple is constructed through the vertex coordinates (x0, y0) and the vertex coordinates (x1, y1), and then the above quadruple (x0, y0, x1, y1) is encoded into a position feature Pi with a dimension d of 128 using the position coordinate editing formula. The aforementioned position feature Pi is the first region coordinate feature of the region to be colored Ri.

[0135] Similarly, for each closed region, the terminal determines the circumscribed rectangle of the closed region, then extracts the vertex coordinates of the rectangle vertices of the circumscribed rectangle of the closed region in the reference line drawing, and performs feature encoding on the vertex coordinates of the rectangle vertices to obtain the second region coordinate feature of the closed region. The specific method is similar to the first region coordinate feature and will not be elaborated here.

[0136] Further, for each area to be colored, the terminal can obtain the second area feature of the area to be colored by adding the second image area feature obtained through the method introduced in the foregoing embodiments and the second area coordinate feature. Similarly, for each reference area, the terminal can obtain the second area feature of the reference area by adding the second image area feature obtained through the method introduced in the foregoing embodiments and the second area coordinate feature.

[0137] It can be understood that considering the correlation between areas in the same line drawing, the correlation between areas can be considered to interact and enhance the area features. That is, the terminal adds the first image area feature and the first area coordinate feature to obtain the first area feature to be processed. Feature interaction and feature enhancement processing are performed on the first area features to be processed of each area to be colored, so as to obtain the first area feature of each area to be colored. Similarly, the terminal adds the second image area feature and the second area coordinate feature to obtain the second area feature to be processed. Feature interaction and feature enhancement processing are performed on the second area features to be processed of each reference area, so as to obtain the second area feature of each reference area. The foregoing feature interaction and feature enhancement processing can be specifically performed through a Transformer structure.

[0138] Further, the terminal compares the area feature similarity between each first area feature and each second area feature to obtain an area similarity matrix. That is, for the line drawing to be colored, the terminal constructs a first area matrix through the first area features of the areas to be colored, and constructs a second area matrix through the second area features of the closed areas for the reference line drawing. Then, multiplying the first area matrix by the second area matrix can obtain the area similarity matrix. The obtaining method of the area similarity matrix is similar to that of the key point similarity matrix, which will not be elaborated here.

[0139] Based on this, the terminal further performs normalization processing on the area similarity matrix to obtain a second normalized matrix. That is, the terminal first performs row normalization processing on the area similarity matrix to obtain a row-normalized area similarity matrix, and performs column normalization processing on the area similarity matrix to obtain a column-normalized area similarity matrix. Then, the row-normalized values in the row-normalized area similarity matrix and the column-normalized values in the column-normalized area similarity matrix are added to form the second normalized matrix. The obtaining method of the second normalized matrix is similar to that of the first normalized matrix, which will not be elaborated here.

[0140] Based on this, for each area to be colored, the terminal filters out the target normalized values that meet the numerical filtering conditions from the second normalization matrix, and uses the closed area represented by the target normalized values as the target closed area that successfully matches the area to be colored. That is, for each area to be colored, the second normalization value that is greater than the normalization threshold and has the largest value in the row corresponding to the area to be colored is determined as the target normalization value. Since the second normalization value represents the regional similarity between the area to be colored and the closed area, the target normalization value represents the regional similarity between the area to be colored and the closed area. At this time, the closed area represented by the target normalization value is the target closed area that successfully matches the area to be colored.

[0141] For example, for the area F1 to be colored, and the closed areas H1, H2, and H3, if the regional feature similarity between the first regional feature of the area F1 to be colored and the second regional feature of the closed area H1 is 90%, the regional feature similarity between the first regional feature of the area F1 to be colored and the second regional feature of the closed area H2 is 70%, and the regional feature similarity between the first regional feature of the area F1 to be colored and the second regional feature of the closed area H3 is 30%, it can be determined at this time that the key points of the area F1 to be colored and the closed area H1 match successfully. Then, the closed area H1 is determined as the target closed area that successfully matches the area F1 to be colored.

[0142] Based on this, in step 608, based on the first filling color of at least one closed area occluded by the mapping area in the reference color draft diagram, determine the target filling color of the area to be colored corresponding to the mapping area, specifically including:

[0143] In step 1404, based on the first filling color of at least one closed area occluded by the mapping area in the reference color draft diagram, and the second filling color of the target closed area in the reference color draft diagram, determine the target filling color of the area to be colored corresponding to the mapping area.

[0144] Specifically, the terminal determines the target filling color of the area to be colored corresponding to the mapping area based on the first filling color of at least one closed area blocked by the mapping area in the reference color draft diagram and the second filling color of the target closed area in the reference color draft diagram. That is, the terminal determines the first filling color based on the filling color of at least one closed area blocked by the mapping area in the reference color draft diagram. The determination of the first filling color is similar to the foregoing embodiments and will not be elaborated here. Secondly, the terminal also needs to determine the second filling color of the target closed area in the reference color draft diagram, that is, the terminal determines the filling color of the target closed area in the reference color draft diagram as the second filling color. For example, through the foregoing example, it can be seen that the closed area H1 is determined as the target closed area successfully matched with the area to be colored F1. If the filling color of the target closed area (i.e., the closed area H1) in the reference color draft diagram is red, then the second filling color of the target closed area in the reference color draft diagram is red.

[0145] Based on this, the terminal needs to compare the first filling color with the second filling color, and determine the target filling color of the area to be colored corresponding to the mapping area according to the color comparison result of the first filling color and the second filling color. The following is a detailed introduction:

[0146] In a specific embodiment, determining the target filling color of the area to be colored corresponding to the mapping area based on the first filling color of at least one closed area blocked by the mapping area in the reference color draft diagram and the second filling color of the target closed area in the reference color draft diagram includes: if the first filling color is the same as the second filling color, determining either the first filling color or the second filling color as the target filling color; if the first filling color is different from the second filling color and neither the first filling color nor the second filling color is the background filling color, determining the background filling color as the target filling color; if the first filling color is different from the second filling color and there is a background filling color among the first filling color and the second filling color, determining the reference color that is not the background filling color as the target filling color.

[0147] Specifically, the terminal first determines whether the first filling color is the same as the second filling color. If they are the same, it means that the filling colors determined by the area matching and the key point matching are consistent. At this time, the accuracy of the filling color is relatively high. Therefore, either the first filling color or the second filling color is determined as the target filling color, that is, the first filling color can be determined as the target filling color, or the second filling color can be determined as the target filling color. For example, if through the method introduced in the foregoing embodiments, the first filling color is red and the second filling color is also red, then the target filling color is determined as red at this time.

[0148] Further, if the first filling color is different from the second filling color, it indicates that the filling colors determined by region matching and key point matching are consistent. At this time, the terminal needs to further determine whether there is a background filling color in the first filling color and the second filling color. If not, that is, both the first filling color and the second filling color are not background filling colors, then the background filling color is determined as the target filling color. For example, if the first filling color is red and the second filling color is blue, then the target filling color is determined as the background filling color (white). If there is, it means that one of the region matching and key point matching is unsuccessful and there is no filling color. Therefore, the reference color that is not the background filling color is determined as the target filling color. For example, if the first filling color is red and the second filling color is the background filling color (white), then the target filling color is determined as the non-background filling color, that is, red.

[0149] To facilitate understanding of the foregoing coloring judgment process, as Figure 15 shown in the flow diagram of determining the target filling color through the first filling color and the second filling color, key point matching 1502 for the area to be colored 1501 can determine the first filling color 1503, and region matching 1504 for the area to be colored 1501 can determine the second filling color 1505. In step 1506, first determine whether the first filling color 1503 is the same as the second filling color 1504. If so, in step 1507, either the first filling color 1503 or the second filling color 1504 is determined as the target filling color, and in step 1508, the area to be colored is colored according to the target filling color. If not, in step 1509, further determine whether both the first filling color 1503 and the second filling color 1505 are not background filling colors. If so, in step 1510, the background filling color is determined as the target filling color, and in step 1511, the area to be colored is left blank, that is, the area to be colored is not colored in the case of the background filling color. If not, that is, there is a background filling color in the first filling color 1503 and the second filling color 1505, then in step 1512, the reference color that is not the background filling color is determined as the target filling color, and at this time, in step 1508, the area to be colored is colored according to the target filling color.

[0150] It can be understood that the foregoing examples are all for understanding the present solution and should not be construed as specific limitations of the present solution.

[0151] In this embodiment, region matching is performed between the closed regions in the reference line drawing and the regions to be colored in the line drawing to be colored, and the region similarity between the regions is characterized in terms of the region dimension. Thus, the closed region that completes the region matching with the region to be colored is determined through the region similarity, and another filling color for the region to be colored is determined from the region dimension. Then, in combination with the filling color determined by the mapping region determined by the similarity between the key points characterized by the key point dimension, matching is performed both from the key point dimension and the region dimension, and in a manner that combines key point matching coloring and region matching coloring, so that the determined target filling color is more accurate than that determined by a single dimension, and thus the coloring accuracy can be improved.

[0152] Based on the detailed introduction of the foregoing embodiment, the complete process of the method for image coloring processing in the embodiments of the present application will be introduced below. In one embodiment, as Figure 16 shown, a method for image coloring processing is provided. This method is described by taking the terminal 102 in Figure 1 as an example. It can be understood that this method can also be applied to the server 104, and can also be applied to a system including the terminal 102 and the server 104, and is implemented through the interaction between the terminal 102 and the server 104. In this embodiment, the method includes the following steps:

[0153] Step 1601: Perform region segmentation on multiple initial line drawings respectively to obtain the closed regions and the number of closed regions of each initial line drawing; the initial line drawing with the largest number of closed regions that matches is determined as the reference line drawing, and the initial line drawings that are not the reference line drawing are determined as the line drawings to be colored.

[0154] Step 1602: Obtain a reference color drawing obtained by coloring the reference line drawing according to the closed regions.

[0155] Step 1603: Perform key point recognition on the reference line drawing and the line drawing to be colored respectively to determine the initial reference key points in the reference line drawing and the key points to be matched in the line drawing to be colored.

[0156] Step 1604: Compare the key point feature similarities between the first key point features of each initial reference key point and the second key point features of each key point to be matched to obtain a key point similarity matrix.

[0157] Step 1605: Perform row normalization processing on the key point similarity matrix to obtain a row-normalized matrix, and perform column normalization processing on the key point similarity matrix to obtain a column-normalized matrix; add the row-normalized values in the row-normalized matrix and the column-normalized values in the column-normalized matrix to form a first normalized matrix.

[0158] Step 1606: Screen out the target normalized values that meet the numerical screening conditions from the first normalized matrix; determine the to-be-matched key points represented by the normalized values and the initial reference key points as the already-matched key points and reference key points respectively; construct key point matching pairs according to the matching relationship between the reference key points and the already-matched key points.

[0159] Step 1607: For each to-be-colored area in the to-be-colored line drawing, associate the to-be-colored area with the already-matched key points within the to-be-colored area; determine the area contour line of the to-be-colored area, and determine the distance between the already-matched key points not within the to-be-colored area and the area contour line, and associate the to-be-colored area with the already-matched key points whose distance is less than the distance threshold.

[0160] Step 1608: Construct a homography matrix through the reference key points that form key point matching pairs with the already-matched key points and the association relationship between the to-be-colored areas and the already-matched key points; map the to-be-colored areas to the reference line drawing through the homography matrix to obtain the mapped areas of the to-be-colored areas in the reference line drawing.

[0161] Step 1609: Determine the first filling color of at least one closed area occluded by the mapped area in the reference color drawing.

[0162] Step 1610: Extract the first area features of each to-be-colored area and the second area features of each closed area respectively; compare the area feature similarities between each first area feature and each second area feature to obtain an area similarity matrix; perform normalization processing on the area similarity matrix to obtain a second normalized matrix.

[0163] Step 1611: For each to-be-colored area, screen out the target normalized values that meet the numerical screening conditions from the second normalized matrix; use the closed area represented by the target normalized value as the target closed area that matches the to-be-colored area successfully.

[0164] Step 1612: Determine the second filling color of the target closed area in the reference color drawing.

[0165] Step 1613: If the first filling color is the same as the second filling color, determine either the first filling color or the second filling color as the target filling color; if the first filling color is different from the second filling color and neither the first filling color nor the second filling color is the background filling color, determine the background filling color as the target filling color; if the first filling color is different from the second filling color and there is a background filling color among the first filling color and the second filling color, determine the reference color that is not the background filling color as the target filling color.

[0166] Step 1614, color the area to be colored according to the target filling color.

[0167] It should be understood that the specific implementation manners of Steps 1601 to 1614 are similar to those of the foregoing embodiments, and will not be described herein again.

[0168] It should be understood that although the steps in the flowcharts involved in the foregoing embodiments are shown in sequence according to the indication of the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise clearly stated in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the foregoing embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be executed alternately or alternately with at least a part of other steps or steps in other steps.

[0169] Based on the same inventive concept, an embodiment of the present application also provides an image coloring processing device for implementing the method for image coloring processing involved above. The solution provided by this device for solving problems is similar to the solution recorded in the above method. Therefore, the specific limitations in one or more embodiments of the image coloring processing device provided below can refer to the limitations on the method for image coloring processing in the foregoing text, and will not be described herein again.

[0170] In one embodiment, as Figure 17 shown, an image coloring processing device is provided, including: an image acquisition module 1702, a key point association module 1704, a region mapping module 1706, a target filling color determination module 1708, and a coloring processing module 1710, where:

[0171] The image acquisition module 1702 is configured to acquire a reference color draft image obtained by coloring a reference line drawing according to a closed area, and a line drawing to be colored having multiple key point matching pairs with the reference line drawing; each closed area in the reference line drawing is associated with at least one reference key point in the reference line drawing;

[0172] The key point association module 1704 is configured to determine, for each area to be colored in the line drawing to be colored, a plurality of matched key points associated with the area to be colored; the matched key points are the key points belonging to the key point matching pairs in the line drawing to be colored;

[0173] The region mapping module 1706 is configured to determine a mapped region of the region to be colored in the reference line drawing based on reference key points that form key point matching pairs with the matched key points, and the association relationship between the region to be colored and the matched key points;

[0174] The target filling color determination module 1708 is configured to determine the target filling color of the region to be colored corresponding to the mapped region based on the first filling color of at least one closed region occluded by the mapped region in the reference color drawing;

[0175] The coloring processing module 1710 is configured to perform coloring processing on the region to be colored according to the target filling color.

[0176] In one embodiment, the key point association module is specifically configured to, for each region to be colored in the line drawing to be colored, associate the region to be colored with the matched key points within the region to be colored; determine the region contour line of the region to be colored, and determine the distance between the matched key points not within the region to be colored and the region contour line, and associate the region to be colored with the matched key points whose distance is less than the distance threshold.

[0177] In one embodiment, the region mapping module is specifically configured to construct a homography matrix through the reference key points that form key point matching pairs with the matched key points, and the association relationship between the region to be colored and the matched key points; map the region to be colored to the reference line drawing through the homography matrix to obtain the mapped region of the region to be colored in the reference line drawing.

[0178] In one embodiment, the image coloring processing device further includes a key point matching pair construction module;

[0179] The key point matching pair construction module is configured to respectively perform key point recognition on the reference line drawing and the line drawing to be colored to determine the initial reference key points in the reference line drawing and the key points to be matched in the line drawing to be colored; perform key point matching based on the first key point features of each initial reference key point and the second key point features of each key point to be matched; use the initial reference key points with successful key point matching as reference key points, and use the key points to be matched with successful key point matching as the matched key points; construct key point matching pairs according to the matching relationship between the reference key points and the matched key points.

[0180] In one embodiment, the key-point matching pair construction module is specifically configured to compare the key-point feature similarity between the first key-point features of each initial reference key-point and the second key-point features of each key-point to be matched, so as to obtain a key-point similarity matrix; perform normalization processing on the key-point similarity matrix to obtain a first normalization matrix; the first normalization value in the first normalization matrix represents the key-point similarity between the key-point to be matched and the initial reference key-point; screen out target normalization values that meet the numerical screening conditions from the first normalization matrix; and respectively determine the key-point to be matched and the initial reference key-point represented by the normalization value as the matched key-point and the reference key-point.

[0181] In one embodiment, the key-point matching pair construction module is specifically configured to perform row normalization processing on the key-point similarity matrix to obtain a row normalization matrix, and perform column normalization processing on the key-point similarity matrix to obtain a column normalization matrix; and add the row normalization values in the row normalization matrix and the column normalization values in the column normalization matrix to form a first normalization matrix.

[0182] In one embodiment, the image colorization processing device further includes a key-point feature acquisition module;

[0183] The key-point feature acquisition module is configured to, for each initial reference key-point, determine the coordinate coding feature of the initial reference key-point in the reference line drawing and extract the key-point coding feature of the initial reference key-point; add the coordinate coding feature and the key-point coding feature to construct a first key-point feature.

[0184] In one embodiment, the image colorization processing device further includes a region matching module;

[0185] The region matching module is configured to, for each region to be colored, determine a target closed region that is successfully matched with the region to be colored from each closed region;

[0186] In one embodiment, the target filling color determination module is specifically configured to determine the target filling color of the region to be colored corresponding to the mapping region based on the first filling color of at least one closed region occluded by the mapping region in the reference color drawing and the second filling color of the target closed region in the reference color drawing.

[0187] In one embodiment, the target filling color determination module is specifically configured to, if the first filling color is the same as the second filling color, determine any one of the first filling color and the second filling color as the target filling color.

[0188] In one embodiment, the target filling color determination module is specifically configured to, if the first filling color is different from the second filling color and both the first filling color and the second filling color are not the background filling color, determine the background filling color as the target filling color.

[0189] In one embodiment, the target filling color determination module is specifically configured to, if the first filling color is different from the second filling color and there is a background filling color among the first filling color and the second filling color, determine the reference color that is not the background filling color as the target filling color.

[0190] In one embodiment, the region matching module is specifically configured to extract the first region feature of each region to be colored and the second region feature of each closed region; compare the region feature similarity between each first region feature and each second region feature to obtain a region similarity matrix; perform normalization processing on the region similarity matrix to obtain a second normalized matrix; the second normalized value in the second normalized matrix represents the region similarity between the region to be colored and the closed region; for each region to be colored, filter out the target normalized value that meets the numerical filtering condition from the second normalized matrix; and use the closed region represented by the target normalized value as the target closed region that matches the region to be colored successfully.

[0191] In one embodiment, the image acquisition module is specifically configured to perform region segmentation on multiple initial line drawing images respectively to obtain the closed regions and the number of closed regions of each initial line drawing image; and determine the initial line drawing image with the largest number of closed regions as the reference line drawing image.

[0192] In one embodiment, the image acquisition module is further configured to perform object recognition on each initial line drawing image to obtain the objects to be colored in each initial line drawing image; divide multiple initial line drawing images whose object feature similarity between the objects to be colored meets the grouping condition into the same line drawing group; determine the reference line drawing image included in each line drawing group, and determine the initial line drawing image that is not the reference line drawing image in the line drawing group as the line drawing image to be colored that has multiple key point matching pairs with the reference line drawing image.

[0193] In one embodiment, a computer device is provided. The computer device can be a server or a terminal. In this embodiment, the computer device is taken as an example of a terminal for introduction, and its internal structure diagram can be as Figure 18As shown in the figure. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit, and an input device. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface, the display unit, and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (Near Field Communication), or other technologies. The computer program, when executed by the processor, implements a method for image coloring processing. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer covering the display screen, or a button, a trackball, or a touchpad provided on the computer device housing, or an external keyboard, touchpad, or mouse, etc.

[0194] Those skilled in the art can understand that Figure 18 the structure shown in the figure is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0195] In one embodiment, a computer device is further provided, including a memory and a processor. A computer program is stored in the memory, and when the processor executes the computer program, the steps in the above method embodiments are implemented.

[0196] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by the processor, the steps in the above method embodiments are implemented.

[0197] In one embodiment, a computer program product is provided, including a computer program. When the computer program is executed by the processor, the steps in the above method embodiments are implemented.

[0198] It should be noted that the image colorization processing involved in this application (including but not limited to object device information, object personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) are all information and data authorized by the object or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of relevant countries and regions.

[0199] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memories. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in this application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., and are not limited thereto. The processors involved in the embodiments provided in this application can be general-purpose processors, central processors, graphics processors, digital signal processors, programmable logic devices, etc., and are not limited thereto.

[0200] The technical feature information of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical feature information in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical feature information, it should be considered as within the scope described in this specification.

[0201] The above-described embodiments merely represent several implementation manners of the present application. The description thereof is relatively specific and detailed, but it should not be construed as a limitation on the patent scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all fall within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.

Claims

1. A method for image coloring processing, characterized in that, The method includes: Obtaining a reference color sketch obtained by coloring a reference line sketch according to closed regions, and a line sketch to be colored that has multiple key point matching pairs with the reference line sketch; each closed region in the reference line sketch is associated with at least one reference key point in the reference line sketch; For each region to be colored in the line sketch to be colored, determining multiple matched key points associated with the region to be colored; the matched key points are the key points in the line sketch to be colored that belong to the key point matching pairs; Based on the reference key points that form key point matching pairs with the matched key points, and the association relationship between the region to be colored and the matched key points, determining the mapped region of the region to be colored in the reference line sketch; Based on the first filling color of at least one closed region occluded by the mapped region in the reference color sketch, determining the target filling color of the region to be colored corresponding to the mapped region; Performing a coloring process on the region to be colored according to the target filling color.

2. The method according to claim 1, wherein The determining, for each region to be colored in the line sketch to be colored, multiple matched key points associated with the region to be colored includes: For each region to be colored in the line sketch to be colored, associating the region to be colored with the matched key points within the region to be colored; Determining the contour line of the region to be colored, and determining the distance between the matched key points not within the region to be colored and the contour line, and associating the region to be colored with the matched key points whose distance is less than a distance threshold.

3. The method according to claim 2, wherein The determining, based on the reference key points that form key point matching pairs with the matched key points, and the association relationship between the region to be colored and the matched key points, the mapped region of the region to be colored in the reference line sketch includes: Constructing a homography matrix through the reference key points that form key point matching pairs with the matched key points, and the association relationship between the region to be colored and the matched key points; Mapping the region to be colored to the reference line sketch through the homography matrix to obtain the mapped region of the region to be colored in the reference line sketch.

4. The method according to claim 1, characterized in that The construction method of the key point matching pairs includes: Performing key point recognition on the reference line sketch and the line sketch to be colored respectively, to determine the initial reference key points in the reference line sketch and the key points to be matched in the line sketch to be colored; Performing key point matching based on the first key point feature of each initial reference key point and the second key point feature of each key point to be matched; Taking the initial reference key points with successful key point matching as reference key points, and taking the key points to be matched with successful key point matching as matched key points; Constructing the key point matching pairs according to the matching relationship between the reference key points and the matched key points.

5. The method according to claim 4, wherein The performing key point matching based on the first key point feature of each initial reference key point and the second key point feature of each key point to be matched includes: Obtain a key-point similarity matrix by comparing the key-point feature similarities between the first key-point features of each of the initial reference key-points and the second key-point features of each of the key-points to be matched; Regarding the initial reference key-points with successfully matched key-points as reference key-points, and regarding the key-points to be matched with successfully matched key-points as the already-matched key-points, includes: Perform a normalization process on the key-point similarity matrix to obtain a first normalization matrix; the first normalization values in the first normalization matrix represent: the key-point similarity between the key-points to be matched and the initial reference key-points; Screen out the target normalization values that meet the numerical screening conditions from the first normalization matrix; Respectively determine the key-points to be matched and the initial reference key-points represented by the normalization values as the already-matched key-points and the reference key-points.

6. The method according to claim 5, wherein The performing a normalization process on the key-point similarity matrix to obtain a first normalization matrix includes: Perform a row normalization process on the key-point similarity matrix to obtain a row normalization matrix, and perform a column normalization process on the key-point similarity matrix to obtain a column normalization matrix; Add the row normalization values in the row normalization matrix and the column normalization values in the column normalization matrix to form the first normalization matrix.

7. The method according to claim 4, wherein The obtaining method of the first key-point feature includes: For each of the initial reference key-points, determine the coordinate coding feature of the initial reference key-point in the reference line drawing and extract the key-point coding feature of the initial reference key-point; Add the coordinate coding feature and the key-point coding feature to construct the first key-point feature.

8. The method according to claim 1, wherein The method further includes: For each of the areas to be colored, determine the target closed area that is successfully matched with the area to be colored from each of the closed areas; The determining the target filling color of the area to be colored corresponding to the mapping area based on the first filling color of at least one closed area occluded by the mapping area in the reference color drawing includes: Based on the first filling color of at least one closed area occluded by the mapping area in the reference color drawing and the second filling color of the target closed area in the reference color drawing, determine the target filling color of the area to be colored corresponding to the mapping area.

9. The method according to claim 8, wherein The determining the target filling color of the area to be colored corresponding to the mapping area based on the first filling color of at least one closed area occluded by the mapping area in the reference color drawing and the second filling color of the target closed area in the reference color drawing includes: If the first filling color is the same as the second filling color, determine any one of the first filling color and the second filling color as the target filling color; If the first filling color is different from the second filling color, and both the first filling color and the second filling color are not the background filling color, determine the background filling color as the target filling color; If the first filling color is different from the second filling color, and there is a background filling color among the first filling color and the second filling color, determine the reference color that is not the background filling color as the target filling color.

10. The method according to claim 8, wherein For each of the areas to be colored, determining a target closed area that matches the area to be colored from each of the closed areas includes: Extracting the first area feature of each area to be colored and the second area feature of each closed area; Comparing the area feature similarity between each first area feature and each second area feature to obtain an area similarity matrix; Performing a normalization process on the area similarity matrix to obtain a second normalization matrix; the second normalization value in the second normalization matrix represents the area similarity between the area to be colored and the closed area; For each area to be colored, screening out target normalization values that meet the numerical screening conditions from the second normalization matrix; Taking the closed area represented by the target normalization value as the target closed area that matches the area to be colored.

11. The method according to any one of claims 1 to 10, characterized in that, The ways to obtain the reference line drawing include: Performing area segmentation on multiple initial line drawings respectively to obtain the closed areas and the number of closed areas of each initial line drawing; Determining the initial line drawing with the largest number of closed areas as the reference line drawing.

12. The method according to any one of claims 1 to 10, characterized in that, The method further includes: Performing object recognition on each initial line drawing to obtain the objects to be colored in each initial line drawing; Dividing multiple initial line drawings whose object feature similarity between the objects to be colored meets the grouping condition into the same line drawing group; Determining the reference line drawing included in each line drawing group, and determining the initial line drawing of the non-reference line drawing in the line drawing group as the line drawing to be colored that has multiple key point matching pairs with the reference line drawing.

13. An image colorization processing device, characterized in that, The device includes: An image acquisition module, configured to acquire a reference color drawing obtained by coloring a reference line drawing according to closed areas, and a line drawing to be colored that has multiple key point matching pairs with the reference line drawing; each closed area in the reference line drawing is associated with at least one reference key point in the reference line drawing; A key point association module, configured to determine, for each area to be colored in the line drawing to be colored, multiple matched key points associated with the area to be colored; the matched key points are the key points belonging to the key point matching pairs in the line drawing to be colored; An area mapping module, configured to determine the mapping area of the area to be colored in the reference line drawing based on the reference key points that form key point matching pairs with the matched key points and the association relationship between the area to be colored and the matched key points; A target filling color determination module, configured to determine the target filling color of the area to be colored corresponding to the mapping area based on the first filling color of at least one closed area blocked by the mapping area in the reference color drawing; A coloring processing module, configured to perform coloring processing on the area to be colored according to the target filling color.

14. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 12.

15. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the method according to any one of claims 1 to 12.

16. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the method according to any one of claims 1 to 12.

Images