Image processing method, electronic device, readable storage medium and program product

Abstract

The application relates to the technical field of image processing, in particular to an image processing method, an electronic device, a readable storage medium and a program product. In the method, a user operation (denoted as a first user operation) of starting rapid coloring in an interface of a drawing application running on the electronic device is detected, M frame images recommended to the user are displayed, and then at least one reference frame image is determined from the M frame images based on the selection of the user, which is used as a coloring reference. The electronic device can perform batch coloring on the to-be-colored images in the multiple frame images of the first animation according to the at least one reference frame image selected by the user, and generate coloring images corresponding to the multiple frame images of the first animation. In this way, when an image sequence containing multiple frame images (such as animation sketches) is subjected to coloring processing, the efficiency of the coloring processing can be improved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, specifically to an image processing method, electronic device, readable storage medium, and program product. Background Technology

[0002] With the rapid development of digital image processing technology, a variety of drawing tools are now available for users (such as illustrators). For example, illustrators can use professional drawing tools to draw images on electronic canvases and can use image processing tools to assist in coloring the images. However, drawing production may involve a large number of objects to be colored, which requires illustrators to color each object one by one, thus requiring a great deal of manpower and time.

[0003] Therefore, improving the efficiency of image coloring is an urgent problem to be solved. Summary of the Invention

[0004] This application provides an image processing method, electronic device, readable storage medium, and program product for improving the efficiency and accuracy of color processing of images.

[0005] In a first aspect, this application provides an image processing method applied to an electronic device. The method includes: running a painting application and displaying a first interface of the painting application, the first interface including at least one frame of a multi-frame image of a first animation; in response to a first user operation instructing the initiation of quick coloring, displaying M-frame images and a target image, wherein the M-frame images are thumbnails of at least one frame of the multi-frame images of the first animation, and the target image is an image to be colored from the multi-frame images of the first animation or an enlarged image of any other image from the M-frame images; acquiring at least one reference frame image selected by the user from the M-frame images, wherein the at least one reference frame image includes coloring information; and generating a coloring image corresponding to the multi-frame images of the first animation based on the at least one reference frame image.

[0006] For example, the aforementioned electronic device can be an electronic device with relevant drawing tools (such as drawing applications (APPs)) installed, such as tablets, mobile phones (such as foldable screen phones), etc. The aforementioned first interface can be, for example, a drawing interface displayed by the electronic device that includes controls supporting quick coloring. The aforementioned first user operation can be, for example, a click operation performed on the aforementioned controls supporting quick coloring. In response to the first user operation, the electronic device can display a drawing interface including recommended M-frame images and prompt the user to select a reference frame image from the recommended multi-frame images. The drawing interface can display images recommended by the electronic device based on the image processing method provided in this application (such as the aforementioned M-frame images) as alternatives to the reference frame images. The user can select a portion or an image from the recommended M-frame images in the aforementioned first interface as a reference frame image, i.e., at least one reference frame image. The reference frame image needs to be a coloring image. In this way, the electronic device can divide other images to be colored into image sets and perform coloring processing based on the reference frame image selected by the user, thereby improving the efficiency of the coloring processing.

[0007] It is understood that the M-frame images recommended by the aforementioned electronic device may include either colored or uncolored images. If the user selects an uncolored image as the reference frame image, the electronic device may output relevant prompts, suggesting that the user first colorize the selected uncolored image (e.g., manually colorize it), and then select the colorized image as the reference frame image.

[0008] In some embodiments, the target image may be a magnified image of the currently drawn or selected image displayed by the electronic device system according to the user's instruction to open the animation function, or an image in the layer where the user first opens the animation function. In other embodiments, the target image may be a magnified image of any frame image determined by the electronic device system from the recommended M-frame images according to the user's instruction to open the animation function. In still other embodiments, the target image may be a magnified image of the image selected by the user from the M-frame images, displayed by the electronic device system according to the user's instruction to open the animation function and the user's click operation on the displayed thumbnail. Among the recommended M-frame images, the images other than the target image may be images displayed as small images (such as thumbnails).

[0009] In one possible implementation of the first aspect above, the multi-frame images of the first animation are continuous multi-frame images with continuously changing image content, wherein the multi-frame images of the first animation include thumbnails of the target image.

[0010] For example, a user can draw multiple frames of images in the drawing interface of a relevant drawing application. These multiple frames can be located on different layers. Thus, the multiple frames drawn by the user can generate a continuously changing animation, i.e., the first animation, according to the layer order. The content of these multiple frames can change continuously, therefore they can also be called continuous multi-frame images. In the embodiments of this application, the aforementioned continuous multi-frame images can also be described as an image sequence, without limitation.

[0011] In one possible implementation of the first aspect described above, the M-frame image includes the target image, and all or part of the target image has coloring information.

[0012] For example, the target image can be any frame image from the recommended M-frame images. The target image can be a partially colored image with color information, or it can be a colored image with color information in all areas. The color information can be the color information that already exists when the user selects the reference frame image, or it can be the color information that is filled in when the user selects the reference frame image; there are no restrictions here.

[0013] In one possible implementation of the first aspect above, the at least one reference frame image includes the target image, and generating a colorized image corresponding to multiple frames of the first animation based on the at least one reference frame image includes: performing batch colorization processing on all or part of the multiple frames of the first animation according to the colorization information in the target image.

[0014] For example, if at least one of the reference frames is a colored image or a partially colored image, the electronic device can perform batch coloring on the images to be colored in the first animation generated corresponding to multiple consecutive frames based on the colored image or partially colored image used as the reference frame. This allows for rapid completion of the coloring process and improves coloring efficiency.

[0015] In one possible implementation of the first aspect above, the image to be colored in the multi-frame images includes the image to be colored in at least two image sets, and each of the at least two image sets includes a reference frame image; generating the colored image corresponding to the multi-frame images of the first animation based on the at least one reference frame image includes: generating the colored image corresponding to the image to be colored in each image set based on the reference frame images in the at least two image sets.

[0016] For example, each of the aforementioned image sets can include a reference frame image. The electronic device can color the corresponding regions of the image to be colored in each image set based on the coloring information of each region of the reference frame image. That is, the regions included in the image to be colored in each image set match the regions of the reference frame image in the corresponding image set. In this way, the electronic device can process the coloring of at least two divided image sets in parallel, which helps to improve the coloring processing efficiency. Furthermore, coloring each image set based on the coloring information of different reference frame images also helps to improve the accuracy of the coloring processing.

[0017] As an example, an electronic device can use a pre-trained neural network model to extract color channel data from each region of a reference frame image and convert it into corresponding high-dimensional features (vectors), and extract color channel data from each region of the image to be colored and convert it into high-dimensional features (vectors) with the same dimension. Then, by matching the above high-dimensional features (vectors), the coloring process of the image to be colored can be realized based on the reference frame image.

[0018] In one possible implementation of the first aspect described above, the first interface for displaying the drawing application includes:

[0019] In response to a second user operation of opening the animation function on the above-mentioned painting application, the first interface is displayed, wherein the first interface includes thumbnails of the above-mentioned consecutive multi-frame images and the target image, the target image being an enlarged image of the first image in the above-mentioned consecutive multi-frame images of the first animation, and the first image being an image selected and viewed by the user from the layer where the above-mentioned consecutive multi-frame images are located or from the thumbnails of the above-mentioned consecutive multi-frame images.

[0020] For example, the second user operation described above could be an action by which a user instructs the activation of the "Animation" function within the relevant drawing interface. This operation could cause the electronic device to display an interface that includes at least one frame from a multi-frame image of the first animation, i.e., the first interface described above.

[0021] In one possible implementation of the first aspect above, the aforementioned consecutive multi-frame images include N frames, and the aforementioned M frames are images to be recommended determined from the N frames, where N is a positive integer and M is a positive integer less than N; and the aforementioned M frames are images in the aforementioned consecutive multi-frame images whose content difference compared to adjacent frames meets the recommendation criteria.

[0022] For example, the number of image frames in the first animation may be N. These N frames can form an image sequence with a fixed order. The degree of difference mentioned above can be measured by the magnitude of inter-frame variation in this embodiment.

[0023] In one possible implementation of the first aspect above, the M-frame images are images to be recommended determined from the N-frame images, including: the M-frame images are images determined from the N-frame images based on the number of image frames to be recommended M and the ranking result of the weighted scores of the N-frame images, wherein the number of image frames to be recommended M is related to the total change amplitude of the N-frame images and the number of image frames N included in the consecutive multi-frame images, the total change amplitude of the N-frame images is the sum of the first change amplitudes of (N-1) frame pairs determined based on the N-frame images, each of the (N-1) frame pairs includes two adjacent frame images in the consecutive multi-frame images, and the first change amplitude is used to indicate the degree of content difference between the two adjacent frame images.

[0024] For example, the recommended M-frame images mentioned above can be images selected from all images (e.g., N-frame images) in the image sequence to be processed (such as the aforementioned consecutive multi-frame images or image sequence), based on the above recommendation criteria. The above recommendation criteria may be related to, for example, the cumulative result of the inter-frame variation amplitude between adjacent images in the consecutive multi-frame images or image sequence (i.e., the total variation amplitude), or it may be related to the total number of images in the consecutive multi-frame images or image sequence, or it may be related to the weighted score corresponding to each image in the consecutive multi-frame images or image sequence.

[0025] The aforementioned first change amplitude is the inter-frame change amplitude. In this embodiment, as an example, the inter-frame change amplitude can be determined as a percentage change corresponding to the pixel difference between two adjacent frames in a series of consecutive images or an image sequence. In other embodiments, as examples, the inter-frame change amplitude can also be the pixel difference between two adjacent frames in a series of consecutive images or an image sequence, and this is not limited. In determining the M frames to be recommended, (N-1) frame pairs can be formed by pairing N frames from the first frame in a series of consecutive images or an image sequence, and then calculating the inter-frame change amplitude between each frame pair and the total change amplitude S of the (N-1) frame pairs. This allows for a comprehensive analysis of the content differences between frames in a series of consecutive images or an image sequence, which is beneficial for reasonably determining the M frames to be recommended.

[0026] In this way, M frames of images that meet the recommendation criteria selected from the image sequence will be recommended to the user as alternative reference frames. This can take into account the content differences between the frames and the number of images to be processed. A reasonable number of recommended images will help improve the efficiency of colorization processing, and a reasonable number of recommended images as reference frames will help improve the accuracy of colorization processing.

[0027] In one possible implementation of the first aspect above, the number of image frames M to be recommended is related to the total variation of the N frames and the number of image frames N included in the consecutive multi-frame images, including: the number of image frames M to be recommended is the minimum of the first number of image frames and the second number of image frames, wherein the first number of image frames is determined based on the total variation and the first reference coefficient, and the second number of image frames is determined based on the number of image frames N included in the consecutive multi-frame images and the second reference coefficient.

[0028] For example, if the first image frame count is denoted as Ks and the first reference coefficient is the change sensitivity coefficient C, then the first image frame count can be determined based on the ratio between the total change amplitude S and the change sensitivity coefficient C, i.e., Ks = S / C. As another example, if the second image frame count is denoted as Kn and the second reference coefficient is the frame number density coefficient F, then the second image frame count can be determined based on the ratio between the image frame count N and the frame number density coefficient F of all images in a continuous multi-frame image or image sequence, i.e., Kn = N / F. Thus, by determining the number of images M to be recommended as the minimum value between Ks and Kn, the content differences between each frame and the number of images to be processed can be reasonably considered. A reasonably recommended number of images M will help improve the efficiency of colorization processing.

[0029] In one possible implementation of the first aspect above, the M-frame images are determined from the N-frame images based on the number of image frames M to be recommended and the sorting result of the weighted scores of the N-frame images, including: the M-frame images are the images corresponding to the first M weighted scores in the first scoring sequence, wherein the first scoring sequence is the result of the weighted scores of the N-frame images arranged in descending order from high to low, and the weighted scores of the N-frame images are related to the first change amplitude between each frame image and the previous frame image in the consecutive multi-frame images and the sequence position of each frame image in the consecutive multi-frame images.

[0030] For example, the recommended M-frame images mentioned above can be the M-frame images with higher weighted scores among the N-frame images, that is, the images corresponding to the first M weighted scores in the first scoring sequence mentioned above. In this way, based on the reasonable balance between the content differences between the recommended number M images and the number of images to be processed, using the M-frame images selected based on the weighted scores as reference frame images will help improve the accuracy of colorization processing.

[0031] In one possible implementation of the first aspect above, the at least one reference frame image includes a first reference frame image and a second reference frame image. The first reference frame image is located in a sequence position in the consecutive multi-frame images before the second reference frame image is located in a sequence position in the consecutive multi-frame images. The step of generating a colorized image corresponding to the image to be colorized in each image set based on the reference frame images in the at least two image sets includes: determining the first reference frame image and the multi-frame images in the consecutive multi-frame images whose sequence position is before the second reference frame image as a first image set; determining the second reference frame image and the multi-frame images in the consecutive multi-frame images whose sequence position is after the second reference frame image as a second image set; generating a colorized image corresponding to the image to be colorized in the first image set based on the first reference frame image; and generating a colorized image corresponding to the image to be colorized in the second image set based on the second reference frame image.

[0032] For example, if a user selects two reference frame images, the electronic device can divide the images to be colored from multiple consecutive frames or image sequences into two image sets, and then perform coloring processing on the images to be colored in the two image sets respectively.

[0033] In one possible implementation of the first aspect described above, the at least one reference frame image further includes a third reference frame. The process of generating colorized images corresponding to the images to be colored in each image set based on the reference frame images in the at least two image sets includes: determining the second image set as the second reference frame image and a series of frames whose sequence positions are between the second and third reference frame images; determining the third image set as the third reference frame image and a series of frames whose sequence positions are after the third reference frame image; generating colorized images corresponding to the images to be colored in the first image set based on the first reference frame image; generating colorized images corresponding to the images to be colored in the second image set based on the second reference frame image; and generating colorized images corresponding to the images to be colored in the third image set based on the third reference frame image.

[0034] For example, if the user selects 3 reference frames, the electronic device can divide the images to be colored from multiple consecutive frames or image sequences into 3 image sets, and then perform coloring processing on the images to be colored in each of the 3 image sets. As another example, if the user selects 4 reference frames, the electronic device can divide the images to be colored from multiple consecutive frames or image sequences into 4 image sets based on the user's selection of 4 reference frames. This process continues until the user selects M reference frames, at which point the electronic device can divide the images to be colored into M image sets, and then perform coloring processing on the images to be colored in each of the M image sets, where M is a positive integer greater than or equal to 2.

[0035] Thus, coloring the images to be colored in parallel across different image sets can improve the efficiency of coloring, while using reference frame images within each set for coloring can improve the accuracy of coloring.

[0036] In one possible implementation of the first aspect described above, N ≥ 3 and M ≤ N / 3. That is, when the total number of frames N in the aforementioned consecutive multi-frame images or image sequence is not less than 3 (i.e., N ≥ 3), the recommended number of image frames M can be no greater than N / 3, i.e., M ≤ N / 3. If the total number of frames N in the consecutive multi-frame images or image sequence is small, such as N < 3, then reference frame images of up to 3 frames can be recommended based on the degree of content difference between each frame, or no reference frame images can be recommended. This helps to save the computing resources of electronic devices.

[0037] In one possible implementation of the first aspect described above, M ≤ 10. It is understood that the larger the recommended number of image frames M, the more reference frame images the user can select, and the greater the computational resources consumed by the electronic device during the colorization process of the relevant algorithms on each image set. Therefore, in some embodiments, the electronic device system can preset a corresponding recommended upper limit threshold, such as 10 frames (i.e., M ≤ 10) or other reasonable values. This helps to control the consumption of computational resources by the electronic device during the execution of the image processing method provided in this application within a reasonable range, and is not limited here.

[0038] In one possible implementation of the first aspect above, obtaining at least one reference frame image selected by the user from the M-frame images includes: detecting the user's selection operation on at least one frame image in the M-frame images, and detecting the user's confirmation operation on the selection result; and determining the at least one frame image as the at least one reference frame image.

[0039] For example, the process of a user selecting a reference frame image can include two stages: pre-selection and confirmation. The pre-selection stage involves the user checking off boxes corresponding to one or more frames in the M-frame image set. If the pre-selection stage detects that the selected image is uncolored, a prompt can be displayed to encourage the user to colorize the selected image first. The confirmation stage involves the user confirming the pre-selection result. In this way, the user can select a more suitable reference frame image through both pre-selection and confirmation stages, improving the accuracy of reference frame image selection and consequently increasing the accuracy of subsequent colorization processing based on the reference frame image.

[0040] In one possible implementation of the first aspect described above, the painting interface displaying the M-frame image and the target image may include a first control that supports the user in confirming the selection result. The detection of the user's confirmation operation on the selection result includes detecting a click operation applied to the first control. For example, the first control may be a "Next" control displayed in the relevant painting interface or other controls indicating confirmation, etc., without limitation. For details, please refer to the relevant interface diagrams and descriptions below, which will not be elaborated upon here.

[0041] In one possible implementation of the first aspect above, before acquiring at least one reference frame image selected by the user from the M-frame images, the method further includes: detecting that the at least one reference frame image selected by the user from the M-frame images is an image to be colored; and outputting first prompt information, wherein the first prompt information is used to prompt the user to color the at least one reference frame image.

[0042] For example, if a user selects an uncolored image as a reference frame image from all images 242, the tablet 10 can output the aforementioned first prompt information, such as displaying a pop-up window or prompting text, prompting the user to first colorize the uncolored image selected as the reference frame image, and then select the colorized image as the reference frame image.

[0043] In one possible implementation of the first aspect above, the N frames in the above-mentioned consecutive multi-frame images or image sequence are binary mask images, or they can be processed into binary mask images first, and then batch coloring processing is performed based on the image processing method provided in this application.

[0044] For example, an electronic device can preprocess the input image, such as extracting alpha channel data and performing binarization, to obtain a binary mask image. Then, the electronic device can apply the image processing method provided in this application to the preprocessed binary mask image for rapid colorization. Since binary mask images facilitate the calculation of inter-frame variations between adjacent images, the efficiency of the electronic device performing the image processing method provided in the first aspect based on the binary mask image can be higher, which is beneficial for improving the efficiency of colorization processing of batch images.

[0045] In a second aspect, this application provides an electronic device, comprising: one or more processors; one or more memories; wherein the one or more memories store one or more programs, and when the one or more programs are executed by the one or more processors, the electronic device performs the image processing method provided in the first aspect and various possible implementations of the first aspect.

[0046] Thirdly, this application provides a readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the image processing method provided in the first aspect and various possible implementations of the first aspect.

[0047] Fourthly, this application provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the image processing method provided in the first aspect and various possible implementations of the first aspect.

[0048] Fifthly, this application provides an apparatus comprising: a module for performing the image processing methods provided in the first aspect and various possible implementations thereof.

[0049] In a sixth aspect, this application provides a chip including at least one processor and at least one memory coupled for executing a computer program stored in the at least one memory, such that the chip performs the image processing method provided in the first aspect and various possible implementations of the first aspect.

[0050] The beneficial effects of the second to sixth aspects mentioned above can be referred to the relevant descriptions in the first aspect and its various possible implementations, which will not be repeated here. Attached Figure Description

[0051] Figure 1 The image shown is a schematic diagram of an image processing scenario involving coloring a large number of objects, according to an embodiment of this application.

[0052] Figure 2aThe diagram shown is a schematic representation of a colorization process for an image sequence with small inter-frame variations provided in an embodiment of this application.

[0053] Figure 2b The image shown is a schematic diagram of a drawing interface provided in an embodiment of this application.

[0054] Figure 2c The diagram shown is a schematic of a drawing interface with more operation functions provided in an embodiment of this application.

[0055] Figure 2d The image shown is a schematic diagram of a drawing interface displaying an animation creation window, provided in an embodiment of this application.

[0056] Figure 2e The image shown is a schematic diagram of a drawing interface that prompts the selection of a reference frame image, according to an embodiment of this application.

[0057] Figure 2f The image shown is a schematic diagram of a drawing interface that prompts users to select an image to be colored, according to an embodiment of this application.

[0058] Figure 2g The image shown is a schematic diagram of a painting interface for completing coloring processing, provided in an embodiment of this application.

[0059] Figure 3a The diagram shown is a schematic of a drawing interface with more operation functions provided in an embodiment of this application.

[0060] Figure 3b The diagram shown is a schematic of a drawing interface displaying an animation production window provided in an embodiment of this application.

[0061] Figure 3c The diagram shown is a schematic of another drawing interface for prompting the selection of a reference frame image provided in an embodiment of this application.

[0062] Figure 3d The image shown is a schematic diagram of a drawing interface that prompts users to select an image to be colored, according to an embodiment of this application.

[0063] Figure 3e The image shown is a schematic diagram of a painting interface for completing coloring processing, provided in an embodiment of this application.

[0064] Figure 4a The diagram shown is a schematic representation of a colorization process for an image sequence with a large number of frames and significant inter-frame variations, as provided in an embodiment of this application.

[0065] Figure 4b The diagram shown is a schematic of a drawing interface with more operation functions provided in an embodiment of this application.

[0066] Figure 4cThe image shown is a schematic diagram of a drawing interface displaying an animation creation window, provided in an embodiment of this application.

[0067] Figure 4d The diagram shown is a schematic of another drawing interface provided in this application embodiment, which prompts the user to select a reference frame image from recommended multi-frame images.

[0068] Figure 4e The image shown is a schematic diagram of a drawing interface that prompts users to select an image to be colored, according to an embodiment of this application.

[0069] Figure 4f The image shown is a schematic diagram of a painting interface for completing coloring processing, provided in an embodiment of this application.

[0070] Figure 4g The image shown is a schematic diagram of an interface that displays the first prompt information in the form of a pop-up window, according to an embodiment of this application.

[0071] Figure 5a The diagram shown is a schematic of a drawing interface with more operation functions provided in an embodiment of this application.

[0072] Figure 5b The diagram shown is a schematic of a drawing interface displaying an animation production window provided in an embodiment of this application.

[0073] Figure 5c The diagram shown is a schematic of another drawing interface provided in this application embodiment, which prompts the user to select a reference frame image from recommended multi-frame images.

[0074] Figure 5d The image shown is a schematic diagram of a drawing interface that prompts users to select an image to be colored, according to an embodiment of this application.

[0075] Figure 5e The diagram shown is a schematic of another drawing interface for completing the coloring process provided in an embodiment of this application.

[0076] Figure 6 The diagram shown is a schematic representation of the implementation flow of an image processing method provided in an embodiment of this application.

[0077] Figure 7 The diagram shown is a schematic representation of a processing flow for acquiring multiple consecutive frames of images or image sequences according to an embodiment of this application.

[0078] Figure 8 The diagram shown is a schematic representation of a processing flow for determining M frames of images to be recommended from a series of consecutive images or image sequences, according to an embodiment of this application.

[0079] Figure 9a The image shown is a schematic diagram of N frames in a continuous multi-frame image or image sequence provided in an embodiment of this application.

[0080] Figure 9b The figure shown represents a portion of the frame pairs provided in this application embodiment, including the percentage change calculated between two adjacent frame images.

[0081] Figure 10 The diagram shown is a schematic diagram of N frames in a series of consecutive frames of images or an image sequence with a determined reference frame image, provided in an embodiment of this application.

[0082] Figure 11 The diagram shown is a schematic representation of a system architecture for an operating system applicable to this application, as provided in an embodiment of this application.

[0083] Figure 12 The diagram shown is a schematic block diagram of a drawing application provided in an embodiment of this application.

[0084] Figure 13 The figure shown is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0085] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions in the embodiments of this application will be described in detail below with reference to the accompanying drawings and specific implementation methods.

[0086] The method provided in this application can be applied to any electronic device, including but not limited to mobile stations (MS) and mobile terminals (MT). For example, the electronic device can be a mobile phone (including foldable phones), a smart TV or large-screen device, a wearable device, a tablet computer, a desktop computer, a laptop computer (e.g., a laptop), a virtual reality (VR) device, an augmented reality (AR) device, a terminal in a smart city, or a terminal in a smart home. This application does not limit the specific form of the electronic device.

[0087] To facilitate understanding of the solutions in the embodiments of this application by those skilled in the art, some concepts and terms involved in the embodiments of this application will be explained below.

[0088] (1) A binary mask image is a special type of image in which the pixel values ​​have only two possible values, usually 0 and 1. Among them, 0 usually represents the "background" or "region of no interest" in the image. 1 usually represents the "foreground" or "region of interest (ROI)" in the image.

[0089] Figure 1 A schematic diagram of an image processing scenario involving coloring a large number of objects is shown.

[0090] like Figure 1 As shown, the scenario includes a tablet 10, on which an installed drawing application 102 is displayed on the desktop 101. The tablet 10 can respond to user actions such as clicking the drawing application 102 or creating a new artwork, entering the drawing interface 103, which is the user interface (UI) that supports drawing or animation design. Users can draw images in the drawing interface 103 and can click the layer controls 104 to create multiple layers, draw images, and complete coloring.

[0091] Afterwards, the user can click on the operation control 105 to open the animation function, causing the animation production window 106 to appear in the drawing interface 103 displayed on the tablet 10. The images drawn in each layer can present foreground elements such as people, animals, scenery, and buildings. The position, shape, action, color, and texture of the foreground elements in each layer can be customized. Actions can be drawn according to the changing needs of the animation content, such as a cat sitting, jumping, or a butterfly flying, as shown in the multi-frame images 107 in the animation production window 106.

[0092] Continue to refer to Figure 1 As shown in the multi-frame image 107, after the user completes the drawing, the images in different layers need to be colored separately. This means the user needs to repeatedly color the same object (such as a kitten or butterfly). Furthermore, the more images involved in the animation, the greater the workload of repeated coloring operations, resulting in low efficiency.

[0093] Therefore, improving the processing efficiency of image coloring is an urgent problem to be solved.

[0094] To address the aforementioned problems, this application provides an image processing method. This method detects a user operation (referred to as the first user operation) initiating quick coloring within the interface of a drawing application running on an electronic device. It then displays M recommended frames of images to the user and, based on the user's selection, determines at least one reference frame image from these M frames for use as a coloring reference. The electronic device can then batch-color the images to be colored in multiple frames of a first animation based on the user-selected reference frame image, generating colored images corresponding to the multiple frames of the first animation. Thus, this application improves the efficiency of coloring processing when coloring image sequences containing multiple frames (such as anime line art).

[0095] In the case where the user selects at least two reference frame images, each of the at least two image sets can include one reference frame image, and the inter-frame variation between images within each image set can be relatively small. This helps improve the accuracy of the colorization process.

[0096] In some embodiments, the electronic device may recommend at least one frame image to the user as a reference frame image based on the inter-frame variation amplitude between adjacent images in a multi-frame image sequence of the first animation. The reference frame image may be a colorized image. Furthermore, based on the reference frame image selected by the user, the images to be colorized in the image sequence may be divided into two or more image sets, and then the colorization processing may be performed on each image set in parallel.

[0097] It can be understood that the aforementioned inter-frame variation amplitude refers to the degree of difference in the image content (i.e., image content) between consecutive frames, including the sum of differences in various aspects such as position, shape, motion, color, and texture. For example, when there is a fast-moving target in the scene, the inter-frame variation amplitude may be large, manifesting as significant differences in the target's position, shape, motion, or color.

[0098] The multi-frame images of the first animation mentioned above can be a series of consecutive multi-frame images or an image sequence arranged in the order of layers.

[0099] For example, for consecutive multi-frame images or image sequences with small inter-frame variations, this application can recommend fewer reference frame images, such as one reference frame image. Correspondingly, the user can select one reference frame image as a coloring reference. In other embodiments, for consecutive multi-frame images or image sequences with small inter-frame variations, this application can also display all images in the consecutive multi-frame images or image sequence for the user to select a reference frame image. In this case, based on the user-selected reference frame image, the electronic device can divide the other images to be colorized in the consecutive multi-frame images or image sequence into image sets and then colorize them separately.

[0100] In this process, the areas to be colored in the image to be colored correspond to the colored areas in the reference frame image. Electronic devices can use pre-trained neural network models to extract color channel data from each region of the reference frame image and generate corresponding high-dimensional features (vectors), and to extract color channel data from each region of the image to be colored and generate high-dimensional features (vectors) with the same dimensions. Then, by matching the results of these high-dimensional features (vectors), the coloring process of the image to be colored can be achieved based on the reference frame image.

[0101] For consecutive multi-frame images or image sequences with small inter-frame variations, the differences between the content of the image to be colored and the reference frame image are small, and the correspondence between the areas to be colored in the image to be colored and the already colored areas in the reference frame image can be relatively accurate. Therefore, when an electronic device divides an image sequence with small inter-frame variations into image sets based on a user-selected reference frame image and performs colorization processing, it can improve the efficiency of the colorization process.

[0102] For example, for consecutive multi-frame images or image sequences with significant inter-frame variations, this application can recommend a number of reference frame images, which the user can then select as coloring references. These recommended reference frame images can be those with significant inter-frame variations compared to the previous frame. In this case, based on the user-selected reference frame images, the electronic device can group the images to be colored in the relevant image sequence and then perform coloring processing on each group of images separately. Thus, although the image content of the reference frames within different groups differs significantly, and the image content of the images to be colored within different groups also differs significantly, the content difference between the images to be colored in each group and the corresponding reference frame images is small. By performing coloring processing on each group of images separately, the electronic device can improve the efficiency of coloring processing while ensuring accurate coloring.

[0103] It is understood that if the user does not select all of the above-mentioned recommended multi-frame reference images, the electronic device, based on the image processing method provided in this application, may output relevant prompts, prompting the user that the currently recommended images include images with significant differences in content within the relevant image sequence, and that selecting only a portion of the reference frame images may result in poor color processing effects, suggesting that the user select all of the recommended reference frame images, etc., without any restrictions.

[0104] It is understood that the process of the aforementioned electronic device coloring other images to be colorized in the image sequence based on a reference frame image, or the process of the electronic device coloring each set of images to be colorized based on multiple reference frame images, can be processed in parallel using multiple threads created by the system. For example, each sub-thread can execute the coloring process for one set of images. In other embodiments, the electronic device can also use the main thread or UI thread created by the system for serial processing. For example, the main thread can first execute the coloring process for one set of images, and after completing the coloring, execute the coloring process for another set of images. There are no restrictions here. Among these, the coloring efficiency of parallel processing can be higher.

[0105] As an example, Figure 2a An embodiment of this application illustrates a colorization process for an image sequence with small inter-frame variations.

[0106] like Figure 2a As shown, an electronic device, such as the aforementioned tablet 10, can sequentially execute S1 to S3 to perform rapid colorization processing on image sequences with small inter-frame variations. Specifically:

[0107] S1: Obtain a reference frame image selected by the user. This reference frame can be a representative image recommended by the tablet 10, or an image selected by the user from multiple frames that make up the image sequence, and it must be a colored image with color information among the multiple frames input to the tablet 10. The colored image can have color information in a partial or complete area; there are no restrictions on this.

[0108] S2: Obtain the multi-frame images to be colored selected by the user. As mentioned above, the multi-frame images to be colored can be other images in the relevant image sequence besides the reference frame image selected by the user. In some embodiments, the multi-frame images to be colored may also exclude the colored images with coloring information in the relevant image sequence.

[0109] S3: Display the coloring result. This coloring result can be referenced from the above. Figure 2g The drawing interface 260 shown includes a colored image 262 that has been colored.

[0110] Regarding the above Figure 2a The interface changes involved in the coloring process shown below are explained in conjunction with... Figures 2b to 2g The interface changes shown on the drawing interface displayed on the tablet 10, or in combination with... Figures 3a to 3e The interface change process displayed on the tablet 10 shown is illustrated by way of example.

[0111] It should be understood that the various interface diagrams provided in the embodiments of this application do not constitute a specific limitation on the relevant interfaces. In other embodiments of this application, the electronic device may also display other forms of drawing interfaces and other related interfaces based on the image processing method provided in this application. The content of the related interfaces may include more or less prompt information and / or controls, etc., which are not limited here.

[0112] As an example, Figures 2b to 2g A schematic diagram of the interface is shown for colorizing image sequences with small inter-frame variations.

[0113] As an example, Figure 2b A schematic diagram of a drawing interface is shown according to an embodiment of this application.

[0114] like Figure 2b As shown, the drawing interface 210 displayed on the tablet 10 can include images drawn by the user, which can be any frame image from a first animation designed by the user. In some embodiments, the user-drawn images included in the drawing interface 210 can also be superimposed images of multiple frames from a first animation designed by the user, wherein the topmost image can be an image in the layer currently viewed by the user.

[0115] Continue to refer to Figure 2b The optional functional area of ​​the drawing interface 210 may include layer controls 211 that support user editing of layers and operation controls 212 that support users to perform more operations. When the tablet 10 detects that the user clicks on the layer control 211 in the drawing interface 210, it can display the layer information of each layer in which the user's drawn image is located, including the layer order indicated by layer 1, layer 2, layer 3, layer 4, layer 5, layer 6, layer 7, etc., as well as the images in each layer and icons 214 indicating whether the images 213 in each layer are visible.

[0116] Continue to refer to Figure 2b When the tablet 10 detects that the user clicks on the operation control 212 in the drawing interface 210, it can display the following text. Figure 2c The drawing interface shown is 220.

[0117] As an example, Figure 2c An embodiment of this application illustrates a schematic diagram of a drawing interface displaying more operational functions.

[0118] like Figure 2cAs shown, the drawing interface 220 displayed on the tablet 10 can include multiple operation functions, such as "Import Image," "Reference," "Drawing Guide," "Animation," "Time-lapse Recording," and "Preference Settings," supporting users to perform various drawing or design operations. The "Animation" function allows users to view the animation effect of the first animation generated corresponding to the currently drawn multi-frame images. The drawing interface 220 can include a switch control 221 to enable the "Animation" function. The tablet 10 can respond to the user clicking the switch control 221 to enable the "Animation" function, displaying the following text... Figure 2d The drawing interface 230 shown has an animation creation window.

[0119] Figure 2d An embodiment of this application illustrates a schematic diagram of a drawing interface displaying an animation creation window.

[0120] like Figure 2d As shown, the drawing interface 230 displayed on the tablet 10 may include an animation creation window 231. This animation creation window may include thumbnails of multiple frames drawn by the user. These multiple frames can generate a first animation, and therefore can also be referred to as multiple frames of the first animation. These multiple frames, arranged in layer order, can form consecutive multiple frames or an image sequence. These multiple frames may include a colored reference frame image (such as the image indicated by the thumbnail marked "1") and images to be colored (such as the image indicated by the thumbnail marked "2," etc.).

[0121] Continue to refer to Figure 2d The animation production window 231 can also display a control 232 that supports quick coloring. The tablet 10 can respond to the user's click on the control 232 and display the following text... Figure 2e The prompt shown indicates the drawing interface for selecting a reference frame image.

[0122] Continue to refer to Figure 2d ,exist Figure 2a In the image sequence shown, the differences in content between the images indicated by the thumbnail marked "1," "2," "3," and "4" can be small. When a user clicks the thumbnail marked "2" in the animation creation window 231, the drawing interface 230 can display a magnified image 233 of the image indicated by the thumbnail marked "2." As an example, this magnified image can be an uncolored image. In this case, the magnified image 233 can be referred to as the target image. In other embodiments, when the user clicks other thumbnails, the drawing interface 230 can also display magnified images of the images indicated by those other thumbnails. These magnified images can also be colored images or partially colored images, etc., without limitation.

[0123] As an example, Figure 2e A schematic diagram of a drawing interface that prompts the selection of a reference frame image is shown according to an embodiment of this application.

[0124] like Figure 2e As shown, the animation creation window 241 in the drawing interface 240 displayed on the tablet 10 may include content prompting the selection of a reference frame image, such as the prompt text "Please select a reference frame image," and may also include all images 242 in the relevant image sequence. The all images 242 may include a colored image 243. In response to the user's selection of the colored image 243 as the reference frame image, the tablet 10 may display a checked mark in the checkbox corresponding to the colored image 243. In other embodiments, if the user selects an uncolored image from the all images 242 as the reference frame image, the tablet 10 may output relevant prompt information, such as displaying a pop-up window or prompt text, prompting the user to first color the uncolored image selected as the reference frame image, and then select the colored image as the reference frame image. This will not be elaborated upon further in this application.

[0125] Continue to refer to Figure 2e The animation production window 241 can also display a "Next" control 244. After the checked mark is displayed in the checkbox corresponding to the colored image 243, the tablet 10 can respond to the user's click operation on the "Next" control 244 and display the following text. Figure 2f The prompt shown indicates the drawing interface for the image to be colored.

[0126] As an example, Figure 2f An embodiment of this application illustrates a drawing interface that prompts users to select an image to be colored.

[0127] like Figure 2f As shown, the animation creation window 251 in the drawing interface 250 displayed on the tablet 10 may include content prompting the selection of images to be colored, such as the prompt text "N frames of images have been selected," and may also include images 252 to be colored from the relevant image sequence. In this embodiment, the aforementioned images 252 to be colored may be other images in the relevant image sequence besides the reference frame image selected by the user. In some embodiments, the aforementioned images 252 to be colored may also not include colored images with coloring information in the relevant image sequence.

[0128] In this embodiment, the animation production window 251 may further include multiple controls related to coloring, such as a "replace" control 253, a "select all" control 254, and a "start coloring" control 255. The "replace" control 253 can support the tablet 10 returning to the above-mentioned... Figure 2bThe drawing interface 220 or other related drawing interface shown allows the user to reselect the reference frame image. The "Select All" control 254 allows the user to select all images 252 to be colored with a single click. The "Start Coloring" control 255 allows the tablet 10 to color the selected images 252 in the current drawing interface 250, and this coloring process can be performed by the user in the above-mentioned... Figure 2e The reference frame image selected in the painting interface 240 shown is used as a coloring reference.

[0129] Continue to refer to Figure 2f The tablet 10 responds to the user's click on the "Start Coloring" control 255, and can display the following text. Figure 2g The completed painting interface 260 is shown.

[0130] Figure 2g An embodiment of this application illustrates a schematic diagram of a painting interface after coloring.

[0131] like Figure 2g As shown, the animation production window 261 in the drawing interface 260 displayed on the tablet 10 can include a colored image 262 with colored information after the colored image has been colored.

[0132] As mentioned above, the drawing interface prompting the selection of a reference frame image can also be displayed on electronic devices such as tablet 10. In response to the user's operation of opening the animation function, based on the image processing method provided in this application, a process of recommending fewer reference frame images can also be displayed, or other different interface change processes can be displayed.

[0133] As another example Figures 3a to 3e This diagram illustrates another interface for colorizing image sequences with small inter-frame variations.

[0134] As an example, Figure 3a An embodiment of this application illustrates a schematic diagram of a drawing interface displaying more operational functions.

[0135] like Figure 3a As shown, in response to a user clicking the operation control 311 in the drawing interface 310, the tablet 10 can display a function window 312 that includes multiple operation functions. The multiple operation functions displayed in this function window 312 are the same as those described above. Figure 2c The drawing interface 220 shown includes many of the same operation functions, which can be referred to above. Figure 2c The descriptions and related information are omitted here. The function window 312 may include a toggle control 313 for enabling the "Animation" function. The tablet 10 can respond to the user's click of the toggle control 313 to enable the "Animation" function, displaying the information described below. Figure 3bThe drawing interface 320 shown has an animation creation window.

[0136] As an example, Figure 3b An embodiment of this application illustrates a schematic diagram of a drawing interface displaying an animation creation window.

[0137] like Figure 3b As shown, the drawing interface 320 displayed on the tablet 10 may include an animation creation window 321. This animation creation window 321 can be connected to the aforementioned... Figure 2d The content displayed in the animation creation window 231 of the drawing interface 230 is the same as that in the animation creation window 231. The control 322 that supports quick coloring in the animation creation window 231 is also the same as the control 232 in the animation creation window 231. For details, please refer to the above. Figure 2d The descriptions and related information are omitted here. Tablet 10 can respond to user clicks on control 322 and display the information below. Figure 3c The prompt shown indicates the selection of a reference frame image in the drawing interface 330.

[0138] As an example, Figure 3c According to an embodiment of this application, another schematic diagram of a drawing interface that prompts for selecting a reference frame image is shown.

[0139] like Figure 3c As shown, the animation creation window 331 in the drawing interface 330 displayed on the tablet 10 may include prompts for selecting a reference frame image, such as the prompt text "Please select a reference frame image," and may also include an image 332 recommended based on the image processing method provided in this application. This image 332 may be a representative image recommended by the tablet 10 based on the image processing method provided in this application, such as the first or second frame image in an image sequence formed by multiple frames of the first animation, or a colored image with coloring information from the multiple frames of the first animation.

[0140] In other embodiments, electronic devices such as tablet 10 may also display other forms of drawing interfaces based on the image processing method provided in this application, prompting users to select reference frame images, without limitation.

[0141] Continue to refer to Figure 3c The animation production window 331 can also display a "Next" control. After displaying the checked mark in the checkbox corresponding to the recommended image 332, the tablet 10 can respond to the user's click on the "Next" control and display the drawing interface for selecting the image to be colored, as shown in 3d below.

[0142] As an example, Figure 3d An embodiment of this application illustrates a drawing interface that prompts users to select an image to be colored.

[0143] like Figure 3d As shown, the animation creation window 341 in the drawing interface 340 displayed on the tablet 10 may include prompts for selecting images to be colored, such as the prompt text "N frames of images selected," and may also include images to be colored from the relevant image sequence 342. The animation creation window 341 may also include multiple controls related to the coloring process, such as a "replace" control 343, a "select all" control 344, and a "start coloring" control 345. Figure 3d The animation creation window and its controls in the drawing interface 340 shown are respectively related to the above. Figure 2f The animation creation window and its controls in the drawing interface 250 shown are the same; please refer to the above for details. Figure 2f The descriptions shown and related information will not be repeated here.

[0144] Continue to refer to Figure 3d In response to the user clicking the "Start Coloring" control 345, the tablet 10 can display the following text. Figure 3e The completed painting interface 350 is shown.

[0145] As an example, Figure 3e An embodiment of this application illustrates a schematic diagram of a painting interface after coloring.

[0146] like Figure 3e As shown, the animation production window 351 in the drawing interface 350 displayed on the tablet 10 can include a colored image 352 with colored information after the colored image has been colored.

[0147] As another example Figure 4a According to an embodiment of this application, a schematic diagram of a colorization process for an image sequence with a large number of frames and a large range of inter-frame variations is shown.

[0148] With the above Figure 2a The difference between the coloring process shown is that... Figure 4a The coloring process shown can recommend multiple frames as alternative reference frames and allows users to select multiple reference frames to color other images to be colored.

[0149] like Figure 4aAs shown, when there are many image frames in the image to be colored, the electronic device (e.g., tablet 10) can first calculate the inter-frame variation amplitude between each image and another adjacent image in the related image sequence. Then, based on the percentage variation calculated from the inter-frame variation amplitude, such as 1.15% or 9.08%, the electronic device can calculate the total variation amplitude of the related image sequence, determine the number of image frames to be recommended, and determine multiple reference frame images based on the user's selection operation. As an example, these multiple reference frame images could be... Figure 4a The three frames shown are colorized images with significant variations.

[0150] Continue to refer to Figure 4a The process shown illustrates the coloring of images from multiple image sets based on multiple reference frame images. The variation between each reference frame image and the image to be colored in its respective image set can be relatively small. The image to be colored in each image set can be colored by referring to the colors of the reference frame images in that image set. After coloring, each image set is output sequentially according to the index information of each frame image in the aforementioned consecutive multiple frame images or image sequence, thus outputting a second image sequence that has been colorized.

[0151] Thus, for image sequences with a large number of frames and significant inter-frame variations, the image processing method provided in this application can also achieve high color processing efficiency and accuracy.

[0152] Regarding the above Figure 4a The interface changes involved in the coloring process shown below are explained in conjunction with... Figures 4b to 4f The interface changes shown on the drawing interface displayed on the tablet 10, or in combination with... Figures 5a to 5e The interface change process displayed on the tablet 10 shown is illustrated by way of example.

[0153] It should be understood that the various interface diagrams provided in the embodiments of this application do not constitute a specific limitation on the relevant interfaces. In other embodiments of this application, the electronic device may also display other forms of drawing interfaces and other related interfaces based on the image processing method provided in this application. The content of the related interfaces may include more or less prompt information and / or controls, etc., which are not limited here.

[0154] As an example, Figures 4b to 4f A schematic diagram of the interface is shown for colorizing image sequences with large inter-frame variations.

[0155] As an example, Figure 4b An embodiment of this application illustrates a schematic diagram of a drawing interface displaying more operational functions.

[0156] like Figure 4bAs shown, in response to a user clicking the operation control 411 in the drawing interface 410, the tablet 10 can display a function window 412 that includes multiple operation functions. The multiple operation functions displayed in this function window 412 are the same as those described above. Figure 2c The drawing interface 220 shown includes many of the same operation functions, which can be referred to above. Figure 2c The descriptions and related information are omitted here. The function window 412 may include a toggle control 413 for enabling the "Animation" function. The tablet 10 can respond to the user's click of the toggle control 413 to enable the "Animation" function, displaying the information described below. Figure 4c The drawing interface 420 shown has an animation creation window.

[0157] As an example, Figure 4c An embodiment of this application illustrates a schematic diagram of a drawing interface displaying an animation creation window.

[0158] like Figure 4c As shown, the drawing interface 420 displayed on the tablet 10 may include an animation creation window 421. This animation creation window may include frames from a related image sequence, including images labeled "1", "2", ..., "8", etc. When a user clicks on a thumbnail labeled "7" in the animation creation window 421, it can be a magnified image of the image indicated by that thumbnail displayed in the drawing interface 420. This image can be an uncolored image or a colored image with coloring information; there are no restrictions here. Figure 4c In the image sequence shown, the image indicated by the thumbnail labeled "6" has little content difference from the images indicated by the thumbnails labeled "5", "4", and "3", but the image indicated by the thumbnail labeled "7" has a significant content difference from the image indicated by the thumbnail labeled "6".

[0159] Continue to refer to Figure 4c The animation production window 421 can also display a control 422 that supports quick coloring. The tablet 10 can respond to the user's click on the control 422 and display the following text... Figure 4d The prompt shown indicates the drawing interface for selecting a reference frame image.

[0160] As an example, Figure 4d According to an embodiment of this application, a schematic diagram of a drawing interface is shown that prompts a user to select a reference frame image from a recommended set of multi-frame images.

[0161] like Figure 4dAs shown, the animation creation window 431 in the drawing interface 430 displayed on the tablet 10 may include content prompting the selection of a reference frame, such as the prompt text "M frames of images with large variation have been recommended for you. Please select a reference frame image." In some other embodiments, the prompt text displayed in the animation creation window 431 may also be other content, such as "M frames of images have been recommended for you. Please select a reference frame image," etc., and there is no limitation here.

[0162] The animation production window 431 may also include a multi-frame image 432 recommended by the image processing method provided in this application for the tablet 10, as an option for reference frames. The multi-frame image 432 may be, for example, an image with a large variation range in the M frames of the related image sequence, such as the image indicated by the thumbnail marked "1", the image indicated by the thumbnail marked "4", the image indicated by the thumbnail marked "5", the image indicated by the thumbnail marked "6", the image indicated by the thumbnail marked "7", and the image indicated by the thumbnail marked "8".

[0163] As an example, refer to the above. Figure 4c The thumbnail marked "1" indicates the first frame of the relevant image sequence to be colored. The image marked "4" shows a significant change in the kitten's tail shape compared to the image marked "3". The image marked "5" shows a significant change in the kitten's tail shape compared to the image marked "4". The image marked "6" shows a significant change in the kitten's tail shape and head-raising posture compared to the image marked "5". The image marked "7" shows a significant change in the kitten's front paw movements and facial expression compared to the image marked "6". The image marked "8" shows a significant change in the kitten's front paw movements and tail shape compared to the image marked "7".

[0164] That is, in the relevant image sequence, the recommended M-frame images have a larger inter-frame variation compared to the previous frame image. In some embodiments, the weighted score of the recommended M-frame images may also be higher than that of other images. The recommended multi-frame images 432 may include colored images with coloring information, such as images indicated by thumbnails marked "1", "6", and "8". The tablet 10 may display a checked mark in the checkbox corresponding to the relevant image in response to the user's selection of a colored image with coloring information as a reference frame image. In other embodiments, if the user selects an uncolored image as a reference frame image from the recommended multi-frame images 432, the tablet 10 may output relevant prompts, such as displaying a pop-up window or prompting text, prompting the user to first colorize the uncolored image selected as the reference frame image, and then select the colored image as the reference frame image. This application will not elaborate on this further.

[0165] Continue to refer to Figure 4d The animation production window 431 can also display a "Select All" control 433 and a "Next" control 434. The "Select All" control 433 allows selecting all recommended multi-frame images 432 as reference frame images. After the selected reference frame image is marked with a checkmark, the tablet 10 can respond to the user's click on the "Next" control 434 and display the following text... Figure 4e The prompt shown indicates the drawing interface for the image to be colored.

[0166] As an example, Figure 4e An embodiment of this application illustrates a drawing interface that prompts users to select an image to be colored.

[0167] like Figure 4e As shown, the animation creation window 441 in the drawing interface 440 displayed on the tablet 10 may include content prompting the selection of images to be colored, such as the prompt text "N frames of images have been selected," and may also include images 442 to be colored from the relevant image sequence. In this embodiment, the aforementioned images 442 to be colored may be other image frames in the relevant image sequence besides the reference frame image selected by the user. In some embodiments, the aforementioned images 442 to be colored may also not include colored images with coloring information in the relevant image sequence.

[0168] In this embodiment, the animation production window 441 may further include multiple controls related to coloring, such as a "replace" control 443, a "select all" control 444, and a "start coloring" control 445. The "replace" control 443 can support the tablet 10 returning to the above-mentioned... Figure 4dThe drawing interface 430 or other related drawing interface shown allows the user to reselect the reference frame image. The "Select All" control 444 allows the user to select all images 442 to be colored with a single click. The "Start Coloring" control 445 allows the tablet 10 to color the selected images 442 in the current drawing interface 440, and this coloring process can be based on the above user input. Figure 4d The reference frame image selected in the painting interface 430 shown is used as a coloring reference, and the user can utilize the above-mentioned... Figure 4d The reference frame image selected in the painting interface 430 first groups the above-mentioned image 442 to be colored, and then performs coloring processing on each group of images to improve the coloring processing efficiency.

[0169] Continue to refer to Figure 4e In response to the user clicking the "Start Coloring" control 445, the tablet 10 can display the following text. Figure 4f The completed painting interface 450 is shown.

[0170] Figure 4f An embodiment of this application illustrates a schematic diagram of a painting interface after coloring.

[0171] like Figure 4f As shown, the animation production window 451 in the drawing interface 450 displayed on the tablet 10 can include each image frame 452 with coloring information after coloring is completed.

[0172] In other embodiments, after determining the recommended M-frame images based on the image processing method provided in this application, the tablet 10 and other electronic devices may also output relevant abnormal prompt information for the M-frame images to inform the user that if a uniform reference frame image is used to colorize the images to be colorized in the relevant image sequence, the colorization effect of the aforementioned M-frame images may be poor. The tablet 10 and other electronic devices may also output colorization suggestions for the aforementioned M-frame images at the same time as or after outputting the aforementioned abnormal prompt information, such as suggesting that the user manually colorize the aforementioned M-frame images and then use the colorized images as reference frame images for colorizing other images to be colorized in the relevant image sequence, etc., without limitation.

[0173] In some embodiments, if the user is in the above Figure 4d In the drawing interface 430 shown, an uncolored image is selected as the reference frame image. The tablet 10 can also display relevant prompts (such as a first prompt), such as a pop-up window or prompting text, prompting the user to first color the uncolored image selected as the reference frame image, and then select the colored image as the reference frame image. In other embodiments, the tablet 10 can also display the above-mentioned... Figure 4dBefore the drawing interface 430 shown, the aforementioned first prompt message is displayed.

[0174] As an example, Figure 4g According to an embodiment of this application, a schematic diagram of an interface that displays a first prompt message in the form of a pop-up window is shown.

[0175] like Figure 4g As shown, the drawing interface 460 displayed on the tablet 10 can be compared with the above-mentioned... Figure 4d The drawing interface 430 shown is the same. When the tablet 10 detects that the user selects the image indicated by the thumbnail marked "1" (colored image), the image indicated by the thumbnail marked "5" (uncolored image), the image indicated by the thumbnail marked "6" (colored image), and the image indicated by the thumbnail marked "8" (uncolored image) as reference frame images in the drawing interface 460, a pop-up window 461 can be displayed. This pop-up window 461 can display the thumbnail marked "5" and the thumbnail marked "8". For comparison, this pop-up window 461 can also display the thumbnail marked "1", indicating that the thumbnail marked "1" has a larger inter-frame variation compared to the thumbnails marked "5" and "8".

[0176] In this embodiment of the application, the pop-up window 461 may also display prompting text, such as "Recommended multi-frame reference images" and "In order to ensure the coloring effect of images with large changes, please color the recommended images", so as to prompt the user to color the uncolored images selected as reference frame images first.

[0177] As another example Figures 5a to 5e A schematic diagram of the interface is shown for colorizing image sequences with large inter-frame variations.

[0178] As an example, Figure 5a An embodiment of this application illustrates a schematic diagram of a drawing interface displaying more operational functions.

[0179] like Figure 5a As shown, in response to a user clicking the operation control 511 in the drawing interface 510, the tablet 10 can display a function window 512 that includes multiple operation functions. The multiple operation functions displayed in this function window 512 are the same as those described above. Figure 4b The drawing interface 410 shown includes many of the same operation functions, which can be referred to above. Figure 4b The descriptions and related information are omitted here. The function window 512 may include a toggle control 513 for enabling the "Animation" function. The tablet 10 can respond to the user's click of the toggle control 513 to enable the "Animation" function, displaying the information described below. Figure 5bThe drawing interface 520 shown has an animation creation window.

[0180] Figure 5b An embodiment of this application illustrates a schematic diagram of a drawing interface displaying an animation creation window.

[0181] like Figure 5b As shown, the drawing interface 520 displayed on the tablet 10 may include an animation creation window 521. This animation creation window 521 can be connected to the aforementioned... Figure 4c The content displayed in the animation creation window 421 of the drawing interface 420 shown is the same. The control 522 that supports quick coloring in the animation creation window 421 is also the same as the control 422 in the animation creation window 421. For details, please refer to the above. Figure 4c The descriptions and related information shown are omitted here. Tablet 10 can respond to user clicks on control 522 and display the information below. Figure 5c The prompt shown indicates the selection of a reference frame image in the drawing interface 530.

[0182] As an example, Figure 5c According to an embodiment of this application, another schematic diagram of a drawing interface is shown that prompts the user to select a reference frame image from a recommended multi-frame image.

[0183] like Figure 5c As shown, the animation creation window 531 in the drawing interface 530 displayed on the tablet 10 may include content prompting the selection of reference frames, such as the prompt text "3 / M frame reference frame images have been selected," and may also include M frame images 532 recommended based on the image processing method provided in this application. The "3" in "3 / M" may correspond to the number of reference frame images selected by the user. Figure 5c The M-frame image 532 shown can be a representative multi-frame image recommended by the image processing method provided in this application for the tablet 10, such as an image with a large inter-frame variation amplitude between itself and the adjacent previous frame image.

[0184] In other embodiments, electronic devices such as tablet 10 may also display other forms of drawing interfaces based on the image processing method provided in this application, prompting users to select reference frame images, without limitation.

[0185] exist Figure 5c In the illustrated embodiment, the reference frame image selected by the user may include a partially colored image. For example, the image indicated by the thumbnail marked "1" may be a partially colored image where the kitten's tail and legs are uncolored, while other areas have colored information; similarly, the image indicated by the thumbnail marked "8" may be a partially colored image where the butterfly is uncolored, while the entire body of the kitten has colored information. Figure 5cIn the illustrated embodiment, the reference frame image selected by the user may also include a colored image with color information for the entire area, such as the image indicated by the thumbnail marked "6".

[0186] Continue to refer to Figure 5c The animation production window 531 can also display a "Select All" control and a "Next" control. After displaying the checked mark in the checkbox corresponding to the reference frame image selected by the user, the tablet 10 can respond to the user's click operation on the "Next" control and display the following text. Figure 5d The prompt shown indicates the drawing interface for the image to be colored.

[0187] As an example, Figure 5d An embodiment of this application illustrates a drawing interface that prompts users to select an image to be colored.

[0188] like Figure 5d As shown, the animation creation window 541 in the drawing interface 540 displayed on the tablet 10 may include prompts for selecting images to be colored, such as the prompt text "N frames of images selected," and may also include images to be colored from the relevant image sequence 542. The animation creation window 541 may also include multiple controls related to the coloring process, such as a "replace" control 543, a "select all" control 544, and a "start coloring" control 545. Figure 5d The animation creation window and its controls in the drawing interface 540 shown are respectively related to the above. Figure 4e The animation creation window and its controls in the drawing interface 440 shown are the same; please refer to the above for details. Figure 4e The descriptions shown and related information will not be repeated here.

[0189] Continue to refer to Figure 5d In response to the user clicking the "Start Coloring" control 545, the tablet 10 can display the following text. Figure 5e The completed painting interface shown is 550.

[0190] As an example, Figure 5e According to an embodiment of this application, another schematic diagram of a painting interface after coloring is shown.

[0191] like Figure 5e As shown, the animation production window 551 in the drawing interface 550 displayed on the tablet 10 can include a colored image 552 with colored information after the colored image has been colored.

[0192] As mentioned earlier, electronic devices (such as tablet 10) can group the images to be colored in a relevant image sequence based on multiple reference frame images selected by the user, and then perform coloring processing on each group of images to be colored separately. Referring to the above... Figure 4a and Figure 5c The tablet 10 can divide the images to be colored in the multi-frame images (i.e., the related image sequence) of the first animation into three image sets, based on two locally colored images and one colored image selected by the user as reference frame images. Based on this, the tablet 10 can perform coloring processing on each of the three image sets using the reference frame image in each set, and then generate... Figure 5e The image shown is a colored image 552 with colored information after coloring. Specifically, the partially colored image 552 where the kitten's tail and legs are uncolored while other areas have colored information can be obtained by coloring an image from the first image set based on the thumbnail labeled "1". The partially colored image 552 where all areas have colored information can be obtained from the second image set based on the thumbnail labeled "6". The partially colored image 552 where the butterfly is uncolored while all areas of the kitten's body have colored information can be obtained from the third image set based on the thumbnail labeled "8".

[0193] Thus, although the content of the reference frame images in different groups differs greatly, and the content of the images to be colored in different groups also differs greatly, the content difference between the images to be colored in each group and the reference frame images in the corresponding group is small. The electronic device can perform coloring processing on each set of images to be colored separately, which can improve the efficiency of coloring processing while ensuring accurate coloring.

[0194] The following section, in conjunction with flowcharts and related figures, provides a detailed description of the specific implementation process of the image processing method provided in this application.

[0195] It should also be stated that the steps in the methods and processes in this application are numbered for ease of reference, not to limit the order of steps. If there is an order between the steps, the textual description shall prevail.

[0196] Figure 6 An implementation flowchart of an image processing method is shown according to an embodiment of this application.

[0197] It should be noted here that Figure 6 In the illustrated process, the executing entity for each step can be an electronic device with drawing tools installed (such as drawing applications like "Born to Draw"). The executing entity for each step will not be repeated when describing the implementation process below. As mentioned earlier, the execution... Figure 6 The electronic devices used in the illustrated implementation process include, but are not limited to, the aforementioned tablet 10, mobile phones (including foldable screen phones), smart TVs or large-screen devices, and laptop computers (e.g., notebook computers). This application describes the tablet 10 as an example.

[0198] Specifically, such as Figure 6 As shown, the implementation process may include:

[0199] S601: Run the drawing application and display the first interface of the drawing application, the first interface including at least one frame of the multi-frame images of the first animation.

[0200] For example, the first interface described above may be a drawing interface displayed on the tablet 10, including controls that support quick coloring. In this embodiment of the application, the first interface described above may refer to the above... Figure 2d , Figure 3b , Figure 4c Or the drawing interface shown in 5b.

[0201] S602: In response to a first user operation instructing the initiation of fast coloring, display an M-frame image and a target image, wherein the M-frame image is a thumbnail of at least one of the multiple frames in the first animation, and the target image is an enlarged image of the image to be colored in the multiple frames of the first animation or any other image in the M-frame image.

[0202] For example, the first user operation described above could be a click operation on the control that supports quick coloring. In response to this first user operation, the tablet 10 can display a drawing interface including recommended M-frame images and prompt the user to select a reference frame image from the recommended multi-frame images, referring to the above... Figure 2e , Figure 3c , Figure 4d or Figure 5c The drawing interface shown.

[0203] In some embodiments, the M-frame images can be images to be recommended determined from N-frame images, where M is a positive integer less than N, and the M-frame images meet the recommendation criteria, which are related to the degree of content difference between each frame in the consecutive multi-frame images or image sequence. The N-frame images are images in the consecutive multi-frame images or image sequence, where N is a positive integer, and the consecutive multi-frame images or image sequence includes images to be colored.

[0204] Specifically, the aforementioned recommendation criteria may be related to, for example, the cumulative result of the inter-frame variation amplitude between adjacent images in a series of consecutive frames (i.e., the total variation amplitude), the total number of images in a series of consecutive frames (i.e., the total number of images), or the weighted score corresponding to each image in a series of consecutive frames (i.e., the total number of images). In other embodiments, the aforementioned recommendation criteria may also be associated with other reasonable parameters to filter out the aforementioned M frames from the aforementioned N frames, which is not limited here.

[0205] In some embodiments, the interface for displaying the M-frame image and the target image described above may include a first control for determining the selection result, as described above. Figure 2e , Figure 3c , Figure 4d Or, as shown in 5c, the "Next" control in the drawing interface. Correspondingly, the aforementioned detection of user confirmation of the selection result can be a click operation applied to the aforementioned first control.

[0206] In other embodiments, the interface for displaying the M-frame image and the target image, as well as the first control, may take other forms of interface and control, which are not limited here.

[0207] In some embodiments, the target image may be a magnified image of the currently drawn or selected image displayed by the electronic device system according to the user's instruction to open the animation function, or an image in the layer where the user first opens the animation function. In other embodiments, the target image may be a magnified image of any frame image determined by the electronic device system from the recommended M-frame images according to the user's instruction to open the animation function. In other embodiments, the target image may be a magnified image of the image selected by the user from the M-frame images, displayed by the electronic device system according to the user's instruction to open the animation function and the user's click operation on the displayed thumbnail. Among the recommended M-frame images, the images other than the target image may be images displayed as small images (such as thumbnails). The user's instruction to open the animation function may be referred to as a second user operation.

[0208] It is understood that the target image can be a partially colored image with coloring information, or a colored image with coloring information in all areas. The aforementioned coloring information can be the coloring information already present when the user selects the reference frame image, or the coloring information filled in when the user selects the reference frame image; there are no restrictions here.

[0209] S603: Obtain at least one reference frame image selected by the user from the M-frame images.

[0210] For example, when the tablet 10 detects a user's selection operation on at least one frame of the M-frame images and detects a user's confirmation operation on the selection result, it determines the at least one frame as the at least one reference frame image, that is, it obtains the at least one reference frame image selected by the user from the M-frame images.

[0211] It is understood that the aforementioned at least one reference frame image can be a colored image. In some embodiments, if the aforementioned at least one reference frame image includes an uncolored image, the tablet 10 can output relevant prompts, such as displaying a pop-up window or prompting text, prompting the user to first colorize the uncolored image selected as the reference frame image, and then use the colored image as the reference frame image selected by the user.

[0212] It can be understood that the aforementioned at least one reference frame image may include one reference frame image, as described above. Figure 2e The colorized image 243 with coloring information selected by the user in the drawing interface 240 shown; for example... Figure 3c The image 332 selected by the user in the drawing interface 330 shown; for example... Figure 4d The drawing interface 430 shown includes images indicated by thumbnails labeled "1", "6", and "8" selected by the user; and so on. Figure 5c The drawing interface shown includes images indicated by thumbnails labeled "1", "6", and "8", etc., selected by the user. In other embodiments, the aforementioned at least one reference frame image may include a greater number of images, which are not limited or elaborated here. The number of reference frame images selected by the user may be less than or equal to the recommended number of image frames M.

[0213] S604: Based on at least one reference frame image, generate colorized images corresponding to multiple frames of the first animation.

[0214] For example, based on the aforementioned at least one reference frame image, the images to be colored in a series of consecutive frames or image sequences can be divided into at least two image sets. Each image set may include multiple frames of images to be colored, i.e., uncolored images. Based on this, the tablet 10 can perform coloring processing on the at least two divided image sets separately. For example, the tablet 10 can use multiple threads to perform coloring processing on the aforementioned at least two image sets in parallel, thereby improving the efficiency of image coloring processing.

[0215] It is understood that when the user selects at least two reference frame images, each of the at least two image sets can include one reference frame image, and the inter-frame variation between images within each image set can be relatively small. This helps improve the accuracy of the colorization process.

[0216] After colorization, each frame in the relevant image sequence can be considered a colorized image, and this image sequence can be referred to as the second image sequence. Unlike the aforementioned continuous multi-frame images or image sequences, the second image sequence does not include uncolorized images.

[0217] In other embodiments, the main thread or UI thread in the system of the tablet 10, which is responsible for coloring the images, can also execute relevant image processing algorithms to colorize the at least two image sets respectively, thereby improving the accuracy of coloring the images in different image sets.

[0218] It is understood that the above-mentioned at least two image sets may include two image sets (such as the first image set and the second image set), or three image sets (such as the first image set, the second image set and the third image set), or more than three image sets, without any limitation.

[0219] The specific image processing procedure, which divides at least two image sets based on at least one reference frame image selected by the user, will be described in detail below with reference to the relevant figures, and will not be repeated here.

[0220] It is understood that before executing the above-described S601 process of running the drawing application and displaying the first interface of the drawing application, the tablet 10 can first acquire a series of consecutive frames of images of the first animation. This series of consecutive frames can be referred to as a series of consecutive frames or an image sequence in this embodiment. The series of consecutive frames or an image sequence can include N frames, where N is a positive integer. The above-described series of consecutive frames or an image sequence can be an ordered collection of images, where each frame can have a corresponding sequence position within the series of consecutive frames or an image sequence. This sequence position can also serve as index information for each frame. The N frames can include images to be colored, such as line drawings completed by the artist but not yet colored, or colored images, such as line drawings completed by the artist and then colored. In this embodiment, the N frames in the above-described series of consecutive frames or an image sequence can be binary mask images, such as binary mask images generated based on the line drawings drawn by the artist. In some embodiments, each frame in the above-mentioned consecutive multi-frame images or image sequence may also include a pre-processed binary mask image corresponding to an image imported by the user on the relevant interface of the painting application, without limitation.

[0221] The specific preprocessing and other processes involved in acquiring multiple consecutive frames of images or image sequences will be described in detail below with reference to the accompanying figures, and will not be repeated here.

[0222] In some embodiments, the M-frame images are images to be recommended determined from N-frame images, and may include: determining the M-frame images from the N-frame images based on the determined number of images to be recommended M and the ranking result of the weighted scores of the N-frame images. The number of images to be recommended M may be related to the total variation amplitude (denoted as S) of the N-frame images and the number of image frames N (i.e., the total number of frames) included in the consecutive multi-frame images or image sequence. The total variation amplitude of the N-frame images is the sum of the inter-frame variation amplitudes (which may be denoted as the first variation amplitude) of (N-1) frame pairs determined based on the N-frame images. Each of the (N-1) frame pairs includes two adjacent frames in the consecutive multi-frame images or image sequence, and the first variation amplitude is used to indicate the degree of content difference between the two adjacent frames.

[0223] The number of image frames M to be recommended can be determined as follows: First, based on the total change amplitude S and the corresponding preset reference frame coefficient C (denoted as the first reference coefficient), an image frame number Ks (denoted as the first image frame number) is determined, and based on the number of image frames N included in the above-mentioned consecutive multi-frame images or image sequence and the corresponding preset reference frame coefficient F (denoted as the second reference coefficient), another image frame number Kn (denoted as the second image frame number) is determined, and then the minimum value between the first image frame number and the second image frame number is determined as the number of image frames M to be recommended.

[0224] The specific process of determining the recommended number of image frames M will be explained in detail below with reference to the relevant figures, and will not be repeated here.

[0225] In some embodiments, when the total number of frames N of the above-mentioned consecutive multi-frame images or image sequences is not less than 3 (i.e., N≥3), the recommended number of image frames M may not be greater than N / 3, i.e., M≤N / 3. If the total number of frames N of the above-mentioned consecutive multi-frame images or image sequences is small, such as N<3, then reference frame images of up to 3 frames may be recommended based on the degree of content difference between each frame image, or no reference frame images may be recommended, without any restrictions.

[0226] It is understood that the larger the recommended number of image frames M is, the more reference frame images the user can select, and the greater the processing load of the tablet 10 and other electronic devices in performing the relevant algorithms to colorize each image set. Therefore, in order to control the computational resources consumed by the tablet 10 and other electronic devices in performing the image processing method provided in this application within a reasonable range, the system of the tablet 10 and other electronic devices can also preset a corresponding recommended upper limit threshold, such as 10 frames (i.e., M≤10) or other reasonable values, which are not limited here.

[0227] Based on the aforementioned determined number of image frames to be recommended, the process of determining the M images from the N images may include a process of determining the images corresponding to the first M weighted scores in the first scoring sequence as the M images. The first scoring sequence is the weighted scores of the N images arranged in descending order from high to low. The weighted scores of the N images are related to the first change magnitude between each image and the previous image in the consecutive multi-frame images or image sequence, and the sequence position of each image in the consecutive multi-frame images or image sequence.

[0228] It is understood that the tablet 10 can determine the index information of each frame image based on the sequence position of each frame image in the aforementioned continuous multi-frame images or image sequence, including the index information of the aforementioned M-frame images determined based on preset recommendation conditions. If the user selects all of the M-frame images recommended by the tablet 10 as reference frame images, a marker indicating the reference frame images can be added to the index information of the M-frame images as identification information for the reference frame images. If the user selects a portion of the M-frame images recommended by the tablet 10 as reference frame images, a marker indicating the reference frame images can be added to the index information of the selected portion of the images, which will not be elaborated here.

[0229] The above processing procedure for multiple consecutive frames of images or image sequences will be explained in detail below with reference to the relevant accompanying drawings.

[0230] Figure 7 An embodiment of this application illustrates a processing flow diagram for acquiring multiple consecutive frames of images or image sequences.

[0231] like Figure 7 As shown, the processing flow may include:

[0232] S701: Acquire a series of consecutive frames of images input by the user, which includes N frames of images.

[0233] For example, the N frames of images input by the user can be line drawings created by the user (such as a illustrator) in multiple layers in the drawing interface, such as those described above. Figure 4a The examples shown are images labeled "1" to "8". These N frames can include either uncolored or colored images; there is no limitation on this. Colored images can serve as reference frames for subsequent colorization processing, while uncolored images can serve as images to be colored in subsequent processing.

[0234] In other embodiments, the N frames of images input by the user may be, for example, multiple frames of images included in a painting artwork imported by the user on the relevant interface of the painting application, or images imported by the user on the relevant interface of the painting application, etc., and there is no limitation here.

[0235] S702: Preprocess N frames of images to obtain N frames of binary mask images.

[0236] For example, the preprocessing described above may include extracting alpha channel data and binarizing the N frames of images respectively, thereby obtaining a binary mask image for each frame. As mentioned earlier, the pixel values ​​of the binary mask image have only two possible values, usually 0 and 1, which facilitates the calculation of the inter-frame variation amplitude between each frame.

[0237] Taking a user-drawn line drawing as an example, the line drawing can be composed of black (RGB(0,0,0)) or other darker colors (where the values ​​of each color channel are small or close to 0). This part corresponds to channel values ​​greater than 0, such as 1, that can be extracted during the Alpha channel data extraction process. The background and other parts of the image correspond to channel values ​​of 0 that can be extracted during the Alpha channel data extraction process. After Alpha channel data extraction, the Alpha channel values ​​can be converted into a binary mask image using a corresponding binarization function (such as a thresholding function). For example, pixels with Alpha channel values ​​greater than 0 are assigned a value of 255 to indicate the presence of the line drawing; pixels with Alpha channel values ​​equal to 0 are assigned a value of 0 to represent the background part excluding the line drawing.

[0238] For example, the above preprocessing of N frames of images can be performed according to the image data stream formed by the arrangement order of the N frames of images input by the user. The resulting binary mask image is also an ordered image, i.e., the above-mentioned consecutive multi-frame images or image sequence. The arrangement order of the above-mentioned N frames of images can correspond to the overlay order of the layers where the user-drawn line drawings are located, or it can correspond to the time order of the user drawing or importing related images, and there is no limitation here.

[0239] The processing procedure for outputting multiple consecutive frames of images or M frames in an image sequence, as described in S602 above, will be explained in detail below with reference to the relevant accompanying drawings.

[0240] Figure 8 According to an embodiment of this application, a schematic diagram of a processing flow for determining M frames of images to be recommended from a series of consecutive images or an image sequence is shown.

[0241] like Figure 8 As shown, the processing flow may include:

[0242] S801: Determine the inter-frame variation amplitude P based on the pixel difference between the (i+1)th frame and the ith frame in a series of consecutive frames or image sequences. iWhere 1≤i≤(N-1) and are integers.

[0243] For example, the aforementioned consecutive multi-frame images or image sequence may include N frames, thus the total number of frames in the consecutive multi-frame images or image sequence is N. When calculating the inter-frame variation amplitude, any two adjacent frames (i.e., adjacent frames) in the consecutive multi-frame images or image sequence can be formed into a frame pair, such as the frame pair formed by the i-th frame image and the (i+1)-th frame image mentioned above. Furthermore, the tablet 10 can calculate the difference of each pixel in each frame pair based on the fact that both frames in each frame pair are binary mask images, and determine the accumulated result of the differences of each pixel as the pixel difference between the two frames.

[0244] In this embodiment, the tablet 10 can calculate the percentage change corresponding to the pixel difference between two frames as the inter-frame change amplitude P. i .

[0245] It is understandable that for N frames of images, the inter-frame variation amplitude P calculated by the tablet 10 is... i It can also include (N-1) values, where the (N-1) inter-frame variation magnitude P i This can form a data set of percentage changes {P1, P2, ..., P}. n-1}

[0246] As an example, Figure 9a An embodiment of this application illustrates a schematic diagram of N frames in a series of consecutive images or an image sequence.

[0247] like Figure 9a As shown, a series of consecutive frames or image sequences can include frame 1, frame 2, ..., frame 49, frame 50, i.e., taking N=50 as an example. Adjacent frames 1 and 2 can form a frame pair, adjacent frames 2 and 3 can form a frame pair, and so on, up to adjacent frames 49 and 50, for a total of 49 frame pairs, or (N-1) frame pairs.

[0248] The inter-frame variation amplitude P calculated for each of the above frame pairs i For example, it may include P1, which can indicate the degree of content difference between frame 1 and frame 2; P2, which can indicate the degree of content difference between frame 2 and frame 3; ...; and P, which can indicate the degree of content difference between frame 49 and frame 50. n-1 .

[0249] Understandably, the greater the content differences, the greater the corresponding inter-frame variation. (Continue to refer to...) Figure 9aThe content difference between frame 1 and frame 2 is small, and the corresponding inter-frame change amplitude P1 is also small. However, the content difference between frame 32 and frame 33 is large, and the corresponding inter-frame change amplitude P1 is large. 32 The values ​​are also relatively large. They will not be enumerated here. In the embodiments of this application, the aforementioned inter-frame variation amplitude can be the pixel difference between two adjacent frames included in each frame pair, and this pixel difference can be an absolute value. Based on this pixel difference, the percentage change in the content of the two adjacent frames can be further calculated as the inter-frame variation amplitude P. i .

[0250] As an example, Figure 9b According to an embodiment of this application, a portion of the frame pairs includes the inter-frame variation amplitude between two adjacent frames.

[0251] like Figure 9b As shown in (1), based on the pixel difference between the first and second frames, the percentage change in the difference map can be calculated to be 1.15%. At this time, the inter-frame change amplitude P1 = 1.15%.

[0252] like Figure 9b As shown in (2), based on the pixel difference between frame 32 and frame 33, the percentage change in the difference map can be calculated to be 9.08%. At this time, the inter-frame change amplitude P 32 =9.08%.

[0253] In other embodiments, the tablet 10 may also determine the pixel difference between the two frames of each frame pair as the inter-frame variation amplitude P. i No restrictions are imposed here.

[0254] S802: Calculate the inter-frame variation amplitude P i The total change S is obtained by summing the results.

[0255] For example, the tablet 10 can execute a preset accumulation algorithm to calculate the N-1 inter-frame change amplitudes P obtained in S801 above. i The total change amplitude of the relevant image sequence is obtained by summing the values, denoted as S. The specific calculation process can be found in the formula S=ΣPᵢ(i=1,…,N-1).

[0256] It is understandable that the total change magnitude S can reflect the cumulative difference of the relevant image sequences and is an important parameter for determining the recommended number of image frames.

[0257] S803: Determine the first image frame number Ks based on the total change amplitude S, and determine the second image frame number Kn based on the total number of frames N.

[0258] For example, the tablet 10 can execute a preset image frame recommendation algorithm to calculate the recommended frame number based on the total change amplitude S and the recommended frame number based on the total number of frames N. The recommended frame number based on the total change amplitude can be denoted as the first image frame number Ks. The recommended frame number based on the total number of frames N can be denoted as the second image frame number Kn. The specific calculation process can refer to the formulas Ks=S / C and Kn=N / F, where C is the change sensitivity coefficient and F is the frame density coefficient.

[0259] It is understandable that the size of C can be related to the inter-frame variation amplitude P mentioned above. i The value of F is related to the rate of change and can be determined empirically, such as within the range of [5, 10]. The magnitude of F can be related to the total number of frames N. The larger the total number of frames, the denser the image distribution in the corresponding image sequence, and the larger F will be. F can also be determined empirically, such as within the range of [3, 6].

[0260] S804: The minimum value between the first image frame number Ks and the second image frame number Kn is determined as the recommended image frame number M.

[0261] For example, the tablet 10 can determine the minimum value of Ks and Kn calculated in S803 above as the recommended number of image frames M. That is, the recommended number of image frames is the minimum value between the recommended number of frames based on the total change amplitude S and the recommended number of frames based on the total number of frames N.

[0262] As mentioned earlier, the total number of frames in the image sequence to be colored and the total variation between adjacent images in the sequence can significantly influence the recommended number of image frames M as the reference frame images. For an image sequence consisting of N frames, with the same number of frames, a greater variation between frames results in a greater total variation, and thus a higher recommended number of image frames. Conversely, with the same variation between frames, a higher number of frames can also lead to a higher recommended number of image frames. Specific effects can be seen in the examples shown in Table 1 below.

[0263] Table 1

[0264] Animated image sequence characteristics Total change S Total frames N The range of values ​​for M Recommendation results Large inter-frame variation and a large number of frames 80%-100% 30+ 8-10 More images recommended Large inter-frame variation and few frames 80%-100% 10-15 5-7 Recommended image quantity Small inter-frame variation and large number of frames 20%-40% 30+ 4-6 Recommended image quantity Small inter-frame variation and few frames 20%-40% 10-15 2-4 Recommend a small number of images

[0265] Referring to Table 1 above, if the inter-frame variation is large and the number of frames is high (e.g., more than 30 frames), the total variation S is large, and the range of values ​​for M is also large. For example, M=8, and more images can be recommended as alternative reference frames in the recommendation results. In this case, the recommended number of image frames M can be classified as the first tier. If the inter-frame variation is large and the number of frames is low (e.g., the total number of frames is some value between 10 and 15), the total variation S is large, and the range of values ​​for M is moderate. For example, M=5, and a suitable number of images can be recommended as alternative reference frames in the recommendation results. In this case, the recommended number of image frames M can be classified as the second tier. If the inter-frame variation is small and the number of frames is high, the total variation S is small, and the range of values ​​for M is moderate. For example, M=4, and a suitable number of images can be recommended as alternative reference frames in the recommendation results. In this case, the recommended number of image frames M can be classified as the third tier. If the inter-frame variation is small and the number of frames is small, the total variation S is small, and the value range of M is moderate. For example, if M=2, a small number of images can be recommended as alternative reference frames in the recommendation results. In this case, the number of recommended image frames M can belong to the fourth level. In other embodiments, the correspondence between the inter-frame variation, total variation, and total number of frames and the value of M, as well as the degree of influence on the recommendation results, can also be reflected in other values ​​and numerical relationships. These values ​​and numerical relationships may differ from those in Table 1 above, and will not be limited or elaborated here.

[0266] Based on the recommended number of frames M determined above, the tablet 10 can continue to execute S805 to S807 to determine at least one reference frame image selected by the user from the relevant image sequence. The following example illustrates this process, where the user selects M reference frame images from the recommended M frames.

[0267] S805: Calculate the weighted score T for each frame in a series of consecutive images or an image sequence. i .

[0268] For example, the tablet 10 can execute a preset weighted scoring algorithm to calculate a weighted score T for each frame of binary masked image in a series of consecutive frames or image sequences. i For detailed calculations, please refer to formula T. i =P i *W i Among them, W i The weight factor Wᵢ is used as a weighting factor. In the embodiments of this application, the calculation formula for the weighting factor Wᵢ can be referred to as Wᵢ=1+i / N, where N is the total number of frames, and i=1,…,N.

[0269] S806: The calculated weighted score T i Sort the scores in descending order from largest to smallest to obtain the first score sequence.

[0270] For example, the tablet 10 can execute a preset sorting algorithm to sort the weighted score T calculated in S805 above. i Arrange the scores in descending order from largest to smallest to obtain the corresponding score sequence, which can be denoted as the first score sequence.

[0271] S807: Determine the images corresponding to the first M weighted scores in the first scoring sequence as the M frames to be recommended.

[0272] For example, the tablet 10 can execute a preset scoring and filtering rule to determine the images corresponding to the top M weighted scores in the first scoring sequence, i.e., the M frames with higher weighted scores, as the M frames to be recommended. This scoring and filtering rule can be a preset sequence position or a preset scoring threshold, etc., and is not limited here.

[0273] In this embodiment, the first frame of the M-frame images to be recommended can be the first frame of a series of consecutive frames or an image sequence, or a colored image with color information among the earlier frames in the series of consecutive frames or an image sequence with smaller changes. When determining the M-frame images to be recommended, the tablet 10 can first determine the first frame or the colored image as the image to be recommended, and then determine (M-1) frames from the first scoring sequence as the images to be recommended. In other embodiments, the first frame of the M-frame images to be recommended can also be the image corresponding to the maximum weighted score in the first scoring sequence.

[0274] It is understandable that the user should select the colored image as the reference frame image. If the recommended M-frame images include an image to be colored (i.e., an uncolored image), then when the user selects the uncolored image as the reference frame image, the tablet 10 can output relevant prompts, prompting the user to first color the uncolored image selected as the reference frame image, and then select the colored image as the reference frame image.

[0275] Based on the above processing steps S801 to S807, the image processing method provided by this application can reasonably determine the recommended number of image frames and suitable reference frame images according to the image change amplitude and total number of frames in the image sequence to be processed, so that users can select a reasonable number of appropriate reference frame images. This is beneficial to improving the efficiency and accuracy of coloring the images to be colorized in the relevant image sequence.

[0276] As an example, Figure 10 An embodiment of this application illustrates a schematic diagram of N frames in a series of consecutive frames of images or an image sequence with a determined reference frame image.

[0277] Continuing with N=50 as an example, the reference frame image has been determined to consist of 50 consecutive frames or image sequences. For example... Figure 10 As shown, the first, 32nd, and 45th colored images in the consecutive multi-frame images or image sequence can be reference frame images selected by the user. It is understood that the number M of image frames recommended by the electronic device from N frames in the consecutive multi-frame images or image sequence based on the image processing method provided in this application can be 3 or more frames, and the corresponding recommended M frames can include, but are not limited to, […]. Figure 10 The three reference frames shown are: the first frame, which can be the default recommended image; the 32nd frame, which can be an image with a large inter-frame variation and a high weighted score compared to the adjacent 33rd frame; and the 45th frame, which can be an image with a large inter-frame variation and a high weighted score compared to the 46th frame.

[0278] Continue to refer to Figure 10 Based on reference frames 1, 32, and 45, the image processing method provided in this application can determine frames 1 to 31 as a first image set, frames 32 to 44 as a second image set, and frames 45 to 50 as a third image set. That is, based on the aforementioned three reference frames, the images to be colored in a series of consecutive frames or image sequences can be divided into three image sets.

[0279] In other embodiments, the user can also select more reference frame images from the recommended M frames. For example, the user can select another frame (such as frame 24) from the recommended images between frames 2 and 31 as a reference frame image. In this case, based on the 4 reference frame images selected by the user, the images to be colored in a series of consecutive frames or image sequences can be divided into 4 image sets. Similarly, based on the M reference frame images selected by the user, the images to be colored in a series of consecutive frames or image sequences can be divided into M image sets, where M is a positive integer greater than or equal to 2. No restrictions or further details are provided here.

[0280] Figure 11 A schematic diagram of a system architecture applicable to an operating system according to an embodiment of this application is shown.

[0281] It is understood that the electronic devices provided in this application embodiment (such as the tablet 10 mentioned above) can run an operating system (OS). This operating system can be various operating systems used in the industry, such as operating systems developed based on OpenHarmony, like HarmonyOS; or other operating systems such as Android™, iOS mobile operating systems; it can also be various open-source operating systems or their derivatives, such as Linux OS, and other embedded operating systems; or it can be a future new operating system, such as an AI operating system based on artificial intelligence (AI). An operating system is a set of interconnected system software programs that manage and control the operation of electronic devices, utilize and run hardware and software resources, and provide public services to organize user interaction. In electronic devices, the operating system connects downwards to the physical devices at the hardware layer and provides a runtime environment for application software upwards.

[0282] An operating system typically includes a kernel layer, a middleware layer, and an application layer. The application layer includes applications, which can include system applications and third-party applications. The middleware layer includes a suite of software providing various services to application developers, or frameworks providing services such as databases, multimedia, and graphics, or capabilities such as distributed scheduling and system scaling. For example, the middleware layer may include a framework layer and / or a system service layer. The framework layer provides application programming interfaces (APIs) and programming frameworks for applications in the application layer. The system service layer includes the system's core capabilities, providing services to applications through the framework layer. The kernel layer is the layer between hardware and software. The kernel layer may include hardware drivers and the operating system kernel. In addition to providing hardware drivers, the kernel layer also supports functions such as memory management and system process management.

[0283] The electronic devices we use in our daily lives come in various types and forms, and are applied in a wide range of scenarios. Therefore, based on the different forms and functions of electronic devices, different application scenarios, and different user needs, the operating systems used in these devices may also differ. The basic functions implemented by the electronic device provided in this application can be implemented using a general-purpose operating system or a dedicated operating system. To more clearly illustrate the implementation of the embodiments of this application under a specific operating system, the architecture of HarmonyOS is shown below. Those skilled in the art can deduce the implementation of the embodiments of this application under other specific operating systems, such as Android™.

[0284] The software architecture of electronic devices can be divided into several layers. In some embodiments, from bottom to top, these layers are: kernel layer, system service layer, framework layer, and application layer. Layers communicate with each other through software interfaces. System functions can be tailored, added, or combined at the subsystem level depending on the deployment scenario of different device forms. Each subsystem can also be tailored, added, or combined at the functional level.

[0285] The kernel layer can include the kernel abstract layer (KAL), kernel subsystems, and driver subsystems. Among them, the KAL provides basic kernel capabilities to the upper layers by shielding the differences between multiple kernels, including but not limited to process / thread management, memory management, file system, network management, and peripheral device management.

[0286] Kernel Subsystem: Supports the selection of a suitable OS kernel for different resource-constrained devices, including but not limited to Linux™ kernel, Harmony™ kernel, and Lite OS.

[0287] Driver Subsystem: The driver framework is the foundation for the open system hardware ecosystem, providing unified peripheral access capabilities and a framework for driver development and management. The driver framework includes: display drivers, camera drivers, audio drivers, Bluetooth drivers, sensor drivers, etc.

[0288] The system service layer comprises the core capabilities of the system, providing services to applications through the framework layer. This layer includes, but is not limited to, the following subsystems:

[0289] The system's basic capability subsystem set provides the foundational capabilities for the operation, scheduling, and migration of distributed applications across multiple devices. This set may include a distributed soft bus, distributed data management, distributed task scheduling, and the Ark multi-language runtime. It also includes a graphics subsystem, a multi-modal input subsystem (not shown), a security subsystem (not shown), and an AI subsystem (not shown).

[0290] Basic software service subsystem set: provides public and general software services; the basic software service subsystem set may include event notification subsystem, telephone service subsystem, multimedia subsystem, etc.

[0291] Enhanced software service subsystem suite: Provides differentiated enhanced software services for different devices; the enhanced software service subsystem suite may include smart screen proprietary business subsystem, wearable proprietary business subsystem, IoT proprietary business subsystem, etc.

[0292] Hardware service subsystem set: Provides hardware services; the hardware service subsystem set may include location service subsystem, unified identity and access management (IAM) subsystem, wearable proprietary hardware service subsystem, biometric identification, IoT proprietary hardware service, and other subsystems.

[0293] Distributed task scheduling enables distributed service management (discovery, synchronization, registration, and invocation), supporting remote startup, remote invocation, remote connection, and migration of applications across devices.

[0294] Distributed data management enables data synchronization, data storage, data sharing, and data access across all scenarios and devices.

[0295] The distributed soft bus provides communication-related capabilities for seamless interconnection between multiple devices, including: WLAN service capabilities, Bluetooth service capabilities, soft bus, remote procedure call (RPC), and StarFlash communication capabilities.

[0296] Ark Multilingual Runtime is a unified compilation runtime platform designed to support the joint compilation and execution of multiple programming languages ​​and multiple chip platforms.

[0297] The framework layer provides application programming interfaces (APIs) and programming frameworks for applications in the application layer. The framework layer includes: the Ark UI framework (which provides a complete infrastructure for UI development of system applications, including UI functions such as components, layouts, animations, and interactive events, as well as interface preview tools), the user application framework, and the Ability framework (an Ability is a lightweight application; the Ability framework schedules and manages the operation and lifecycle of Abilities). Different devices may have different operating systems, and the APIs they support may also differ.

[0298] The HarmonyOS API is a series of open capabilities provided to support HarmonyOS application development. The HarmonyOS API can be set at the framework layer or independently of the framework layer. The HarmonyOS API includes the Audio API (audio service), Push API (push service), and Account API (account service), among others.

[0299] Applications at the application layer can include system applications and extended / third-party applications. System applications can include the desktop, control bar, settings, contacts, phone, camera, etc., while extended / third-party applications can include social applications, travel applications, etc. In this embodiment, extended / third-party applications can include drawing applications, such as "Born to Draw," etc., and are not limited thereto.

[0300] Figure 12 A schematic block diagram of the structure of a painting application is shown according to an embodiment of this application.

[0301] like Figure 12 As shown, the painting application may include at least an inter-frame variation amplitude calculation module 1210, an image recommendation module 1220, and a coloring processing module 1230.

[0302] The inter-frame variation amplitude calculation module 1210 can be used to calculate the aforementioned inter-frame variation amplitude P. i It can also be used to execute a preset accumulation algorithm to calculate the total change magnitude S.

[0303] The image recommendation module 1220 can be used to execute a preset image frame recommendation algorithm, calculate the first image frame number Ks based on the total change amplitude S, calculate the second image frame number Kn based on the total number of frames N, and determine the number of image frames M to be recommended. Furthermore, the image recommendation module 1220 can also be used to execute preset weighted scoring and ranking algorithms, calculating a weighted score T for each frame of the binary masked image in a series of consecutive images or image sequences. i And the calculated weighted score T i The images are sorted in descending order from largest to smallest, and the images corresponding to the first M weighted scores in the first scoring sequence are determined as the M frames to be recommended.

[0304] The coloring processing module 1230 can be used to colorize images in multiple image sets divided based on reference frame images. The coloring processing module 1230 can use the color of the reference frame image in each image set as a reference to colorize the images in each image set.

[0305] Specifically, the painting application, based on the aforementioned inter-frame variation amplitude calculation module 1210, image recommendation module 1220, and coloring processing module 1230, can execute the image processing method provided in this application when running on an electronic device (such as a tablet 10), achieving the purpose of quickly and accurately coloring the images to be colored in the image sequence to be processed. The specific execution process can be referred to the above. Figure 6 , Figure 7 as well as Figure 8 The process and related descriptions shown are not repeated here.

[0306] It is understood that the aforementioned pre-defined accumulation algorithm, frame recommendation algorithm, weighted scoring algorithm, and sorting algorithm can be integrated, in whole or in part, into a single image processing algorithm to implement the image processing method provided in this application. The image processing functions implemented by the aforementioned image processing algorithms can also be achieved through model training, and this is not a limitation.

[0307] Figure 13 A schematic diagram of the hardware structure of an electronic device is shown according to an embodiment of this application.

[0308] In the embodiments of this application, such as Figure 13 The electronic device 100 shown can be the tablet 10 described above. In other embodiments, the electronic device 100 can also be other electronic devices, such as foldable screen phones, laptops, etc., and there is no limitation here.

[0309] Figure 13 The electronic device 100 shown may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identity module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, etc.

[0310] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0311] Processor 110 may include one or more processing units, such as application processors (APs), modem processors, graphics processing units (GPUs), image signal processors (ISPs), controllers, video codecs, digital signal processors (DSPs), baseband processors, and / or neural network processing units (NPUs). These different processing units may be independent devices or integrated into one or more processors.

[0312] The controller can generate operation control signals based on the instruction opcode and timing signals to complete the control of instruction fetching and execution.

[0313] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the aforementioned memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0314] In this embodiment, the processor 110 can use the aforementioned controller to generate operation control signals to complete the operation. Figure 6 , Figure 7 as well as Figure 8 The instructions related to each step in the process shown are used to control the fetching and execution of instructions in order to realize the image processing method provided in this application.

[0315] The electronic device 100 implements its display function through a GPU, a display screen 194, and an application processor. In this embodiment, the display function can be used to display a drawing interface, as detailed above. Figures 2a to 2f , Figures 3a to 3e , Figures 4b to 4f or Figures 5a to 5e The drawing interface shown is not limited here. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0316] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Mini-LED, a Micro-LED, a Micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, electronic device 100 may include one or N displays 194, where N is a positive integer greater than 1.

[0317] The external storage interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external storage interface 120 to perform data storage functions. For example, music, video, and other files can be saved on the external memory card.

[0318] Internal memory 121 can be used to store computer executable program code, including instructions. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback, image playback, etc.), etc. The data storage area may store data created during the use of electronic device 100 (such as audio data, phonebook, etc.). In addition, internal memory 121 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc. Processor 110 executes various functional applications and data processing of electronic device 100 by running instructions stored in internal memory 121 and / or instructions stored in memory disposed in the processor.

[0319] This application also provides an apparatus, including a module for performing the image processing method provided in the specific embodiments described above.

[0320] This application also provides a chip, including a processor coupled to a memory, for executing a computer program or instructions stored in the memory, causing the chip to perform the image processing method provided in the above specific embodiments.

[0321] This application also provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the image processing method provided in the above-described specific embodiments.

[0322] This application also provides a computer program product, including a computer program / instructions, which, when executed by a processor, are used to implement the image processing method provided in the specific embodiments described above.

[0323] Various embodiments of the mechanisms disclosed in this application can be implemented in hardware, software, firmware, or combinations of these implementation methods. Embodiments of this application can be implemented as computer program modules or module code executable on a programmable system, the programmable system including at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device.

[0324] Computer program modules or module code can be applied to input instructions to perform the functions described in this application and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, the processing system includes any system having a processor such as, for example, a digital signal processor (DSP), a microcontroller, an application-specific integrated circuit (ASIC), or a microprocessor.

[0325] Module code can be implemented using a high-level modular language or an object-oriented programming language to communicate with the processing system. Assembly language or machine language can also be used to implement module code when needed. The mechanisms described in this application are not limited to any particular programming language. In either case, the language can be a compiled language or an interpreted language.

[0326] In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments may also be implemented as instructions carried or stored thereon on one or more temporary or non-temporary machine-readable (e.g., computer-readable) storage media, which may be read and executed by one or more processors. For example, the instructions may be distributed via a network or via other computer-readable storage media. Therefore, machine-readable storage media may include any mechanism for storing or transmitting information in a machine-readable (e.g., computer-readable) form, including but not limited to floppy disks, optical disks, optical discs, magneto-optical disks, read-only memory (ROM), random access memory (RAM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic cards or optical cards, flash memory, or tangible machine-readable storage for transmitting information (e.g., carrier waves, infrared signals, digital signals, etc.) using the Internet in the form of electrical, optical, acoustic, or other forms of propagated signals. Therefore, machine-readable storage media include any type of machine-readable storage media suitable for storing or transmitting electronic instructions or information in a machine-readable (e.g., computer-readable) form.

[0327] In this specification, the reference to "an embodiment" or "an embodiment" means that a specific feature, structure, or characteristic described in connection with the embodiment is included in at least one exemplary implementation or technology disclosed according to an embodiment of this application. The appearance of the phrase "in an embodiment" in various places in the specification does not necessarily refer to the same embodiment.

[0328] The disclosure of embodiments of this application also relates to means for performing operations in text. This means may be specifically constructed for the claimed purpose or may include a general-purpose computer selectively activated or reconfigured by a computer program stored in a computer. Such a computer program may be stored in a computer-readable storage medium, such as, but not limited to, read-only memory (ROM), random access memory (RAM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), any type of disk, such as floppy disk, optical disk, CD-ROM, magneto-optical disk, magnetic or optical card, application-specific integrated circuit (ASIC), or any type of medium suitable for storing electronic instructions, and each may be coupled to a system bus. Furthermore, the computer mentioned in the specification may include a single processor or may be an architecture employing multiple processors for increased computing power.

[0329] Furthermore, the language used in this specification has been primarily chosen for readability and instructional purposes and may not have been chosen to depict or limit the disclosed subject matter. Therefore, the embodiments disclosed herein are intended to illustrate, and not limit, the scope of the concepts discussed herein.

Claims

1. An image processing method, characterized in that, Applied to electronic devices, the method includes: Run the drawing application and display the first interface of the drawing application, the first interface including at least one frame of the multi-frame images of the first animation; In response to a first user operation instructing the initiation of quick coloring, an M-frame image and a target image are displayed, wherein the M-frame image is a thumbnail of at least one frame of a multi-frame image in the first animation, and the target image is an enlarged image of the image to be colored in the multi-frame image of the first animation or any other image in the M-frame image; Obtain a first reference frame image and a second reference frame image selected by the user from the M-frame images, wherein the first reference frame image is located in the sequence position of the multi-frame images in the first animation and is located before the second reference frame image in the sequence position of the multi-frame images in the first animation; The first image set is defined as the first reference frame image and the multi-frame images in the first animation whose sequence position is before the second reference frame image. The second image set is determined by the second reference frame image and the multi-frame images in the first animation whose sequence position is after the second reference frame image. Based on the first reference frame image, a colorized image corresponding to the image to be colorized in the first image set is generated; and based on the second reference frame image, a colorized image corresponding to the image to be colorized in the second image set is generated.

2. The method according to claim 1, characterized in that, The first animation consists of multiple frames of images with continuously changing image content, and the first animation includes thumbnails of the target image.

3. The method according to claim 1 or 2, characterized in that, The M-frame image includes the target image, and all or part of the target image has color information.

4. The method according to claim 3, characterized in that, The first reference frame image includes the target image, and, The step of generating a colorized image corresponding to the image to be colorized in the first image set based on the first reference frame image includes: Based on the coloring information in the target image, batch coloring processing is performed on all or part of the images to be colored in the first image set.

5. The method according to claim 2, characterized in that, The first interface for displaying the drawing application includes: In response to a second user action to enable the animation function on the painting application, the first interface is displayed, wherein the first interface includes thumbnails of the consecutive multi-frame images and the target image, the target image being a magnified image of the first image in the consecutive multi-frame images of the first animation, and the first image being an image selected by the user from the layer where the consecutive multi-frame images are located or from the thumbnails of the consecutive multi-frame images.

6. The method according to claim 5, characterized in that, The consecutive multi-frame images include N frames, and the M frames are images to be recommended determined from the N frames, where N is a positive integer and M is a positive integer less than N; and... The M-frame images are those whose content differences compared to adjacent frames among the consecutive multi-frame images meet the recommended criteria.

7. The method according to claim 6, characterized in that, The M-frame images are the images to be recommended determined from the N-frame images, including: The M-frame images are images determined from the N-frame images based on the number M of images to be recommended and the ranking result of the weighted scores of the N-frame images. The number M of images to be recommended is related to the total change amplitude of the N-frame images and the number N of images included in the consecutive multi-frame images. The total change amplitude of the N-frame images is the sum of the first change amplitudes of (N-1) frame pairs determined based on the N-frame images. Each of the (N-1) frame pairs includes two adjacent frames in the consecutive multi-frame images. The first change amplitude is used to indicate the degree of content difference between the two adjacent frames.

8. The method according to claim 7, characterized in that, The number of image frames M to be recommended is related to the total variation of the N frames and the number of image frames N included in the consecutive multi-frame images, including: The number of image frames M to be recommended is the minimum value between the first number of image frames and the second number of image frames, where, The first number of image frames is determined based on the total change amplitude and the first reference coefficient, and the second number of image frames is determined based on the number of image frames N included in the consecutive multi-frame images and the second reference coefficient.

9. The method according to claim 7, characterized in that, The M-frame images are determined from the N-frame images based on the number M of image frames to be recommended and the ranking result of the weighted scores of the N-frame images, and include: The M-frame images are the images corresponding to the first M weighted scores in the first scoring sequence, wherein, The first scoring sequence is the weighted scores of the N frames of images arranged in descending order from high to low. The weighted scores of the N frames of images are related to the first change magnitude between each frame and the previous frame in the consecutive multi-frame images and the sequence position of each frame in the consecutive multi-frame images.

10. The method according to claim 9, characterized in that, The first reference frame image is positioned in the sequence of the consecutive multi-frame images before the second reference frame image is positioned in the sequence of the consecutive multi-frame images.

11. The method according to claim 10, characterized in that, The method further includes: Obtain the third reference frame image selected by the user from the M-frame images; The second image set is defined as the second reference frame image and the multi-frame images whose sequence position is located between the second reference frame image and the third reference image in the consecutive multi-frame images; The third reference frame image and the multi-frame images whose sequence position is located after the third reference frame image in the consecutive multi-frame images are determined as the third image set; Based on the first reference frame image, a colorized image corresponding to the image to be colorized in the first image set is generated; based on the second reference frame image, a colorized image corresponding to the image to be colorized in the second image set is generated; and based on the third reference frame, a colorized image corresponding to the image to be colorized in the third image set is generated.

12. The method according to any one of claims 6 to 11, characterized in that, N≥3 and M≤N / 3.

13. The method according to claim 12, characterized in that, M≤10。 14. The method according to any one of claims 1, 2, 5 to 11, characterized in that, The step of obtaining the first reference frame image and the second reference frame image selected by the user from the M-frame images includes: The system detects a user's selection operation on the first reference frame image and the second reference frame image in the M-frame images, and detects a user's confirmation operation on the selection result. Obtain the first reference frame image and the second reference frame image.

15. The method according to claim 14, characterized in that, Before acquiring the first reference frame image and the second reference frame image selected by the user from the M-frame images, the method further includes: It was detected that the first reference frame image and the second reference image selected by the user in the M-frame image were images to be colored; Output a first prompt message, wherein the first prompt message is used to prompt the user to colorize the first reference frame image and the second reference frame image.

16. An electronic device, characterized in that, include: One or more processors; One or more memories; the one or more memories storing one or more programs, which, when executed by the one or more processors, cause the electronic device to perform the image processing method of any one of claims 1 to 15.

17. A computer-readable storage medium, characterized in that, The readable storage medium stores instructions that, when executed on a computer, cause the computer to perform the image processing method according to any one of claims 1 to 15.

18. A computer program product, characterized in that, It includes a computer program / instructions that, when executed by a processor, implement the image processing method according to any one of claims 1 to 15.

Images