A multi-role recognition and image rendering method based on color label mapping

Abstract

This invention belongs to the field of artificial intelligence image generation technology, specifically a multi-role recognition and image rendering method based on color mark mapping. The method includes: assigning distinct non-natural marker colors to multiple roles in the content to be processed, using these non-natural marker colors as intermediate codes for role identities; generating a sketch image with marker colors based on the input content; performing color detection on the sketch image to identify the actual roles appearing in the current scene; constructing a mapping relationship between marker colors, role identities, and appearance reference information in the current scene based on the recognition results; color rendering the sketch image while keeping it essentially unchanged, replacing the marker color areas with the corresponding roles' actual appearances; and performing structure maintenance verification and cross-scene consistency verification on the rendering results. This invention decouples role identity identification from appearance rendering by introducing non-natural marker colors as an intermediate coding layer for role identities.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence image generation technology, specifically a multi-role recognition and image rendering method based on color mark mapping. Background Technology

[0002] With the continuous development of image generation technologies based on diffusion models and multimodal large models, text-driven generation of multi-character scene images has become an important application direction, especially in film and television storyboard generation, animation production, and digital content production. However, in the process of achieving stable generation of multi-character scenes in the same frame, existing technologies usually rely on text prompts or reference images to constrain the characters. This approach lacks clear and structured means of identity expression, which leads to problems such as character feature confusion, appearance drift, and identity misalignment during the generation process.

[0003] However, existing technologies are mainly limited by the lack of a unified and computable intermediate encoding mechanism for character identities when achieving accurate identification and consistent rendering in multi-role scenarios. This makes it difficult to balance the accuracy of multi-role identity differentiation with cross-screen appearance consistency in practical applications, thus affecting the overall generation quality and result stability. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides a multi-role recognition and image rendering method based on color mark mapping, thereby solving the problem of the lack of a unified and computable intermediate encoding mechanism for role identity in the prior art.

[0005] A multi-role recognition and image rendering method based on color mark mapping includes the following steps: construction and tolerance verification of a marker color sketch based on color partition constraints, specifically:

[0006] A set of unnatural marker colors is constructed, and color partitioning constraints are applied to the constructed set of unnatural marker colors. At the same time, a set of storyboards is constructed, and a corresponding sketch image is created for each storyboard. A color detection tolerance is introduced to verify the pixels in the sketch image. The sketch images that pass the verification are filled with marker colors. The unnatural marker colors are used for intermediate encoding of character identity.

[0007] The scene character recognition and dynamic mapping construction based on marker color detection is as follows:

[0008] Marker color detection is performed on the sketch image after the marker color is filled. Based on the detection results, the set of actual characters appearing in the storyboard is determined. At the same time, the pixel area ratio corresponding to the marker color is calculated. Based on the calculation results, it is determined whether the current marker color appears in the current storyboard image. A mapping subset between the marker color and the characters appearing in the storyboard is constructed. Only the characters that are detected as actually appearing in the current storyboard image are included in the mapping subset and their appearance is bound. Characters that are not detected as actually appearing do not participate in the rendering of the current storyboard image.

[0009] Rendering and dynamic correction based on dual constraints of structure and consistency are specifically as follows:

[0010] Color rendering is performed based on a subset of the mapping between the marked colors and the characters. While keeping the basic composition of the sketch unchanged, the structural constraints are used to verify the structural integrity of the rendering results. For images that pass the structural integrity verification, the storyboard consistency verification is performed, and re-rendering is dynamically executed based on the consistency verification results.

[0011] Preferably, the color partitioning constraint applied to the constructed non-natural marker color set is as follows:

[0012] Set the set of marker colors , Indicates the first A marker color, Indicates the total number of characters;

[0013] By applying color partitioning constraints using a color discrimination threshold, we have:

[0014] ;

[0015] in, This indicates that a color differentiation threshold is set to ensure that the marker colors of different roles are significantly distinguishable.

[0016] Preferably, the construction of the storyboard set and the creation of a corresponding sketch image for each storyboard are as follows:

[0017] Set up storyboard ,in, Indicates the first Each storyboard, This indicates the total number of storyboards.

[0018] For each storyboard, a corresponding sketch image is constructed, then:

[0019] ;

[0020] in, Indicates the first A collection of character positions and layouts in each storyboard. Represents a set of character identities. This represents the sketch generation function.

[0021] Preferably, the introduction of color detection tolerance to verify pixels in the sketch image is as follows:

[0022] Setting color detection tolerance And calculate the Euclidean distance between the marked color and the pixel color values ​​in the sketch image. Based on the calculation results, verify the pixels in the sketch image. Then:

[0023] If the calculation result satisfies the formula This indicates that in the sketch image Pixel belongs to A colored marker indicates that a pixel in the sketch image has passed verification, while a darker color indicates that a pixel in the sketch image has failed verification.

[0024] Preferably, the calculation of the pixel area ratio corresponding to the marker color is as follows:

[0025] ;

[0026] in, Indicates the marker color The number of pixels, Indicates the first The total number of pixels in each storyboard image. This indicates the percentage of pixel area corresponding to the calculated marker color.

[0027] Preferably, the step of determining whether the current marker color appears in the current storyboard image based on the calculation result is as follows:

[0028] The pixel area ratio corresponding to the marker color is compared with a set pixel ratio threshold. Based on the comparison result, it is determined whether the current marker color appears in the current storyboard image. Then:

[0029] If the comparison results satisfy the formula This indicates that the marker color appears in the current storyboard image;

[0030] If the comparison results satisfy the formula If , it means that the marker color will not appear in the current storyboard image.

[0031] Preferably, the structural preservation verification of the rendering result through structural constraints is performed as follows:

[0032] Extract the pre-render sketch Each character is marked with a color area. The geometric features are used to construct the corresponding set of character positions. ;

[0033] Extract the rendered image The geometric features of the corresponding character regions are used to construct a set of corresponding character positions. ;

[0034] By introducing structural constraints to validate the rendering results, we have:

[0035] ;

[0036] in, This represents the set of character positions before rendering. This represents the set of rendered character positions. This indicates the set structural deviation threshold.

[0037] Preferably, the step of performing the storyboard consistency verification is as follows:

[0038] ;

[0039] in, This represents the function for extracting character features. Indicates role The global average characteristics, This represents the output of the consistency evaluation function, quantifying the first... The degree of variation in the appearance of each character in different scenes.

[0040] Preferably, the step of dynamically performing re-rendering based on the consistency verification result is as follows:

[0041] The output of the consistency evaluation function With consistency threshold By comparing the results and dynamically re-rendering based on them, we have:

[0042] If the comparison results satisfy the formula This indicates that the cross-scene appearance consistency of character i meets the requirements and no additional processing is needed;

[0043] If the comparison results satisfy the formula , indicating the first The appearance differences of a character in different storyboards are beyond acceptable limits, and the relevant storyboards need to be corrected.

[0044] Preferably, character identification is based on the color information of the marker in the sketch image, rather than on the character's spatial position or body movements in the picture.

[0045] Compared with the prior art, the present invention has the following beneficial effects:

[0046] 1. This invention achieves the technical effect of assigning a unique and computable intermediate identity code to each role by constructing a set of non-natural marker colors and constraining the marker colors by color partitioning. This fundamentally avoids the confusion and false detection of role identities caused by overlapping color spaces or similar colors.

[0047] 2. This invention achieves robust recognition of marker colors even under engineering distortion conditions such as image compression, scaling, and anti-aliasing rendering by adopting a technical means of verifying the pixels of the sketch image by introducing color detection tolerance. This solves the technical problem of existing technologies that rely too much on ideal input conditions and have poor recognition stability in practical applications.

[0048] 3. This invention achieves the technical effect of effectively filtering out color detection noise, jagged edges, and false detections in very small areas by adopting a technique based on a pixel area ratio threshold to determine whether the marked color actually appears in the scene, thus accurately identifying the set of real characters appearing in each scene.

[0049] 4. This invention achieves independent and dynamic adjustment of the mapping relationship between different scenes by adopting the technical means of independently constructing a dynamic mapping subset between the marker color and the character appearing in each scene. It does not require maintaining a global fixed mapping table and avoids the technical effect of identity misalignment and mapping conflict caused by changes in the appearance of characters.

[0050] 5. This invention achieves the technical effect of ensuring that the rendering process does not destroy the character spatial layout set in the original sketch by adopting the technical means of introducing structural constraints to verify the deviation of the character position set before and after rendering, thus solving the technical problem of the narrative relationship disorder between the storyboard caused by the displacement of character positions due to rendering.

[0051] 6. This invention achieves the technical effect of automatically detecting and correcting the appearance drift of the same character in different scenes by constructing a cross-scene consistency evaluation function and dynamically performing local re-rendering based on the comparison between the evaluation result and the consistency threshold, thus ensuring the stable and consistent appearance features of the character across scenes. Attached Figure Description

[0052] Figure 1 This is a schematic diagram of the overall method steps of a multi-role recognition and image rendering method based on color mark mapping according to the present invention. Detailed Implementation

[0053] The embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and should not be construed as limiting the scope of the invention.

[0054] Example 1

[0055] Reference Figure 1As an embodiment of the present invention, a multi-role recognition and image rendering method based on color mark mapping is provided, comprising the following steps:

[0056] S1: Construction and tolerance verification of marker color sketches based on color partitioning constraints.

[0057] Specifically, the construction and tolerance verification of the color-coded sketch based on color partitioning constraints involves establishing a set of color-coded markers that do not involve semantic interference, encoding the character's identity, and generating a storyboard sketch based on the encoding results. The specific implementation is as follows:

[0058] First, we construct a set of non-natural marker colors to avoid overlap with the color space of real images, then we have:

[0059] Set the set of marker colors as follows:

[0060] ;

[0061] in, Indicates the first A marker color, Indicates the total number of characters. Represents a set of marker colors;

[0062] By constraining the set of marked colors by minimum distance, we have:

[0063] ;

[0064] in, This represents the color differentiation threshold, used to ensure that the marker colors of different roles are significantly distinguishable, so as to avoid subsequent detection and recognition errors due to colors that are too similar.

[0065] In addition, non-natural marker colors are colors that appear very infrequently in natural images (portraits, landscapes, indoor scenes, etc.), specifically:

[0066] Select colors from the RGB color space that are highly saturated, highly bright, and whose Euclidean distance from common natural colors (skin tone, sky blue, grass green, earth brown, etc.) is greater than [value missing]. Color values, such as pure magenta RGB(255,0,255), pure cyan RGB(0,255,255), pure yellow RGB(255,255,0), etc.

[0067] It should be noted that in this embodiment, the marker color serves as a unique identity encoding carrier across storyboards and is used throughout the entire process of role detection, mapping construction, and consistency evaluation. It is represented using the RGB color space, with each color channel ranging from 0 to 255, and a color differentiation threshold. Set to 50, which is the marker color for any two different characters. and satisfy:

[0068] ;

[0069] Furthermore, the color differentiation threshold was verified experimentally, specifically as follows:

[0070] When the threshold is below 30, color misjudgment is likely to occur in scenarios with changes in lighting or image compression; when the threshold is above 70, the number of available colors is significantly reduced, limiting the maximum number of roles supported by the system; 50 is the preferred value that balances robustness of differentiation and coding capacity.

[0071] As an alternative embodiment, the marker color can also be defined in the CIELAB color space. The CIELAB space possesses perceptual uniformity, meaning that the Euclidean distance between two points in the color space is linearly related to the color difference perceived by human vision. In the CIELAB space, the color discrimination threshold... It can be set to 20; the conversion between RGB space and CIELAB space adopts the standard conversion formula formulated by the International Commission on Illumination (CIE).

[0072] Secondly, based on the script information, a storyboard mesh structure is constructed, and sketch images are generated, then:

[0073] The storyboard set is set as follows:

[0074] ;

[0075] in, Indicates the first Each storyboard, This indicates that the total number of storyboards is set by the implementers based on the actual application scenario. This represents the set of storyboards constructed.

[0076] Furthermore, for each storyboard, a corresponding sketch image is constructed, resulting in:

[0077] ;

[0078] in, Indicates the first A collection of character positions and layouts in each storyboard. Represents a set of character identities. This represents the sketch generation function, and its implementation is as follows:

[0079] Based on the script description, determine the set of characters appearing in each storyboard and their expected positions on the screen. Generate a blank canvas. According to the position information defined in the character position layout set, fill the area corresponding to each character with the corresponding marker color. For the background area, fill with neutral gray (such as RGB(128,128,128)) to distinguish it from the marker color area. If there are multiple characters in the same storyboard, determine the filling order according to the occlusion relationship defined in the character position layout set. The overlapping parts of the occluded character's area are not filled.

[0080] It should be noted that when performing color detection on sketch images, due to factors such as image compression, scaling, anti-aliasing rendering, or storage format conversion, the pixel values ​​of the original marker colors may shift slightly. If a strict equality judgment is adopted, a large number of pixels that should belong to the same marker color will not be correctly identified.

[0081] Therefore, a color detection tolerance is introduced to verify the pixels in the sketch image, specifically:

[0082] Setting color detection tolerance And calculate the Euclidean distance between the marked color and the pixel color values ​​in the sketch image. Based on the calculation results, verify the pixels in the sketch image. Then:

[0083] If the calculation result satisfies the formula This indicates that in the sketch image Pixel belongs to A colored marker indicates that a pixel in the sketch image has passed verification, while a darker color indicates that a pixel in the sketch image has failed verification.

[0084] It should be noted that the color detection tolerance setting is based on the RGB color space, and the specific settings are as follows:

[0085] After performing JPEG compression (75% quality factor), bilinear scaling (0.9 scaling factor), and anti-aliasing rendering on a set of standard marker color images, the Euclidean distance distribution between the pixel values ​​and the original marker colors was statistically analyzed, and the distance value corresponding to the 95th quantile was taken as... The measured value is Between these values, 10 is taken as the preferred value;

[0086] For applications requiring higher precision, Set to 5;

[0087] For scenarios requiring higher robustness but with a smaller number of characters, it is advisable to... Set to 15.

[0088] Furthermore, by filling the character areas in the sketch with a marker color, we have:

[0089] ;

[0090] in, Indicates the first In the first scene The pixel area corresponding to each character This represents the sketch before rendering.

[0091] S2: Storyboard character recognition and dynamic mapping construction based on marker color detection.

[0092] Specifically, the storyboard character recognition and dynamic mapping construction based on marker color detection identifies the actual characters appearing in the scene through color analysis and region clustering, and constructs a dynamic mapping relationship between characters and appearance information. The specific implementation is as follows:

[0093] First, perform color-coded detection and region extraction on each storyboard image, then:

[0094] ;

[0095] in, Indicates the first The color marked in each scene The number of pixels, This indicates the set color detection tolerance;

[0096] Based on the detection results, the actual set of characters appearing in the storyboard is determined, then:

[0097] ;

[0098] in, This represents the pixel percentage threshold, used to determine whether a certain marker color truly represents an "actually present" role, rather than a false recognition result caused by color detection noise, jagged edges, or false detections in extremely small areas, as detailed below:

[0099] For the A storyboard image Calculate the marker color Number of pixels After that, set the first The total number of pixels in each storyboard image is ;

[0100] Calculating the pixel area ratio corresponding to the marked color, we have:

[0101] ;

[0102] in, Indicates the marker color The number of pixels, Indicates the first The total number of pixels in each storyboard image. This indicates the percentage of pixel area corresponding to the calculated marker color;

[0103] The determination of whether the current marker color appears in the current storyboard image is based on a pixel proportion threshold.

[0104] The pixel area ratio corresponding to the marker color is compared with a set pixel ratio threshold. Based on the comparison result, it is determined whether the current marker color appears in the current storyboard image. Then:

[0105] If the comparison results satisfy the formula This indicates that the marker color appears in the current storyboard image;

[0106] If the comparison results satisfy the formula If , it means that the marker color will not appear in the current storyboard image.

[0107] It should be noted that, regarding the setting of the pixel ratio threshold, this embodiment uses... To clarify, in practical applications, the settings are adaptively adjusted based on the image resolution, specifically as follows:

[0108] For an image with a resolution of 1920×1080, the total number of pixels is about 2 million. One-thousandth of that corresponds to about 2,000 pixels, which is enough to form the smallest character area that can be recognized by the naked eye.

[0109] For high-resolution images (such as 4K), the resolution can be appropriately reduced. Up to 0.0005;

[0110] For low-resolution images (such as 512×512), it can be improved. Up to 0.005.

[0111] Secondly, by establishing the mapping relationship between the marker color and the character's appearance reference information, we have:

[0112] ;

[0113] in, Indicates the first The appearance feature vectors of each character, including clothing, hairstyle, and facial features, are extracted using the following method:

[0114] From one or more character reference images, use a pre-trained identity feature extraction network (such as ArcFace, CLIP image encoder) to extract embedding vectors;

[0115] Character appearance attributes (such as clothing color, hairstyle, and facial key features) are manually labeled and encoded into structured feature vectors;

[0116] For simple scenarios, the character reference image itself is used directly as the appearance feature vector, and texture transfer is performed through image similarity matching during rendering.

[0117] Furthermore, based on the segment recognition results, the mapping relationships are filtered, resulting in:

[0118] ;

[0119] in, Indicates the first A subset of the mapping of each storyboard.

[0120] It should be noted that the constructed mapping subset only includes the marker color-appearance mapping pairs corresponding to the characters actually appearing in the current storyboard; mapping relationships for characters not appearing are explicitly excluded. Furthermore, the mapping subsets for different storyboards are independent of each other; the mapping relationship of one storyboard will not interfere with the rendering decisions of other storyboards. Simultaneously, as storyboards switch... Content based on It dynamically adjusts to changes, without the need to maintain a globally fixed mapping table.

[0121] S3: Rendering and dynamic correction based on dual constraints of structure and consistency.

[0122] Specifically, the dual-constraint rendering and dynamic correction based on structure and consistency replaces the marker colors with the actual appearance while keeping the sketch structure unchanged, and optimizes and controls this through a consistency function. The specific implementation is as follows:

[0123] First, if we perform color replacement while maintaining spatial structure constraints, then:

[0124] ;

[0125] in, This indicates the rendering function that maps the marker color to the corresponding appearance texture.

[0126] It should be noted that the rendering function is used to replace the marked color areas with the corresponding character's real appearance texture and color. This embodiment provides two optional implementation methods, including: local redrawing based on the diffusion model, and texture mapping and color transfer based on the reference image. Implementers can choose one or a combination of them according to the specific application scenario, as follows:

[0127] Local redrawing based on the diffusion model treats the marked color regions as masks to be generated, using the character's appearance feature vectors. As a condition, a pre-trained diffusion model (such as the Stable Diffusion Inpainting model) is invoked to perform local generation, as follows:

[0128] sketch image Marked areas Character appearance feature vector As input data;

[0129] Pixels within the masked area are set to random noise or the original marker color is preserved as spatial guides for the CLIP image encoder. The appearance embeddings extracted from them are processed as cross-attention conditions;

[0130] The processed result is a filled, realistic texture image patch.

[0131] Texture mapping and color transfer based on reference images are applications for scenarios with a simple appearance, as detailed below:

[0132] From the character appearance feature vector Extract the dominant color tone, texture statistical features (such as histogram of oriented gradients), or directly extract the character region under the standard pose as a texture template;

[0133] Mark the area Within the pixels, the texture template is deformed and adapted to the geometry of the region through image affine transformation or thin plate spline interpolation;

[0134] Adjust the lighting consistency of the adapted texture (e.g., histogram matching) to make it consistent with the background lighting in the sketch.

[0135] Furthermore, after the color replacement is completed, structural constraints are set to verify the structural preservation of the rendering result, ensuring that the rendering process does not destroy the character space layout set in the original sketch. The specific implementation is as follows:

[0136] Extract the pre-render sketch Each character is marked with a color area. The geometric features (such as region centroid coordinates and bounding box vertex positions) are used to construct the corresponding character position set. ;

[0137] Extract the rendered image The geometric features of the corresponding character regions are used to construct a set of corresponding character positions. ;

[0138] If the character positions shift significantly before and after rendering, it will disrupt the narrative relationships between scenes. Therefore, structural constraints are introduced to validate the rendering results, specifically:

[0139] ;

[0140] in, This represents the set of character positions before rendering. This represents the set of rendered character positions. This represents the set structural deviation threshold. The specific value is set by the implementer based on the actual application scenario.

[0141] In this embodiment, the Euclidean distance (normalized by image size) of the centroid coordinates of the character region is used as the measure of positional deviation, then:

[0142] Structural Deviation Threshold Set as That is, the normalized displacement of the character's centroid before and after rendering does not exceed 5% of the image width or height;

[0143] The structural deviation threshold is determined experimentally, and thus:

[0144] when At that time, users can clearly perceive the shift in the character's position;

[0145] when At times, overly strict constraints can easily lead to rendering failures;

[0146] The optimal value is chosen to balance structural preservation and generation success rate.

[0147] In addition, if the positional deviation of the characters before and after rendering meets the structural constraints, the structure is deemed to be maintained; otherwise, the current scene rendering is deemed to have failed and the rendering needs to be re-executed until the structural constraints are met.

[0148] Constructing a cross-scene consistency evaluation function, we have:

[0149] ;

[0150] in, This represents the function for extracting character features. Indicates role The global average characteristics, This represents the output of the consistency evaluation function, quantifying the first... The degree of variation in the appearance of each character across different storyboards is as follows:

[0151] The smaller the value, the more stable and consistent the character's appearance is across different scenes;

[0152] The larger the value, the more significant the appearance drift.

[0153] To automate decision-making based on this quantification result, a consistency threshold is introduced. The specific criteria for judgment are as follows:

[0154] The output of the consistency evaluation function With consistency threshold By making a comparison and executing automated decisions based on the comparison results, we have:

[0155] If the comparison results satisfy the formula This indicates that the cross-scene appearance consistency of character i meets the requirements and no additional processing is needed;

[0156] If the comparison results satisfy the formula , indicating the first The appearance differences of a character in different storyboards exceed the acceptable range, and the relevant storyboards need to be corrected, specifically:

[0157] When corrections are needed for relevant storyboards, the location of one or more storyboards that contribute the most to the error (i.e., The largest value ), only for the first scene in the storyboard Perform local re-rendering on the marked color areas corresponding to each character; after re-rendering, recalculate. And again with Compare until the formula is satisfied. until.

[0158] Local re-rendering is applied only to the first scene in the storyboard. The marked area for each character Perform a partial re-render, specifically:

[0159] Keeping the rendering results of other characters in the storyboard unchanged, clear the rendering results in the area, restore the original marker color, use the area as a mask, and use the character appearance feature vector as a condition to call the rendering function again to generate. After the re-rendering is completed, re-verify the structural constraints and consistency constraints.

[0160] Furthermore, the introduced consistency threshold is determined based on the accuracy requirements of the application scenario, specifically as follows:

[0161] In a typical embodiment, a pre-trained ArcFace network is used to extract 512-dimensional feature vectors. The cosine distance between two images with the same identity in the feature space is usually between 0.2 and 0.4, while the distance between images with different identities is usually above 0.6.

[0162] This embodiment is set , where d is the feature vector dimension and N is the number of scenes in which the character appears, with the specific values ​​determined by experiments.

[0163] It should be noted that there are multiple ways to extract character features using the function, and the implementer can choose the appropriate method based on the actual application scenario. Specifically:

[0164] Use pre-trained face recognition networks (such as ArcFace, FaceNet) to extract feature vectors of the character's facial regions;

[0165] After locating the region where the character is located using a semantic segmentation model, the CLIP visual features of that region are extracted.

[0166] For distant or side-view scenes where facial features cannot be extracted, handcrafted features such as color histograms and directional gradient histograms of the character region can be used as alternatives.

[0167] Furthermore, if the aforementioned function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0168] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-including system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0169] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0170] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of protection claimed by the present invention.

Claims

1. A multi-role recognition and image rendering method based on color mark mapping, characterized in that: Includes the following steps: The construction and tolerance verification of the marker color sketch based on color partitioning constraints are as follows: A set of unnatural marker colors is constructed, and color partitioning constraints are applied to the constructed set of unnatural marker colors. At the same time, a set of storyboards is constructed, and a corresponding sketch image is created for each storyboard. A color detection tolerance is introduced to verify the pixels in the sketch image. The sketch images that pass the verification are filled with marker colors. The unnatural marker colors are used for intermediate encoding of character identity. The scene character recognition and dynamic mapping construction based on marker color detection is as follows: Marker color detection is performed on the sketch image after the marker color is filled. Based on the detection results, the set of actual characters appearing in the storyboard is determined. At the same time, the pixel area ratio corresponding to the marker color is calculated. Based on the calculation results, it is determined whether the current marker color appears in the current storyboard image. A mapping subset between the marker color and the characters appearing in the storyboard is constructed. Only the characters that are detected as actually appearing in the current storyboard image are included in the mapping subset and their appearance is bound. Characters that are not detected as actually appearing do not participate in the rendering of the current storyboard image. Rendering and dynamic correction based on dual constraints of structure and consistency are specifically as follows: Color rendering is performed based on a subset of the mapping between the marked colors and the characters. While keeping the basic composition of the sketch unchanged, the structural constraints are used to verify the structural integrity of the rendering results. For images that pass the structural integrity verification, the storyboard consistency verification is performed, and re-rendering is dynamically executed based on the consistency verification results.

2. The multi-role recognition and image rendering method based on color mark mapping as described in claim 1, characterized in that: The color partitioning constraint applied to the constructed set of non-natural marker colors is as follows: Set the set of marker colors , Indicates the first A marker color, Indicates the total number of characters; By applying color partitioning constraints using a color discrimination threshold, we have: ； in, This indicates that a color differentiation threshold is set to ensure that the marker colors of different roles are significantly distinguishable.

3. The multi-role recognition and image rendering method based on color mark mapping as described in claim 2, characterized in that: The process of constructing a set of storyboards and creating a corresponding sketch image for each storyboard is as follows: Set up storyboard ,in, Indicates the first Each storyboard, This indicates the total number of storyboards. For each storyboard, a corresponding sketch image is constructed, then: ； in, Indicates the first A collection of character positions and layouts in each storyboard. Represents a set of character identities. This represents the sketch generation function.

4. The multi-role recognition and image rendering method based on color mark mapping as described in claim 3, characterized in that: The introduction of color detection tolerance to verify pixels in the sketch image is as follows: Setting color detection tolerance And calculate the Euclidean distance between the marked color and the pixel color values ​​in the sketch image. Based on the calculation results, verify the pixels in the sketch image. Then: If the calculation result satisfies the formula This indicates that in the sketch image Pixel belongs to A colored marker indicates that a pixel in the sketch image has passed verification, while a darker color indicates that a pixel in the sketch image has failed verification.

5. The multi-role recognition and image rendering method based on color mark mapping as described in claim 4, characterized in that: The calculation of the pixel area ratio corresponding to the marker color is as follows: ； in, Indicates the marker color The number of pixels, Indicates the first The total number of pixels in each storyboard image. This indicates the percentage of pixel area corresponding to the calculated marker color.

6. The multi-role recognition and image rendering method based on color mark mapping as described in claim 5, characterized in that: The process of determining whether the current marker color appears in the current storyboard image based on the calculation results is as follows: The pixel area ratio corresponding to the marker color is compared with a set pixel ratio threshold. Based on the comparison result, it is determined whether the current marker color appears in the current storyboard image. Then: If the comparison results satisfy the formula This indicates that the marker color appears in the current storyboard image; If the comparison results satisfy the formula If , it means that the marker color will not appear in the current storyboard image.

7. The multi-role recognition and image rendering method based on color mark mapping as described in claim 6, characterized in that: The structural preservation verification of the rendering results through structural constraints is as follows: Extract the pre-render sketch Each character is marked with a color area. The geometric features are used to construct the corresponding set of character positions. ; Extract the rendered image The geometric features of the corresponding character regions are used to construct a set of corresponding character positions. ; By introducing structural constraints to validate the rendering results, we have: ； in, This represents the set of character positions before rendering. This represents the set of rendered character positions. This indicates the set structural deviation threshold.

8. The multi-role recognition and image rendering method based on color mark mapping as described in claim 7, characterized in that: The process of verifying the consistency of the storyboard is as follows: ； in, This represents the function for extracting character features. Indicates role The global average characteristics, This represents the output of the consistency evaluation function, quantifying the first... The degree of variation in the appearance of each character in different scenes.

9. The multi-role recognition and image rendering method based on color mark mapping as described in claim 8, characterized in that: The re-rendering is dynamically performed based on the consistency verification result, as detailed below: The output of the consistency evaluation function With consistency threshold By comparing the results and dynamically re-rendering based on them, we have: If the comparison results satisfy the formula This indicates that the cross-scene appearance consistency of character i meets the requirements and no additional processing is needed; If the comparison results satisfy the formula , indicating the first The appearance differences of a character in different storyboards are beyond acceptable limits, and the relevant storyboards need to be corrected.

10. The multi-role recognition and image rendering method based on color mark mapping as described in claim 1, characterized in that: Character identification is based on the color information marked in the sketch image, rather than on the character's spatial position or body movements in the picture.

Images