A controllable scene sketch generation method and system

Abstract

This invention belongs to the field of computer graphics technology and provides a controllable scene sketch generation method and system. The method includes: receiving input information of a sketch to be drawn, including text segments, hand-drawn sketches, and reference images; performing scene analysis and content analysis on the input information; based on the scene analysis results and content analysis results, decomposing each target object in the sketch to be drawn into multiple independent brush information and determining the weight of each brush; drawing the sketch to be drawn using piecewise Bézier curves, and outputting the brush path of each brush layer by layer, specifically including: dynamically adjusting the parameters of the Bézier curves according to the shape and curvature of each brush stroke; generating an initial drawing according to user instructions and based on the brush path of each brush stroke, or recalculating each brush path to optimize the initial drawing. This invention can efficiently meet user customization needs and improve the usability and operability of drawing.

Description

Technical Field

[0001] This invention relates to the field of computer graphics technology, and more specifically to a method and system for generating controllable scene sketches. Background Technology

[0002] Computer-aided drawing (CAD) technology has made significant strides over the past few decades, becoming an important tool for artistic creation, industrial design, and visual expression. Traditional drawing software, such as Adobe Photoshop and Illustrator, offers powerful drawing and image processing capabilities, enabling users to create detailed hand-drawn images. However, these software programs rely on manual operation and lack intelligent automated drawing capabilities. Users must manually adjust parameters such as brushes, colors, and layers, and create based on their own drawing experience. While this method is flexible, it demands a high level of drawing skill from the user, and often consumes a significant amount of time and effort in drawing and editing complex patterns.

[0003] In recent years, with the development of deep learning, AI-based drawing tools have gradually emerged. For example, Generative Adversarial Networks (GANs) and Diffusion Models, such as Stable Diffusion, have demonstrated remarkable image generation capabilities. These models can learn artistic styles and generate highly realistic stylized images, widely used in illustration, concept art, and virtual character design. However, a major problem with these models is the lack of controllability. While GANs and Diffusion Models can generate high-quality artistic images, users find it difficult to directly specify specific details of the drawing, such as the shape and color of objects, and the order of strokes. Furthermore, these models are typically based on pixel-level image synthesis rather than step-by-step stroke drawing, making precise control over the drawing process difficult. Although the conditional generation capabilities of Diffusion Models have improved, for example, by allowing text prompts to guide the generation, they still cannot flexibly adjust individual strokes, thus limiting their applicability in sketch generation tasks.

[0004] Existing sketch generation technologies suffer from several key problems, one of the most significant being the inability to precisely control stroke order. Current AI generation methods typically optimize the drawing order based on pixel-by-pixel or overall style rather than stroke-by-stroke reasoning. This results in poor logical consistency in the sketch, especially when drawing complex objects, leading to issues such as illogical line overlaps and disordered stroke layers. For example, when generating a sketch of an animal, the AI ​​might draw internal details before the external outline, or ignore necessary stroke order, resulting in an image that structurally contradicts natural drawing logic. Furthermore, existing technologies are deficient in scene analysis. Most sketch generation models focus only on local images, lacking intelligent understanding of the global scene, which easily leads to illogical object placement. For instance, if AI generates a sketch containing multiple objects, it might incorrectly arrange their positions, causing disproportionate proportions, or even resulting in overlapping or improper occlusion of elements. This lack of intelligent analysis severely impacts the overall composition of the sketch, limiting its practical applications. Furthermore, existing methods use random color filling or sample colors from existing artworks through style transfer. This approach lacks control over the overall color scheme, leading to inconsistencies in color style and potentially resulting in overly chaotic and inconsistent colors in the generated sketches. Additionally, many sketching styles involve gradations, shading, and special brushstrokes, but current AI generation methods struggle to maintain the fluidity of the drawing while capturing these details. Moreover, there is a limitation on freedom of expression, primarily manifested in the fact that the generated content is limited to the outline of the input image, failing to provide more flexible creative space.

[0005] Although existing systems can automatically extract edge lines or path features, the generated results are highly dependent on the original input image. Users cannot personalize the details, style, or structure of the lines, resulting in a lack of autonomy in the creative process.

[0006] Finally, existing sketch generation methods remain relatively limited in terms of output format. Most AI-generated sketches are output only as static images, such as PNG or JPG. While this format is suitable for direct display or printing, it lacks a hierarchical structure, preventing users from deeply editing the generated content. For example, if a user wants to modify the shape of a single stroke, adjust a color area, or move the position of an element, they usually have to rely on manual redrawing, rather than simple parameter adjustments. This lack of editability significantly limits the application value of sketches in fields such as design, animation, and interactive applications.

[0007] Therefore, it is necessary to provide a new method and system for generating controllable scene sketches to solve the above problems. Summary of the Invention

[0008] This invention aims to provide a controllable scene sketch generation method and system to solve the problems of existing technologies, such as the inability to accurately control the stroke order and the lack of intelligent analysis capabilities, which seriously affect the overall composition of the sketch. The use of random color filling or sampling colors from existing artworks through style transfer leads to a lack of control over the overall color matching, resulting in a lack of harmony in color style. This can cause the generated sketch to have problems such as overly messy colors and inconsistent styles. It is also impossible to maintain the smoothness of the drawing while taking into account the technical issues of sketch style, such as gradients, shadows, and special brushstrokes. The technical problems to be solved by this invention are achieved through the following technical solutions.

[0009] The first aspect of this invention provides a controllable scene sketch generation method, comprising: receiving input information of a sketch to be drawn, the input information including at least one of the following information data: text segment, hand-drawn sketch, and reference image; performing scene analysis and content analysis on the input information; when the input information contains a hand-drawn sketch, identifying connected regions in the hand-drawn sketch and performing curvature analysis and intersection detection to obtain specific contours and key features for dynamic path tracking; when the input information contains a reference image, extracting object contours, position, color information, and style information from the reference image, and then... When extracting color information, the K-Means clustering algorithm is used to analyze the main color tone of the reference image; based on the scene analysis results and content analysis results, each target object in the sketch to be drawn is divided into multiple independent brush information, and the weight of each brush is determined; piecewise Bézier curves are used to draw the sketch to be drawn, and the brush path of each brush is output layer by layer, specifically including: dynamically adjusting the parameters of the Bézier curve according to the shape and curvature of each brush; generating an initial drawing according to user instructions and based on the brush path of each brush, or recalculating each brush path to optimize the initial drawing.

[0010] A second aspect of this invention provides a controllable scene sketch generation system, which uses the controllable scene sketch generation method described in the first aspect of this invention. The controllable scene sketch generation system includes: an input receiving module for receiving input information of a sketch to be drawn, the input information including at least one of the following information data: text segment, hand-drawn sketch, and reference image; and a parsing and processing module for performing scene parsing and content parsing on the input information. When the input information contains a hand-drawn sketch, the module identifies connected regions in the hand-drawn sketch and performs curvature analysis and intersection detection to obtain specific contours and key features for dynamic path tracking; when the input information contains a reference image, the module extracts object wheels from the reference image. The system extracts outline, position, color, and style information. When extracting color information, the K-Means clustering algorithm is used to analyze the dominant color tone of the reference image. A splitting module, based on scene and content analysis results, splits each target object in the sketch to be drawn into multiple independent brush information entries and determines the weight of each brush. A layered output module uses piecewise Bézier curves to draw the sketch, outputting the brush path of each brush in layers. Specifically, this includes dynamically adjusting the parameters of the Bézier curves based on the shape and curvature of each brush stroke. A generation module, based on user instructions and the brush path of each brush, generates an initial drawing or recalculates the brush paths to optimize the initial drawing.

[0011] A third aspect of the present invention provides an electronic device, comprising: one or more processors; a storage device for storing one or more programs; wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the controllable scene sketch generation method described in the first aspect of the present invention.

[0012] A fourth aspect of the present invention provides a computer-readable medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the controllable scene sketch generation method described in the first aspect of the present invention.

[0013] The beneficial effects of this invention are:

[0014] Compared to existing technologies, this invention achieves highly controllable sketching by understanding and parsing various input information, improving the controllability, drawing quality, and editability of generated sketches, thereby realizing an intelligent painting mechanism. Specifically, it employs an intelligent agent architecture to process the painting process in stages, using piecewise Bézier curves for drawing. Combined with AI software, it can progressively analyze the scene, objects, and their stroke layers according to a logical framework. Within this framework, AI software first determines the overall scene layout, then draws the main object structures, and finally fills in the detailed strokes, ensuring that the drawing process is more in line with human drawing habits. Unlike traditional one-step image generation methods, this invention makes the sketch generation process more controllable and enhances the rationality of the overall composition. This invention achieves highly controllable sketching by understanding and parsing various input information, improving the controllability, drawing quality, and editability of generated sketches, thereby realizing an intelligent painting mechanism.

[0015] Furthermore, in terms of stroke generation, this invention employs piecewise Bézier curves for drawing to enhance the smoothness and naturalness of the sketch. Compared to traditional polyline drawing methods, Bézier curves provide smoother curve transitions, making strokes more natural and reducing breaks and harshness. In addition, an intelligent stroke optimization mechanism is introduced, automatically adjusting the detail level based on the curvature, length, and weight of the stroke, making the sketch more layered. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating an example of the controllable scene sketch generation method of the present invention;

[0017] Figure 2 This is a schematic diagram illustrating an example of the controllable scene sketch generation method of the present invention;

[0018] Figure 3 This is a schematic diagram of a specific implementation of the controllable scene sketching generation method of the present invention;

[0019] Figure 4 yes Figure 3 A schematic diagram of the split curve;

[0020] Figure 5 This is a schematic diagram of an example of the controllable scene sketch generation system of the present invention;

[0021] Figure 6 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention;

[0022] Figure 7 This is a schematic diagram of a computer-readable medium embodiment according to the present invention. Detailed Implementation

[0023] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0024] In view of the above problems, this invention provides a controllable scene sketch generation method. It employs an intelligent agent architecture to process the drawing process in stages and uses piecewise Bézier curves for drawing. Combined with AI software, it can progressively analyze the scene, objects, and their stroke layers according to a reasonable logic. Within this framework, AI software first determines the overall scene layout, then draws the main object structures, and finally fills in the detailed strokes, thus ensuring that the drawing process is more in line with human drawing habits. Unlike traditional one-step image generation methods, this invention makes the sketch generation process more controllable and enhances the rationality of the overall composition. By understanding and parsing various input information, this invention can achieve highly controllable sketch drawing, improving the controllability, drawing quality, and editability of the generated sketch, thereby realizing an intelligent drawing mechanism.

[0025] Furthermore, in terms of stroke generation, this invention employs piecewise Bézier curves for drawing to enhance the smoothness and naturalness of the sketch. Compared to traditional polyline drawing methods, Bézier curves provide smoother curve transitions, making strokes more natural and reducing breaks and harshness. In addition, an intelligent stroke optimization mechanism is introduced, automatically adjusting the detail level based on the curvature, length, and weight of the stroke, making the sketch more layered.

[0026] It should be noted that the method of this invention supports multiple input methods, enabling users to flexibly guide AI software in sketching creation. This invention has a wide range of applications, particularly suitable for drawing sketches requiring specific outlines or compositions. Specifically, it is applicable to individual users (such as artists, illustrators, and designers), and can also be applied in various scenarios, including specific design fields (such as UI / UX design, game art, and advertising design) and education and training (such as digital painting courses and intelligent painting assistance). For example, in the field of digital illustration, vector sketches generated by AI software can be directly imported into design software, helping artists quickly complete drafts and providing editable layered graphics. In UI / UX design, intelligent sketching can be used to generate interactive prototypes, helping designers quickly iterate on interface layouts and visual elements; in game art production, it can be used to generate controllable hand-drawn style art resources, improving game development efficiency; in children's art education, it provides learners with intelligent painting guidance and work optimization suggestions, and can also serve as an AI-assisted painting tool.

[0027] Furthermore, in terms of scalability, this invention can not only be deployed on existing computing platforms but also integrated into existing design software, web applications, or mobile applications via APIs or plugins. It supports a combination of JSON and SVG formats, allowing AI-generated content to be flexibly adjusted in different environments and ensuring compatibility with mainstream design software and programming frameworks. In addition, it can be combined with cloud computing resources to support online drawing generation, improving computational efficiency, reducing the computational load on terminal devices, and providing intelligent drawing services to more users.

[0028] Example 1

[0029] The following reference Figure 1 , Figure 2 , Figure 3 and Figure 4 The present invention will provide a detailed description of the simulation display and control method for determining the task area.

[0030] Figure 1 This is a flowchart illustrating an example of the controllable scene sketch generation method of the present invention.

[0031] First, in step S101, input information for the sketch to be drawn is received, and the input information includes at least one of the following information data: text segment, hand-drawn sketch, and reference image.

[0032] Specifically, it receives input information for the sketch to be drawn, which may include one or more types of information data, such as text segments, hand-drawn sketches, reference images, etc.

[0033] exist Figure 2 In application examples, such as using AI to draw sketches, inputting a hand-drawn sketch (e.g.) Figure 2 (Simplified sketch in the text), text segment (corresponding to) Figure 2 The system parses the input information, including text and reference images, and uses prompts and multiple, progressively layered intelligent agents (scene agent, object agent, detail agent, stroke agent, and color agent) to extract and analyze data from different dimensions to obtain data corresponding to the sketch to be drawn. Then, the data is stored in a combined format and used for drawing on the canvas, with the relevant data exported from the user interface.

[0034] In one specific implementation, a user can input creative requirements into an application system via plain text descriptions, such as "a small cabin in a forest." The application system will parse the text and automatically generate scenes and objects that match the description. The application system comprises multiple intelligent agents in a multi-level, progressive manner; see details below. Figure 2 .

[0035] In another specific implementation, for example, a sample sketch can be input, where the user can provide a simple hand-drawn sketch. AI software, for instance, can then complete and optimize the structure of this sketch to generate a more complete drawing. For example, in concept design or hand-drawn sketching, the user outlines the general shape with simple lines, then refines and polishes it using software such as AI.

[0036] It should be noted that, in addition to text and sketches, reference images are also supported. Users can provide an image as a style or color scheme reference, and the AI ​​will analyze the key features in the image and apply them during the sketch generation process. For example, if a user uploads a street sketch, the AI ​​can learn its color scheme and stroke style and apply them to the newly generated sketch. This allows the AI ​​to combine the user's personalized needs to create a more customized sketch that meets their expectations. The above is only an optional example and should not be construed as limiting the invention.

[0037] Next, in step S102, the input information is parsed for scene and content. When the input information contains a hand-drawn sketch, the connected regions in the hand-drawn sketch are identified, and curvature analysis and intersection detection are performed to obtain specific contours and key features for dynamic path tracking. When the input information contains a reference image, the object contour, position, color information and style information are extracted from the reference image. When extracting color information, the K-Means clustering algorithm is used to analyze the main color tone of the reference image.

[0038] Specifically, when the application system receives input information, it may use a scene intelligence agent to perform scene parsing and content parsing on the input information (specifically including parsing of sketch language and natural language).

[0039] Determine the scene elements for each application scenario, including background, environmental layout, and object distribution, to form a scene intelligent agent.

[0040] In this invention, the scene agent, acting as the parsing and structuring module for input information, is primarily responsible for extracting scene element information from user input and generating a complete and standardized scene configuration. For natural language input, the scene agent directly integrates a large language model as its core parsing engine, automatically recognizing the intent of the input description and extracting scene elements. Leveraging the powerful contextual understanding and reasoning capabilities of the LLM model, it can accurately extract core scene elements such as "background," "environmental layout," and "main objects." For example, given a user inputting the description "At dusk on the beach, there are two seagulls on the sand, and a lighthouse in the distance," the LLM directly outputs structured scene elements, including scene type (beach), background features (dusk, sky), environmental layout (layered structure of beach-sea-sky), a list of objects (seagulls, lighthouse), and their basic attributes (quantity, location, distance, orientation, etc.). The background is automatically generated through the semantic understanding capabilities of the LLM model, the environmental layout is automatically divided into foreground, midground, and background layers based on the spatial representation mode of the LLM model, and the main objects are supplemented with attribute information and then uniformly archived into the scene configuration. Compared to existing methods of "dependency parsing and rule extraction", the parsing method based on large language models has stronger language understanding and complex relationship modeling capabilities, can efficiently handle diverse and complex natural language inputs, reduce manual rule design, and improve parsing accuracy.

[0041] First, the user's input information is parsed to extract key scene information, including cities, forests, and indoor spaces. When determining the scene structure, the following layout analysis is performed based on perspective rules and spatial logic: the relative positions of multiple spatial elements are determined, and the multiple spatial elements are ensured to conform to perspective rules; when generating natural landscapes, the arrangement of multiple terrain elements is analyzed.

[0042] In one specific implementation, when the input information includes a hand-drawn sketch, the connected regions in the hand-drawn sketch are identified, and curvature analysis and intersection detection are performed to obtain specific contours and key features to achieve dynamic path tracking.

[0043] For example, when a simplified sketch is input, AI software might break down the overall line into individual strokes for later refinement and completion. First, connected component detection is performed, using the Flood-Fill algorithm to identify and separate different connected components, ensuring that different strokes are not confused. Then, curvature analysis and intersection detection are performed: for intersecting strokes, AI software might use curvature change rate analysis (e.g., using the second derivative to determine the degree of curve curvature) combined with Canny edge detection to determine how to break down the intersection points.

[0044] First, Canny edge detection is used to extract edge points in the image. Then, the connected curves of the edges are obtained through the edge points, and all points in the contours are collected into a set. traversal At each point in Within a given area, the number of neighboring edge points is calculated. Polar coordinate analysis is performed on the distribution direction of these neighboring points. If three directions are activated, it is considered an intersection point. Each intersection point has multiple edge points in its vicinity. Connecting the intersection point with edge points in different directions forms multiple direction clusters. Determining the splitting method for edge points involves pairing these direction clusters. The principle of pairing is that for the same natural curve segment, the curvature values ​​at its left and right ends should be similar, and the rate of change of curvature of connected curves should be gradual. For intersection points… ,exist Within a cluster of directions, there are pairwise combinations. For example, with four directions, there are three possible combinations. A cost function is defined for each combination (A, B): ,in, These are the average curvatures within the two directional clusters, respectively. Let be the rate of curvature change of the connecting segment between the two clusters along the two directions. These represent the weights corresponding to the two directional clusters. In this invention, the rate of change of curvature is particularly important. The selection range is 0.2 to 0.4. It ranges from 0.6 to 0.8.

[0045] Preferably, It is 0.3. It is 0.7.

[0046] For example, when three strokes intersect at a point, and the user inputs a hand-drawn sketch, the system first extracts the edge points of the sketch. For ease of display, only a portion of the edge points are shown evenly. First, it determines whether a point is an intersection point. Then, it selects edge points within a certain range centered on that point. Next, it establishes a coordinate system with that point as the origin, the horizontal direction as the x-axis, and the vertical direction (perpendicular to the horizontal direction) as the y-axis. Finally, it checks whether each selected point lies in more than two quadrants. Figure 3 For example, determine Figure 3 Is G1 an intersection point? First, select points G2 and G3 around G1, and then establish a coordinate system with G1. We find that G2 and G3 only activate the first and third quadrants, so G1 is not an intersection point. Next, determine point P1. Select four points P2, P3, P4, and P5 around P1, and establish a coordinate system with P1 as the origin. We find that all four quadrants are activated, so P1 is an intersection point. Next, use curvature to determine the curve splitting. First, connect this point with other points to form direction clusters, resulting in four direction clusters. Then, combine these direction clusters in pairs, resulting in three different splitting methods. Figure 4In the diagram, curves Z1 and Z2 (represented by gray lines), curves Z3 and Z4 (represented by black lines), and line segments ab and cd represent three types of strokes. Calculate the value of each stroke. We can see that the cost function of the method for dividing lines ab and cd is the lowest, so the stroke division method is determined.

[0047] Next, dynamic path tracking is performed, specifically using Bézier curve fitting to model each stroke of the simplified sketch, ensuring the application system can understand the stroke order. After obtaining the stroke information, the large model is used to convert it into a sketch language based on SVG format, which is then used as the final feature extracted from the simplified sketch.

[0048] In one specific implementation, when the input information includes a reference image, object outline, position, color information, and style information are extracted from the reference image. When extracting color information, the K-Means clustering algorithm is used to analyze the main color tone of the reference image.

[0049] For example, when a reference image is input, a series of computer vision techniques are employed for object detection, color extraction, and style analysis to extract useful information and apply it to the sketch generation process. In object detection, the YOLOv8 object recognition model is used to automatically analyze the main objects in the reference image and extract their contours and positions. For instance, in a street scene image, different elements such as buildings, trees, and pedestrians can be identified and adjusted accordingly based on the style requirements of the sketch.

[0050] For color extraction, the K-Means clustering algorithm is used to analyze the dominant color tone of the reference images, and the color scheme is optimized during the sketch generation process. Specifically, the pixels of all images are extracted into a feature matrix. , This represents the i-th pixel, where i is a positive integer, specifically 1, 2, ..., N; This represents the coordinate space of the pixels. Clustering is performed within the RGB space. for The vectors of points in the 3D RGB space are used to perform K-Means clustering calculations using the following expression to obtain the three primary colors. (j is 3, (This represents three primary colors), and each pixel is assigned to its respective primary color: Where 'c' represents the primary color. This represents the j-th primary color, where j is a positive integer, specifically 1, 2, or 3; This represents the i-th pixel, where i is a positive integer, specifically 1, 2, ..., N; for The vector of a point in the 3D RGB space, where 'r' refers to the RGB space, i.e., the color coordinate system. This formula calculates the sum of the squared minimum Euclidean distances from each pixel in the RGB space to its nearest dominant color center.

[0051] All pixels are divided into three primary color gamuts based on their color labels. After the primary color of each pixel is determined based on the minimum distance, each pixel is labeled with its primary color, which is denoted as L. i Assuming there are three main colors, then L i There are three possible values: 1, 2, and 3, represented by the following expression: , Represents j primary color gamuts, L i This represents the i-th primary color.

[0052] Perform a separate K-Means clustering calculation within each color gamut, using the following expression to extract three sub-colors: Where S represents the subcolor, S Q This indicates the seed color Q, where Q=3 in this example; This represents the i-th pixel, where i is a positive integer, specifically 1, 2, ..., N; for A vector of a point in 3D RGB space, where 'r' refers to the RGB space, i.e., the color coordinate system. This represents j primary color gamuts.

[0053] In this example, three colors are selected each time a primary color is chosen. After determining the three primary colors, three sub-primary colors are then selected from them, resulting in a total of nine colors. These nine colors can be used as color scheme choices. Simultaneously, the color proportions of the primary colors are calculated. The dominant color with the highest proportion is the main color tone of the reference image. N represents the total number of pixels belonging to the primary color j. 像素 This represents the total number of pixels in the current image.

[0054] For example, if the reference image's main color tone is warm, the colors in the generated sketch will be automatically adjusted to maintain consistency with the original image's style. Furthermore, users can manually adjust the color extraction results to ensure the final artwork meets their personalized needs.

[0055] In terms of style analysis, CLIP (Contrastive Language-Image Pretraining) and VGG19 are combined to extract deep style features, identify the overall style, brushstroke features and light and shadow changes of the reference image, and allow AI to imitate the artistic style of the reference image in a more intelligent way, thereby generating more harmonious and expected sketches.

[0056] Next, the information on each object, color, and style is converted into natural language that can be understood by the large model and incorporated into a custom prompt.

[0057] The system receives structured descriptive input, typically including information in three dimensions: object, color, and style. Since large language models are better at processing natural language, the first step is to convert the structured information into expressions that conform to natural language habits. Specifically, object and color information are combined into a concrete scene description, while style information is embedded as an overall visual or textual representation. The specific method involves dividing a complete prompt into a system prompt, an example prompt, and a user prompt. The system prompt defines the model's behavior and role; the large model understands the task to be completed and the steps to complete it through the system prompt, while also determining the format of the output. The example prompt provides a complete output example, allowing the large model to learn the specific content of the example. The user prompt is where information is integrated. Since the reference image mainly serves as a supplement to the text input, after adding text, "Object: Cat," "Main Color: Black," and "Style: Cute" are added to the user prompt.

[0058] The customized prompt primarily revolves around the generation of controllable color scene sketches. Through hierarchical information organization, it guides the large model to parse user input and generate drawing instructions that meet expectations. First, it defines the basic format of the sketch language, clarifying the representation of the sketch, including how to use piecewise Bézier curves to describe stroke paths, setting control points, and adjusting the parameter t value to ensure curve smoothness. Simultaneously, the customized prompt also limits the hierarchical structure of strokes, allowing strokes of different importance to be expressed using curves of varying precision, ensuring that the generated image is both smooth and visually logical.

[0059] Regarding color settings, the customized prompt details how to determine the color scheme for the subject, supporting the extraction of keywords from text descriptions to infer colors, and also enabling the extraction of primary hues from natural language from reference images to ensure that the final sketch's color style matches the user's expectations. Simultaneously, the color allocation logic has been optimized to ensure overall harmony and unity, adapting to different drawing needs, whether for realistic or minimalist styles.

[0060] Extract color keywords from the text information in the input information, extract hue information from the reference image, and match the corresponding colors for each object in the sketch to be drawn based on the extracted color keywords and hue information.

[0061] In addition, the detail agent is responsible for the specific division of strokes and sets the drawing priority and complexity according to the importance of different strokes, so that the sketch has a sense of layering.

[0062] The module corresponding to the detail agent is driven by a large-scale language model (LLM). It is primarily responsible for intelligently parsing the structural information and semantic description of the input object, completing the fine division and hierarchical planning of strokes, thereby generating a sketch with good hierarchy and visual guidance. The input to the module corresponding to the detail agent is the semantic description information of the object (e.g., "This is a vintage bicycle" or "This is a portrait of a woman"). The detail agent analyzes the information through LLM to establish the semantic priority and structural hierarchy of each region in the image.

[0063] The detail agent uses LLM to perform semantic parsing of object descriptions, understanding the object's composition and features. For example, LLM is used to determine the main structure of an object (such as outlines and skeletal lines, which are high-priority areas) and requires them to be drawn first to ensure the structural integrity of the sketch; while for special details mentioned in the description, the detail agent can increase the detail priority of the corresponding area, making the details more detailed.

[0064] In terms of stroke complexity control, LLM combines the overall style keywords of the scene (such as "rough," "delicate," and "simplified") with the visual complexity of each area (such as local texture variations and edge density) to make a comprehensive judgment and determine the appropriate level of detail for each layer of strokes. For example, in areas with high complexity and emphasis on detail, dense strokes are used to restore realism; while in areas where the description emphasizes simplicity or background areas, simplified lines are used to highlight the hierarchy and avoid visual redundancy.

[0065] To enable large models to accurately parse and generate sketches, the customized Prompt employs an agent collaboration mechanism. In the drawing process, different agents are assigned specific tasks: the scene agent analyzes the overall layout, determining the main objects in the scene and their relative positions; the object agent is responsible for dissecting the structures of objects in the scene to ensure their reasonable forms; the detail agent further breaks down the constituent strokes of objects and assigns drawing weights to optimize drawing accuracy; the stroke agent generates stroke paths based on piecewise Bézier curves; and the color agent adjusts the color matching scheme based on text or images. This multi-level architecture allows large models to gradually construct complete and logically sound sketches.

[0066] Next, a combination of structured data and visual storage formats is used, specifically including: storing drawing information in JSON format, including the following detailed parameters: stroke order, position, control points, and pen weight;

[0067] The final image is stored, rendered, and displayed using SVG. JSON parameters can be adjusted during dynamic drawing or secondary generation without modifying the SVG code.

[0068] In terms of output, the customized prompt uses a combination of JSON and SVG formats, ensuring that the generated sketches can not only be directly visualized but also easily edited by users. The JSON structure stores detailed data for all strokes, including path, color, and weight information, while the SVG code provides layered drawing results, allowing users to flexibly adjust the shape and color of objects. Furthermore, to help large models better understand the drawing task, the customized prompt includes a detailed example demonstrating how to parse object information from natural language input converted from text or images and gradually build complete drawing instructions, making the entire process more intuitive and clear.

[0069] By optimizing the Prompt to create a customized Prompt, the multimodal large model can more effectively parse user input and rationally coordinate the collaboration between agents. The customized Prompt adopts a structured design, clearly defining the input format, drawing logic, stroke generation rules, color adaptation scheme, etc., enabling the large model to follow a strict logical structure when understanding the task and complete the sketch generation step by step according to the preset process. Specifically, the customized Prompt details how to parse user input (text, sketches, images), how to perform agent task allocation (such as scene parsing, object decomposition, stroke calculation, color matching, etc.), and how to output editable drawing results in a combination of JSON and SVG formats. In this way, the large model can not only generate high-quality sketches according to user needs but also dynamically update the drawing content after the user adjusts parameters, ensuring that the generated results meet user expectations and improving the controllability and interactivity of AI-generated content.

[0070] It should be noted that the above is only an optional example and should not be construed as a limitation of the present invention.

[0071] Next, in step S103, based on the scene analysis results and content analysis results, each target object in the sketch to be drawn is split into multiple independent brush information, and the weight of each brush is determined.

[0072] Bézier curve fitting is used to model the individual strokes of the sketch to be drawn, so that the large model understands the order of the strokes and is converted into a sketch language based on SVG format for extracting key features.

[0073] For strokes formed by a single n-order Bézier curve, strokes with different importance weights are represented by curves of different precision. Specifically, the general expression for a single n-order Bézier curve is: .in, Let be the order of the Bézier curve. ,in, These are the minimum and maximum orders, determined by the weights. This is a coefficient that adjusts the influence of the weighting, with a value ranging from 0.5 to 0.7, preferably 0.6. This is a coefficient that adjusts the influence of curvature, with a value ranging from 0.6 to 0.8, preferably 0.7. The curvature estimates given for the large model are selected from low to high, and from... The values ​​are taken from the middle, and the weights are determined by the larger model and given at the same time when outputting the strokes. They are stored in the stroke information, and the weights are divided into high, medium, and low. High weights are... Medium weight low weight ;P i Let i represent the control point of the i-th Bézier curve, where i is a positive integer.

[0074] The large model makes a judgment: if it needs to generate standard geometric strokes (such as triangles, circles, rectangles, etc.), then the above calculation process will not be performed.

[0075] When determining that a stroke is not a geometric stroke, the `path` tag is used to perform the above calculation process. A method for segmenting each stroke is then applied.

[0076] It should be noted that the above is only an optional example and should not be construed as a limitation of the present invention.

[0077] Next, in step S104, a piecewise Bézier curve is used to draw the sketch to be drawn, and the stroke path of each stroke is output in layers. Specifically, the parameters of the Bézier curve are dynamically adjusted according to the shape and curvature of each stroke.

[0078] Specifically, piecewise Bézier curves are used to describe the stroke path, control points are set, and the parameter t value is adjusted to optimize the curve shape. At the same time, a prompt (i.e., a customized prompt) is used to limit the hierarchical structure of the stroke.

[0079] When drawing a sketch using Bézier curve fitting, the connection between adjacent strokes is detected. If a broken or discontinuous line is identified, the following steps are performed.

[0080] Step S201: Determine two adjacent Bézier curves (hereinafter referred to as "curves") in the sketch to be drawn. The two adjacent Bézier curves include the first curve and the second curve.

[0081] Specifically, the first and second adjacent curves (e.g., adjacent curve 1 and curve 2) are determined from the sketch to be drawn. A point set is established with the endpoints and start points of all curves as nodes. The nearest neighbor is found using Euclidean distance, and the curves that can match each other are designated as adjacent curves.

[0082] Step S202: When determining whether a break occurs between the first curve and the second curve in two adjacent Bézier curves, determine whether the difference between the end point of the first curve and the starting point of the second curve is greater than a specified value.

[0083] Specifically, if there are two curves, a first curve and a second curve, in the sketch to be drawn, the Euclidean distance, i.e. the difference, from the end point (i.e., the first control point) of the first curve (e.g., curve 1) to the starting point (the second control point) of the second curve (e.g., curve 2) is detected. If the difference is greater than a specified value (specifically 6px to 12px, preferably 10px), then there is an obvious break.

[0084] Step S203: Using the method of adjusting the first and last control points, the first control point and the second control point are moved towards the midpoint of the gap to reduce the gap and correct the first control point and the second control point, thus completing the first correction process.

[0085] Specifically, the first control point and the second control point are moved towards the midpoint of the gap, i.e., moved closer to each other, and the midpoint of the gap is calculated. ,in, Indicates the coordinate position of the first control point. This indicates the coordinates of the second control point, and the two control points move towards the midpoint: P Among them, P1' end P1 is the coordinate information of the first control point after correction along the vertical direction. end The coordinates of the first control point before correction. The selection range is 0.2 to 0.5; P2' start P2 provides the coordinate information of the second control point after vertical correction. start The coordinates of the second control point before correction.

[0086] Step S204: Determine whether there are discontinuous curves in the sketch to be drawn, calculate the first tangent vector of the endpoint of the first curve and the second tangent vector of the starting point of the second curve, and determine whether the angle formed by the first tangent vector and the second tangent vector is greater than a specific value to determine whether the first curve and the second curve are continuous.

[0087] Specifically, calculate the first tangent vector at the endpoint of the first curve: ,in, The first tangent vector representing the endpoint of the first curve; This indicates the coordinates of the endpoint of the first curve, i.e., the first control point. This indicates the coordinates of the midpoint of the gap between the first and second control points.

[0088] Calculate the second tangent vector at the starting point of the second curve: ,in, The second tangent vector represents the starting point of the second curve; This indicates the coordinates of the endpoint of the first curve, i.e., the first control point. This indicates the coordinates of the midpoint of the gap between the first and second control points.

[0089] Next, if the angle between the first tangent vector and the second tangent vector... If the angle between the first and second tangent vectors is greater than a specific value (specifically between 40 and 50 degrees, preferably 45 degrees), then the first and second curves are considered discontinuous. Furthermore, if the angle between the first and second tangent vectors is greater than a specific value, then the first and second curves are considered discontinuous. If the value is less than or equal to a specific value (specifically between 40 and 50 degrees, preferably 45 degrees), then the first curve and the second curve are considered to be continuous.

[0090] Step S205: Based on the first tangent vector and the second tangent vector, perform secondary modification processing on the discontinuous first curve and second curve.

[0091] When the first curve and the second curve are discontinuous, calculate the average tangent vector of the first tangent vector and the second tangent vector: ,in, This represents the average tangent vector of the first and second tangent vectors; The first tangent vector representing the endpoint of the first curve; The second tangent vector represents the starting point of the second curve.

[0092] Next, the control points are rotated to the average tangent vector, which is a secondary modification of the discontinuous first and second curves. The first and second control points after the secondary modification are represented as follows:

[0093] (1)

[0094] in, This indicates the coordinate position of the first control point after the second correction process. This indicates the coordinates of the endpoint of the first curve, i.e., the first control point. The coordinates of the midpoint of the gap between the first control point and the second control point; This represents the average tangent vector of the first and second tangent vectors.

[0095] (2)

[0096] in, This indicates the coordinate information of the second control point after secondary correction processing; This indicates the coordinate position of the starting point of the second curve, i.e., the second control point; The coordinates of the midpoint of the gap between the first control point and the second control point; This represents the average tangent vector of the first and second tangent vectors.

[0097] Through the above two correction processes, a more natural and effective transition effect between adjacent curves can be obtained.

[0098] Distributing t values ​​evenly, i.e., at equal intervals, can lead to insufficient sampling in high-curvature areas and oversampling in low-curvature areas, resulting in insufficient smoothness. Therefore, this invention employs a dynamic distribution, with denser t values ​​at high curvature and sparser t values ​​at low curvature. t is a parameter controlling the position on the Bézier curve, representing the progress from the starting point to the ending point, and its value ranges from 0 to 1. Typically, when drawing a Bézier curve, sampling is automatically performed from t=0 to t=1, collecting a large number of t values, calculating the points on the curve, and connecting them to draw a smooth curve.

[0099] An intelligent stroke optimization mechanism is employed to optimize each stroke. First, the length and total curvature of the target Bézier curve for each stroke are calculated. When calculating the Bézier curve length, there is usually no analytical solution for high-order Bézier curve integrals, but this invention only needs to obtain a comparison of the lengths of each curve, so a piecewise accumulation method is used. First, the... The interval is divided into N' smaller intervals, and then the discrete points of the Bézier curve are used. Approximate a continuous curve and calculate the distance between adjacent points: ,in, Indicates the t-th i Discrete points at time; Indicates the t-th i+1 The discrete point at time; L represents the discrete point. and discrete points The distance between them. The overall curvature of a Bézier curve is calculated using the total amount of bending: ,in, Let be the curvature of the curve at time t. Since this integral usually has no analytical solution, a piecewise accumulation method is also used, i.e. .

[0100] The weight of each stroke is assigned during the generation of the large model and is divided into three levels: high, medium, and low. If only curvature is used for judgment, long and smooth main contour lines will be judged as low priority, while small but sharply turning detail lines will be ignored if only length is used for judgment. Therefore, stroke priority is introduced in this invention. The concept of priority is expressed by the following expression:

[0101] (3)

[0102] Where S represents the priority of the current stroke; The length of the current stroke. This represents the maximum length of all strokes in the sketch to be drawn. R represents the total curvature of the current stroke. max This represents the maximum total curvature of all strokes in the sketch to be drawn. and These represent the first and second adjustable coefficients, used to control the relative importance of length and curvature, respectively. Since the outline is more important in a sketch, the influence of length should be given greater weight. In this example, The selection range is 0.6~0.8. The value range is 0.2 to 0.4.

[0103] Next, we will optimize each stroke, starting with prioritizing the strokes with high weight. The strokes are sorted, with the top 30% forming the main outline. The program then thickens and widens this main outline, and prioritizes the strokes with a weight of medium. The strokes are sorted, with the top 30% designated as main detail lines and thickened to medium. Finally, the number of low-weight strokes is determined. If the number of low-weight strokes in the target object's sketch exceeds 5, these low-weight strokes are prioritized, and the bottom 20% are deleted to avoid visual interference and reduce the difficulty of subsequent coloring. For example, when generating a sketch of a girl, her high-weight body outlines, due to their length and high weight, are defined as main outlines and thickened, while the girl's hair, being detail lines, is widened to medium, and the patterns on the skirt are unimportant strokes and can be deleted.

[0104] Specifically, first roughly estimate the arc length of each segment (adjacent control points) of the sketch to be drawn (i.e. each stroke), then calculate the total curvature of the j-th segment, calculate the weighted length of each segment, and calculate the cumulative weighted length, and then normalize to generate a t-distribution.

[0105] Specifically, the total curvature of the sketch to be drawn is calculated using the following expression: Simultaneously, calculate the total curvature of the j-th segment of the curve: .in, This represents the curvature of the curve at the parameter t; Indicates the distance between two adjacent control points; This represents the (j+1)th control point; This represents the control point corresponding to the j-th curve segment.

[0106] Calculate the weighted length for each segment: ,in, This represents the weighted length of the current curve segment and the j-th curve segment. It considers both the arc length and curvature between two control points, and the weighted sum of these two values ​​yields a value that displays the arc length and total curvature. The longer the arc length and the greater the total curvature, the larger this value will be. Indicates the distance between two adjacent control points; Let be the total curvature of the j-th segment of the curve, representing the degree of curvature of the j-th segment of the curve. The larger this value is, the greater the degree of curvature. This indicates the weight of curvature in the weighted length; The value ranges from 0.4 to 1.

[0107] Calculate the cumulative weighted length: Finally, normalization generates a t-distribution: ,in, This indicates the number of t-values ​​sampled within the j-th segment of the curve. This indicates the total weighted length of the current stroke. This represents the weighted length of the j-th segment of the curve in the current stroke, where j is a positive integer.

[0108] The metric for smoothness is the curvature square integral energy: Since there is usually no analytical solution, it is simplified to discrete calculation: ,when It was deemed to meet the requirements at that time. This represents the curvature of the curve at point t. The norm represents the tangent vector of the curve at the parameter t. This represents the magnitude of the velocity of the curve at parameter t, in discrete calculations. It is the curvature value at the midpoint of the i-th segment of the curve. This represents the square of the curvature value. is the length of the i-th segment, where i is a positive integer. E is the curvature square integral energy, used to measure the overall curvature complexity of the curve. After calculating the value of E, in addition to normalizing the total arc length, if the value of E is less than 0.05, the smoothness is considered to meet the requirements; if the value of E is greater than or equal to 0.05, the smoothness is considered to fail to meet the requirements. In this case, increase... Continue the calculation until the smoothness meets the requirements.

[0109] It should be noted that the above is only an optional example and should not be construed as a limitation of the present invention.

[0110] Next, in step S105, an initial drawing is generated according to the user's instructions and the pen path of each pen, or the pen paths are recalculated to optimize the initial drawing.

[0111] Specifically, an initial drawing is generated based on user instructions and the pen path of each pen.

[0112] The recalculation of each brush path is activated when the user directly modifies or provides modification suggestions for the initially generated sketch (i.e., the initial sketch). If the user directly modifies, the corresponding tools are invoked, such as panning, changing control points, scaling, and adjusting color and stroke thickness. If a large model is used for modification, all tools are first summarized into a toollist. Then, the large model decomposes the task into sub-tasks, matches the tools required for each sub-task, and inputs the modified parameters into the tools to modify the initial sketch.

[0113] In one specific implementation, the user inputs, for example, "draw a kitten in the sketch to be drawn" on the user interface, and then determines the brush and brush path corresponding to "kitten" to generate the initial drawing.

[0114] It's worth noting that this example allows users to interact at different stages, including dragging and adjusting object positions, modifying color schemes, and selecting different stroke styles. These modifications are achieved by directly altering the generated JSON+SVG programmatically. This interactive approach ensures that the drawing results meet the user's personalized needs.

[0115] Furthermore, a progressive generation mechanism is employed. After the user inputs initial instructions, a basic sketch is first generated. The user can then make adjustments, such as changing the composition, adjusting proportions, and optimizing details. Subsequently, based on the user's adjustments and combined with the simplified sketch and text input, the stroke paths, color matching, and hierarchical structure are recalculated, ultimately outputting an optimized, high-quality drawing result. To enhance the user experience, multiple export formats are supported, such as SVG, PNG, and PSD, allowing users to continue editing and adjusting in different post-processing tools. This ensures the flexibility of the drawing process, enabling AI-generated sketches to not only meet the needs of different users but also adapt to various application scenarios, thereby improving the practicality and operability of the drawing.

[0116] For example, when generating complex sketch fonts or street art pieces with smooth curves, the optimization algorithm for Bézier curves can ensure that each stroke conforms to the expected visual style.

[0117] In different application scenarios, we can improve the rendering effect while satisfying object edge requirements and rendering efficiency.

[0118] For object edge evaluation, the ability of a sketch to reproduce the contours of key objects is assessed through connected component detection, curvature analysis, and stroke priority calculation. Specific quantitative standards are as follows:

[0119] High-priority stroke extraction rate:

[0120] (4)

[0121] The stroke priorities S and S' are calculated using the above expression (3), where the stroke priority S corresponds to S in the above expression (3). When calculating the stroke priority S', S in the above expression (3) is replaced with S'. The number of strokes with high S value refers to the number of strokes that are theoretically arranged from largest to smallest according to the calculated S value. The first 30% of these strokes are called the number of strokes with high S value. In practice, this refers to arranging all strokes from largest to smallest according to the calculated S-value, and selecting the top 30% of strokes; this portion of strokes is called the "high" strokes. Number of strokes.

[0122] For evaluating rendering efficiency, the number of rendering path segments and curvature smoothness are used together to assess rendering efficiency, which is specifically quantified as follows:

[0123] The lower the average number of control points for each piecewise Bézier curve, the higher the drawing efficiency.

[0124] The curvature square integral energy (i.e., smoothness E) is calculated using the curvature square integral energy calculation method described in step S104 above, which uses the smoothness metric as the index.

[0125] The evaluation of drawing quality is based on a comprehensive assessment of stroke layering, main outline enhancement, and detail pruning proportions. Specifically:

[0126] The proportion of the main contour is calculated using the following expression:

[0127] (5)

[0128] After sorting the main outline strokes according to their stroke priority, select the top-ranked portion (e.g., 20%–30%) and thicken it.

[0129] Calculate the detail pruning rate and prune low-priority strokes (e.g., remove the lower-ranking parts, specifically 10% to 20%) to avoid visual clutter and improve the overall effect.

[0130] Next, we will illustrate this with examples from two different scenarios. The quantized values ​​for each scenario are provided by a large model. The scenario involves generating an interior space sketch. Interior spaces have structural lines (walls, windows, furniture edges), and their contours have obvious low curvature (mostly straight lines or gentle curves), requiring high accuracy and clear edges. For object edges to meet quantization requirements, the success rate of edge extraction mainly depends on a high extraction rate of connected regions (because the boundaries are clear and there are few overlaps). Regarding stroke priority distribution: strokes of the main contours (wall contours, sofa contours) have high priority to ensure complete extraction. A score greater than 95% is required to ensure complete extraction.

[0131] Specifically, drawing efficiency is quantified based on drawing time and curvature square integral energy, while drawing time is approximated by the average number of control points per curve (mostly straight lines or simple arcs). The average number of control points is 2 or 3, and the E value is less than 0.02 (because the overall lines are straight).

[0132] For example, thicken the main objects such as walls and sofas, making them account for more than 60% of the total stroke length, and remove 20% of weak strokes of unnecessary lines (such as floor textures) for detail pruning. Only when the main outline accounts for more than or equal to 60% and the reduction rate of the number of S-value strokes is greater than or equal to 20% can complete extraction be guaranteed.

[0133] For generating natural forest sketches: Forest scenes are characterized by complex edges (branches, grass, terrain), large curvature variations, and high requirements for layering, but tolerate some detail loss to improve efficiency. Due to the large number of connected regions, curvature analysis is more complex. Stroke priority S-distribution: main trunks and roof outlines have high S-value stroke counts, while thin branches and grass have low S-value stroke counts. An extraction rate greater than or equal to 90% is required to ensure complete extraction. Because the average number of curve control points per stroke in natural forest sketches is slightly higher (due to fitting curved branches), the curvature square integral energy E-value is higher (because of the many natural curves), but after optimization, it is expected to be controlled within 0.04. The average number of control points is 4 or 5, and the E-value is less than 0.04. The main outline and trunk of the cabin are thickened, accounting for more than 50% of the overall stroke length, and 30% of low-priority thin lines such as grass are pruned to reduce visual clutter and perform detail pruning. Complete extraction can only be guaranteed when the main outline accounts for more than or equal to 50% and the reduction rate of low S-value strokes is greater than or equal to 30%.

[0134] It should be noted that the above is only an optional example and should not be construed as a limitation of the present invention.

[0135] Compared to existing technologies, this invention achieves highly controllable sketching by understanding and parsing various input information, improving the controllability, drawing quality, and editability of generated sketches, thereby realizing an intelligent painting mechanism. Specifically, it employs an intelligent agent architecture to process the painting process in stages, using piecewise Bézier curves for drawing. Combined with AI software, it can progressively analyze the scene, objects, and their stroke layers according to a logical framework. Within this framework, AI software first determines the overall scene layout, then draws the main object structures, and finally fills in the detailed strokes, ensuring that the drawing process is more in line with human drawing habits. Unlike traditional one-step image generation methods, this invention makes the sketch generation process more controllable and enhances the rationality of the overall composition. This invention achieves highly controllable sketching by understanding and parsing various input information, improving the controllability, drawing quality, and editability of generated sketches, thereby realizing an intelligent painting mechanism.

[0136] Furthermore, in terms of stroke generation, this invention employs piecewise Bézier curves for drawing to enhance the smoothness and naturalness of the sketch. Compared to traditional polyline drawing methods, Bézier curves provide smoother curve transitions, making strokes more natural and reducing breaks and harshness. In addition, an intelligent stroke optimization mechanism is introduced, automatically adjusting the detail level based on the curvature, length, and weight of the stroke, making the sketch more layered.

[0137] Example 2

[0138] The following are system embodiments of the present invention, which can be used to execute the method embodiments of the present invention. For details not disclosed in the system embodiments of the present invention, please refer to the method embodiments of the present invention.

[0139] Figure 5 This is a schematic diagram of an example of a controllable scene sketching generation system according to the present invention.

[0140] The following will refer to Figure 5 The present invention describes a controllable scene sketch generation system that uses the controllable scene sketch generation method described in the first aspect of the present invention.

[0141] Specifically, the controllable scene sketch generation system 500 includes an input receiving module 510, a parsing processing module 520, a splitting processing module 530, a layered output module 540, and a generation processing module 550.

[0142] In one specific embodiment, the input receiving module 510 is used to receive input information of the sketch to be drawn, the input information including at least one of the following information data: text segment, hand-drawn sketch, and reference image. The parsing processing module 520 is used to perform scene parsing and content parsing on the input information. When the input information contains a hand-drawn sketch, it identifies connected regions in the sketch and performs curvature analysis and intersection detection to obtain specific contours and key features for dynamic path tracking. When the input information contains a reference image, it extracts object contours, positions, color information, and style information from the reference image. When extracting color information, it uses the K-Means clustering algorithm to analyze the dominant color tone of the reference image. The splitting processing module 530, based on the scene parsing results and content parsing results, splits each target object in the sketch to be drawn into multiple independent brush information and determines the weight of each brush. The layered output module 540 uses piecewise Bézier curves to draw the sketch to be drawn, and outputs the brush path of each brush layered, specifically including: dynamically adjusting the parameters of the Bézier curves according to the shape and curvature of each brush stroke. The generation processing module 550 generates an initial drawing based on user instructions and the drawing path of each pen, or recalculates each pen path to optimize the initial drawing.

[0143] According to an optional implementation, the step of drawing the sketch to be drawn using piecewise Bézier curves and outputting the pen path of each pen stroke in layers includes:

[0144] Bézier curve fitting is used to model the individual strokes of the sketch to be drawn, so that the large model understands the order of the strokes and is converted into a sketch language based on SVG format for extracting key features.

[0145] According to an optional implementation, the method further includes: using piecewise Bézier curves to describe stroke paths, configuring control points, and adjusting the parameter t value to optimize the curve shape; simultaneously, using a prompt board to define the hierarchical structure of strokes, so that strokes with different importance weights are expressed using curves of different precision.

[0146] For a single-segment n-order Bézier curve, the expression is: ,in, Let be the order of the Bézier curve. ,in, These are the minimum and maximum orders, determined by the weights. It is a coefficient that adjusts the influence of the weight, and its value ranges from 0.5 to 0.7, with 0.6 being preferred. It is a coefficient that adjusts the curvature influence, and its value ranges from 0.6 to 0.8, preferably 0.7; The curvature estimates given for the large model are selected from low to high, and from... The values ​​are categorized into high, medium, and low weights, with high weight being the most significant. Medium weight low weight ;P i Let i represent the control point of the i-th Bézier curve, where i is a positive integer.

[0147] According to an optional implementation, when drawing the sketch to be drawn using Bézier curve fitting, the connection between adjacent strokes is detected. When a broken or discontinuous line is determined to be present, the following steps are performed:

[0148] Step S201: Determine two adjacent Bézier curves in the sketch to be drawn, wherein the two adjacent Bézier curves include a first curve and a second curve.

[0149] Step S202: When determining whether a break occurs between the first curve and the second curve in two adjacent Bézier curves, determine whether the difference between the end point of the first curve and the starting point of the second curve is greater than a specified value.

[0150] Step S203: Using the method of adjusting the first and last control points, the first control point and the second control point are moved towards the midpoint of the gap to reduce the gap and correct the first control point and the second control point, thus completing the first correction process.

[0151] Step S204: Determine whether there are discontinuous curves in the sketch to be drawn, calculate the first tangent vector of the endpoint of the first curve and the second tangent vector of the starting point of the second curve, and determine whether the angle formed by the first tangent vector and the second tangent vector is greater than a specific value to determine whether the first curve and the second curve are discontinuous.

[0152] Step S205: Based on the first tangent vector and the second tangent vector, perform secondary modification processing on the discontinuous first curve and second curve.

[0153] According to an optional implementation, the priority of each stroke is represented by the following expression:

[0154]

[0155] Where S represents the priority of the current stroke; The length of the current stroke. This represents the maximum length of all strokes in the sketch to be drawn. This represents the total curvature of the current stroke. This represents the maximum total curvature of all strokes in the sketch to be drawn. and These represent the first and second adjustable coefficients, used to control the relative importance of length and curvature, respectively. Since the outline is more important in a sketch, the influence of length should be given greater weight. In this example, The selection range is 0.6~0.8. The value range is 0.2 to 0.4.

[0156] According to the optional implementation method, the scene elements of each application scenario are determined, specifically including background, environmental layout, and object distribution, to form a scene intelligent agent.

[0157] First, analyze the user's input information to extract key scene information, specifically including city, forest, and indoor space. When determining the scene structure, perform the following layout analysis based on perspective rules and spatial logic:

[0158] Determine the relative positions of multiple spatial elements and ensure that the multiple spatial elements conform to the laws of perspective; when generating natural landscapes, analyze the arrangement of multiple terrain elements.

[0159] According to the optional implementation method, color keywords are extracted from the text information in the input information, hue information is extracted from the reference image, and corresponding colors are matched for each object in the sketch to be drawn based on the extracted color keywords and the extracted hue information.

[0160] According to an optional implementation, a combined format of structured data and visual storage is used, specifically including:

[0161] Drawing information is stored in JSON format, including detailed parameters such as stroke order, position, control points, and pen weight. The final image is stored, rendered, and displayed using SVG. During dynamic drawing or secondary generation, the JSON parameters can be adjusted without modifying the SVG code.

[0162] It should be noted that, due to Figure 5 The controllable scene sketching generation system executes a controllable scene sketching generation method and Figure 1 The controllable scene sketch generation method in the examples is largely the same, therefore, the descriptions of the same parts have been omitted.

[0163] Compared to existing technologies, this invention achieves highly controllable sketching by understanding and parsing various input information, improving the controllability, drawing quality, and editability of generated sketches, thereby realizing an intelligent painting mechanism. Specifically, it employs an intelligent agent architecture to process the painting process in stages, using piecewise Bézier curves for drawing. Combined with AI software, it can progressively analyze the scene, objects, and their stroke layers according to a logical framework. Within this framework, AI software first determines the overall scene layout, then draws the main object structures, and finally fills in the detailed strokes, ensuring that the drawing process is more in line with human drawing habits. Unlike traditional one-step image generation methods, this invention makes the sketch generation process more controllable and enhances the rationality of the overall composition. This invention achieves highly controllable sketching by understanding and parsing various input information, improving the controllability, drawing quality, and editability of generated sketches, thereby realizing an intelligent painting mechanism.

[0164] Furthermore, in terms of stroke generation, this invention employs piecewise Bézier curves for drawing to enhance the smoothness and naturalness of the sketch. Compared to traditional polyline drawing methods, Bézier curves provide smoother curve transitions, making strokes more natural and reducing breaks and harshness. In addition, an intelligent stroke optimization mechanism is introduced, automatically adjusting the detail level based on the curvature, length, and weight of the stroke, making the sketch more layered.

[0165] Example 3

[0166] Figure 6 This is a schematic diagram of an embodiment of an electronic device according to the present invention.

[0167] like Figure 6 As shown, the electronic device is embodied in the form of a general-purpose computing device. There can be one or more processors working collaboratively. This invention also does not preclude distributed processing, meaning that processors can be distributed across different physical devices. The electronic device of this invention is not limited to a single entity, but can also be the sum of multiple physical devices.

[0168] The memory stores a computer-executable program, typically machine-readable code. The computer-readable program can be executed by the processor to enable the electronic device to perform the method of the present invention, or at least some steps of the method.

[0169] The memory includes volatile memory, such as random access memory (RAM) and / or cache memory, and may also be non-volatile memory, such as read-only memory (ROM).

[0170] Optionally, in this embodiment, the electronic device further includes an I / O interface for exchanging data with external devices. The I / O interface can represent one or more of several bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0171] It should be understood that Figure 6 The electronic device shown is merely one example of the present invention, and the electronic device of the present invention may also include elements or components not shown in the above examples. For example, some electronic devices also include display units such as displays, and some electronic devices also include human-computer interaction elements such as buttons and keyboards. Any electronic device capable of executing a computer-readable program in memory to implement the method of the present invention or at least some steps of the method can be considered as an electronic device covered by the present invention.

[0172] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software, or by combining software with necessary hardware. Therefore, as... Figure 7 As shown, the technical solution according to the embodiments of the present invention can be embodied in the form of a software product. The software product can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, mobile hard drive, etc.) or on a network, and includes several commands to cause a computing device (such as a personal computer, server, or network device, etc.) to execute the above-described method according to the embodiments of the present invention.

[0173] The software product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0174] The computer-readable storage medium may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The readable storage medium may also be any readable medium other than a readable storage medium, capable of transmitting, propagating, or transmitting programs for use by or in connection with a command execution system, apparatus, or device. The program code contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0175] Program code for performing the operations of this invention can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0176] The aforementioned computer-readable medium carries one or more programs, which, when executed by a device, enable the computer-readable medium to implement the data interaction method of this disclosure.

[0177] Those skilled in the art will understand that the above modules can be distributed in the device as described in the embodiments, or they can be modified accordingly and placed in one or more devices that are unique to this embodiment. The modules in the above embodiments can be combined into one module, or they can be further divided into multiple sub-modules.

[0178] Through the description of the above embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions of the embodiments of the present invention can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, portable hard drive, etc.) or on a network, including several commands to cause a computing device (such as a personal computer, server, mobile terminal, or network device, etc.) to execute the methods according to the embodiments of the present invention.

[0179] It should be noted that the above detailed descriptions are exemplary and intended to provide further explanation of this application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0180] In the detailed description above, reference has been made to the accompanying drawings, which form part of this document. In the drawings, similar symbols typically identify similar parts unless the context otherwise indicates otherwise. The illustrated embodiments described in the detailed specification, drawings, and claims are not intended to be limiting. Other embodiments may be used and other changes may be made without departing from the spirit or scope of the subject matter presented herein.

[0181] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A controllable scene sketch generation method, characterized by, Comprising: receiving input information of a sketch to be drawn, the input information including at least one of the following information data: a text segment, a hand-drawn sketch, a reference picture; performing scene analysis and content analysis on the input information, when the input information contains a hand-drawn sketch, identifying the connected regions in the hand-drawn sketch, and performing curvature analysis and intersection detection to obtain the outline and key features to obtain dynamic path tracking; when the input information contains a reference picture, extracting object outlines, positions, color information and style information from the reference picture, and using K-Means clustering algorithm to analyze the dominant color of the reference picture when extracting color information; According to the scene analysis result and the content analysis result, the target objects in the sketch to be drawn are divided into multiple independent brush information, and the weight of each brush is determined; using a segmented Bezier curve to draw the sketch to be drawn, and outputting the brush path of each brush in layers, specifically including: dynamically adjusting the parameters of the Bezier curve according to the shape and curvature of each stroke; According to the user instruction, and according to the brush path of each brush, an initial drawing is generated, or the brush path is recalculated to optimize the initial drawing; Wherein, the using a segmented Bezier curve to draw the sketch to be drawn, and outputting the brush path of each brush in layers, includes: using Bezier curve fitting to model each stroke divided from the sketch to be drawn, so that the large model understands the order of each stroke, and is converted into a sketch language based on SVG format for extracting key features, further including: using a segmented Bezier curve to describe the stroke path, setting control points, and adjusting the parameter t value to satisfy the optimization of the curve shape, and using the hint version to define the hierarchical structure of the stroke, so that strokes with different importance weights are expressed with different precision curves, For single segment n order Bezier curve, the expression is: Wherein, n is the order of Bezier curve, Wherein, Respectively, the minimum order, the maximum order, determined by the weight, W is the coefficient of adjusting the influence of weight, the value range is 0.5-0.7; is the coefficient of the curvature influence, the value range is 0.6-0.8; K is the curvature degree estimation given by the large model, selected from low to high, and takes value from [0, 2], the weight is divided into high, medium and low, n min =4, n max =8, n min =3, n max =6, n min =2, n max =4, P i represents the control point of the i-th Bezier curve, i is a positive integer.

2. The controllable scene sketch generation method of claim 1, wherein, Comprising: When drawing the sketch to be drawn using Bezier curve fitting, the connection between adjacent strokes is detected, and when it is determined that there is a broken or incoherent line, the following steps are performed: Step S201: judging two adjacent Bezier curves in the sketch to be drawn, the two adjacent Bezier curves including a first curve and a second curve; Step S202: when judging whether there is a break between the first curve and the second curve in the two adjacent Bezier curves, judging whether the difference between the end point of the first curve and the start point of the second curve is greater than a specified value; Step S203: using the method of adjusting the first and second control points, moving the two control points of the first and second control points to the midpoint of the difference to reduce the difference and correct the first and second control points, that is, complete the first correction processing; Step S204: judging whether there is an incoherent curve in the sketch to be drawn, calculating the first tangent vector of the end point of the first curve and the second tangent vector of the start point of the second curve, and determining whether the included angle formed by the first tangent vector and the second tangent vector is greater than 45 degrees to determine whether the first curve and the second curve are coherent; Step S205: according to the first tangent vector and the second tangent vector, performing secondary modification processing on the incoherent first curve and the second curve.

3. The controllable scene sketch generation method of claim 1, wherein, Comprising: The priority of each stroke is expressed by the following expression: Wherein, S represents the priority of the current stroke; L is the length of the current stroke, L max is the maximum length of all strokes in the sketch to be drawn, R is the total curvature of the current stroke, R max is the maximum total curvature of all strokes in the sketch to be drawn, and respectively represent the first adjustable coefficient and the second adjustable coefficient, respectively, to control the influence proportion of the length and the curvature, The selection range of is 0.6~0.8, The value range of is 0.2~0.

4.

4. The controllable scene sketch generation method of claim 1, wherein, Comprising: Determine the scene elements of each application scenario, including background, environmental layout, and object distribution, to form a scene agent; First, analyze the user's input information and extract key scene information, including city, forest, and indoor space. When determining the scene structure, follow the perspective rules and spatial logic to perform the following layout analysis: Determine the relative positions of multiple spatial elements and ensure that they comply with the perspective rules. When generating natural landscapes, analyze the arrangement of multiple terrain elements.

5. The controllable scene sketch generation method of claim 1, wherein, This includes: Extract color keywords from the text information in the input information, extract color information from the reference pictures, and match the appropriate colors for each object in the sketch to be drawn based on the extracted color keywords and the extracted color information.

6. The controllable scene sketch generation method of claim 1, wherein, This includes: Using a combination of structured data and visual storage formats, including: Storing drawing information in JSON format, including the following detailed parameters: stroke order, position, control points, and brush weight; Using SVG to store the rendered and displayed final image; When dynamically drawing or generating, you can adjust the JSON parameters without modifying the SVG code.

7. A controllable scene sketch generation system, characterized by, The use of the controllable scene sketch generation method of any one of claims 1 to 6, the controllable scene sketch generation system comprises: An input receiving module for receiving input information of a sketch to be drawn, the input information including at least one of the following information data: text segment, hand-drawn sketch, and reference picture; An analysis processing module for scene analysis and content analysis of the input information. When the input information contains a hand-drawn sketch, identify the connected regions in the hand-drawn sketch and perform curvature analysis and intersection detection to obtain the outline and key features for dynamic path tracking. When the input information contains a reference picture, extract object outlines, positions, color information, and style information from the reference picture. When extracting color information, use the K-Means clustering algorithm to analyze the dominant color of the reference picture; A splitting processing module that splits each target object in the sketch to be drawn into multiple independent brush information based on the scene analysis results and content analysis results, and determines the weight of each brush; A hierarchical output module that uses segmented Bezier curves to draw the sketch to be drawn and outputs the brush path of each brush, including dynamically adjusting the parameters of the Bezier curve based on the shape and curvature of each stroke. A generation processing module that generates an initial drawing based on user instructions and the brush path of each brush, or recalculates the brush path to optimize the initial drawing; Wherein, the use of segmented Bezier curves to draw the sketch to be drawn and output the brush path of each brush includes: using Bezier curve fitting to model each stroke split from the sketch to be drawn, so that the large model understands the order of each stroke and converts it into a sketch language based on the SVG format for extracting key features, further including: The stroke path is described by using a segmented Bezier curve, control points are arranged, and the parameter t value is adjusted to satisfy the optimized curve shape, and the hierarchical structure of the stroke is defined by using a hint version, so that strokes with different importance weights are expressed by curves with different precisions, For single segment n order Bezier curve, the expression is: Wherein, n is the order of Bezier curve, Wherein, n min , n max are minimum order, maximum order respectively, determined by weight, W is the coefficient of adjusting weight influence, the value range is 0.5-0.7; is the coefficient of the curvature influence, the value range is 0.6-0.8; K is the curvature degree estimation given by the large model, selected from low to high, and takes value from [0, 2], the weight is divided into high, medium and low, n min = 4, n max = 8, n min = 3, n max = 6, n min = 2, n max = 4, P i represents the control point of the i-th Bezier curve, i is a positive integer.

Images