A method for assisting artistic drawing based on deep learning

Abstract

The application provides a kind of method based on deep learning auxiliary artistic painting, belong to painting intelligent auxiliary technical field, the application is through the construction of covering the stage difference auxiliary framework of whole process of starting, laying color, in-depth and refinement, make the auxiliary strategy of AI evolve with the user creation process synchronization.Starting stage only provides composition guide and perspective correction, color laying stage is automatically switched to color harmony analysis and light reconstruction, in-depth stage is converted into basic modeling enhancement, refinement stage is adjusted to local completion and overall adjustment, the auxiliary function of each stage accurately matches the current needs of the user and does not interfere with each other;At the same time, combined with the intention of brush stroke level prediction completion and adaptive intensity adjustment based on proficiency, ensure that the whole process of auxiliary intensity is moderate, the presentation mode is non-interference, fundamentally solves the core problem that the auxiliary function is disconnected with the creation stage in the prior art, realizes the whole process of adaptive assistance that conforms to the real painting logic, and retains the user's creation leading right throughout the process.

Description

Technical Field

[0001] This invention relates to the field of intelligent assistive technology in painting, specifically to a method for assisting artistic painting based on deep learning. Background Technology

[0002] Painting is an important way of expressing thoughts. However, in the process of painting, especially when using computer painting, most ordinary people cannot express their thoughts and feelings well while also achieving aesthetic appeal due to limitations in technology and equipment.

[0003] Existing digital painting assistance technologies mainly fall into two categories. One is end-to-end AI image generation, where the model directly outputs a complete image after the user inputs text. Essentially, this replaces rather than assists the user, preventing user participation in the drawing process and severely deviating from the progressive creative logic of real painting. The other category consists of single auxiliary tools, such as perspective grids, color recommendations, and brushstroke smoothing. While these can provide some assistance in certain areas, their functions are fragmented and isolated. The main problem is that these technologies completely lack the ability to recognize the stages of the painting creation process. They cannot perceive whether the user is currently in the sketching, color-laying, detailing, or refinement stage, resulting in fixed and unchanging assistance strategies that are misaligned with the user's actual needs. For example, displaying a perspective grid from the sketching stage during the color-laying stage, or providing color-filling suggestions for the color-laying stage during the detailing stage, is not only unhelpful but also visually distracting, contradicting the objective principle that different stages in real painting require different auxiliary methods. Therefore, this paper proposes a method for assisting artistic painting based on deep learning. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides a method for assisting artistic painting based on deep learning. This method solves the core problem of the disconnect between auxiliary functions and the creative stage in existing technologies, achieving full-process adaptive assistance that aligns with the logic of real painting while preserving the user's creative control throughout the process.

[0005] The technical solution adopted in this invention is as follows: A method for assisting art painting based on deep learning includes the following steps: Step S1: Real-time acquisition of canvas image data and user pen stroke operation data to obtain multi-source raw acquisition data; Step S2: Preprocess and extract features from the multi-source raw data to obtain global canvas features and brush stroke behavior features. Global canvas features include coverage, color entropy and edge proportion, while brush stroke behavior features include brush stroke coordinates, pressure value, speed value and temporal information. Step S3: Input the global features of the canvas and the brush stroke behavior features into the pre-trained lightweight fusion judgment model to identify the current painting stage and obtain the stage identification result. The painting stages include the sketching stage, the color laying stage, the deepening stage, and the refinement stage. Step S4: Based on the stage recognition results, match the corresponding intelligent assistance strategy for the stage, and perform auxiliary calculation processing on the current canvas image data to obtain the auxiliary calculation results; Step S5: Overlay the auxiliary calculation results onto the canvas in a non-interference manner as a semi-transparent layer or guide lines to generate auxiliary rendering output; Step S6: During the execution of steps S3 to S5, the migration status of the stage recognition results is monitored in real time. When migration of the painting stage is detected, the intelligent assistance strategy corresponding to the stage after migration is automatically switched, and steps S4 to S5 are repeated.

[0006] Furthermore, in step S3, the specific method for identifying the current painting stage using the lightweight fusion judgment model is as follows: Step S3-1: Input the global features of the canvas into the first decision unit to obtain the first-stage probability distribution based on the overall state of the canvas; Step S3-2: Input the pen stroke behavior features into the second determination unit to obtain the second-stage probability distribution based on user operation behavior; Step S3-3: Combine the first-stage probability distribution and the second-stage probability distribution to obtain the fused stage probability distribution; Step S3-4: Based on the probability distribution of the fused stages, select the stage with the highest probability value as the stage identification result; The lightweight fusion judgment model adopts a sliding window mechanism, which selects a series of consecutive feature sequences within a preset time window centered on the current time for comprehensive judgment, and verifies the confidence of the judgment result through a confidence threshold. When the confidence is lower than the preset threshold, the recognition result of the previous stage is maintained.

[0007] Furthermore, in step S6, the specific method for monitoring the migration status of the identification results in the real-time monitoring stage is as follows: Step S6-1: At each time step, record the current stage identification result and the corresponding confidence value; Step S6-2: Compare the current stage identification result with the previous stage identification result to determine whether there is a stage change; Step S6-3: When a stage change is detected, calculate the average confidence level and the number of consecutive frames of the new stage identification results within a preset number of consecutive frames; Step S6-4: When the average confidence level is greater than the first preset threshold and the number of consecutive frames is greater than the second preset threshold, confirm that the stage transition has occurred and output the stage identifier after the transition. Step S6-5: Based on the post-migration stage identifier, retrieve the corresponding intelligent auxiliary strategy parameters from the auxiliary strategy library to complete the automatic switching of auxiliary strategies.

[0008] Further, in step S4, the intelligent assistance strategy corresponding to the stage is matched and auxiliary calculation processing is performed, specifically including: When the stage recognition result is the drafting stage, the drafting assistance strategy is executed: the composition balance is evaluated on the current canvas image data to generate a composition evaluation score; perspective correction parameters and proportion optimization parameters are calculated based on the composition evaluation score; symmetry auxiliary lines are generated based on the perspective correction parameters and proportion optimization parameters, and the symmetry auxiliary lines are output as auxiliary calculation results. When the stage recognition result is the color laying stage, the color laying auxiliary strategy is executed: perform color harmony analysis on the current canvas image data to generate color distribution analysis data; reconstruct the brightness relationship and adjust the tone according to the color distribution analysis data to obtain the adjusted color mapping matrix, and output the adjusted color mapping matrix as the auxiliary calculation result; When the stage recognition result is the in-depth stage, the in-depth auxiliary strategy is executed: the current canvas image data is subjected to volume enhancement processing, material texture generation processing and light and shadow consistency verification processing to obtain enhanced canvas analysis data, and the enhanced canvas analysis data is output as the auxiliary calculation result; When the stage recognition result is the refinement stage, a refinement auxiliary strategy is executed: local detail completion processing, brush stroke smoothing optimization processing, and overall image harmonization processing are performed on the current canvas image data to obtain the refined canvas data, and the refined canvas data is output as an auxiliary calculation result.

[0009] Furthermore, the method for assisting artistic painting also includes a brushstroke intention prediction and completion step, which is executed in parallel during step S4. The specific method is as follows: Step P1: Real-time acquisition of continuous motion trajectory data of the user's current pen stroke. The continuous motion trajectory data includes trajectory coordinate sequence, pressure sequence, velocity sequence and direction sequence. Step P2: Input the continuous motion trajectory data into the pre-trained time-series trend analysis model to predict the user's complete drawing intention for the current stroke and obtain the stroke trend prediction trajectory. Step P3: Detect whether the current stroke is incomplete. If it is determined to be incomplete, generate a semi-transparent preview completion suggestion based on the stroke trend prediction trajectory. Step P4: Overlay the semi-transparent preview completion suggestions onto the canvas for users to refer to or ignore.

[0010] Furthermore, in step P3, the specific method for detecting whether the current stroke is incomplete is as follows: Step P3-1: Obtain the drawn trajectory segment of the current stroke, and calculate the length and direction change rate of the drawn trajectory segment; Step P3-2: Compare the length and direction change rate of the drawn trajectory segment with the corresponding parameters of the stroke trend prediction trajectory, and calculate the trajectory matching degree; Step P3-3: When the trajectory matching degree is lower than the third preset threshold and the end speed of the drawn trajectory segment is lower than the fourth preset threshold, it is determined that the current stroke is incomplete.

[0011] Furthermore, it also includes an adaptive auxiliary intensity adjustment step, which is performed synchronously during the execution of steps S4 and S5. The specific method is as follows: Step Q1: Acquire the user's historical pen stroke operation data sequence in real time, and extract proficiency assessment features from the historical pen stroke operation data sequence. The proficiency assessment features include pen stroke stability index, line continuity index, and color matching consistency index. Step Q2: Input the proficiency assessment features into the pre-trained proficiency analysis model to obtain the user's current drawing proficiency level; Step Q3: Determine the corresponding auxiliary intervention weight coefficient based on the user's current drawing proficiency level. The auxiliary intervention weight coefficient is used to control the output weight of the intelligent assistance strategy. Step Q4: Weight the auxiliary calculation results using the auxiliary intervention weight coefficients to obtain the modulated auxiliary output; Step Q5: In response to the user's manual adjustment command, correct the auxiliary intervention weight coefficient, and re-execute step Q4 based on the corrected auxiliary intervention weight coefficient.

[0012] Furthermore, in step S1, the specific method for real-time acquisition of canvas image data and user pen stroke operation data is as follows: The pixel matrix data of the canvas area is captured through the graphical user interface and used as canvas image data, with a sampling frequency of N frames per second. The pen touch event stream of the user's input device is captured in real time by the input device driver. The pen touch coordinates, pressure values, speed values ​​and timing information are parsed from the pen touch event stream as user pen touch operation data. The canvas image data and user pen stroke operation data at the same timestamp are time-aligned to form a synchronized multi-source data frame sequence.

[0013] This invention also proposes a system for deep learning-assisted art painting, comprising: The data acquisition module is used to collect canvas image data and user pen stroke operation data in real time to obtain multi-source raw data. The stage recognition module is used to preprocess and extract features from multi-source raw data to obtain global canvas features and brushstroke behavior features. The global canvas features include coverage, color entropy, and edge proportion, while the brushstroke behavior features include brushstroke coordinates, pressure value, speed value, and temporal information. The global canvas features and brushstroke behavior features are then input into a pre-trained lightweight fusion judgment model to identify the current painting stage and obtain the stage recognition result. The painting stages include the sketching stage, the color laying stage, the deepening stage, and the refinement stage. The phased assistance module is used to match the corresponding intelligent assistance strategy based on the phase recognition result, and to perform auxiliary calculation processing on the current canvas image data to obtain the auxiliary calculation result. The interactive presentation module is used to overlay the auxiliary calculation results onto the canvas in a non-interfering manner, generating auxiliary presentation output; The stage migration control module is used to monitor the migration status of the stage recognition results in real time during the execution of the stage recognition module, the stage-by-stage assistance module, and the interactive presentation module. When a migration of the drawing stage is detected, it automatically switches to the intelligent assistance strategy corresponding to the migrated stage and controls the stage-by-stage assistance module and the interactive presentation module to continue execution based on the switched intelligent assistance strategy.

[0014] Furthermore, the phased support modules include: The drafting assistance submodule is used to evaluate the composition balance of the current canvas image data when the stage recognition result is the drafting stage. It calculates the perspective correction parameters and proportion optimization parameters based on the composition evaluation score, and generates symmetry auxiliary lines based on the perspective correction parameters and proportion optimization parameters. The color-layout auxiliary submodule is used to perform color harmony analysis on the current canvas image data when the stage recognition result is the color-layout stage. Based on the color distribution analysis data, it reconstructs the brightness relationship and adjusts the tone uniformity, and outputs the adjusted color mapping matrix. The Deep Auxiliary Submodule is used to perform volume enhancement, material texture generation, and light and shadow consistency verification on the current canvas image data when the stage recognition result is the Deep stage, and outputs the enhanced canvas analysis data. The refinement auxiliary submodule is used to perform local detail completion processing, brush stroke smoothing optimization processing, and overall image harmony processing on the current canvas image data when the stage recognition result is the refinement stage, and output the refined canvas data. The stroke intent prediction submodule is used to acquire the continuous motion trajectory data of the user's current stroke in real time, input the continuous motion trajectory data into the pre-trained time-series trend analysis model, predict the complete drawing intent of the user's current stroke to obtain the stroke trend prediction trajectory, detect whether the current stroke is incomplete, and generate a semi-transparent preview completion suggestion based on the stroke trend prediction trajectory when it is determined to be incomplete. The adaptive intensity adjustment submodule is used to acquire the user's historical brush stroke operation data sequence in real time and extract proficiency assessment features. The proficiency assessment features are input into the pre-trained proficiency analysis model to obtain the user's current drawing proficiency level. The auxiliary intervention weight coefficient is determined based on the user's current drawing proficiency level. The auxiliary intervention weight coefficient is used to weight and modulate the auxiliary calculation results, and the auxiliary intervention weight coefficient is corrected in response to the user's manual adjustment command.

[0015] The beneficial effects of this invention are: This invention constructs a phased, differentiated assistance framework covering the entire process from sketching, color application, deepening, and refinement, allowing the AI's assistance strategy to evolve synchronously with the user's creative progress. The sketching stage only provides composition guidance and perspective correction; the color application stage automatically switches to color harmony analysis and light and shadow reconstruction; the deepening stage shifts to basic shape enhancement; and the refinement stage adjusts to local completion and overall harmony. Each stage's assistance function precisely matches the user's current needs without interfering with one another. Simultaneously, it combines brushstroke-level intention prediction and completion with proficiency-based adaptive intensity adjustment, ensuring moderate assistance intensity and non-intrusive presentation throughout. This fundamentally solves the core problem of existing technologies where assistance functions are disconnected from the creative stage, achieving full-process adaptive assistance that aligns with the logic of real painting while preserving the user's creative control throughout. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating the data acquisition process according to an embodiment of the present invention. Figure 2 This is a flowchart of the stage migration detection and strategy switching according to an embodiment of the present invention; Figure 3 This is an auxiliary calculation flowchart of one embodiment of the present invention; Figure 4 A block diagram illustrating an embodiment of the present invention. Figure 1 ; Figure 5 A block diagram illustrating an embodiment of the present invention. Figure 2 . Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] Example 1 like Figures 1-5 As shown, an embodiment of the present invention provides a system for deep learning-assisted art painting, comprising: The data acquisition module is used to collect canvas image data and user pen stroke operation data in real time to obtain multi-source raw data. The stage recognition module is used to preprocess and extract features from multi-source raw data to obtain global canvas features and brushstroke behavior features. The global canvas features include coverage, color entropy, and edge proportion, while the brushstroke behavior features include brushstroke coordinates, pressure value, speed value, and temporal information. The global canvas features and brushstroke behavior features are then input into a pre-trained lightweight fusion judgment model to identify the current painting stage and obtain the stage recognition result. The painting stages include the sketching stage, the color laying stage, the deepening stage, and the refinement stage. The phased assistance module is used to match the corresponding intelligent assistance strategy based on the phase recognition result, and to perform auxiliary calculation processing on the current canvas image data to obtain the auxiliary calculation result. The interactive presentation module is used to overlay the auxiliary calculation results onto the canvas in a non-interfering manner, generating auxiliary presentation output; The stage migration control module is used to monitor the migration status of the stage recognition results in real time during the execution of the stage recognition module, the stage-by-stage assistance module, and the interactive presentation module. When a migration of the drawing stage is detected, it automatically switches to the intelligent assistance strategy corresponding to the migrated stage and controls the stage-by-stage assistance module and the interactive presentation module to continue execution based on the switched intelligent assistance strategy.

[0019] The system synchronously acquires canvas images and pen stroke data at 30 frames per second. The canvas images are captured as a pixel matrix, and the pen stroke data includes coordinates, pressure, speed, and timing information.

[0020] Three global quantization features are extracted from the canvas image: coverage ,in This represents the number of pixels already drawn. Total number of pixels on the canvas; color entropy ,in To quantify the number of color categories, For the first Frequency of color occurrence; edge proportion ,in The number of edge pixels detected by the Sobel operator. To prevent division by zero of extremely small positive numbers. These three elements constitute the global feature vector of the canvas. Simultaneously, the mean coordinates, mean pressure, mean speed, speed variance, and duration are extracted from the stroke data to construct a stroke behavior feature vector. .

[0021] Two feature vectors are input into a dual-path fully connected network, and each outputs a four-dimensional probability distribution. and This corresponds to the four stages of sketching, color application, in-depth study, and refinement, and is achieved through weighted fusion. ( Preferred , The fusion probability is obtained, and the maximum value corresponds to the stage. This is the recognition result for the current frame.

[0022] To suppress misjudgments in a single frame, a sliding window length is introduced. Frame, frequency of occurrence of each stage within the statistical window (Freq) Only when the highest frequency exceeds the confidence threshold Output the result of this stage when it is ready, otherwise maintain the result of the previous time step.

[0023] After the phase identification stabilizes and outputs, the system monitors the migration in real time, employing triple conditional logic and judgment:

[0024] The first item concerns whether the testing phase has changed; the second item... For the most recent Frame confidence mean, Conf For the first frame The maximum value, The migration confidence threshold; in the third term For the most recent The number of frames in the frame that is determined to be in the new phase. This is the minimum number of consecutive frames required for identification.

[0025] When all three conditions are met simultaneously, the migration is confirmed, and the system hot-swaps from the auxiliary strategy library to the corresponding stage: the sketching stage outputs composition evaluation scores, perspective correction parameters, and symmetry guides; the color laying stage outputs color distribution analysis data and adjusted color mapping matrix; the detailing stage outputs enhanced canvas data; and the refinement stage outputs refined canvas data.

[0026] Parallel execution of stroke intent prediction and completion: Obtain the current stroke trajectory point Calculate the drawn length and the rate of change of direction ,in atan The heading angle is used. The trajectory is input into the LSTM time series model to predict the complete trajectory, and the predicted length is calculated by taking the prefix. and the predicted rate of change of direction Define the matching degree:

[0027] in , This is the weighting coefficient. When Sim < 0.6 and the terminal instantaneous velocity... If the stroke is incomplete when measured in pixels per second, a semi-transparent preview completion is triggered, and it is recommended to display it overlay.

[0028] Adaptive intensity adjustment is performed simultaneously: three proficiency assessment features are extracted from the user's historical strokes—stroke stability (trajectory curvature variance), line coherence (number of breakpoints per unit length), and color matching consistency (degree of deviation from the main color tone). These features are input into a proficiency analysis regression model, and a continuous score is output. Based on this, the auxiliary intervention weighting coefficient is determined. hour , hour , hour , hour .

[0029] Let the correction amount for a certain auxiliary function calculation be... Actual output ,in The original canvas state is used to implement linear interpolation modulation of the auxiliary intensity. Users can also manually adjust the input. Corrected weighting coefficients Clip The system is based on Modulation is re-executed. The modulated auxiliary result is overlaid on the canvas in a semi-transparent, non-interference manner, and the new data generated by the user's continued drawing serves as the input for the next frame, forming a closed loop.

[0030] Example 2 The method for deep learning-assisted art painting based on this embodiment of the invention is as follows: like Figures 1-5 As shown, in implementation, the system acquires pixel matrix data of the canvas area at a fixed sampling frequency through a graphical user interface. The sampling frequency can be set according to the actual application scenario and hardware performance, typically between 15 and 60 frames per second. For the acquisition of user pen stroke data, the system captures event streams from input devices such as styluses, mice, or graphics tablets in real time through the input device driver. Each pen stroke event includes at least the position information of the pen stroke in the canvas coordinate system, the pressure value of the pen tip on the canvas, the instantaneous speed of the pen stroke movement, and the timestamp corresponding to the event. To ensure a strict temporal correspondence between canvas features and pen stroke features in the subsequent fusion judgment model, the system aligns the canvas image data and pen stroke data acquired at the same timestamp to form a synchronized multi-source data frame sequence for subsequent processing.

[0031] After obtaining the aligned multi-source data frame sequence, the system needs to extract quantized features for discrimination. For canvas image data, the system calculates three global features in sequence: coverage, color entropy, and edge proportion.

[0032] Coverage is calculated by comparing the number of pixels in the canvas that differ from the initial background color to the total number of pixels in the canvas. Coverage is zero when the canvas is completely blank, and this value gradually increases as drawing progresses. Color entropy is calculated by first quantizing the canvas image to reduce computational complexity, then counting the frequency of each color category in the drawn area, and substituting this frequency into the definition of entropy. This value reflects the uniformity and richness of the color distribution on the canvas. Edge proportion is calculated by detecting edge pixels in the canvas using gradient operators and comparing their proportion to the total number of drawn pixels. Gradient operators can include the Sobel operator, Canny operator, or other equivalent edge detection algorithms.

[0033] For pen stroke operation data, the system extracts the statistical characteristics of pen stroke coordinates within the current time window, the mean and fluctuation range of pressure, the mean and fluctuation range of speed, and the duration of the current pen stroke from the moment it is written to the current moment, as well as other behavioral characteristics.

[0034] After obtaining the above features, the system uses a lightweight decision network in dual parallel paths for processing. One path processes the global features of the canvas, and the other path processes the brush stroke behavior features. Each of the two paths outputs a probability distribution for the four painting stages. The two outputs are then combined according to preset weights to obtain a fused probability distribution. The stage corresponding to the highest probability is taken as the stage determination result of the current frame.

[0035] To eliminate potential noise disturbances in single-frame decision-making, the system introduces a sliding window mechanism. The frequency of each stage is counted in the decision-making sequence of multiple consecutive frames. When the frequency of a certain stage exceeds a preset confidence threshold, it is taken as a stable output result; otherwise, the result of the previous moment is used, thereby achieving temporal smoothing of the decision-making results.

[0036] During the continuous output of stage recognition results, the system needs to determine in real time whether a substantial transfer has occurred during the drawing stage. In practice, the system maintains a buffer that records the stage recognition results of the most recent few frames and their corresponding confidence values. The confidence value, which is the maximum value in the fused probability distribution, represents the model's degree of certainty regarding the current judgment result.

[0037] When the system detects that the current stage category is different from the previous stage, it does not immediately trigger the migration action, but instead enters the migration confirmation process: the system counts the number of frames in the recent frames where the new stage has appeared and the average confidence level of these frames. Only when the frequency of the new stage is high enough and the average confidence level is large enough will the migration be confirmed to have actually occurred.

[0038] The system incorporates differentiated auxiliary calculation logic for each of the four painting stages. In the initial sketching stage, the system performs a compositional balance assessment of the current canvas. Specifically, it detects the distribution center of gravity of the drawn lines and shapes in the image, compares it with the geometric center of the canvas, generates a compositional assessment score based on the direction and degree of offset, and calculates perspective correction parameters and proportion optimization parameters accordingly. Subsequently, it generates corresponding symmetry auxiliary lines and perspective auxiliary grids on the canvas.

[0039] During the color application stage, the system performs a harmony analysis on the color distribution of the canvas. Specifically, it extracts the main hues and their distribution areas in the canvas, detects whether there are isolated color blocks that conflict with the main color tone or abrupt areas that lack transition, and calculates a color adjustment mapping matrix based on the analysis results to reconstruct and unify the light and dark relationships and hues.

[0040] While the user is drawing a single stroke, the system performs stroke intent prediction and completion in parallel in the background. During implementation, the system acquires continuous trajectory data of the current stroke from the moment it is first drawn in real time at a sampling rate higher than the canvas acquisition frequency. This includes the coordinate sequence of trajectory points, the pressure value, velocity, and direction of movement at each point. The drawn trajectory point sequence is then input into a pre-trained temporal analysis model. This model, employing a recurrent neural network or its variants, is trained on a large dataset of strokes with complete geometric shapes and is capable of predicting the most likely complete trajectory shape based on the first half of the stroke's trajectory.

[0041] At the same time, the system makes a real-time determination of the completeness of the current pen stroke. Specifically, it calculates the length and direction change rate of the drawn trajectory and compares them with the length and direction change rate of the corresponding prefix of the predicted trajectory. When the two deviate significantly and the movement speed of the pen stroke end decreases significantly, the current pen stroke is determined to be incomplete.

[0042] During the phased assisted calculation and stroke intent prediction completion process, the system simultaneously performs adaptive adjustment of the assisted intensity. During implementation, the system continuously collects historical stroke data completed by the user, extracting quantitative indicators that reflect the user's drawing proficiency, including stroke stability, line coherence, and color consistency.

[0043] Pen stroke stability is obtained by statistical analysis of the curvature variance of a large number of pen stroke trajectories. The smaller the variance, the more stable the wrist control when the user draws curves. Line continuity is obtained by statistical analysis of the number of pen stroke breaks or lifting points per unit length. The fewer the breaks, the more smoothly the user writes the pen. Color matching consistency is obtained by analyzing the degree of deviation between the user's selected color and the main color tone of the picture. The smaller the deviation, the better the user's control over the overall color relationship.

[0044] The above indicators constitute a multi-dimensional feature vector input proficiency analysis model, which outputs a continuous score representing the user's overall proficiency level. Based on the range of this score, the system determines the applicable auxiliary intervention weight coefficient. The lower the score, the larger the weight coefficient, indicating that the system provides stronger auxiliary guidance; the higher the score, the smaller the weight coefficient, indicating that the system only provides weaker reference prompts, avoiding interference with the independent creation of professional users.

[0045] After determining the weighting coefficients, the system applies them to all auxiliary corrections obtained from the phased auxiliary calculations, and presents each auxiliary output to the user after weighted modulation. Furthermore, the system provides a manual adjustment entry in the user interface, allowing users to manually adjust the weighting coefficients at any time. The system then re-executes the weighted modulation based on the adjusted coefficients, combining automatic evaluation with user subjective preferences.

[0046] The above steps are continuously and iteratively executed during system operation. The data acquisition module continuously generates multi-source data frames at a fixed frequency. The stage identification module extracts features and determines the stage for each frame. The stage migration control module monitors the temporal changes of the determination results in real time and triggers strategy switching when migration occurs. The stage-based assistance module continuously calculates the assistance correction amount according to the current strategy. The stroke intent prediction module provides completion suggestions in parallel during each stroke drawing process. The adaptive intensity adjustment module dynamically updates the weight coefficients based on accumulated historical data.

[0047] The modules communicate with each other through standardized data interfaces. The output of the auxiliary calculations is modulated and overlaid on the canvas as a semi-transparent layer or guide lines. The new data generated by the user's continued drawing under the guidance of the system serves as the input for the next round of processing, forming a complete data loop. Throughout the process, every drawing operation by the user participates in feature calculation and auxiliary decision-making in real time. The system's auxiliary output evolves continuously with the user's operation and changes in the canvas state until the user completes the creation.

[0048] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Furthermore, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0049] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of the invention pertain.

[0050] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0051] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0052] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0053] Furthermore, the functional units in the various embodiments of the present invention can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0054] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for assisting artistic painting based on deep learning, characterized in that, Includes the following steps: Step S1: Real-time acquisition of canvas image data and user pen stroke operation data to obtain multi-source raw acquisition data; Step S2: Preprocess and extract features from the multi-source raw data to obtain global canvas features and brush stroke behavior features. Global canvas features include coverage, color entropy and edge proportion, while brush stroke behavior features include brush stroke coordinates, pressure value, speed value and temporal information. Step S3: Input the global features of the canvas and the brush stroke behavior features into the pre-trained lightweight fusion judgment model to identify the current painting stage and obtain the stage identification result. The painting stages include the sketching stage, the color laying stage, the deepening stage, and the refinement stage. Step S4: Based on the stage recognition results, match the corresponding intelligent assistance strategy for the stage, and perform auxiliary calculation processing on the current canvas image data to obtain the auxiliary calculation results; Step S5: Overlay the auxiliary calculation results onto the canvas in a non-interference manner as a semi-transparent layer or guide lines to generate auxiliary rendering output; Step S6: During the execution of steps S3 to S5, the migration status of the stage recognition results is monitored in real time. When migration of the painting stage is detected, the intelligent assistance strategy corresponding to the stage after migration is automatically switched, and steps S4 to S5 are repeated.

2. The method for deep learning-assisted art painting according to claim 1, characterized in that, In step S3, the specific method for identifying the current painting stage using the lightweight fusion judgment model is as follows: Step S3-1: Input the global features of the canvas into the first decision unit to obtain the first-stage probability distribution based on the overall state of the canvas; Step S3-2: Input the pen stroke behavior features into the second determination unit to obtain the second-stage probability distribution based on user operation behavior; Step S3-3: Combine the first-stage probability distribution and the second-stage probability distribution to obtain the fused stage probability distribution; Step S3-4: Based on the probability distribution of the fused stages, select the stage with the highest probability value as the stage identification result; The lightweight fusion judgment model adopts a sliding window mechanism, which selects a series of consecutive feature sequences within a preset time window centered on the current time for comprehensive judgment, and verifies the confidence of the judgment result through a confidence threshold. When the confidence is lower than the preset threshold, the recognition result of the previous stage is maintained.

3. The method for deep learning-assisted art painting according to claim 2, characterized in that, In step S6, the specific method for monitoring the migration status of the identification results in the real-time monitoring stage is as follows: Step S6-1: At each time step, record the current stage identification result and the corresponding confidence value; Step S6-2: Compare the current stage identification result with the previous stage identification result to determine whether there is a stage change; Step S6-3: When a stage change is detected, calculate the average confidence level and the number of consecutive frames of the new stage identification results within a preset number of consecutive frames; Step S6-4: When the average confidence level is greater than the first preset threshold and the number of consecutive frames is greater than the second preset threshold, confirm that the stage transition has occurred and output the stage identifier after the transition. Step S6-5: Based on the post-migration stage identifier, retrieve the corresponding intelligent auxiliary strategy parameters from the auxiliary strategy library to complete the automatic switching of auxiliary strategies.

4. The method for deep learning-assisted art painting according to claim 3, characterized in that, In step S4, the intelligent assistance strategy corresponding to the stage is matched and auxiliary calculation processing is performed, specifically including: When the stage recognition result is the drafting stage, the drafting assistance strategy is executed: the composition balance is evaluated on the current canvas image data to generate a composition evaluation score; perspective correction parameters and proportion optimization parameters are calculated based on the composition evaluation score; symmetry auxiliary lines are generated based on the perspective correction parameters and proportion optimization parameters, and the symmetry auxiliary lines are output as auxiliary calculation results. When the stage recognition result is the color laying stage, the color laying auxiliary strategy is executed: perform color harmony analysis on the current canvas image data to generate color distribution analysis data; reconstruct the brightness relationship and adjust the tone according to the color distribution analysis data to obtain the adjusted color mapping matrix, and output the adjusted color mapping matrix as the auxiliary calculation result; When the stage recognition result is the in-depth stage, the in-depth auxiliary strategy is executed: the current canvas image data is subjected to volume enhancement processing, material texture generation processing and light and shadow consistency verification processing to obtain enhanced canvas analysis data, and the enhanced canvas analysis data is output as the auxiliary calculation result; When the stage recognition result is the refinement stage, a refinement auxiliary strategy is executed: local detail completion processing, brush stroke smoothing optimization processing, and overall image harmonization processing are performed on the current canvas image data to obtain the refined canvas data, and the refined canvas data is output as an auxiliary calculation result.

5. The method for deep learning-assisted art painting according to claim 4, characterized in that, It also includes a stroke intent prediction and completion step, which is executed in parallel during step S4. The specific method is as follows: Step P1: Real-time acquisition of continuous motion trajectory data of the user's current pen stroke. The continuous motion trajectory data includes trajectory coordinate sequence, pressure sequence, velocity sequence and direction sequence. Step P2: Input the continuous motion trajectory data into the pre-trained time-series trend analysis model to predict the user's complete drawing intention for the current stroke and obtain the stroke trend prediction trajectory. Step P3: Detect whether the current stroke is incomplete. If it is determined to be incomplete, generate a semi-transparent preview completion suggestion based on the stroke trend prediction trajectory. Step P4: Overlay the semi-transparent preview completion suggestions onto the canvas for users to refer to or ignore.

6. The method for deep learning-assisted art painting according to claim 5, characterized in that, In step P3, the specific method for detecting whether the current stroke is incomplete is as follows: Step P3-1: Obtain the drawn trajectory segment of the current stroke, and calculate the length and direction change rate of the drawn trajectory segment; Step P3-2: Compare the length and direction change rate of the drawn trajectory segment with the corresponding parameters of the stroke trend prediction trajectory, and calculate the trajectory matching degree; Step P3-3: When the trajectory matching degree is lower than the third preset threshold and the end speed of the drawn trajectory segment is lower than the fourth preset threshold, it is determined that the current stroke is incomplete.

7. The method for deep learning-assisted art painting according to claim 4, characterized in that, It also includes an adaptive auxiliary intensity adjustment step, which is performed synchronously during the execution of steps S4 and S5. The specific method is as follows: Step Q1: Acquire the user's historical pen stroke operation data sequence in real time, and extract proficiency assessment features from the historical pen stroke operation data sequence. The proficiency assessment features include pen stroke stability index, line continuity index, and color matching consistency index. Step Q2: Input the proficiency assessment features into the pre-trained proficiency analysis model to obtain the user's current drawing proficiency level; Step Q3: Determine the corresponding auxiliary intervention weight coefficient based on the user's current drawing proficiency level. The auxiliary intervention weight coefficient is used to control the output weight of the intelligent assistance strategy. Step Q4: Weight the auxiliary calculation results using the auxiliary intervention weight coefficients to obtain the modulated auxiliary output; Step Q5: In response to the user's manual adjustment command, correct the auxiliary intervention weight coefficient, and re-execute step Q4 based on the corrected auxiliary intervention weight coefficient.

8. The method for deep learning-assisted art painting according to any one of claims 1 to 7, characterized in that, In step S1, the specific method for real-time acquisition of canvas image data and user pen stroke operation data is as follows: The pixel matrix data of the canvas area is captured through the graphical user interface and used as canvas image data, with a sampling frequency of N frames per second. The pen touch event stream of the user's input device is captured in real time by the input device driver. The pen touch coordinates, pressure values, speed values ​​and timing information are parsed from the pen touch event stream as user pen touch operation data. The canvas image data and user pen stroke operation data at the same timestamp are time-aligned to form a synchronized multi-source data frame sequence.

9. A system for assisting artistic painting based on deep learning, characterized in that, include: The data acquisition module is used to collect canvas image data and user pen stroke operation data in real time to obtain multi-source raw data. The stage recognition module is used to preprocess and extract features from multi-source raw data to obtain global canvas features and brushstroke behavior features. The global canvas features include coverage, color entropy, and edge proportion, while the brushstroke behavior features include brushstroke coordinates, pressure value, speed value, and temporal information. The global canvas features and brushstroke behavior features are then input into a pre-trained lightweight fusion judgment model to identify the current painting stage and obtain the stage recognition result. The painting stages include the sketching stage, the color laying stage, the deepening stage, and the refinement stage. The phased assistance module is used to match the corresponding intelligent assistance strategy based on the phase recognition result, and to perform auxiliary calculation processing on the current canvas image data to obtain the auxiliary calculation result. The interactive presentation module is used to overlay the auxiliary calculation results onto the canvas in a non-interfering manner, generating auxiliary presentation output; The stage migration control module is used to monitor the migration status of the stage recognition results in real time during the execution of the stage recognition module, the stage-by-stage assistance module, and the interactive presentation module. When a migration of the drawing stage is detected, it automatically switches to the intelligent assistance strategy corresponding to the migrated stage and controls the stage-by-stage assistance module and the interactive presentation module to continue execution based on the switched intelligent assistance strategy.

10. The system for deep learning-assisted art painting according to claim 9, characterized in that, The phased support module includes: The drafting assistance submodule is used to evaluate the composition balance of the current canvas image data when the stage recognition result is the drafting stage. It calculates the perspective correction parameters and proportion optimization parameters based on the composition evaluation score, and generates symmetry auxiliary lines based on the perspective correction parameters and proportion optimization parameters. The color-layout auxiliary submodule is used to perform color harmony analysis on the current canvas image data when the stage recognition result is the color-layout stage. Based on the color distribution analysis data, it reconstructs the brightness relationship and adjusts the tone uniformity, and outputs the adjusted color mapping matrix. The Deep Auxiliary Submodule is used to perform volume enhancement, material texture generation, and light and shadow consistency verification on the current canvas image data when the stage recognition result is the Deep stage, and outputs the enhanced canvas analysis data. The refinement auxiliary submodule is used to perform local detail completion processing, brush stroke smoothing optimization processing, and overall image harmony processing on the current canvas image data when the stage recognition result is the refinement stage, and output the refined canvas data. The stroke intent prediction submodule is used to acquire the continuous motion trajectory data of the user's current stroke in real time, input the continuous motion trajectory data into the pre-trained time-series trend analysis model, predict the complete drawing intent of the user's current stroke to obtain the stroke trend prediction trajectory, detect whether the current stroke is incomplete, and generate a semi-transparent preview completion suggestion based on the stroke trend prediction trajectory when it is determined to be incomplete. The adaptive intensity adjustment submodule is used to acquire the user's historical brush stroke operation data sequence in real time and extract proficiency assessment features. The proficiency assessment features are input into the pre-trained proficiency analysis model to obtain the user's current drawing proficiency level. The auxiliary intervention weight coefficient is determined based on the user's current drawing proficiency level. The auxiliary intervention weight coefficient is used to weight and modulate the auxiliary calculation results, and the auxiliary intervention weight coefficient is corrected in response to the user's manual adjustment command.

Images