AR-based intelligent auxiliary drawing system and method thereof

Abstract

The invention discloses an intelligent auxiliary drawing system and method based on AR, and the system combines an augmented reality technology and a real drawing environment, and provides real-time creation feedback and guidance for a user through a visual interaction module, an auditory interaction module, a touch interaction module and a real-time tutoring module. A user can create in a real drawing environment, and the AR technology displays virtual creation suggestions and auxiliary information (such as stroke demonstration, color matching suggestions, composition guidance and the like) to perform real-time superposition with the actual creation process of the user. And the AI analysis module provides personalized tutoring suggestions according to the drawing progress of the user, and enhances the learning experience through multi-sensory feedback. The system not only provides accurate art creation guidance, but also effectively improves the creation expressive force of the user and the efficiency of art learning.

Description

Technical Field

[0001] The present invention relates to the field of augmented reality (AR) technology and art education, and in particular to an AR-based intelligent auxiliary painting system and method thereof. Background Art

[0002] The development of augmented reality (AR) technology has brought revolutionary changes to traditional art creation and education. With the continuous advancement of technology, AR technology has been able to combine virtual information with the real environment to bring an immersive experience to users. The development of this technology, especially in the field of education, has provided new tools and methods for traditional art teaching, making digital transformation possible.

[0003] At present, the application of AR technology in the education industry has covered multiple disciplines, including geography, physics, chemistry, etc., and has enhanced students' spatial thinking, imagination and hands-on ability through AR sandbox, optical experiments and mechanical experiments. In the field of art education, although there are digital painting tools and virtual environments, these tools still cannot fully simulate the immersive interactive experience in the real environment, and most of them are aimed at users with a certain foundation, lacking guidance functions for zero-based learners. Especially in traditional painting learning, it is impossible to effectively solve the problems of high learning threshold, lack of guidance and insufficient user experience.

[0004] Therefore, it is urgent to develop a system that combines the real painting environment with AR technology. The system can use AR devices to provide students with personalized, real-time creative suggestions and feedback, thereby enhancing the immersiveness of creation and learning effects. Summary of the invention

[0005] In order to solve the above problems in the prior art, the present invention provides an AR-based intelligent assisted painting system and method thereof, which provides users with real-time, immersive creative feedback and guidance based on a visual interaction module, an auditory interaction module, a tactile interaction module and a real-time coaching module by combining AR technology with a real painting environment. The system includes:

[0006] The visual interaction module is used to obtain the user's visual data in the creation scene and provide real-time guidance from the visual feedback;

[0007] Auditory interaction module, used to obtain user voice data in the creation scene and provide auditory feedback on user questions and other auxiliary information;

[0008] The tactile interaction module is used to obtain the user's tactile data in the creation scene and provide real-time feedback on the brush operation information through tactile sense;

[0009] The real-time assistance module is used to analyze the user's painting behavior in real time and provide personalized improvement suggestions or practice techniques in real time.

[0010] The above system fully obtains user data and provides real-time feedback through AR technology, so that users can obtain immersive real-time feedback and guidance during the creation process. The cleverness of the present invention lies in that it fully implements the data interaction between the environment, users, and wearable equipment, and provides real-time guidance suggestions based on the AI ​​model.

[0011] As a further preferred embodiment of the AR-based intelligent assisted painting system and method.

[0012] Preferably, the visual interaction module mainly includes AR glasses, a data processing device, a sensor device, and a communication device;

[0013] The AR glasses are configured such that when a user wears the AR glasses and creates in a real painting environment, the AR glasses capture the user's painting movements, brushstroke positions and details through a camera, and the AR glasses also display virtual creation guidance information in real time, and superimpose the virtual painting suggestions with the actual creation process on the canvas;

[0014] The data processing device is configured to identify and analyze the visual data collected by the AR glasses. The specific method includes the following steps:

[0015] Step S100, pre-processing the image, first converting the color image captured by the camera into a gray image to reduce the amount of calculation, and then using Gaussian filtering and median filtering to denoise the image;

[0016] Step S200, performing feature point detection, using Harris corner detector, Canny edge detector and SIFT algorithm to detect corner points, edges and scale information respectively;

[0017] Step S300, generating a description vector for each detected feature point, which can uniquely describe the feature of the point and maintain scale and rotation invariance;

[0018] Step S400, matching feature descriptors of different images to find nearest neighbor matches, and using ratio test to exclude wrong matches to improve matching accuracy;

[0019] Step S500, performing posture estimation using matched feature points to determine the position and direction of the AR glasses relative to the recognition image;

[0020] Step S600, integrating the real world data into the virtual environment according to the results of feature point matching and posture estimation;

[0021] Step S700, tracking feature points in a dynamic environment, and using a RANSAC optimization algorithm to improve tracking stability and accuracy;

[0022] The sensor device is configured for spatial positioning and calibration to ensure synchronization of the AR glasses with the real creation environment and other modules;

[0023] The communication device is configured to communicate with other modules wirelessly and transmit data in real time.

[0024] Preferably, the auditory interaction module mainly includes a microphone, an earphone and a data processing device, wherein the microphone is mainly used to obtain the user's voice data, that is, the user's voice command, and the system's built-in voice assistant answers and feeds back the user's voice command through the earphone;

[0025] The data processing device is configured to process and recognize the acquired voice data, and mainly comprises the following steps:

[0026] Step A100, using a Gaussian filter to remove background noise, improve the quality of the voice signal, and automatically adjust the volume to ensure the consistency of the voice signal;

[0027] Step A200, extracting key features from the voice signal, matching the extracted features with a preset voice command template, and recognizing the user's voice command;

[0028] Step A300, parsing the recognized voice command, determining the user's intention, and performing corresponding operations or services according to the user's command;

[0029] Step A400, converting the system's answer or feedback into speech, playing the synthesized speech through headphones, and providing feedback to the user;

[0030] Preferably, the tactile interaction module mainly includes a tactile sensing device, a data processing device, and a communication device;

[0031] The tactile sensing device is configured as a smart paintbrush and drawing board, and is used to obtain the user's tactile data, capture the position and movement trajectory of the paintbrush on the drawing board through the built-in sensor; sense the painting force, monitor the user's line thickness and force control; through force feedback technology, through vibration or resistance simulation function, provide tactile reminders when the user deviates from the recommended path;

[0032] The data processing device is configured to process and identify the sensed tactile data in real time, and mainly includes the following steps:

[0033] Step B100, removing unnecessary noise and interference through filtering to improve data accuracy, and converting the captured tactile data into recognizable input commands, including the thickness of the stroke or the depth of color, etc.;

[0034] Step B200, calculating the required force feedback strength according to the user's touch pressure and movement speed, generating a corresponding force feedback signal, and simulating the real brushstroke painting feeling;

[0035] Step B300, converting the calculated force feedback signal into tactile feedback perceptible to the user through force feedback technology, and dynamically adjusting the strength of the force feedback according to the user's preference and actual experience;

[0036] The communication device is configured to connect to the visual interaction module in real time via Bluetooth or Wi-Fi, synchronize data, and transmit brush trajectory and force data.

[0037] Preferably, the core of the real-time auxiliary module is an AI intelligent analysis algorithm, which analyzes the user's painting behavior through AI, and automatically provides personalized improvement suggestions and automatically recommends suitable creation tasks or skill improvement exercises according to the user's creation progress and painting style. The module operation mainly includes the following steps:

[0038] Step C100, loading the pre-trained AI model, ensuring that the algorithm is ready, initializing the user's visual, auditory, and tactile interaction modules, and preparing to record the user's painting behavior;

[0039] Step C200, collecting the user's painting data in real time through sensors or input devices, including visual, auditory, tactile data, etc., and storing the collected data in a user database for subsequent analysis;

[0040] Step C300, extracting key features from the collected painting data, such as brushstroke patterns, color usage habits, etc., and using an AI model to analyze the user's painting style and behavior patterns;

[0041] Step C400, based on the user's creative progress and painting style, evaluate the completion of the current work and generate personalized improvement suggestions, analyze the user's strengths and weaknesses in painting, design a task-based learning model based on the user's skill analysis results, and provide practice progress records and learning reports;

[0042] Step C500, presenting improvement suggestions and task recommendations to the user, and collecting user feedback on the suggestions and tasks for optimizing subsequent suggestions;

[0043] Step C600, continuously training and optimizing the AI ​​model based on user feedback and new painting data, and adjusting the suggestion generation algorithm based on the model optimization results to improve the accuracy and relevance of the suggestions;

[0044] A second aspect of the present invention provides an AR-based intelligent assisted painting method, comprising the following steps:

[0045] Step E100, system initialization and calibration, including setting the initial position and initial state of each module;

[0046] Step E200, drawing mode selection, including 3D model mode, picture copying mode, and still life sketching mode;

[0047] Step E300, real-time painting guidance, including visual-based dynamic step-by-step prompts, tactile-based smart brush interaction support, and auditory-based voice interaction;

[0048] Step E400, painting completion and optimization, the system intelligently analyzes the user's completed work and provides optimization suggestions.

[0049] Beneficial effects of the present invention:

[0050] (1) AR application in real painting environment: AR technology is used to superimpose virtual creation guidance information and feedback information in the real environment, thus achieving an immersive learning experience of artistic creation.

[0051] (2) Real-time personalized creative guidance: The system uses AR technology to analyze the user's painting data in real time and provide personalized creative suggestions to promote the user's skill improvement.

[0052] (3) Immersive multimodal interactive learning experience: Combining visual interaction, auditory interaction, touch interaction and other technologies to provide users with a full range of learning experience and enhance the fun and effectiveness of creation.

[0053] (4) The system breaks down painting learning into multiple levels through task module design. According to the user's learning progress and habits, the system dynamically adjusts the teaching content to provide a personalized learning experience.

[0054] (5) Innovative painting experience that combines tradition and intelligence: The smart brush retains the feel of traditional painting and combines digital technology to achieve intelligent guidance, allowing users to experience the efficiency improvement brought by technology in a familiar painting method. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0056] Figure 1 It is a flow chart of an AR-based intelligent assisted painting system and method thereof of the present invention;

[0057] Figure 2 It is a schematic diagram of the framework of an AR-based intelligent assisted painting system of the present invention;

[0058] Figure 3It is a schematic diagram of corner point detection, scale transformation and feature point matching in a visual interaction module of an AR-based intelligent assisted painting system of the present invention;

[0059] Figure 4 It is a flow chart of a voice interaction module system of an AR-based intelligent assisted painting system of the present invention;

[0060] Figure 5 It is a system flow chart of a real-time tutoring module of an AR-based intelligent auxiliary painting system of the present invention; DETAILED DESCRIPTION

[0061] The present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the relevant invention, rather than to limit the invention. It is also necessary to explain that, for ease of description, only the parts related to the relevant invention are shown in the accompanying drawings.

[0062] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0063] The present invention provides an AR-based intelligent assisted painting system and method thereof, comprising the following contents:

[0064] The visual interaction module is used to obtain the user's visual data in the creation scene and provide real-time guidance from the visual feedback;

[0065] Auditory interaction module, used to obtain user voice data in the creation scene and provide auditory feedback on user questions and other auxiliary information;

[0066] The tactile interaction module is used to obtain the user's tactile data in the creation scene and provide real-time feedback on the brush operation information through tactile sense;

[0067] The real-time assistance module is used to analyze the user's painting behavior in real time and provide personalized improvement suggestions or practice techniques in real time.

[0068] In order to more clearly explain the AR-based intelligent assisted painting system and method of the present invention, the following is combined with Figure 1 Each step in the embodiment of the present invention is described in detail.

[0069] The steps of the AR-based intelligent assisted painting method of the first embodiment of the present invention are described in detail as follows:

[0070] Step E100, system initialization and calibration, including setting the initial position and initial state of each module;

[0071] The user wears AR glasses, headphones, and a microphone, uses a smart brush, and selects the type of drawing board (real or virtual).

[0072] The AR glasses use the camera to identify the position of the drawing board and synchronize it with the initial state of the brush;

[0073] The brush transmits the initial position data to the system via wireless transmission;

[0074] Load pre-trained AI analysis models.

[0075] Step E200, drawing mode selection, including 3D mode, picture copying mode, and still life sketching mode;

[0076] The three-dimensional model mode is that the system provides preset three-dimensional painting objects (such as apples and vases) and breaks them down into multiple steps for guidance;

[0077] The picture copying mode is that the user uploads a picture, and the system analyzes the outline, light and shadow, and structure of the picture and generates drawing prompts;

[0078] The still life sketching mode is that the user scans the real still life through the AR glasses camera, and the system generates guidance information in real time.

[0079] Step E300, real-time painting guidance, including visual-based dynamic step-by-step prompts, tactile-based smart brush interaction support, and auditory-based voice interaction;

[0080] The dynamic step-by-step prompt based on vision is that the system displays the current painting step in the AR glasses, including: structural line drawing, that is, displaying the outline or key area; light and shadow expression, that is, prompting the light and shadow distribution through dynamic highlighting; color analysis, that is, prompting the relevant information of hue, brightness, and purity based on the PCCS color system. After the user completes the current step, he confirms through voice or gesture and proceeds to the next step.

[0081] The color analysis comprises the following steps:

[0082] Step E310, color space conversion, that is, converting the image from the RGB color space to other color spaces (such as HSV, Lab, etc.) for better color analysis;

[0083] Step E320, color quantization, i.e., quantizing the image so that only a limited number of color intervals are retained in the image;

[0084] Step E330, histogram calculation, that is, calculating the frequency distribution of each color in the image to form a color histogram;

[0085] Step E340, threshold setting and segmentation, i.e. setting a threshold and segmenting the image into different regions according to a color histogram;

[0086] Step E350, morphological operation, that is, using morphological operation to remove noise and optimize the segmentation result;

[0087] Step E360, feature extraction, extracts color features, such as hue, brightness, purity, etc., to facilitate further analysis.

[0088] Step E400, painting completion and optimization, the system intelligently analyzes the user's completed work and provides optimization suggestions.

[0089] The system performs intelligent analysis on the user's completed works, including proportion analysis, which is to check whether the painting content meets the target proportion; light and shadow analysis, which is to detect the layering and detail expression of light and shadow processing; line analysis, which is to evaluate the smoothness and accuracy of lines; color analysis, which is to evaluate the hue, brightness and purity of the work.

[0090] The provision of optimization suggestions includes the system generating an example of an optimized work and demonstrating the improvement process step by step through AR glasses; the user can redraw the optimized part with a brush.

[0091] A second embodiment of the present invention is an AR-based intelligent assisted painting system, such as Figure 2 As shown, it includes: visual interaction module, voice interaction module, tactile interaction module, and real-time tutoring module;

[0092] The visual interaction module mainly includes AR glasses, a data processing device, a sensor device, and a communication device;

[0093] The AR glasses are configured such that when a user wears the AR glasses and creates in a real painting environment, the AR glasses capture the user's painting movements, brushstroke positions and details through a camera, and the AR glasses also display virtual creation guidance information in real time, such as brushstroke demonstrations, composition frameworks, color matching suggestions, etc., and superimpose the virtual painting suggestions with the actual creation process on the canvas;

[0094] The data processing device is configured to identify and analyze the visual data collected by the AR glasses. The specific method includes the following steps:

[0095] Step S100, pre-processing the image, first converting the color image captured by the camera into a gray image to reduce the amount of calculation, and then using Gaussian filtering and median filtering to denoise the image;

[0096] Step S200, feature point detection is performed, using Harris corner detector, Canny edge detector and SIFT algorithm to detect corners, edges and scale information respectively. Harris corners and SIFT scale information are as follows: Figure 3 As shown;

[0097] Step S300, generating a description vector for each detected feature point, which can uniquely describe the feature of the point and maintain scale and rotation invariance;

[0098] Step S400, matching feature descriptors of different images to find nearest neighbor matches, and using ratio test to exclude wrong matches to improve matching accuracy;

[0099] Step S500, performing posture estimation using matched feature points to determine the position and direction of the AR glasses relative to the recognition image;

[0100] Step S600, integrating the real world data into the virtual environment according to the results of feature point matching and posture estimation;

[0101] Step S700, as Figure 3 As shown, feature points are tracked in a dynamic environment, and the RANSAC optimization algorithm is used to improve the stability and accuracy of tracking;

[0102] The sensor device is configured for spatial positioning and calibration to ensure synchronization of the AR glasses with the real creation environment and other modules;

[0103] The communication device is configured to communicate with other modules wirelessly and transmit data in real time.

[0104] The auditory interaction module mainly includes a microphone, headphones and a data processing device, wherein the microphone is mainly used to obtain the user's voice data, that is, the user's voice instructions, such as "How to draw a shadow?", "What's the next step?", the system's built-in voice assistant answers and feedbacks the user's voice instructions through the headphones, and displays the corresponding painting steps in the AR glasses. Figure 4 is a schematic diagram of the auditory interaction module;

[0105] The data processing device is configured to process and recognize the acquired voice data, and mainly comprises the following steps:

[0106] Step A100, using a Gaussian filter to remove background noise, improve the quality of the voice signal, and automatically adjust the volume to ensure the consistency of the voice signal;

[0107] Step A200, extracting key features from the voice signal, matching the extracted features with a preset voice command template, and recognizing the user's voice command;

[0108] Step A300, parsing the recognized voice command, determining the user's intention, and performing corresponding operations or services according to the user's command;

[0109] Step A400, converting the system's answer or feedback into speech, playing the synthesized speech through headphones, and providing feedback to the user;

[0110] The tactile interaction module mainly includes a tactile sensing device, a data processing device, and a communication device;

[0111] The tactile sensing device is configured as a smart paintbrush and drawing board, and is used to obtain the user's tactile data, capture the position and movement trajectory of the paintbrush on the drawing board through the built-in sensor; sense the painting force, monitor the user's line thickness and force control; through force feedback technology, through vibration or resistance simulation function, provide tactile reminders when the user deviates from the recommended path;

[0112] The smart brush has built-in sensors to capture the user's painting trajectory, strength and angle;

[0113] When the user deviates from the recommended trajectory or the force is inappropriate, the brush will remind the user to adjust through slight vibration or resistance;

[0114] The data processing device is configured to process and identify the sensed tactile data in real time, and mainly includes the following steps:

[0115] Step B100, removing unnecessary noise and interference through filtering to improve data accuracy, and converting the captured tactile data into recognizable input commands, including the thickness of the stroke or the depth of color, etc.;

[0116] Step B200, calculating the required force feedback strength according to the user's touch pressure and movement speed, generating a corresponding force feedback signal, and simulating the real brushstroke painting feeling;

[0117] Step B300, converting the calculated force feedback signal into tactile feedback perceptible to the user through the force feedback technology of the handle and the touchpad, and dynamically adjusting the strength of the force feedback according to the user's preference and actual experience;

[0118] The communication device is configured to connect to the visual interaction module in real time via Bluetooth or Wi-Fi, synchronize data, and transmit brush trajectory and force data.

[0119] The core of the real-time assistance module is the AI ​​intelligent analysis algorithm, which analyzes the user's painting behavior through AI, and automatically provides personalized improvement suggestions and recommends suitable tasks based on the user's creative progress and painting style. Figure 5 As shown, the system's real-time tutoring mainly includes the following steps:

[0120] Step C100, loading the pre-trained AI model, ensuring that the algorithm is ready, initializing the user's visual, auditory, and tactile interaction modules, and preparing to record the user's painting behavior;

[0121] Step C200, collecting the user's painting data in real time through sensors or input devices, including visual, auditory, tactile data, etc., and storing the collected data in a user database for subsequent analysis;

[0122] Step C300, extracting key features from the collected painting data, such as brushstroke patterns, color usage habits, etc., and using an AI model to analyze the user's painting style and behavior patterns;

[0123] Step C400, based on the user's creative progress and painting style, evaluate the completion of the current work and generate personalized improvement suggestions, analyze the user's strengths and weaknesses in painting, and recommend suitable creative tasks or exercises based on the user's skill analysis results;

[0124] Step C500, presenting improvement suggestions and task recommendations to the user, and collecting user feedback on the suggestions and tasks for optimizing subsequent suggestions;

[0125] Step C600, continuously training and optimizing the AI ​​model based on user feedback and new painting data, and adjusting the suggestion generation algorithm based on the model optimization results to improve the accuracy and relevance of the suggestions;

[0126] The recommended tasks include basic tasks, such as drawing contours, learning light and shadow processing, and performing color analysis; advanced tasks, such as complex light and shadow and texture expression; and creative tasks, in which users can create freely and the system provides optimization suggestions.

[0127] It should be noted that the AR-based intelligent assisted painting system provided in the above embodiment is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the modules or steps in the embodiments of the present invention can be decomposed or combined. For example, the modules in the above embodiments can be combined into one module, or further divided into multiple sub-modules to complete all or part of the functions described above. The names of the modules and steps involved in the embodiments of the present invention are only for distinguishing the modules or steps and are not regarded as improper limitations of the present invention.

[0128] An electronic device according to a third embodiment of the present invention comprises: at least one processor; and a memory communicatively connected to at least one of the processors; wherein the memory stores instructions executable by the processor, and the instructions are used to be executed by the processor to implement the above-mentioned AR-based intelligent assisted painting system and method.

[0129] The terms "first", "second", etc. are used to distinguish similar objects rather than to describe or indicate a particular order or sequence.

[0130] The term "comprise" or any other similar term is intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus / device that includes a list of elements includes not only those elements but also other elements not expressly listed, or also includes elements inherent to such process, method, article, or apparatus / device.

[0131] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

Claims

1. An AR-based intelligent assisted painting system, characterized in that: Include: The visual interaction module is used to obtain the user's visual data in the creation scene and provide real-time guidance from the visual feedback; Auditory interaction module, used to obtain user voice data in the creation scene and provide auditory feedback on user questions and other auxiliary information; The tactile interaction module is used to obtain the user's tactile data in the creation scene and provide real-time feedback on the brush operation information through tactile sense; Real-time assistance module, which is used to analyze the user's painting behavior in real time and provide personalized improvement suggestions or practice techniques in real time.

2. The AR-based intelligent assisted painting system according to claim 1, characterized in that: The visual interaction module mainly includes AR glasses, a data processing device, a sensor device, and a communication device; The AR glasses are configured such that the AR glasses capture the user's real painting environment and data through a camera, and are also used to display virtual creation guidance information in real time, and to overlay and display the virtual painting suggestions with the actual creation process on the canvas; The data processing device is configured to identify and analyze the visual data collected by the AR glasses. The specific method includes the following steps: Step S100, pre-processing the image, first converting the color image captured by the camera into a gray image to reduce the amount of calculation, and then using Gaussian filtering and median filtering to denoise the image; Step S200, performing feature point detection, using Harris corner detector, Canny edge detector and SIFT algorithm to detect corner points, edges and scale information respectively; Step S300, generating a description vector for each detected feature point, which can uniquely describe the feature of the point and maintain scale and rotation invariance; Step S400, matching feature descriptors of different images to find nearest neighbor matches, and using ratio test to exclude wrong matches to improve matching accuracy; Step S500, performing posture estimation using matched feature points to determine the position and direction of the AR glasses relative to the recognition image; Step S600, integrating the real world data into the virtual environment according to the results of feature point matching and posture estimation; Step S700, tracking feature points in a dynamic environment, and using a RANSAC optimization algorithm to improve tracking stability and accuracy; The sensor device is configured for spatial positioning and calibration to ensure synchronization of the AR glasses with the real creation environment and other modules; The communication device is configured to communicate with other modules wirelessly and transmit data in real time.

3. The AR-based intelligent assisted painting system according to claim 1, characterized in that: The auditory interaction module mainly includes a microphone, an earphone and a data processing device, wherein the microphone is mainly used to obtain the user's voice data, that is, the user's voice command, and the system's built-in voice assistant answers and feedbacks the user's voice command through the earphone; The data processing device is configured to process and recognize the acquired voice data, and mainly comprises the following steps: Step A100, using a Gaussian filter to remove background noise, improve the quality of the voice signal, and automatically adjust the volume to ensure the consistency of the voice signal; Step A200, extracting key features from the voice signal, matching the extracted features with a preset voice command template, and recognizing the user's voice command; Step A300, parsing the recognized voice command, determining the user's intention, and performing corresponding operations or services according to the user's command; Step A400, converting the system's answer or feedback into voice, playing the synthesized voice through headphones, and providing feedback to the user.

4. The AR-based intelligent assisted painting system according to claim 1, characterized in that: The tactile interaction module mainly includes a tactile sensing device, a data processing device, and a communication device; The tactile sensing device is configured as a smart paintbrush and drawing board, and is used to obtain the user's tactile data, capture the position and movement trajectory of the paintbrush on the drawing board through the built-in sensor; sense the painting force, monitor the user's line thickness and force control; through force feedback technology, through vibration or resistance simulation function, provide tactile reminders when the user deviates from the recommended path; The data processing device is configured to process and identify the sensed tactile data in real time, and mainly includes the following steps: Step B100, removing unnecessary noise and interference through filtering to improve the accuracy of data, and converting the captured tactile data into recognizable input commands, including the thickness of the stroke or the depth of color; Step B200, calculating the required force feedback strength according to the user's touch pressure and movement speed, generating a corresponding force feedback signal, and simulating the real brushstroke painting feeling; Step B300, converting the calculated force feedback signal into tactile feedback perceptible to the user through the force feedback technology of the handle and the touchpad, and dynamically adjusting the strength of the force feedback according to the user's preference and actual experience; The communication device is configured to connect to the visual interaction module in real time via Bluetooth or Wi-Fi, synchronize data, and transmit brush trajectory and force data.

5. The AR-based intelligent assisted painting system and method thereof according to claim 1, characterized in that: The core of the real-time assistance module is the AI ​​intelligent analysis algorithm, which analyzes the user's painting behavior through AI, and automatically provides personalized improvement suggestions and automatically recommends suitable creation tasks or skill improvement exercises based on the user's creation progress and painting style. The module operation mainly includes the following steps: Step C100, loading the pre-trained AI model, ensuring that the algorithm is ready, initializing the user's visual, auditory, and tactile interaction modules, and preparing to record the user's painting behavior; Step C200, collecting the user's painting data in real time through sensors or input devices, including visual, auditory, and tactile data, and storing the collected data in a user database for subsequent analysis; Step C300, extracting key features from the collected painting data and using an AI model to analyze the user's painting style and behavior pattern; Step C400, based on the user's creative progress and painting style, evaluate the completion of the current work and generate personalized improvement suggestions, analyze the user's strengths and weaknesses in painting, and recommend suitable creative tasks and exercises based on the user's skill analysis results; Step C500, presenting improvement suggestions and task recommendations to the user, and collecting user feedback on the suggestions and tasks for optimizing subsequent suggestions; Step C600, based on user feedback and new painting data, continuously train and optimize the AI ​​model, and adjust the suggestion generation algorithm based on the model optimization results to improve the accuracy and relevance of the suggestions.

6. An AR-based intelligent assisted painting method, characterized in that: The following steps are involved: Step E100, system initialization and calibration, including setting the initial position and initial state of each module; Step E200, drawing mode selection, including 3D model mode, picture copying mode, and still life sketching mode; Step E300, real-time painting guidance, including dynamic step-by-step prompts based on vision, intelligent brush interaction support based on touch, and voice interaction based on hearing; Step E400, painting completion and optimization, the system intelligently analyzes the user's completed work and provides optimization suggestions.

Images