AR-based embroidery method, system, ar glasses, and storage medium

Abstract

The application belongs to the technical field of embroidery and discloses an AR-based embroidery method, system, AR glasses and storage medium, which comprises the following steps: inputting an embroidery sketch into a cycle generative adversarial network to obtain an embroidery pattern, wherein the cycle generative adversarial network is used for feature fusion of line structures of the embroidery sketch and embroidery patterns and generation of the embroidery pattern; embroidering according to the embroidery pattern by using AR glasses; in the embroidery process, in response to collected eye movement data and brain wave data, inputting the eye movement data and the brain wave data into an emotion model to obtain an emotion quantitative index, searching for the emotion quantitative index in a preset color-emotion association library to obtain a recommended color combination and displaying the recommended color combination. The application can realize convenient and efficient personalized embroidery work creation assistance.

Description

Technical Field

[0001] This invention belongs to the field of embroidery technology, specifically relating to an AR-based embroidery method, system, AR glasses, and storage medium. Background Technology

[0002] Embroidery uses needles as pens and threads as ink, employing various exquisite needlework techniques (such as flat stitch, seed stitch, and goldwork) to cleverly present colors and patterns on fabrics, combining practical value with artistic appreciation.

[0003] In recent years, with the rapid development of augmented reality (AR) technology, its application in traditional handicrafts has gradually attracted attention, with AR embroidery assistance systems being a typical example. Currently, existing AR embroidery assistance technologies mainly focus on providing users with embroidery learning functions. This involves using AR technology to overlay virtual information such as embroidery steps and stitch patterns onto real embroidery fabrics or tools based on existing fixed design patterns, guiding users to imitate the preset patterns and steps. While this approach lowers the barrier to entry for embroidery to some extent, helping beginners quickly master basic embroidery skills, existing systems struggle to provide effective technical support when users want to design unique patterns based on their own creativity, adjust the layout and colors of pattern elements, or customize embroidery content to meet personal needs. This limits the in-depth application of AR technology in the embroidery creation process and fails to fully meet users' needs for personalized and creative embroidery works. Summary of the Invention

[0004] The purpose of this invention is to provide an AR-based embroidery method, system, AR glasses, and computer storage medium for convenient and efficient assistance in creating personalized embroidery works.

[0005] The first aspect of this invention discloses an AR-based embroidery method, comprising:

[0006] The embroidery sketch is input into a recurrent adversarial generative network to obtain an embroidery pattern. The recurrent adversarial generative network is used to fuse the line structure of the embroidery sketch with the embroidery pattern and generate the embroidery pattern.

[0007] Embroidery is performed using AR glasses based on the embroidery pattern;

[0008] During the embroidery process, in response to the collected eye movement data and brainwave data, the eye movement data and brainwave data are input into the emotion model to obtain emotion quantification indicators. The emotion quantification indicators are then searched in a preset color-emotion association library to obtain recommended color combinations and display them.

[0009] In some implementations, the emotion model includes a CNN branch and an LSTM branch. The CNN branch is used to extract eye movement focusing duration and pupil diameter changes from the eye movement data, and the LSTM branch is used to extract brainwave frequency band energy from the brainwave data. The emotion quantification indicators output by the emotion model include pleasure and focus.

[0010] In some implementations, the step of searching for the emotion quantification index in a preset color-emotion association library to obtain recommended color combinations includes:

[0011] Extract the dominant color tone of the current image;

[0012] Based on the main color tone and the emotion quantification index, a preset color-emotion association library is searched to obtain several color combinations;

[0013] Clustering algorithms are used to select recommended color combinations from several color combinations.

[0014] In some implementations, the generator of the recurrent adversarial generative network includes a low-level network, a high-level network, and a fusion network. The low-level network is used to preserve the line structure of the embroidery sketch, the high-level network is used to extract the gradient color pattern, and the fusion network is used to combine the line structure and the gradient color pattern.

[0015] In some implementations, the generator of the recurrent adversarial generative network is further provided with a color attention mechanism, which is used to identify color block regions in the embroidery sketch and establish a correlation between the color block regions and the gradient color pattern.

[0016] In some implementations, the embroidery process also includes:

[0017] Acquire user data, which includes user behavior and / or preference data;

[0018] Based on the user data, the needle usage sequence is extracted according to implicit semantic analysis and encoded into a vector;

[0019] The vectors are input into a hybrid recommendation model to obtain recommended embroidery patterns and / or stitching techniques.

[0020] In some implementations, the hybrid recommendation model includes a collaborative filtering module, a content filtering module, and a feedback reinforcement module. The collaborative filtering module is used to filter embroidery patterns whose user ratings are lower than a set threshold. The content filtering module is used to recommend embroidery patterns that match the vector. The feedback reinforcement module is used to dynamically adjust the recommendation weight of the embroidery patterns based on the user's selection.

[0021] A second aspect of this invention discloses an AR-based embroidery system, comprising:

[0022] The embroidery pattern module is used to input embroidery sketches into a recurrent adversarial generative network to obtain embroidery patterns. The recurrent adversarial generative network is used to fuse the line structure of the embroidery sketches with the embroidery patterns to generate the embroidery patterns.

[0023] An interactive module is used to perform embroidery based on the embroidery pattern using AR glasses;

[0024] The color combination recommendation module is used to respond to the collected eye movement data and brainwave data, input the eye movement data and brainwave data into the emotion model to obtain the emotion quantification index, search the emotion quantification index in the preset color-emotion association library, obtain the recommended color combination and display it.

[0025] A third aspect of this invention discloses an AR glasses system, comprising a gesture recognition sensor, an eye-tracking sensor, an electroencephalogram (EEG) sensor, a memory storing executable program code, and a processor coupled to the memory. The gesture recognition sensor is used to recognize the user's gestures, the eye-tracking sensor is used to acquire the user's eye movement data, and the EEG sensor is used to acquire the user's brainwave data. The processor invokes the executable program code stored in the memory to execute the AR-based embroidery method disclosed in the first aspect.

[0026] A fourth aspect of the present invention discloses a computer-readable storage medium storing a computer program, wherein the computer program causes a computer to perform the AR-based embroidery method disclosed in the first aspect.

[0027] The beneficial effects of this invention are that it uses a recurrent adversarial generative network to fuse embroidery sketches and embroidery patterns to generate embroidery patterns quickly and conveniently. During the embroidery process, it further analyzes user emotion data such as eye movement data and EEG data to recommend color combinations, realizes automatic pattern optimization and color matching suggestions, and can conveniently and efficiently assist in the creation of personalized embroidery works. Attached Figure Description

[0028] The accompanying drawings illustrate specific examples of the technical solutions described in this invention and, together with the detailed embodiments, form part of the specification, serving to explain the technical solutions, principles, and effects of this invention.

[0029] Unless otherwise specified or defined, the same reference numerals in different figures represent the same or similar technical features, and different reference numerals may be used to represent the same or similar technical features.

[0030] Figure 1This is a flowchart of an embodiment of an AR-based embroidery method according to an embodiment of the present invention;

[0031] Figure 2 This is a schematic diagram of the structure of the improved CycleGAN network according to an embodiment of the present invention;

[0032] Figure 3 This is a schematic diagram of the structure of the emotion model according to an embodiment of the present invention;

[0033] Figure 4 This is a flowchart illustrating how to obtain recommended color combinations according to an embodiment of the present invention;

[0034] Figure 5 This is a schematic diagram of the emotion fluctuation curve according to an embodiment of the present invention;

[0035] Figure 6 This is a schematic diagram of a color preference heatmap according to an embodiment of the present invention;

[0036] Figure 7 This is a flowchart of a recommended new pattern element or stitching technique according to an embodiment of the present invention;

[0037] Figure 8 This is a schematic diagram of the AR-based embroidery system according to an embodiment of the present invention;

[0038] Figure 9 This is a schematic diagram of the structure of an AR glasses according to an embodiment of the present invention. Detailed Implementation

[0039] Unless otherwise specified or defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art. When combined with the technical solutions of the invention in a real-world scenario, all technical and scientific terms used herein may also have meanings corresponding to the purpose of achieving the technical solutions of the invention. The terms "first," "second," etc., used herein are merely for distinguishing names and do not represent a specific number or order. The term "and / or," as used herein, includes any and all combinations of one or more of the associated listed items.

[0040] It should be noted that when a component is considered "fixed" to another component, it can be directly fixed to the other component or there can be an intervening component; when a component is considered "connected" to another component, it can be directly connected to the other component or there can be an intervening component; when a component is considered "mounted" on another component, it can be directly mounted on the other component or there can be an intervening component; when a component is considered "placed" on another component, it can be directly placed on the other component or there can be an intervening component.

[0041] Unless otherwise specified or defined, the terms "described" or "the" as used herein refer to the technical features or technical content mentioned or described prior to the relevant section, which may be the same as or similar to the technical features or technical content mentioned herein. Furthermore, the terms "comprising" and "having," and any variations thereof, as used herein, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the steps or units listed, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such processes, methods, products, or apparatus.

[0042] To facilitate understanding of the present invention, specific embodiments of the present invention will be described in more detail below with reference to the accompanying drawings.

[0043] This invention provides an AR-based embroidery method for Cantonese embroidery. This method can be implemented through computer programming, and the executing entity can be an electronic device such as a computer, laptop, or tablet, or a control chip embedded in an electronic device; this invention does not limit this. It should be noted that this invention is not limited to producing Cantonese embroidery works, but can also be used to produce other embroidery works such as Suzhou embroidery and Hunan embroidery.

[0044] This embodiment runs an AR-based embroidery method on the processor of the AR glasses, such as... Figure 1 As shown, the specific steps include:

[0045] Step S100: Input the embroidery sketch into the recurrent adversarial generative network to obtain the embroidery pattern. The recurrent adversarial generative network is used to fuse the line structure of the embroidery sketch with the embroidery pattern and generate the embroidery pattern.

[0046] An embroidery sketch is a preliminary design drawing used in the creation of an embroidery piece. It clearly outlines the shape of the embroidery pattern, including various lines, shapes, and curves. These outlines will serve as the basis for the direction of the stitches during embroidery. For example, when embroidering a butterfly, the sketch will first draw the approximate outlines of the butterfly's wings, body, antennae, and other parts.

[0047] Embroidery sketches can be obtained through the following methods:

[0048] I. AR Gesture Drawing. Specifically, users can directly draw embroidery sketches on an AR virtual canvas using gestures (such as swiping their fingers in the air). The AR glasses have built-in high-precision gesture recognition sensors (such as Leap Motion) that capture hand movements in real time, generating lines and rendering them as embroidery sketches. When drawing embroidery sketches, users can also perform intelligent correction functions such as line smoothing and symmetry adjustments, as well as zoom in / out to observe details, thus lowering the barrier to entry for drawing.

[0049] Second, users can pre-draw or photograph patterns using electronic devices such as mobile phones and tablets, and then upload them to AR glasses via Bluetooth or Wi-Fi. The AR glasses automatically extract the outline and adapt it to the rules of Cantonese embroidery stitches, converting the closed path into an embroidery area.

[0050] 3. Users input keywords (such as "lychee pattern" or "phoenix tail feather") or select emotion tags (such as "festive" or "elegant"). Based on the Cantonese embroidery pattern library and CycleGAN, user preferences are integrated to generate embroidery sketches.

[0051] After obtaining the embroidery sketch, it is input into a Cyclic Adversarial Generative Network (CYG). The CYG fuses the line structure of the user's embroidery sketch with classic Cantonese embroidery patterns, preserving the composition while optimizing color transitions to generate the embroidery design. Embroidery patterns refer to various designs and patterns created on textiles using needles and colored threads in the embroidery process. They possess rich cultural connotations and artistic value, such as dragon and phoenix patterns, butterfly patterns, and peony patterns.

[0052] Traditional CycleGAN achieves unsupervised cross-domain transformation through two generators and two discriminators, such as cross-domain transfer of image styles. However, when converting embroidery sketches into Cantonese embroidery patterns, this embodiment further improves CycleGAN to address the requirements of composition preservation and color optimization in embroidery, such as... Figure 2 As shown, through three core design elements—structural constraints, layered integration, and color attention—the framework remains unchanged while the colors are upgraded.

[0053] 1. During generator training, an Edge Consistency Loss is added to impose structural constraints, forcing the generated embroidery pattern to align with the edge structure of the input embroidery sketch. For example, the Sobel operator is used to extract edge feature maps E_X and E_Y' from the embroidery sketch X and the generated embroidery pattern Y' respectively, and the L1 norm between them is calculated to force the generated pattern to retain the line structure of the input sketch: L_edge = ||E_X - E_Y'||1.

[0054] 2. A low-level network, a high-level network, and a fusion network were designed in the generator. The low-level network extracts lines and contours, directly preserving the line structure of the embroidery sketch; the high-level network extracts texture and color, learning color distribution rules from the Cantonese embroidery pattern library (e.g., warm color gradients require the superposition of 3 stitches), and uses style transfer technology to map classic color patterns to the corresponding areas of the embroidery sketch, thereby injecting the gradient color mode of the Cantonese embroidery pattern; the fusion network is used to weightedly fuse feature maps of different levels, realizing the organic combination of the embroidery sketch skeleton and gradient colors. The fusion network combines the sketch structure and gradient colors, ultimately allowing the generated embroidery pattern to retain the original composition of the embroidery sketch while possessing the traditional aesthetic details of Cantonese embroidery.

[0055] 3. A color attention module has also been added to the generator. The color attention module can dynamically identify abrupt color blocks (such as large areas of red) in the embroidery sketch and associate them with the gradient color patterns of classic Cantonese embroidery patterns (such as the transition of stitches from dark red to light red), guiding the recurrent adversarial generative network to generate more natural color gradients.

[0056] A color attention loss is also introduced, which works in conjunction with the color attention module in the generator. Let A be the attention map output by the color attention module (identifying the color block areas), and C_ref be a reference gradient color pattern extracted from the Guangxi embroidery pattern library. This loss encourages the color distribution of the generated pattern Y' to converge toward C_ref in the areas indicated by the attention map A, where L_color = ||A ⊙ (Y' - C_ref)||2, and ⊙ represents element-wise multiplication.

[0057] The total loss function is: L_total = L_adv + λ_cyc * L_cyc + λ_edge * L_edge + λ_color * L_color, where L_adv is the adversarial loss, L_cyc is the cycle consistency loss, L_edge is the edge consistency loss, L_color is the color attention loss, and λ_cyc, λ_edge, and λ_color are weight parameters.

[0058] The improved CycleGAN decouples the structural information of the embroidery sketch from the color and texture information of Cantonese embroidery at different levels of the neural network, and then fuses them in a targeted manner. This allows it to draw a precise line framework based on the user's embroidery sketch, and then, referring to the color matching rules of classic Cantonese embroidery (such as using the "wine-stitch" technique for plum-red gradients), transforms harsh color blocks into delicate stitch transitions, outputting the embroidery pattern. The output embroidery pattern includes various gradient color modes and the corresponding embroidery stitches for each gradient color mode.

[0059] For example, a user draws a sketch of a lychee embroidery. Although the lines of the sketch are clear, the red color blocks are stiff. The sketch is input into an improved recurrent adversarial generative network, and the output is: the outline completely retains the shape of the lychee, and the red color block area is decomposed into a combination of "wine needle + fluff needle" to generate a fluffy texture and color gradient.

[0060] Step S200: Embroider according to the embroidery pattern using AR glasses;

[0061] First, various embroidery stitches are pre-loaded into the AR glasses. Then, embroidery patterns are loaded into the AR glasses, and the embroidery stitches corresponding to the gradient color modes in the embroidery patterns are displayed on the AR interface. Each embroidery stitch includes several stitch steps. After the user puts on the AR glasses, the AR glasses will display indicators and directional symbols to guide the user's actions. The user can also make gestures under the AR action guidance. The user's gestures can also be compared with pre-saved images of the gestures of artisans. If the gestures are inconsistent, the user will be prompted, making it easy and convenient for the user to learn embroidery. Specifically, the AR glasses capture and track the limb movement trajectory of the instructor and obtain limb movement information. Based on the captured limb movement information, the corresponding gesture interaction information is determined and the gestures are compared. AR glasses have been widely used in AR navigation, AR sports, AR teaching (such as Chinese patent CN106683528A), etc. Gesture capture, gesture comparison, and AR interface display are common technical means in AR glasses and will not be elaborated here.

[0062] In AR embroidery, voice control, gesture recognition, and eye tracking can be used to make it easier for users to operate the AR glasses. For example, users can use voice commands to call up specific Cantonese embroidery patterns or needlework tutorial videos, or use gestures to adjust the size and position of the pattern, zoom in and out, and switch between different colored threads to embroider the desired pattern. Users can also choose their favorite interface and scene, such as their own backyard, to create a personalized scene interface.

[0063] After the embroidery is completed, save it as an image. This image can then be printed onto portable canvas bags, keychains, or other products.

[0064] In some implementations, a library of Cantonese embroidery patterns can be built into the AR glasses. Users can quickly access existing Cantonese embroidery patterns when creating embroidery, according to their creative needs.

[0065] Step S300: During the embroidery process, in response to the collected eye movement data and EEG data, the eye movement data and EEG data are input into the emotion model to obtain emotion quantification indicators. The emotion quantification indicators are searched in the preset color-emotion association library to obtain recommended color combinations and display them.

[0066] Specifically, the AR glasses are equipped with a high-precision eye-tracking sensor that tracks the user's eye movements, records where the user is looking (such as the phoenix head in Cantonese embroidery) and the size of the pupil, and obtains eye movement data including the gaze point and the pupil diameter.

[0067] Brainwave signals are collected non-invasively using dry electrodes, and energy changes in alpha and beta waves are analyzed to obtain brainwave data. These dry electrodes can be attached to a lightweight EEG headband worn by the user, or they can be electrode patches included with AR glasses.

[0068] After collecting eye-tracking and electroencephalogram (EEG) data, timestamp alignment is performed to ensure the synchronization of multimodal data. This data is then input into an emotion model to calculate quantitative emotion indicators. The emotion model includes a Convolutional Neural Network (CNN) branch and a Long Short-Term Memory (LSTM) branch. The CNN branch processes eye-tracking data, identifying details repeatedly observed by the user, such as the magenta region, and extracting features such as eye focus duration and pupil diameter changes. The LSTM branch processes EEG data, analyzing EEG changes over time and extracting features such as EEG frequency band energy to obtain characteristics such as changes in the user's relaxation state. For example, EEG signals contain different frequency bands (such as alpha waves, beta waves, and theta waves), and their energy distribution has a recognized physiological correlation with emotional states. Increased alpha wave energy is usually positively correlated with relaxation and calmness, while increased beta wave energy is usually associated with tension, focus, or anxiety. Therefore, by quantifying the changes in energy across each frequency band over time, changes in the user's relaxation state can be indirectly reflected. For example, tension for the first 5 seconds, relaxation for the next 10 seconds. Specifically: After EEG data is input into the LSTM, energy values ​​of each frequency band are extracted according to time windows to form a multi-dimensional temporal feature vector. A gating mechanism is used to memorize the characteristic changes at key time points, thereby modeling the dynamic patterns of EEG energy. For example, if the β-wave energy is significantly higher than the α-wave energy within time t=0-5 seconds, the LSTM outputs that this time period is characterized by tension; if the α-wave energy increases and the β-wave energy decreases within time t=5-15 seconds, then the output indicates that this time period is characterized by relaxation.

[0069] Combining these two components, the output includes quantitative emotion metrics such as pleasure level (e.g., pleasure level 0.9) and focus level (e.g., focus level 0.1). The emotion model can be deployed in the cloud or compressed (e.g., quantification, pruning, knowledge distillation) and then deployed on AR glasses.

[0070] like Figure 3As shown, in this embodiment, the eye-tracking data includes a time-series sequence of coordinate points and a pupil diameter sequence. First, the eye-tracking data is smoothed using Kalman filtering or a moving average method to eliminate outliers caused by hand tremors or device noise. Then, based on a velocity-threshold algorithm or a dispersion threshold algorithm, the coordinate points in the time-series sequence are classified into fixation points, saccades, and tracking movements. In this embodiment, the focus is primarily on the coordinates of the fixation point category. Using a set time window (e.g., 5 seconds) as the time unit, the embroidery pattern area displayed by the AR glasses is divided into an MxN grid. The total duration for which the fixation point falls into each grid within the time unit is used as the value of each grid cell, generating a focus duration map. The average rate of change of the pupil diameter relative to the pupil baseline diameter when the fixation point falls into each grid within the time unit is used as the value of each grid cell, generating a pupil diameter variation map. Then, the focus duration map and the pupil diameter variation map are stitched together to form a dual-channel image. The dual-channel image corresponding to each time unit is input into a convolutional neural network branch to extract the user's eye-tracking focus duration and pupil diameter variation features. Specifically, the convolutional neural network branch is a lightweight network architecture called SqueezeNet, which includes convolutional layers (Conv2D, kernel=3×3, stride=1), pooling layers, and flattening layers, and outputs a 256-dimensional eye-tracking feature vector.

[0071] After obtaining multi-channel EEG data, the data is first subjected to power frequency interference filtering, bandpass filtering, and artifact removal. Power frequency interference filtering eliminates power supply interference, bandpass filtering removes extremely low-frequency drift and high-frequency electromyographic noise, and artifact removal identifies and removes artifact signals caused by blinking, eye movements, and muscle tension. Then, for each channel and each time window, the total power within each frequency band is calculated by integration to obtain the energy value of each band. The energy values ​​of all channels and frequency bands are arranged in time series to form a multi-dimensional temporal feature vector. This multi-dimensional temporal feature vector is then input into the Long Short-Term Memory (LSTM) network branch to extract the EEG frequency band energy features. The EEG frequency band energy features are feature codes reflecting the user's neural state at the current time point. Specifically, in the LSTM network branch, the multi-dimensional temporal feature vector is first processed through two layers of LSTM units (128 hidden units), and finally output as a 128-dimensional EEG feature vector through a fully connected layer.

[0072] Finally, a fully connected layer is used to map eye-tracking focus duration and pupil diameter variation features to focus level, and EEG frequency band energy features to pleasure level. Specifically, after concatenating the eye-tracking feature vector and the EEG feature vector, multi-task prediction is performed, and then mapped to two emotion quantification indicators, pleasure level and focus level, respectively, through a fully connected layer (ReLU activation). The fully connected layer concatenates the output results of multiple tasks to form an output vector containing multiple results. In some implementations, eye-tracking focus duration and pupil diameter variation features can also be mapped to focus level through one fully connected layer, and EEG frequency band energy features can be mapped to pleasure level through another fully connected layer.

[0073] The annotation process for the emotion model training data is as follows: Participants wear AR glasses and biosensors to view a series of pre-set embroidery patterns with different emotional effects and complexities. Their eye movement and electroencephalogram (EEG) data are recorded simultaneously. Then, participants are asked to rate their current emotional pleasure using a self-assessment Mannick Scale or a 9-point Likert scale. For example, since the activity of the zygomaticus major muscle in facial electromyography (EMG) is positively correlated with pleasure, facial EMG signals can also be collected simultaneously for auxiliary annotation of pleasure. Next, a simple interactive task (such as embroidering a section according to a specific pattern) is set, and attention is scored using task accuracy and reaction time. For example, since specific indicators of skin conductance activity and heart rate variability are highly correlated with cognitive load and attention concentration, these two indicators can be used for auxiliary annotation of attention.

[0074] The synchronously collected raw eye-tracking and EEG data are aligned on the timeline, and a vector of [pleasure label, focus label] is generated for each segment of training data to obtain the training data.

[0075] When training an emotion model using training data, the training data can be preprocessed as follows: For eye-tracking data, remove jitter noise, label the core areas of user focus (such as the fuzzy part of an embroidery pattern), smooth the trajectory, and label regions of interest (AOIs). For EEG data, filter out environmental interference (such as device electrical noise) and extract alpha / beta wave energy values.

[0076] After obtaining the emotion quantification index, the emotion quantification index is searched in the preset color-emotion association library to obtain and display the recommended color combination. For example, the recommended color combination is superimposed on the blank embroidery area of ​​the AR interface; or, a floating icon is used at the edge of the field of view to prompt the needlework operation guide corresponding to the recommended color combination (such as: double-tap gesture to start the multi-layer gold needle embroidery); or, voice prompts are provided.

[0077] The specific process of constructing the color-emotion association library is as follows: Users browse historical Cantonese embroidery works, and EEG and eye-tracking data are obtained. When the user's pleasure and / or focus exceed preset values ​​based on EEG data (e.g., pleasure > 0.8 and focus > 0.7 for more than 3 seconds), the eye focus area is identified based on eye-tracking data. The HSV values ​​of all pixels in the focus area are extracted. K-means clustering is performed on all hue (H channel) values ​​in the focus area, and the central hue value of the cluster with the most pixels at its center point is taken as the primary color. The average saturation (S) and brightness (V) of all pixels in the focus area whose hue differs from the primary color within ±30 degrees are calculated to form one or more core colors. Simultaneously, complementary or contrasting colors (colors differing by 180 or 120 degrees on the color wheel) in the focus area are extracted. The primary color, the emotional index range corresponding to pleasure and / or focus, and the color combinations composed of the core color and complementary color are saved to the color-emotion association library. For example, if the color combination of plum red and peony green triggers a peak in user pleasure, then the plum red + peony green combination is associated with the current emotional quantification index and saved in the color-emotion association library. The fields recorded in the color-emotion association library of this embodiment include: the hue value of the primary color, the pleasure index range (e.g., [0.7, 0.9]), the focus index range (e.g., [0.6, 1.0]), and the recommended color combination (e.g., {"colors": [{"h": 340, "s": 0.85, "v":0.9}, {"h": 120, "s": 0.7, "v": 0.8}], "source_artwork_id": "ART_0012"}), where source_artwork_id is the artwork ID from which the color combination originates.

[0078] like Figure 4 As shown, the specific steps for obtaining the recommended color combination in this embodiment include:

[0079] Step S310: Extract the main color tone of the current image;

[0080] Obtain the image data of the current scene, convert the image data from RGB color space to HSV color space to reduce the interference of color brightness on the extraction of the main color tone, and then extract the main color tone H_current of the current scene.

[0081] In some implementations, the image data of the current scene can be input into a CNN network model, and the CNN network model can extract the main color tone of the current scene.

[0082] Step S320: Based on the main color tone and emotional quantification index, search the preset color-emotion association library to obtain several color combinations;

[0083] The color-emotion association library in this embodiment contains multiple sets of mapping relationships. Each set of mapping relationships includes at least: a primary color tone, an emotion quantification index range, and a color combination. Based on the primary color tone and the emotion quantification index, a search is performed in the preset color-emotion association library to find all records where the difference between hue_value and H_current is within ±20 degrees, and where the pleasure index range and focus index range respectively cover pleasure and focus, thus obtaining several color combinations. In this embodiment, the color-emotion association library stores several color combinations that indicate high pleasure and focus when a user appreciates classic Cantonese embroidery works. If the emotion quantification index is low, a search can be performed in the color-emotion association library based solely on the primary color tone to obtain several color combinations, thereby proactively recommending classic color combinations that historically evoke positive emotions, enhancing the user's creative pleasure and focus, breaking through creative bottlenecks, and optimizing the entire embroidery experience.

[0084] Step S330: Use a clustering algorithm to select recommended color combinations from several color combinations.

[0085] The K-means clustering algorithm is used to cluster color combinations to obtain recommended color combinations. K-means clustering is a common technique in this field and will not be elaborated upon here.

[0086] This embodiment ensures the visual relevance of color recommendations to the original image by accurately extracting the main color tone of the image, avoiding color combinations that are out of touch with the theme; and uses a clustering algorithm to filter color combinations, reducing redundant recommendations and improving the harmony and rationality of color combinations.

[0087] In some implementations, color combinations are recommended when appreciating classic Cantonese embroidery works. Specifically: when a user appreciates a classic Cantonese embroidery work, the dominant color tone of the current scene is extracted, combined with real-time emotion data (such as a pleasure level ≥ 0.8), and the optimal color combination is obtained through a clustering algorithm (such as K-means), and then overlaid on the AR interface prompt area. For example: if your frequently used cool colors are detected, try adding a "peacock blue gradient" to enhance the sense of depth.

[0088] In some implementations, a psychological analysis report is also generated. Specifically, when a user wears AR glasses to appreciate classic Cantonese embroidery works, the system analyzes the user's emotional data, extracts the color combinations corresponding to the peak of pleasure (such as plum red + peony green), and generates a personalized psychological analysis report for the user. For example, a voice prompt might say to the user: "You spent 35% of your time in the plum red area, your brainwave relaxation index increased by 20%, we recommend using more warm colors!" Alternatively, the Python Matplotlib library can be used to generate the report. Figure 5 The emotional fluctuation curve shown Figure 6 The charts shown include color preference heatmaps.

[0089] In some implementations, such as Figure 7 As shown, during the embroidery process, new pattern elements or stitching techniques are intelligently recommended based on the user's creative habits and preferences. Specific steps include:

[0090] Step S400: Obtain user data, wherein the user data includes: user behavior and / or preference data;

[0091] User data can be user behavior and preference data collected during the AR embroidery process. For example, the gesture recognition sensor built into the AR glasses (such as Leap Motion) records the type, frequency, and precision of the stitches used by the user; and the eye-tracking sensor captures the duration of the user's gaze and visual heat area on the pattern elements in the AR interface.

[0092] User data can also be metadata automatically recorded from a user's portfolio, such as color combinations used, stitch combinations, and creation time.

[0093] Step S500: Based on user data, extract the sequence of needles using latent semantic analysis and encode it into a vector;

[0094] Latent Dirichlet Allocation (LDA) is an unsupervised topic model that uses latent topic structures in data to achieve low-dimensional vector representations of discrete sequences. Encoding a sequence of needlework techniques (such as "sprinkled needle → gold-threaded needle → fluffy needle") into a user behavior vector using LDA is equivalent to analogizing the needlework sequence to a document and the needlework to words. By mining latent needlework combination topics, each needlework sequence is transformed into a probability distribution vector, such as [2, 1, 0, 0].

[0095] Step S600: Input the vector into the hybrid recommendation model to obtain recommended embroidery patterns and / or stitching techniques.

[0096] The hybrid recommendation model includes a collaborative filtering module, a content filtering module, and a feedback reinforcement module. The collaborative filtering module is used to filter embroidery patterns whose user ratings are lower than a set threshold (e.g., 0.8); the content filtering module is used to recommend embroidery patterns that match the vector; and the feedback reinforcement module is used to dynamically adjust the recommendation weight of embroidery patterns based on the user's selection.

[0097] Specifically, in the collaborative filtering module, a user-pattern matrix is ​​pre-built based on the user's historical patterns. Values ​​in the matrix can include ratings, number of views, usage duration, etc. Based on the user-pattern matrix, matrix factorization methods (such as Singular Value Decomposition++: SVD++) are used to predict the user's rating for untried pattern elements. For example, the predicted interest rating for the phoenix tail feather pattern is 4.2. In the content filtering module, a nearest neighbor search is performed in the feature library of classic Cantonese embroidery works based on the vectors. The cosine similarity between the user's behavior vector and the content vector of the Cantonese embroidery pattern is calculated. New elements with style matching are recommended based on the cosine similarity. The content vector of the Cantonese embroidery pattern can be obtained by encoding the Cantonese embroidery pattern using an autoencoder. In the feedback reinforcement module, when the user adopts a recommended embroidery pattern, the parameters of the collaborative filtering and content filtering models are dynamically adjusted using the Q-Learning algorithm. The specific process of intelligent recommendation is as follows: First, several recommended embroidery patterns and / or stitching techniques are obtained from the content filtering module. Then, the collaborative filtering module filters out some low-rated content, obtaining the recommended embroidery patterns and / or stitching techniques. For example, based on user historical data, analysis reveals frequently used cool colors, and a hybrid recommendation model recommends phoenix tail feather textures to enhance the sense of depth.

[0098] The entire process of AR embroidery in this embodiment is as follows: the user virtually embroiders using gestures, and the AR glasses translate the movements into Cantonese embroidery stitches in real time, overlaying cultural allusions and voice explanations (such as the "sprinkled needle" often used to depict lychee fuzz), providing a more convenient and efficient tool for Cantonese embroidery artists and enthusiasts to create and experience. After the DIY creation is completed, the self-made image can be printed in real time onto portable canvas bags, keychains, and other cultural and creative products. An AI invisible watermark is added to the Cantonese embroidery canvas bag, and scanning it with a mobile phone can trigger an AR animation explaining the cultural connotations behind the personalized work. The AI ​​invisible watermark is a unique QR code generated by AI (containing an encrypted user ID and work data). After scanning with a mobile phone, the cloud API is called to decrypt and render a report to the mobile AR page, supporting 3D animation to display the color emotion association logic.

[0099] In summary, this embodiment uses a recurrent adversarial generative network to fuse embroidery sketches and patterns to generate embroidery designs, enabling quick and convenient pattern creation. During the embroidery process, it further analyzes user emotion data such as eye movement and EEG data to recommend color combinations and embroidery stitches, achieving automatic pattern optimization and color matching suggestions. This provides convenient and efficient assistance in creating personalized embroidery works.

[0100] like Figure 8 As shown, based on the above-mentioned AR-based embroidery method, this embodiment of the invention discloses an AR-based embroidery system, including:

[0101] Embroidery pattern module 600 is used to input embroidery sketches into a recurrent adversarial generative network to obtain embroidery patterns. The recurrent adversarial generative network is used to fuse the line structure of the embroidery sketches with the embroidery patterns and generate the embroidery patterns.

[0102] Interaction module 610 is used to perform embroidery based on the embroidery pattern using AR glasses;

[0103] The color combination recommendation module 620 is used to respond to the collected eye movement data and brain wave data, input the eye movement data and brain wave data into the emotion model, obtain the emotion quantification index, search the emotion quantification index in the preset color-emotion association library, obtain the recommended color combination and display it.

[0104] like Figure 9 As shown, an embodiment of the present invention discloses an AR glasses, including a gesture recognition sensor, an eye tracking sensor, an electroencephalogram (EEG) sensor, a memory storing executable program code, and a processor coupled to the memory;

[0105] The gesture recognition sensor is used to recognize the user's gestures, the eye tracking sensor is used to acquire the user's eye movement data, the EEG sensor is used to acquire the user's brainwave data, and the processor calls the executable program code stored in the memory to execute the AR-based embroidery method described in the above embodiments.

[0106] This invention also discloses a computer-readable storage medium storing a computer program that causes a computer to perform the AR-based embroidery method described in the above embodiments.

[0107] The purpose of the above embodiments is to reproduce and derive the technical solution of the present invention by way of example, and to fully describe the technical solution, purpose and effect of the present invention. The purpose is to enable the public to have a more thorough and comprehensive understanding of the disclosure of the present invention, and not to limit the scope of protection of the present invention.

[0108] The above embodiments are not an exhaustive list based on the present invention, and there may be many other embodiments not listed. Any substitutions and improvements made without departing from the concept of the present invention are within the protection scope of the present invention.

Claims

1. An AR-based embroidery method, characterized in that, include: The embroidery sketch is input into a recurrent adversarial generative network to obtain an embroidery pattern. The recurrent adversarial generative network is used to fuse the line structure of the embroidery sketch with the embroidery pattern and generate the embroidery pattern. Embroidery is performed using AR glasses based on the embroidery pattern; During the embroidery process, in response to the collected eye movement data and brain wave data, the eye movement data and brain wave data are input into the emotion model to obtain emotion quantification indicators. The emotion quantification indicators are then searched in a preset color-emotion association library to obtain recommended color combinations and display them. The generator of the recurrent adversarial generative network includes a low-level network, a high-level network, and a fusion network. The low-level network is used to preserve the line structure of the embroidery sketch, the high-level network is used to extract the gradient color pattern, and the fusion network is used to combine the line structure and the gradient color pattern. The generator of the recurrent adversarial generative network also includes a color attention mechanism, which is used to identify color block regions in the embroidery sketch and establish a correlation between the color block regions and the gradient color pattern.

2. The AR-based embroidery method as described in claim 1, characterized in that, The emotion model includes a CNN branch and an LSTM branch. The CNN branch is used to extract eye movement focusing duration and pupil diameter changes from the eye movement data, and the LSTM branch is used to extract brainwave frequency energy from the brainwave data. The emotion quantification indicators output by the emotion model include pleasure and focus.

3. The AR-based embroidery method as described in claim 1, characterized in that, The step of searching for the emotion quantification index in a preset color-emotion association library to obtain recommended color combinations includes: Extract the dominant color tone of the current image; Based on the main color tone and the emotion quantification index, a preset color-emotion association library is searched to obtain several color combinations; Clustering algorithms are used to select recommended color combinations from several color combinations.

4. The AR-based embroidery method as described in claim 1, characterized in that, The embroidery process also includes: Acquire user data, which includes user behavior and / or preference data; Based on the user data, latent semantic analysis is used to extract the needle usage sequence and encode it into a vector; The vectors are input into a hybrid recommendation model to obtain recommended embroidery patterns and / or stitching techniques.

5. The AR-based embroidery method as described in claim 4, characterized in that, The hybrid recommendation model includes a collaborative filtering module, a content filtering module, and a feedback reinforcement module. The collaborative filtering module is used to filter embroidery patterns whose user ratings are lower than a set threshold. The content filtering module is used to recommend embroidery patterns that match the vector. The feedback reinforcement module is used to dynamically adjust the recommendation weight of the embroidery patterns based on the user's selection.

6. An AR-based embroidery system, characterized in that, include: The embroidery pattern module is used to input embroidery sketches into a recurrent adversarial generative network to obtain embroidery patterns. The recurrent adversarial generative network is used to fuse the line structure of the embroidery sketches with the embroidery patterns to generate the embroidery patterns. An interactive module is used to perform embroidery based on the embroidery pattern using AR glasses; The color combination recommendation module is used to respond to the collected eye movement data and brain wave data, input the eye movement data and brain wave data into the emotion model, obtain the emotion quantification index, search the emotion quantification index in the preset color-emotion association library, obtain the recommended color combination and display it; The generator of the recurrent adversarial generative network includes a low-level network, a high-level network, and a fusion network. The low-level network is used to preserve the line structure of the embroidery sketch, the high-level network is used to extract the gradient color pattern, and the fusion network is used to combine the line structure and the gradient color pattern. The generator of the recurrent adversarial generative network also includes a color attention mechanism, which is used to identify color block regions in the embroidery sketch and establish a correlation between the color block regions and the gradient color pattern.

7. An AR glasses, characterized in that, The device includes a gesture recognition sensor, an eye-tracking sensor, an electroencephalogram (EEG) sensor, a memory storing executable program code, and a processor coupled to the memory. The gesture recognition sensor is used to recognize the user's gestures, the eye-tracking sensor is used to acquire the user's eye movement data, the EEG sensor is used to acquire the user's brainwave data, and the processor calls the executable program code stored in the memory to execute the AR-based embroidery method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein the computer program causes a computer to perform the AR-based embroidery method according to any one of claims 1 to 5.

Images