An audio-visual cross-modal interaction design method for flat embroidery

Abstract

The application discloses an audio-visual cross-modal interaction design method for flat embroidery needle methods, which comprises the following steps: collecting different flat embroidery needle method graphs, vectorizing and sample augmenting each graph; using a K-means clustering algorithm to cluster graphs with similar features into clusters, screening representative graphs of each cluster to construct a visual modal information library; screening perceptual vocabulary capable of representing the visual modal information of the graphs, including attribute layer, perception layer and association layer vocabulary; scoring the matching degree of perceptual vocabulary at each level and each representative graph to construct the mapping relationship of "visual modal information-perceptual vocabulary" of each representative graph; finding out music bars related thereto, scoring the matching degree of each music bar and the representative graph, thereby constructing the "visual modal information-music bar" mapping relationship based on perceptual vocabulary; obtaining an audio-visual cross-modal information parameter table containing each representative graph and the corresponding music bar parameters based on the mapping relationship, adjusting the music bar parameters to output new music bars, evaluating and optimizing the matching degree of each representative graph and the corresponding new music bar, and outputting the optimized audio-visual cross-modal information library. The application produces a new interactive experience mode through scientific description of different flat embroidery needle methods to meet the needs of people for all-around perception and in-depth experience of embroidery technology.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of cross-stitch needle method interactive design, in particular to an audio-visual cross-modal interactive design method for cross-stitch needle method. BACKGROUND

[0002] Cross-stitch, as a general term for flat embroidery needle method, its basic operation method involves picking up the embroidery thread from a point in the pattern, straightening it to another point, and forming a stitch (i.e. single needle) with the embroidery thread direction being divided into vertical, horizontal and diagonal. Cross-stitch realizes different batches of embroidery through diversified single needle combination, thus presenting the shape of the object, the level of space and the gradient of color. In cross-stitch needle method, the single needles of the same batch show consistent characteristics, where the same batch refers to a plurality of continuous and regularly distributed single needles, which have commonality in direction, length, distribution position or color, etc.

[0003] Although cross-stitch as a traditional technique plays an important role in culture, with the development of new generation information technology and artificial intelligence, traditional embroidery craft is facing challenges in scientific and intelligent representation. On the one hand, the innovation and efficiency improvement of hand embroidery and machine embroidery products are mostly concentrated on the improvement of the supply end process, such as the selection and processing of raw materials, the improvement of embroidery skills or equipment; on the other hand, the existence of technical threshold makes it difficult for the public to deeply understand and experience the embroidery craft in a short time, and they can only touch the finished product and cannot fully perceive the craftsmanship and aesthetics behind it. Therefore, it is necessary to scientifically describe the cross-stitch needle method, promote its integration with modern information technology, and produce a new interactive experience way to meet people's demand for all-around perception and in-depth experience of embroidery craft. SUMMARY

[0004] The purpose of the present application is to provide an audio-visual cross-modal interactive design method for cross-stitch needle method to produce a new interactive experience way to meet people's demand for all-around perception and in-depth experience of embroidery craft.

[0005] Technical solution: To achieve the above-mentioned purpose, the audio-visual cross-modal interactive design method for cross-stitch needle method according to the present application comprises the following steps:

[0006] Step 1: Collect the atlas of different cross-stitch needle methods and construct an atlas sample library;

[0007] Step 2: Perform multi-dimensional vectorization representation on each atlas in the sample library;

[0008] Step 3: When the number N of atlases in the sample library is less than a set value, perform sample augmentation on the vectorized atlas sample library;

[0009] Step 4: Based on step 2 or 3, use the K-means clustering algorithm to cluster the graphs with similar features into clusters, and further filter out the representative graphs of each cluster, thereby constructing the visual modal information library of flat embroidery needle method;

[0010] Step 5: Based on the visual modal information library, filter out the perceptual vocabulary that can represent the visual modal information of the graph, and further classify the perceptual vocabulary into attribute layer, perception layer and association layer;

[0011] Step 6: By testing the matching degree of each level of perceptual vocabulary and each representative graph in the visual modal information library, the mapping relationship between the visual modal information and the perceptual vocabulary of each representative graph is constructed;

[0012] Step 7: Use the association layer vocabulary of each representative graph to search in the music library, find multiple music pieces related to the association layer vocabulary, and form a music set for each representative graph containing the mapping relationship between "perceptual vocabulary-music piece", and find the music piece with the highest matching degree by testing the matching degree of each music piece in the music set and the representative graph, thereby constructing the mapping relationship between "visual modal information-music piece" through the perceptual vocabulary as a bridge;

[0013] Step 8: Based on the mapping relationship between "visual modal information-music piece", obtain the audiovisual cross-modal information parameter table containing each representative graph and its corresponding music piece parameters, adjust the music piece parameters corresponding to each representative graph in the audiovisual cross-modal information parameter table based on the attribute layer and perception layer vocabulary, and output new music pieces;

[0014] Step 9: Evaluate and optimize the matching degree of each representative graph and its corresponding new music piece, and output the optimized audiovisual cross-modal information library.

[0015] Among them, the graph of step 1 shows the specific needle application steps of a certain flat embroidery needle method, including the position of each single needle stitch in each batch; the graphs of different flat embroidery needle methods have the same size, and the two-dimensional coordinates are defined on the graph, and the starting needle application position is positioned with the coordinate origin as the reference point.

[0016] Among them, the multi-dimensional vectorization representation method of step 2 includes:

[0017] Step 201: Set the sample library to include N graphs, where the nth graph has W batches of single needle groups, and the color values of each batch are different, the number of single needles in the ith batch is S i , the color value is HSB i , i=1,2,...,W; then the graph has S1+S2+....+S i +....+S W =S number of single needles;

[0018] Step 202: If the feature value of the jth single needle in the ith batch is v i,j = {I i,j , θ i,j}, j = 1, 2, …, S i , the vectorization matrix of the ith batch is represented as:

[0019] Wherein, I i,j represents the length of the single needle, θ i,j represents the angle of the single needle on the two-dimensional coordinates of the atlas, respectively represent the coordinates of the starting point and the end point of the single needle on the atlas;

[0020] Step 203: The vectorization matrix of the atlas is represented as:

[0021]

[0022] The sample augmentation method described in step 3 is: first, a color library containing X color values is constructed, each color value represents a color, and X is equal to the batch value of the atlas with the highest batch in the atlas sample library; for the same vectorized atlas, the order of the batches is unchanged, and for different batches, all color values in the color library are used for arbitrary permutation and combination, then the nth atlas with W batch single needle groups in the sample library is augmented, if X > W, then X! number of atlas vectorization matrices are obtained, if X = W, then W! number of atlas vectorization matrices are obtained.

[0023] Wherein, step 4 described that the atlas with similar features is gathered into a cluster, which means that the similarity between the atlas samples in the same cluster is the highest, and the similarity between the atlas samples in different clusters is the lowest; using K-means clustering algorithm to cluster the atlas sample library into K clusters, and selecting the atlas closest to the cluster center in the cluster as the representative atlas of the cluster, thereby constructing a visual modal information library with K representative atlases, the clustering process includes:

[0024] Step 401: Based on the matrix of the length, angle and color vectorization representation of each single needle in the atlas in step 2 or 3 as the feature vector, select K atlases as the initial cluster center;

[0025] Step 402: Calculate the Euclidean distance between the feature vectors of the remaining atlases and each cluster center, thereby distributing the remaining atlases to the cluster represented by the nearest cluster center, and then recalculate the mean value of the feature vector of each cluster to obtain a new cluster center;

[0026] ​Step 403: continuously repeat step 402 until a stop condition is met, such as the change in cluster center being less than a set threshold or reaching a preset number of iterations;

[0027] Step 404: output the representative graph of each cluster.

[0028] Wherein, the visual modal information in step 5 refers to the graph information received through the visual sense, including the shape, color, texture, depth, motion and spatial relationship presented on the graph through wiring;

[0029] Wherein, the attribute layer focuses on the basic features of the line visual elements, including color, length and shortness; the perception layer focuses on the visual element relationship between lines, including dynamic and static, neat and messy, light and heavy, smooth and rough, gathering and scattering; the association layer focuses on the emotional, memory and cognitive reactions caused by the line visual elements, including elegant and popular, simple and complex, calm and turbulent, rich and monotonous, happy and sad, romantic and realistic, healing and pain, unique and ordinary, smart and dull.

[0030] Wherein, step 6 described by the tester scores the matching degree of each level of perceptual vocabulary and the representative graph, specifically: based on the representative graph of each cluster, multiple testers score the matching degree of the representative graph with the perceptual layer and the association layer vocabulary respectively, while the attribute layer vocabulary is directly checked, obtaining the subjective evaluation score of multiple testers on the representative graph in multiple evaluation dimensions; thereby based on the attribute layer, the perception layer and the association layer, the perceptual vocabulary with extremely high matching degree for each representative graph is screened out, and the mapping relationship of "visual modal information-perceptual vocabulary" of each representative graph is constructed; wherein extremely high matching degree refers to the average of subjective evaluation scores of a certain representative graph in the perceptual layer or the association layer vocabulary being greater than a set threshold.

[0031] Wherein, step 7 described by the tester scores the matching degree of each music piece in the music set with the representative graph, specifically: based on the music set constructed by the association layer vocabulary of each representative graph, multiple testers score the matching degree of each music piece in the music set with the representative graph, and obtain the subjective evaluation score of multiple testers on the representative graph on the music piece; the music piece with the highest matching degree is screened out, thereby constructing the "visual modal information-music piece" mapping relationship through the perceptual vocabulary as a bridge.

[0032] Wherein, the music piece parameter in step 8 is the music attribute that the music piece has, including rhythm, timbre, loudness, tone length and music style;

[0033] The audiovisual cross-modal information parameter table includes the following information:

[0034] The kth cluster-representative graph α / {attribute layer vocabulary, perceptual layer vocabulary, association layer vocabulary} / new music piece parameter;

[0035] The attribute layer and the perception layer of the music bar parameter are adjusted based on the emotional vocabulary to output a new music bar matching the representative graph emotional characteristics, specifically: the timbre and the length of the new music bar are adjusted based on the representative graph and the attribute layer of the emotional vocabulary, wherein the length of the attribute layer adjusts the length, and the color adjusts the number of timbres; the timbre, the length, the rhythm and the loudness of the new music bar are adjusted based on the perception layer of the emotional vocabulary, wherein the smooth and rough adjust the characteristics of the timbre, the dynamic and static, the light and heavy adjust the change and the weight of the loudness, the neat and disorderly, the gathering and the dispersion adjust the law and the density of the rhythm, and finally a new music bar after the adjustment of the music bar parameter is output; the style of the music bar before the adjustment is taken as the style of the new music bar corresponding to the representative graph based on the association layer of the emotional vocabulary and the mapping relationship between the visual modal information and the music bar constructed in step 7.

[0036] In step 9, the matching degree of each representative graph and the corresponding new music bar is evaluated and optimized, and the specific process is as follows:

[0037] Step 901: The corresponding relationship between each representative graph and the new music bar is randomly disturbed;

[0038] Step 902: The matching degree between each representative graph and the new music bar is calculated by selecting the new music bar with the highest matching degree based on each representative graph by multiple testers;

[0039] Step 903: If the correct rate of the new music bar selected by a certain representative graph is greater than a set threshold, it is considered that the music scheme composed of the representative graph and the corresponding new music bar is effective, and if it is less than or equal to the set threshold, it is considered that the mapping relationship is invalid, at this time, a new corresponding relationship is constructed for the new music bar with the most selection times and the representative graph;

[0040] Step 904: Steps 901-903 are repeatedly performed until all music schemes are effective and stopped;

[0041] Step 905: The optimized audio-visual cross-modal information library is output, and the audio-visual cross-modal information library includes the following information: the kth cluster-representative graph alpha-new music bar.

[0042] Beneficial effects: The method has the following advantages: 1. The method collects and analyzes different flat embroidery needle patterns, converts the visual information of the pattern into a vector representation, makes the differences between different flat embroidery needle patterns more scientific, and provides accurate data support for the feature description and style recognition of flat embroidery needle patterns;

[0043] 2、The method combines the visual information of flat embroidery needle method with music and emotion to produce a new audio-visual cross-modal experience, so that the user can not only appreciate the visual beauty of flat embroidery needle method, but also feel the emotional resonance brought by music, and the interactivity between the user and the embroidery work is enhanced. The cross-modal fusion not only enriches the artistic expression form of flat embroidery needle method and provides more detailed emotional and cognitive description, but also meets the user's demand for comprehensive perception and in-depth experience of embroidery technology. BRIEF DESCRIPTION OF DRAWINGS

[0044] Figure 1 It is a flowchart of the method;

[0045] Figure 2 It is a schematic diagram of the inclined needle atlas with two-dimensional coordinates;

[0046] Figure 3 It is a schematic diagram of different flat embroidery needle method atlas;

[0047] Figure 4 It is a schematic diagram of atlas change when the needle sample is augmented with a wooden comb cover;

[0048] Figure 5 It is a schematic diagram of interactive application with users realized based on the method. DETAILED DESCRIPTION

[0049] The technical solutions of the application will be described in detail below with reference to the embodiments and drawings.

[0050] As shown in the drawings, Figure 1 The audio-visual cross-modal interactive design method for flat embroidery needle method according to the application comprises the following steps:

[0051] Step 1: Collecting atlas of different flat embroidery needle methods to construct an atlas sample library;

[0052] The atlas shows the specific needle application steps of a certain flat embroidery needle method, including the position of each single needle stitch in each batch and the needle application sequence of each single needle, providing intuitive guidance for learners about the characteristics of the needle method; In order to better distinguish the details of different flat embroidery needle methods, the frame size of the displayed atlas is defined to be the same, and two-dimensional coordinates are defined at the same position, with the coordinate origin as the reference point to locate the starting needle application position.

[0053] As shown in the drawings, Figure 2 Take inclined needle as an example, the frame and two-dimensional coordinates of the atlas are displayed, as well as the position of each single needle in the coordinates, to prepare for subsequent atlas vectorization; as shown in the drawings, Figure 3 Some common flat embroidery needle method atlases are listed, clearly showing the differences between different atlas needle methods (two-dimensional coordinates are omitted), including different densities of the same needle method, and different single needle groups of different colors in the same atlas representing different batches.

[0054] Step 2: Perform multi-dimensional vectorization representation of each map in the sample library;

[0055] Step 201: Suppose the sample library contains N spectra, where the nth spectra has W batches of single needle groups, and the color values ​​of each batch are different. The number of single needles in the i-th batch is S. i The color value is HSB. i If i = 1, 2, ..., W, then the total number of graphs is S1 + S2 + ... + S i +....+S W =S number of single needles;

[0056] Step 202: If the characteristic value of the j-th single needle in the i-th batch is v i,j ={I i,j ,θ i,j}, j = 1, 2, ..., S i Then the vectorized matrix of the i-th batch is represented as:

[0057] Among them, I i,j This indicates the length of the single needle. θ i,j This indicates the angle of the single needle on the two-dimensional coordinate system of the graph. These represent the coordinates of the start and end points of the single needle on the chart, respectively;

[0058] Step 203: The vectorized matrix representation of this map is as follows:

[0059]

[0060] Step 3: When the number of spectra N in the sample library is less than the set value of 100, sample augmentation is performed on the vectorized spectra sample library. The method is as follows: First, a color library containing X color values ​​is constructed, where each color value represents a color, and X is equal to the batch value of the spectra with the highest batch in the spectra sample library. For the same vectorized spectra, the batch order remains unchanged. For different batches, all color values ​​in the color library are arbitrarily arranged and combined. Then, the nth spectra with W batches of single needle groups in the sample library is augmented to obtain... If X > W, then the nth map in the sample library with W batches of single needles is augmented to obtain X! map vectorization matrices.

[0061] like Figure 4As shown, taking the wood comb needle sets as an example, if there are four color values (red HSB: 2 / 47 / 60; blue HSB: 205 / 55 / 48; gray HSB: 30 / 1 / 63; pink HSB: 13 / 21 / 84) in the color library, 24 kinds of vectorization matrices with different colors can be obtained by arranging the four colors on the four batches of wood comb needles.

[0062] Step 4: Based on step 2 or 3, the K-means clustering algorithm is used to cluster the graphs with similar features into clusters, and further filter out the representative graphs of each cluster, thereby constructing the visual modal information library about flat embroidery needle method; the clustering process includes:

[0063] Step 401: Based on step 2 or 3, the matrix after the length, angle, and color vectorization of the single needle in each graph is taken as the feature vector, and K graphs are selected as the initial cluster center, K < N;

[0064] Step 402: The Euclidean distance between the feature vectors of the remaining graphs and each cluster center is calculated respectively, so that the remaining graphs are assigned to the cluster represented by the nearest cluster center, and the mean value of the feature vector of each cluster is recalculated to obtain a new cluster center;

[0065] Step 403: Step 402 is repeatedly executed until the stop condition is met, such as the change of the cluster center being less than a set threshold or reaching a preset iteration number;

[0066] Step 404: The representative graph of each cluster is output.

[0067] Step 5: Based on the visual modal information library, the perceptual vocabulary capable of representing the visual modal information of the graph is screened out, and the perceptual vocabulary is further classified into attribute layer, perception layer, and association layer;

[0068] The visual modal information refers to the graph information received through the visual sense, including the shape, color, texture, depth, motion, and spatial relationship presented on the graph through wiring;

[0069] Among them, the attribute layer focuses on the basic features of the line visual elements, including color, length, and shortness; the perception layer focuses on the visual element relationship between lines, including dynamic and static, neat and disorderly; the association layer focuses on the emotional, memory, and cognitive reactions caused by the line visual elements, including elegance and popularization, simplicity and complexity, as shown in Table 1.

[0070] Table 1: Perceptual vocabulary classification table

[0071]

[0072]

[0073] Step 6: The mapping relationship between the visual modal information and the perceptual vocabulary of each representative graph is constructed by scoring the matching degree of the perceptual vocabulary of each level and the representative graph in the visual modal information library. Specifically, based on the representative graph of each cluster, 1000 testers score the matching degree of each representative graph with all the words in the perception layer and the association layer, and the words in the attribute layer are directly selected. The subjective evaluation scores of the representative graph in each evaluation dimension are obtained from 1000 testers. The perceptual vocabulary with high matching degree for each representative graph is selected based on the attribute layer, the perception layer and the association layer (for example, the average subjective evaluation score of the representative graph on a certain word in the perception layer or the association layer is greater than 8 points, and the full score is 10 points), and the mapping relationship between the visual modal information and the perceptual vocabulary of each representative graph is constructed. The mapping relationship includes one or more words in the attribute layer, the perception layer and the association layer with high matching degree for the graph.

[0074] Step 7: The music set is composed by searching the music library with the association layer words of each representative graph to find multiple music pieces related to each word. The music piece with the highest matching degree is found by scoring the matching degree of each music piece in the music set and the association layer words by the testers, so as to construct the mapping relationship between the visual modal information and the music piece by using the perceptual vocabulary as a bridge. Specifically, based on the music set constructed by the association layer words of each representative graph, multiple testers score the matching degree of each music piece in the music set and the representative graph, and the subjective evaluation scores of the representative graph on the music piece are obtained from multiple testers. The music piece with high matching degree for each representative graph is selected, and the mapping relationship between the visual modal information and the music piece is constructed by using the perceptual vocabulary as a bridge. In the mapping relationship, each representative graph includes a music piece with the highest matching degree.

[0075] Step 8: Based on the mapping relationship between the visual modal information and the music piece, the audiovisual cross-modal information parameter table containing the parameters of each representative graph and its corresponding music piece is obtained. The parameters of the music piece corresponding to each representative graph in the audiovisual cross-modal information parameter table are adjusted based on the attribute layer and the perception layer, and a new music piece is output. The music piece parameter is the music attribute of the music piece, including rhythm, timbre, loudness, tone length and music style. The new music piece refers to a series of continuous notes matching the emotional characteristics of the representative graph, which can cause emotional resonance of the user.

[0076] The audiovisual cross-modal information parameter table includes the following information: the kth cluster-representative graph α / {attribute layer words, perception layer words, association layer words} / new music piece parameters. Specifically, as shown in Table 2.

[0077] Table 2 Audiovisual cross-modal information parameter mapping table

[0078]

[0079] The method for adjusting the music bar parameters based on the attribute layer and the perceptual layer vocabulary is as follows: based on the representative atlas, the timbre and the length of the new music bar are adjusted by the attribute layer vocabulary in the perceptual vocabulary, wherein the length adjustment adjusts the length, and the color adjusts the number of timbres; the timbre, the length, the rhythm and the loudness of the new music bar are adjusted by the perceptual layer vocabulary in the perceptual vocabulary, wherein the smooth and rough adjust the characteristics of the timbre, the dynamic and static, the light and heavy adjust the changes and the lightness of the loudness, the neat and disorderly, the gathering and the dispersion adjust the law and the density of the rhythm, and finally a new music bar after the adjustment of the music bar parameters is output; the style of the music bar before the adjustment is taken as the style of the new music bar corresponding to the representative atlas based on the association layer vocabulary in the perceptual vocabulary and the mapping relationship between the visual modal information and the music bar constructed in step 7.

[0080] Step 9: The matching degrees of each representative atlas and the new music bar corresponding thereto are evaluated and optimized, and an optimized audio-visual cross-modal information library is output.

[0081] Step 901: The corresponding relationship between each representative atlas and the new music bar is randomly disturbed;

[0082] Step 902: The matching degrees between each representative atlas and the new music bar are calculated by 1000 testers based on the selection of the new music bar with the highest matching degree corresponding to each representative atlas;

[0083] Step 903: If the correct rate of the new music bar selected by a certain representative atlas is greater than 0.8, it is considered that the music scheme composed of the representative atlas and the new music bar corresponding thereto is effective, and if it is less than or equal to 0.8, it is considered that the music scheme is invalid, at this time, a new corresponding relationship is constructed for the new music bar with the most selection times and the representative atlas;

[0084] Step 904: Steps 901-903 are repeatedly performed until all the music schemes are effective and stopped;

[0085] Step 905: An optimized audio-visual cross-modal information library is output, and the audio-visual cross-modal information library includes the following information: the kth cluster-representative atlas alpha-new music bar.

[0086] As Figure 5As shown, the embodiment based on the above method also provides an application of audio-visual cross-modal interactive design for filling the peony pattern with chicken feather needle pattern. The application interface includes various filling patterns, various representative patterns for filling the patterns, colors of the filling patterns, and density of filling the patterns using the representative patterns, wherein the colors are selected in color groups, including red groups, purple groups, yellow groups, cyan groups, etc., and each group includes several colors with similar color values. Selecting in the form of color groups can make the filled patterns have color changes. Based on the above interface, the user can arbitrarily select the pattern to be filled, the representative pattern, the color group, and the filling density. After selection, the interface will show the filling process, and according to the selected representative pattern, the highest matching new music bar will be played synchronously. So that the user can not only appreciate the visual beauty of the flat embroidery needle method, but also feel the emotional resonance brought by the new music bar, enhancing the interactivity between the user and the embroidery work.

Claims

1. A cross-modal audiovisual interaction design method for flat embroidery stitch, characterized in that, Includes the following steps: Step 1: Collect patterns of different flat embroidery stitches and build a pattern sample library; Step 2: Perform multi-dimensional vectorization representation of each map in the sample library; Step 3: When the number of spectra N in the sample library is less than the set value, perform sample augmentation on the vector-represented spectra sample library; Step 4: Based on Step 2 or 3, use the K-means clustering algorithm to group the graphs with similar features into clusters, and further filter out the representative graphs of each cluster to build a visual modal information database about flat embroidery stitches. Step 5: Based on the visual modality information database, select perceptual words that can represent the visual modality information of the graph, and further classify the perceptual words into attribute layer, perception layer and association layer; Step 6: By having testers score the matching degree between the sensory vocabulary at each level and each representative map in the visual modality information database, the mapping relationship between "visual modality information - sensory vocabulary" of each representative map is constructed. Step 7: Use the associative layer words of each representative graph to search the music library, find multiple music bars related to the associative layer words, and form a music set of each representative graph containing the mapping relationship of "sensory words-music bars". Testers score the matching degree of each music bar in the music set with the representative graph, and find the music bar with the highest matching degree. In this way, the mapping relationship of "visual modal information-music bars" is constructed with sensory words as a bridge. Step 8: Based on the "visual modal information-music bar" mapping relationship, obtain the audiovisual cross-modal information parameter table containing each representative spectrogram and its corresponding music bar parameters. Adjust the music bar parameters corresponding to each representative spectrogram in the audiovisual cross-modal information parameter table based on the vocabulary of the attribute layer and the perception layer, and output the new music bar. Step 9: Evaluate and optimize the matching degree between each representative spectrum and its corresponding new music bar, and output the optimized audiovisual cross-modal information database.

2. The audiovisual cross-modal interaction design method for flat embroidery stitch as described in claim 1, characterized in that, The diagram described in step 1 shows the specific stitching steps of a certain flat embroidery technique, including the position of each single stitch in each batch; the diagrams of different flat embroidery techniques are the same size, and two-dimensional coordinates are defined on the diagrams, with the origin of the coordinates as the reference point to locate the starting stitching position.

3. The audiovisual cross-modal interaction design method for flat embroidery stitch as described in claim 1, characterized in that, The multi-dimensional vectorization representation method described in step 2 includes: Step 201: Suppose the sample library contains N spectra, where the nth spectra has W batches of single needle groups, and the color values ​​of each batch are different. The number of single needles in the i-th batch is S. i The color value is HSB. i If i = 1, 2, ..., W, then the total number of graphs is S1 + S2 + ... + S i +....+S W =S number of single needles; Step 202: If the characteristic value of the j-th single needle in the i-th batch is v i,j ={I i,j ,θ i,j }, j = 1, 2, ..., S i Then the vectorized matrix of the i-th batch is represented as: Among them, I i,j This indicates the length of the single needle. θ i,j This indicates the angle of the single needle on the two-dimensional coordinate system of the graph. These represent the coordinates of the start and end points of the single needle on the chart, respectively; Step 203: The vectorized matrix representation of this map is as follows:

4. The audiovisual cross-modal interaction design method for flat embroidery stitch as described in claim 1, characterized in that, The sample augmentation method described in step 3 is as follows: First, construct a color library containing X color values, where each color value represents a color. X equals the batch value of the highest-batch spectrum in the spectrum sample library. For the same vectorized spectrum, the batch order remains unchanged. For different batches, all color values ​​in the color library are arbitrarily arranged and combined. After augmenting the nth spectrum of single-needle groups with W batches in the sample library, if X > W, then... If X = W, then we obtain W! number of graph vectorization matrices.

5. The audiovisual cross-modal interaction design method for flat embroidery stitch as described in claim 1, characterized in that, Step 4, clustering maps with similar features, means that the similarity between map samples within the same cluster is the highest, and the similarity between map samples in different clusters is the lowest. The K-means clustering algorithm is used to cluster the map sample library into K clusters, and the map closest to the cluster center within each cluster is selected as the representative map of that cluster, thus constructing a visual modality information library with K representative maps. The clustering process includes: Step 401: Based on the matrix obtained by vectorizing the length, angle, and color of the single needle in each pattern in Step 2 or 3, select K patterns as the initial cluster centers. Step 402: Calculate the Euclidean distance between the remaining maps and the feature vectors of each cluster center, thereby assigning the remaining maps to the clusters represented by the nearest cluster centers, and then recalculate the mean of the feature vectors of each cluster to obtain new cluster centers; Step 403: Repeat step 402 continuously until the stopping conditions are met, including the change in the cluster center being less than the set threshold or the preset number of iterations being reached; Step 404: Output the representative map for each cluster.

6. The audiovisual cross-modal interaction design method for flat embroidery stitch as described in claim 1, characterized in that, The visual modal information mentioned in step 5 refers to the map information received through visual senses, including the shape, color, texture, depth, motion, and spatial relationships presented on the map through wiring. The attribute layer focuses on the basic characteristics of visual elements of lines, including color and length; the perception layer focuses on the relationship between visual elements of lines, including dynamic and static, neat and messy, light and heavy, smooth and rough, clustered and scattered; the association layer focuses on the emotional, memory and cognitive responses evoked by visual elements of lines, including elegant and popular, simple and complex, calm and turbulent, rich and monotonous, joyful and sorrowful, romantic and realistic, healing and painful, unique and ordinary, clever and dull.

7. The audiovisual cross-modal interaction design method for flat embroidery stitch as described in claim 1, characterized in that, Step 6, which involves testers scoring the matching degree between the sensory words and representative maps at each level, specifically involves: based on the representative map of each cluster, multiple testers score its matching degree with the words in the perception layer and association layer, while words in the attribute layer are directly selected, and the subjective evaluation scores of multiple testers on the representative map on multiple evaluation dimensions are obtained; thus, based on the attribute layer, perception layer, and association layer, sensory words with extremely high matching degree with each representative map are selected, and the mapping relationship of "visual modal information - sensory words" for each representative map is constructed; Extremely high matching degree means that the average subjective evaluation score of a certain representative graph on the vocabulary of the perception layer or association layer is greater than the set threshold.

8. The audiovisual cross-modal interaction design method for flat embroidery stitch as described in claim 1, characterized in that, Step 7, which involves having testers score the matching degree between each music item in the music set and the representative spectrogram, specifically involves: constructing a music set based on the vocabulary of the associative layer of each representative spectrogram; having multiple testers score the matching degree between each music item in the music set and the representative spectrogram; obtaining the subjective evaluation scores of multiple testers on the representative spectrogram on the music item; and selecting the music item with the highest matching degree, thereby constructing a "visual modal information-music item" mapping relationship using sensory vocabulary as a bridge.

9. The audiovisual cross-modal interaction design method for flat embroidery stitch as described in claim 1, characterized in that, The music bar parameters mentioned in step 8 are the music attributes of the music bar, including rhythm, timbre, loudness, duration, and music bar style; The audiovisual cross-modal information parameter table includes the following information: The k-th cluster represents the graph α / {attribute layer vocabulary, perception layer vocabulary, association layer vocabulary} / new music bar parameters; The process of adjusting the music bar parameters based on attribute layer and perception layer vocabulary to output a new music bar that matches the emotional features of the representative spectrogram is as follows: Based on the representative spectrogram, the timbre and duration of the new music bar are adjusted using attribute layer vocabulary, where length adjusts duration and color adjusts the number of timbres; the timbre, duration, rhythm, and loudness of the new music bar are adjusted using perception layer vocabulary from the emotional layer vocabulary, where smoothness and roughness adjust timbre characteristics, dynamics and statics, lightness and heaviness adjust loudness variations and intensity, and regularity and disorder, clustering and dispersion adjust rhythm regularity and density, ultimately outputting a new music bar with adjusted parameters; based on the "visual modal information-music bar" mapping relationship constructed in step 7 using the associative layer vocabulary from the emotional layer vocabulary, the style of the music bar before adjustment is used as the style of the new music bar corresponding to the representative spectrogram.

10. The audiovisual cross-modal interaction design method for flat embroidery stitch as described in claim 1, characterized in that, Step 9, which involves evaluating and optimizing the matching degree between each representative musical score and its corresponding new musical bar, specifically involves the following process: Step 901: Randomly shuffle the correspondence between each representative musical score and the new musical bars; Step 902: Calculate the accuracy of matching between each representative musical score and the new musical score by having multiple testers select the new musical score that they believe has the highest matching degree based on each representative musical score. Step 903: If the accuracy rate of the new music bar selected by a certain representative graph is greater than the set threshold, the music scheme composed of the representative graph and its corresponding new music bar is considered valid. If it is less than or equal to the set threshold, it is considered invalid. At this time, a new correspondence will be constructed between the new music bar selected most frequently by the representative graph and it. Step 904: Repeat steps 901-903 continuously until all music schemes have been effectively stopped; Step 905: Output the optimized audiovisual cross-modal information database, which includes the following information: the k-th cluster - representative spectrum α - new music strip.

Images