Feature library generation method and apparatus, and electronic device

Abstract

The application provides a feature library generation method and device and electronic equipment, wherein the method comprises: obtaining a plurality of multi-dimensional information sets corresponding to each target character in a character set in N virtual brush calligraphy writing systems, the multi-dimensional information set comprising a first number of sequence sets, the sequence set comprising a sequence representing the pose information of a writing controller, a sequence representing the hand pose of a writer, and a sequence representing character trajectory information; for each target character, performing information extraction on the N multi-dimensional information sets to obtain N feature network sets of the target character, the feature network set comprising M first feature networks associated with all strokes and P second feature networks associated with stroke important parts; and generating a target feature library according to the N feature network sets of each target character in the character set. The application can establish a multi-dimensional and high-integrity feature library that is truly suitable for the characteristics of a virtual brush calligraphy writing system for characters.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method, apparatus and electronic device for generating a feature library. Background Technology

[0002] The promotion of traditional Chinese calligraphy is accelerating. Meanwhile, various virtual calligraphy writing software and hardware systems, such as touchscreen writing or VR (Virtual Reality) writing, are expanding their application scope due to their flexibility and novelty.

[0003] Currently, a common problem in most virtual calligraphy writing systems is the lack of guidance on the process and techniques of virtual writing; they primarily remain at the level of simply enabling users to "write." While existing research and applications have explored various methods to provide guidance—such as directly using tracing, evaluating writing by analyzing the differences between the user's handwriting and that of a model, or beautifying the user's original handwriting to some extent using a standard model template—the overall focus of current methods is primarily on analyzing the virtual text itself.

[0004] Because a crucial aspect of traditional Chinese calligraphy lies in the control of the brush during the writing process—including hand posture, brush direction, and rotation—current technologies are primarily limited to analyzing the written content. Consequently, the feature databases built upon this foundation lack comprehensiveness and completeness. Furthermore, existing research also lacks consideration for the inherent characteristics of virtual calligraphy writing systems, thus limiting their applicability to the increasingly widespread use of virtual calligraphy writing systems. Summary of the Invention

[0005] In view of the above problems, embodiments of this application provide a feature library generation method, apparatus, and electronic device that overcomes or at least partially solves the above problems.

[0006] In a first aspect, embodiments of this application provide a method for generating a feature library, including:

[0007] Obtain the multi-dimensional information set corresponding to each target character in the text set in N virtual calligraphy writing systems. The multi-dimensional information set includes a sequence set corresponding to a first number of calligraphy writers. The sequence set includes a first sequence representing the pose information of the writing controller, a second sequence representing the hand posture of the writer, and a third sequence representing the trajectory information of the text.

[0008] For each target character, information is extracted from the N multi-dimensional information sets corresponding to the target character to obtain the feature network sets corresponding to the target character in the N virtual calligraphy writing systems. The feature network sets include M first feature networks associated with all strokes of the target character and P second feature networks associated with important parts of the strokes of the target character.

[0009] A target feature library is generated based on the N feature network sets corresponding to each target character in the text set;

[0010] Wherein, N is an integer greater than or equal to 1, and the text set includes a preset number of target texts.

[0011] Secondly, embodiments of this application provide a feature library generation apparatus, comprising:

[0012] The first acquisition module is used to acquire the multi-dimensional information set corresponding to each target character in the text set in N virtual calligraphy writing systems. The multi-dimensional information set includes a sequence set corresponding to a first number of calligraphy writers. The sequence set includes a first sequence representing the pose information of the writing controller, a second sequence representing the hand posture of the writer, and a third sequence representing the trajectory information of the text.

[0013] The extraction module is used to extract information from N multi-dimensional information sets corresponding to each target character, and to obtain the feature network sets corresponding to the target character in the N virtual calligraphy writing systems. The feature network sets include M first feature networks associated with all strokes of the target character and P second feature networks associated with important parts of the strokes of the target character.

[0014] The generation module is used to generate a target feature library based on the N feature network sets corresponding to each target character in the text set;

[0015] Wherein, N is an integer greater than or equal to 1, and the text set includes a preset number of target texts.

[0016] Thirdly, embodiments of this application provide an electronic device, including a processor, a memory, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the steps of the feature library generation method described in the first aspect above.

[0017] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the feature library generation method described in the first aspect above.

[0018] The technical solution of this application, during the process of a calligrapher writing text based on a virtual calligraphy writing system, acquires complete state information of the writing controller, complete state information of the calligrapher's hand, and complete writing trajectory information. Based on the characteristics of calligraphy writing, it extracts deep-level features in the dimensions of character shape, hand posture, and writing controller state. This establishes a multi-dimensional and highly complete feature library that is truly suitable for the characteristics of the virtual calligraphy writing system itself. Based on this feature library, it can provide intelligent guidance for subsequent writing processes and provide an important foundation for specific applications such as intelligent beautification of written content. Attached Figure Description

[0019] Figure 1 A schematic diagram illustrating the feature library generation method provided in an embodiment of this application;

[0020] Figure 2A This illustration shows a writer using a writing controller to write text in a virtual brush calligraphy system, as provided in this application embodiment. Figure 1 ;

[0021] Figure 2B Schematic diagram 2 illustrating a writer using a writing controller to write text in a virtual brush calligraphy writing system according to an embodiment of this application;

[0022] Figure 2C This illustration shows a writer using a writing controller to write text in a virtual brush calligraphy system, as provided in this application embodiment. Figure 3 ;

[0023] Figure 3 A schematic diagram showing marker points that can be installed at the end of a stylus for infrared motion capture;

[0024] Figure 4 A diagram illustrating the continuous acquisition of the writing trajectory during the writing process;

[0025] Figure 5 Examples of writing results from participants lacking writing experience;

[0026] Figure 6 This is an example flowchart illustrating the construction of a feature library corresponding to a virtual calligraphy writing system provided in an embodiment of this application.

[0027] Figure 7 This is a schematic diagram of the feature library generation apparatus provided in an embodiment of this application;

[0028] Figure 8 This is a schematic diagram of the electronic device structure provided in the embodiments of this application. Detailed Implementation

[0029] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0030] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Multiple embodiments in this application may include two or more.

[0031] In the various embodiments of this application, it should be understood that the sequence number of each process described below does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0032] This application embodiment aims to enable various virtual calligraphy writing systems to truly play an educational and entertaining role in improving users' calligraphy skills. It constructs a writing feature library adapted to different virtual calligraphy writing systems, containing features of the writing content and control features for pen use during the writing process. This allows for the creation of feature libraries adapted to different virtual calligraphy writing systems based on their own features. The establishment of this feature library provides intelligent guidance for subsequent writing processes and lays an important foundation for specific applications such as intelligent beautification of writing content. The feature library generation method provided in this application embodiment includes two stages: the first stage involves an information acquisition device acquiring multi-dimensional information corresponding to the calligrapher's writing process; the second stage involves an information extraction device extracting information from the acquired multi-dimensional information to generate the target feature library. The information acquisition device and the information extraction device can be the same device or different devices. The feature library generation method provided in this application embodiment is described in detail below. Figure 1 The above includes:

[0033] Step 101: Obtain the multi-dimensional information set corresponding to each target character in the text set in N virtual calligraphy writing systems. The multi-dimensional information set includes a sequence set corresponding to a first number of calligraphy writers. The sequence set includes a first sequence representing the pose information of the writing controller, a second sequence representing the hand posture of the writer, and a third sequence representing the trajectory information of the text. N is an integer greater than or equal to 1. The text set includes a preset number of target characters.

[0034] The information acquisition device acquires multi-dimensional information sets corresponding to each target character in the text set across N virtual calligraphy writing systems, where N is an integer greater than or equal to 1. When N is greater than 1, it can acquire multi-dimensional information sets corresponding to the target character in different virtual calligraphy writing systems, allowing for the acquisition of matching multi-dimensional information sets tailored to the specific characteristics of each system. The text set includes a preset number of target characters, which are pre-determined characters that need to be written by a calligrapher in the virtual calligraphy writing system.

[0035] In this embodiment, the calligrapher writing the target text needs to possess a high level of calligraphy skill. The text written by these calligraphers in a virtual calligraphy system is used as a sample to ensure the quality of the extracted information. Furthermore, there are multiple calligraphers; by acquiring the target text written by multiple calligraphers, the content written by different calligraphers can be aggregated, providing a basis for subsequent information extraction.

[0036] In this embodiment, the number of calligraphers writing characters in the virtual calligraphy writing system is a first number. In each virtual calligraphy writing system, the first number of calligraphers write target characters. The information acquisition device acquires a sequence set corresponding to the target characters written by each calligrapher. This sequence set includes a first sequence representing the pose information of the writing controller, a second sequence representing the hand posture of the calligrapher, and a third sequence representing the trajectory information of the characters. Based on the sequence sets corresponding to the first number of calligraphers in the current virtual calligraphy writing system, a multi-dimensional information set corresponding to the current virtual calligraphy writing system is acquired.

[0037] By obtaining the multi-dimensional information set corresponding to the target text in each virtual calligraphy writing system, it is possible to obtain N multi-dimensional information sets corresponding to the target text in N virtual calligraphy writing systems; by obtaining the corresponding N multi-dimensional information sets for each target text in the text set, it is possible to obtain multiple multi-dimensional information sets corresponding to the text set.

[0038] In current virtual calligraphy writing systems, the writer needs to hold a writing controller to write characters. Different virtual calligraphy writing systems use writing controllers with different hardware forms. For example, such as... Figure 2A As shown, a PC (Personal Computer) peripheral with tracking capabilities, similar in shape to a ballpoint pen, is used for hover writing; as... Figure 2B As shown, writing is done directly on the touchscreen using a real calligraphy brush; Figure 2C As shown, writing is done on a touchscreen using a stylus. If the virtual calligraphy writing system is a VR writing system, then a VR controller can be used directly for hovering writing. Different hardware forms offer different feel and characteristics during writing, which is an important reason why each virtual calligraphy writing system needs to be analyzed individually.

[0039] Step 102: For each target character, extract information from the N multi-dimensional information sets corresponding to the target character to obtain the feature network sets corresponding to the target character in the N virtual calligraphy writing systems. The feature network sets include M first feature networks associated with all strokes of the target character and P second feature networks associated with important parts of the strokes of the target character.

[0040] After completing the first stage of multidimensional information acquisition, the information extraction device performs the second stage. For each target character in the text set, the information extraction device extracts information from the N multidimensional information sets corresponding to the target character, obtaining N feature network sets corresponding to the target character in the N virtual calligraphy writing systems. Each virtual calligraphy writing system corresponds to one feature network set, which includes M first feature networks and P second feature networks. The M first feature networks are associated with all strokes of the target character, and the P second feature networks are associated with the important parts of all strokes of the target character.

[0041] The M first feature networks and P second feature networks are all determined through model training. The set of feature networks corresponding to the target text in the virtual calligraphy writing system obtained by information extraction can be understood as follows: after extracting information at the stroke level, the model is trained at the stroke level based on the extracted information to determine the M first feature networks; after extracting information at the important part of the stroke level, the model is trained at the important part of the stroke level based on the extracted information to determine the P second feature networks.

[0042] The key strokes are determined after the individual strokes of the target character are identified; that is, after determining the individual strokes of the target character, the key strokes are identified based on the individual strokes of the target character. Since the multi-dimensional information set includes a set of sequences corresponding to a first number of calligraphers, and the sequence set includes a first sequence, a second sequence, and a third sequence, with the first sequence associated with the writing controller state, the second sequence associated with hand posture, and the third sequence associated with character shape, M first feature networks are matched with the pose information of the writing controller, the writer's hand posture, and the character trajectory information, and P second feature networks are matched with the pose information of the writing controller, the writer's hand posture, and the character trajectory information.

[0043] Step 103: Generate a target feature library based on the N feature network sets corresponding to each target character in the text set.

[0044] After obtaining N feature network sets corresponding to each target character through information extraction, the N feature network sets corresponding to each target character in the character set are aggregated to generate a target feature library. That is, the target feature library includes N feature network sets corresponding to a predetermined number of target characters.

[0045] The above-described implementation scheme of this application, during the process of a calligrapher writing text based on a virtual calligraphy writing system, acquires complete state information of the writing controller, complete state information of the calligrapher's hand, and complete writing trajectory information. Based on the characteristics of calligraphy writing, it extracts deep-level features in the dimensions of character shape, hand posture, and writing controller state, and establishes a multi-dimensional and highly complete feature library for the written text that is truly suitable for the characteristics of the virtual calligraphy writing system itself. Based on this feature library, it can provide intelligent guidance for subsequent writing processes and provide an important foundation for specific applications such as intelligent beautification of written content.

[0046] The following describes the process by which the information acquisition device acquires multi-dimensional information corresponding to a calligrapher's writing process. The acquisition involves obtaining the multi-dimensional information set corresponding to each target character in the text set across N virtual calligraphy writing systems, including:

[0047] In each virtual calligraphy writing system, for each target character in the character set, the sequence set corresponding to the first number of calligraphy writers writing the target character is obtained. Based on the first number of sequence sets, the multi-dimensional information set corresponding to the target character in the virtual calligraphy writing system is determined. The multi-dimensional information sets corresponding to the preset number of target characters in N virtual calligraphy writing systems are aggregated to obtain the multi-dimensional information set corresponding to the character set.

[0048] or

[0049] For each target character in the text set, obtain N multi-dimensional information sets corresponding to the target characters written by the first number of calligraphy writers in N virtual calligraphy writing systems. Based on the N multi-dimensional information sets corresponding to the preset number of target characters, obtain the multi-dimensional information set corresponding to the text set.

[0050] When acquiring multidimensional information corresponding to the writing process of a calligrapher, the information acquisition device can use the following two methods:

[0051] Method 1

[0052] In each of the N virtual calligraphy writing systems, for each target character in the character set, a first set of sequences corresponding to the writing of that target character by a first number of calligraphers in the current virtual calligraphy writing system is obtained. These first set of sequences are then aggregated to determine the multi-dimensional information set corresponding to the target character in the current virtual calligraphy writing system. After obtaining the multi-dimensional information set corresponding to each target character in the character set for each virtual calligraphy writing system, the multi-dimensional information sets corresponding to a predetermined number of target characters in the N virtual calligraphy writing systems are aggregated to obtain the multi-dimensional information set corresponding to the character set.

[0053] As an example, there are three virtual calligraphy writing systems: Virtual Calligraphy Writing System 1, Virtual Calligraphy Writing System 2, and Virtual Calligraphy Writing System 3. There are 10 calligraphers and the text set includes 100 target characters. In Virtual Calligraphy Writing System 1, for each target character, a set of sequences corresponding to the writing of that target character by 10 calligraphers is obtained. These 10 sequence sets are aggregated to determine the multi-dimensional information set corresponding to that target character in Virtual Calligraphy Writing System 1. After obtaining the multi-dimensional information sets corresponding to each of the 100 target characters, these 100 multi-dimensional information sets are aggregated to obtain the 100 multi-dimensional information sets corresponding to the target character in Virtual Calligraphy Writing System 1. In Virtual Calligraphy Writing System 2, for each target character, a set of sequences corresponding to the writing of that target character by 10 calligraphers is obtained. These 10 sequence sets are aggregated to determine the multi-dimensional information set corresponding to that target character in Virtual Calligraphy Writing System 2. After obtaining the multi-dimensional information sets corresponding to 100 target characters, these 100 sets are aggregated to obtain the corresponding 100 multi-dimensional information sets in Virtual Calligraphy Writing System 2. In Virtual Calligraphy Writing System 3, a similar method is used to obtain the corresponding 100 multi-dimensional information sets. Finally, the multi-dimensional information sets corresponding to the three virtual calligraphy writing systems are aggregated to determine the final set of multi-dimensional information.

[0054] Among them, obtaining the sequence set corresponding to the first number of calligraphy writers writing the target characters includes:

[0055] During the first time period when each calligrapher writes the target text, the pose vector of the writing controller at each time point, the hand posture vector of the calligrapher at each time point, and the writing trajectory vector of the target text at each time point are obtained. The pose vector represents the position and rotation angle of the writing controller.

[0056] For each calligrapher, a first sequence is generated based on multiple pose vectors corresponding to the writing controller in the first time period, a second sequence is generated based on multiple hand posture vectors corresponding to the calligrapher in the first time period, a third sequence is generated based on multiple writing trajectory vectors corresponding to the target text in the first time period, and a sequence set is generated based on the first sequence, the second sequence, and the third sequence.

[0057] The first number of calligraphers can write the same target text simultaneously, or different calligraphers can write the same target text at different times. There are no restrictions here, and the time spent by different calligraphers writing the same target text can vary.

[0058] For each calligrapher, during the first time period of writing the target text, the following methods are used: obtaining the pose vector of the writing controller held by the calligrapher at each time point during the first time period; obtaining the hand posture vector of the calligrapher at each time point during the first time period; and obtaining the writing trajectory vector of the target text at each time point during the first time period. Then, a first sequence is generated based on the multiple pose vectors of the writing controller during the first time period; a second sequence is generated based on the multiple hand posture vectors of the calligrapher during the first time period; and a third sequence is generated based on the multiple writing trajectory vectors of the target text during the first time period, thus obtaining a set of sequences corresponding to each calligrapher. By obtaining the set of sequences corresponding to the writing of the target text by a first number of calligraphers, a multi-dimensional information set corresponding to the target text can be determined based on the first number of sequence sets.

[0059] The pose vector corresponding to the writing controller represents its position and rotation angle. The meaning of the writing controller's position varies in different virtual brush calligraphy writing systems. For example, for... Figure 2A For the virtual calligraphy writing system shown or the virtual calligraphy writing system in VR space, the position of the writing controller is: the relative position of the "virtual brush" (not shown in the screenshot) and the "virtual paper" that are synchronized with the writing controller in the 3D scene, that is, the coordinates in the virtual paper's own coordinate system. For Figure 2B The virtual calligraphy writing system shown depicts the writing controller positioned relative to the actual calligraphy brush in the writer's hand and the "virtual paper." Specifically, in... Figure 2A In the 3D scene, the writing plane is a virtual paper surface. Figure 2B In this context, the touchscreen surface is a virtual paper surface. The reason for needing to obtain the relative position is that, for virtual writing, one of the key factors that substantially generates virtual ink marks and affects the density of those ink marks is the aforementioned relative position.

[0060] Accordingly, the meaning of the rotation angle of the writing controller varies in different virtual brush calligraphy writing systems. For example, for Figure 2A For the virtual calligraphy writing system shown, or the virtual calligraphy writing system in VR space, the rotation angle of the writing controller is: the spatial rotation angle of the virtual brush in the coordinate system of the virtual paper itself; for Figure 2BIn the virtual calligraphy writing system shown, the rotation angle of the writing controller is the spatial rotation angle of the actual brush in the writer's hand within the coordinate system of the touchscreen surface. This rotation angle is necessary because during the writing process, the brush may need to rotate around any one of the X, Y, or Z axes, which significantly impacts the final writing effect. Although the hardware design of the writing controller in different virtual calligraphy writing systems may not always perfectly support rotation around all three axes, it generally supports at least relatively free rotation around two axes. Therefore, obtaining this rotation angle is essential.

[0061] To accurately acquire the position and rotation angle of the writing controller, tailored solutions can be developed for different virtual brush calligraphy writing systems. For example, for... Figure 2A The virtual calligraphy writing system shown, or the virtual calligraphy writing system in VR space, already provides spatial displacement and rotation information acquisition functions through the handheld pen-shaped peripheral device or VR controller. Therefore, no hardware modifications are required; the necessary displacement and rotation information can be directly obtained based on the hardware's SDK (Software Development Kit). As another example, for... Figure 2B The virtual calligraphy writing system shown requires modification of the brush, such as adding marker points, and using a motion capture system to acquire the spatial positions of all marker points. Based on these positions, the spatial displacement and rotation information of the brush can then be calculated. For example,... Figure 2C As shown in the virtual calligraphy writing system, a device such as... can be installed at the end of the stylus. Figure 3 As shown, marker points are used for infrared motion capture, which allows for real-time acquisition of the stylus's position and angle changes at different moments during the writing process.

[0062] The above describes the pose vector corresponding to the writing controller and the acquisition method. The following describes the process of acquiring hand posture information and writing trajectory.

[0063] During the process of a calligrapher writing each target character, it is necessary to recognize the hand posture of the calligrapher holding the writing controller throughout the entire process, such as through a "hand posture recognition module." This module can achieve hand posture recognition in a way that, for example, involves deploying one or more cameras aimed at the calligrapher's hand, and using a deep learning-based gesture recognition network to continuously acquire the coordinate information of various joints in the hand. As an example, it is possible to... Figure 2CDuring the writing process shown, two cameras are placed next to the writer's hand to simultaneously capture images. A gesture recognition network is then used to obtain the overall hand posture changes at different moments during the writing process. The hand posture information obtained at each time point is in vector form; that is, a hand posture vector is obtained at each time point.

[0064] During the process of a calligrapher writing each target character, it is also necessary to acquire the complete writing trajectory on the virtual paper to better grasp the actual writing trajectory. The reason for acquiring the writing trajectory rather than the image of the completed target character is that different virtual calligraphy systems render virtual ink marks based on the original writing trajectory on the virtual paper, combined with their respective ink rendering algorithms. Compared to extracting features from such secondary processed writing content, directly extracting features from the writing trajectory at the moment of the brushstroke better reflects the original writing characteristics. For example... Figure 4 As shown, the dotted lines inside the strokes schematically illustrate the continuously acquired writing trajectory during the writing process. The writing trajectory acquired at each time point is in vector form; that is, a writing trajectory vector is acquired at each time point.

[0065] During the process of a calligrapher writing the target text, the pose (vector form) of the writing controller, the calligrapher's hand posture (vector form), and the writing trajectory (vector form) are continuously acquired and stored. For example, the writing controller's pose is stored in the writing controller information storage module, the calligrapher's hand posture in the hand posture information storage module, and the writing trajectory in the writing trajectory storage module. This allows for the storage of the complete sequence based on the corresponding module. Since the information acquired by each module at each time point is in vector form, after completing the writing of a target character, the information acquired by each module can be considered a complete vector sequence. The sequence content can be stored directly in a binary file as a matrix, or stored in a database or JSON file as records at multiple time points.

[0066] Method 2

[0067] For each target character in the text set, obtain the multi-dimensional information sets corresponding to each of the first number of calligraphy writers writing the current target character in N virtual calligraphy writing systems, thus obtaining N multi-dimensional information sets. That is, at the character level, for a preset number of target characters, according to a preset character arrangement order, obtain N multi-dimensional information sets corresponding to each target character. After obtaining the N multi-dimensional information sets corresponding to the preset number of target characters, aggregate the N multi-dimensional information sets corresponding to the preset number of target characters to obtain the multi-dimensional information set corresponding to the text set.

[0068] Among them, when acquiring the N multi-dimensional information sets corresponding to the target characters written by the first number of calligraphy writers in N virtual calligraphy writing systems, it includes:

[0069] For each calligrapher, obtain N sequence sets corresponding to the target text written by the calligrapher in the N virtual calligraphy writing systems, and aggregate the N sequence sets corresponding to the first number of calligraphers to obtain N multi-dimensional information sets corresponding to the target text.

[0070] or

[0071] In each virtual calligraphy writing system, a multi-dimensional information set, including a first number of sequence sets, is obtained corresponding to the target text written by the first number of calligraphy writers. The multi-dimensional information sets corresponding to the N virtual calligraphy writing systems are aggregated to obtain the N multi-dimensional information sets corresponding to the target text.

[0072] When obtaining N sets of multi-dimensional information corresponding to a target text, one can obtain N sets of sequences corresponding to the writing of the target text by each of the first number of calligraphers using brush calligraphy, across N virtual calligraphy writing systems. This allows obtaining N sets of sequences for each calligrapher from the calligrapher's perspective. After obtaining the N sets of sequences corresponding to the first number of calligraphers, these sets are aggregated to obtain the multi-dimensional information set corresponding to each of the N virtual calligraphy writing systems, thus achieving the acquisition of N sets of multi-dimensional information corresponding to the target text. The acquisition method described in Method 1 can be used when obtaining the sequence sets, and will not be elaborated upon here.

[0073] For example, 10 calligraphy writers write the target character "yong" in 3 virtual calligraphy writing systems. For each calligraphy writer, 3 sequence sets corresponding to the writer writing "yong" in the 3 virtual calligraphy writing systems are obtained. After obtaining the 3 sequence sets corresponding to the 10 calligraphy writers respectively, at the level of the virtual calligraphy writing system, for each virtual calligraphy writing system, based on the 10 sequence sets corresponding to the virtual calligraphy writing system, a multi-dimensional information set corresponding to the virtual calligraphy writing system is obtained, so as to obtain 3 multi-dimensional information sets.

[0074] When obtaining N multi-dimensional information sets corresponding to a target character, for each of the N virtual calligraphy writing systems, a multi-dimensional information set including the first number of sequence sets corresponding to the first number of calligraphy writers writing the target character in the current writing system can also be obtained. That is, in each virtual calligraphy writing system, the first number of sequence sets corresponding to the first number of calligraphy writers writing the target character in the current writing system are obtained, and the first number of sequence sets are aggregated to determine the multi-dimensional information set corresponding to the current virtual calligraphy writing system. Then, the multi-dimensional information sets corresponding to the N virtual calligraphy writing systems are aggregated respectively to obtain N multi-dimensional information sets corresponding to the target character.

[0075] For example, 10 calligraphy writers write the target character "yong" in 3 virtual calligraphy writing systems. In each virtual calligraphy writing system, 10 sequence sets corresponding to the 10 calligraphy writers writing "yong" are obtained, and the 10 sequence sets are aggregated to obtain the multi-dimensional information set corresponding to the target character "yong" in the current virtual calligraphy writing system. Then, the multi-dimensional information sets corresponding to the 3 virtual calligraphy writing systems are aggregated.

[0076] The process of obtaining the multi-dimensional information sets corresponding to the target character in the N virtual calligraphy writing systems respectively is introduced above. By having calligraphy writers write the target character in different virtual calligraphy writing systems, a multi-dimensional information set adapted to the virtual calligraphy writing system can be obtained based on the characteristics of the virtual calligraphy writing system itself, realizing targeted acquisition of multi-dimensional information.

[0077] Next, the process of the second-stage information extraction device extracting information from the obtained multi-dimensional information to generate a target feature library is introduced. Optionally, information extraction is performed on the N multi-dimensional information sets corresponding to the target character respectively to obtain a set of feature networks corresponding to the target character in the N virtual calligraphy writing systems respectively, including:

[0078] In each virtual calligraphy writing system, the target character is divided into single strokes and important parts of strokes in sequence, and the first division details corresponding to the single strokes and the second division details corresponding to the important parts of strokes are obtained. The important parts of strokes include at least one of the following: the starting part, the ending part, the turning part of the single stroke, and the connecting segment of the stroke.

[0079] For each virtual calligraphy writing system, the first sequence, second sequence, and third sequence corresponding to each sequence set in the matched multi-dimensional information set are divided according to the first division details to obtain a first division result set. The second sequence and third sequence corresponding to each sequence set in the matched multi-dimensional information set are divided according to the second division details to obtain a second division result set.

[0080] For each virtual calligraphy writing system, at the single stroke level, network training is performed on the first sequence, second sequence, and third sequence after division in the first division result set to obtain M first feature networks. At the important part of the stroke level, network training is performed on the first sequence, second sequence, and third sequence after division in the second division result set to obtain P second feature networks, so as to obtain the feature network set corresponding to the target character in the virtual calligraphy writing system.

[0081] After completing the first stage of multi-dimensional information acquisition, for each target character in the text set, in each virtual calligraphy writing system, the target character is divided into single strokes to obtain the first matching division details. That is, the target character written in the first stage is divided into its corresponding strokes. Due to the different writing characteristics of different virtual calligraphy writing systems, the results of single stroke division are different. The first matching division details can be determined for each virtual calligraphy writing system. After completing the single stroke division, it is also necessary to divide the important parts of the strokes. The important parts of the strokes include at least one of the following: the starting part, the ending part, the turning part of the single stroke, and the connecting segment of the stroke. The starting part, the ending part, and the turning part of the single stroke are part of the content of the single stroke, and the connecting segment of the stroke is the connection part of two single strokes. It should be noted that the division can be done manually based on the final rendered text. Correspondingly, due to the different writing characteristics of different virtual calligraphy writing systems, the division of the important parts of the strokes is different. The second matching division details can be determined for each virtual calligraphy writing system.

[0082] For each virtual calligraphy writing system, after dividing the target character into single strokes and important parts of strokes sequentially, the first, second, and third sequences of each sequence set in the multi-dimensional information set corresponding to the target character can be divided according to the first division details, obtaining the sequence division results corresponding to the first, second, and third sequences respectively, to determine the first division result set corresponding to the target character in the current virtual calligraphy writing system; the second sequence details can be divided according to the second division details, obtaining the sequence division results corresponding to the first, second, and third sequences respectively, to determine the second division result set corresponding to the target character in the current virtual calligraphy writing system.

[0083] Specifically, when dividing the first sequence, second sequence, and third sequence corresponding to each sequence set in the matched multi-dimensional information set according to the first division details, and obtaining the first division result set, the process includes:

[0084] Based on the first division details, the third sequence of each sequence set in the matched multi-dimensional information set is divided to obtain the first text trajectory division result corresponding to the third sequence;

[0085] For each sequence set, the first and second sequences are divided according to the first text trajectory division result corresponding to the sequence set, and the first pose information division result and the first hand pose division result are obtained.

[0086] In each virtual calligraphy writing system, after dividing the target character into individual strokes, the first division details (associated with individual strokes) corresponding to the current virtual calligraphy writing system can be obtained. Then, in each virtual calligraphy writing system, based on the first division details corresponding to the virtual calligraphy writing system, the third sequence of each sequence set in the matched multi-dimensional information set is divided to obtain the first character trajectory division result corresponding to the third sequence. That is, the division of individual strokes is mapped to the writing trajectory, and the obtained writing trajectory is divided based on individual strokes to obtain the first character trajectory division result.

[0087] Since the multi-dimensional information set corresponding to the virtual calligraphy writing system includes a first number of sequence sets, and each sequence set includes a first sequence, a second sequence, and a third sequence, when dividing the third sequence, for the multi-dimensional information set, the third sequence contained in each sequence set in the first number of sequence sets is divided to obtain the first character trajectory division result corresponding to the third sequence, thereby realizing the acquisition of the first number of first character trajectory division results for the current multi-dimensional information set.

[0088] After obtaining the first text trajectory segmentation result corresponding to the third sequence in each sequence set for the multi-dimensional information set, for each sequence set, the matching first sequence and second sequence are segmented according to the first text trajectory segmentation result corresponding to the sequence set. The first pose information segmentation result corresponding to the first sequence and the first hand pose segmentation result corresponding to the second sequence are obtained. After obtaining the first text trajectory segmentation result, the first sequence representing the writing controller pose and the second sequence representing the writer's hand pose are further segmented.

[0089] Sequences belonging to the same sequence set can be associated based on the sequence set number or the writer's identifier. When dividing the first and second sequences, the division result of the first text trajectory corresponding to the third sequence belonging to the same sequence set is determined, and the first and second sequences are divided based on the division of the matching third sequence.

[0090] As an example, there are three virtual calligraphy writing systems. For virtual calligraphy writing system 1, based on the first division details (associated with single stroke division) of the target character in virtual calligraphy writing system 1, the third sequence in the 10 sequence sets of multi-dimensional information set 1 (matched with virtual calligraphy writing system 1) is divided to obtain 10 first character trajectory division results. For each first character trajectory division result, the associated first and second sequences are divided based on the current first character trajectory division result to obtain the corresponding first pose information division result and first hand pose division result. Then, for virtual calligraphy writing system 2, based on the first division details (associated with single stroke) of the target character in virtual calligraphy writing system 2, the third sequence in the 10 sequence sets of multi-dimensional information set 2 is divided. Based on the first character trajectory division results corresponding to the third sequence, the associated first and second sequences are divided accordingly. A similar operation is performed for virtual calligraphy writing system 3 to achieve sequence division, which will not be elaborated here.

[0091] Accordingly, when dividing the first sequence, second sequence, and third sequence corresponding to each sequence set in the matched multi-dimensional information set according to the second division details, and obtaining the second division result set, the process includes:

[0092] Based on the second division details, the third sequence of each sequence set in the matched multi-dimensional information set is divided to obtain the second text trajectory division result;

[0093] For each sequence set, the matching first sequence and second sequence are divided according to the second text trajectory division result corresponding to the sequence set, and the second pose information division result and the second hand pose division result are obtained.

[0094] In each virtual calligraphy writing system, after dividing the target character into its key strokes, the second division details (associated with the key strokes) corresponding to the current virtual calligraphy writing system are obtained. Then, in each virtual calligraphy writing system, based on the second division details corresponding to the virtual calligraphy writing system, the third sequence of each sequence set in the matched multi-dimensional information set is divided, and the second character trajectory division result corresponding to the third sequence is obtained. That is, the division of key strokes is mapped to the writing trajectory, and the obtained writing trajectory is divided based on the key strokes to obtain the second character trajectory division result.

[0095] Since the multi-dimensional information set includes a first number of sequence sets, when dividing the third sequence, it is necessary to divide the third sequence of each sequence set in the multi-dimensional information set to obtain the second text trajectory division result corresponding to each third sequence, thereby obtaining the first number of second text trajectory division results. After obtaining the second text trajectory division results corresponding to the third sequence in each sequence set, for each sequence set, the matched first sequence and second sequence are divided according to the second text trajectory division results corresponding to the sequence set, obtaining the second pose information division result corresponding to the first sequence and the second hand pose division result corresponding to the second sequence. After obtaining the second text trajectory division results, the obtained first sequence representing the writing controller pose and the second sequence representing the writer's hand pose are further divided.

[0096] It should be noted that due to the different writing characteristics of different virtual calligraphy writing systems, the division of single strokes and the division of important parts of strokes may differ in different virtual calligraphy writing systems for the same target character. Since the third sequence is based on the first division details (associated with single strokes) and the second division details (associated with important parts of strokes), the division of the third sequence differs between different virtual calligraphy writing systems. Correspondingly, the division of the first and second sequences also differs.

[0097] For N virtual calligraphy writing systems, after dividing the first, second, and third sequences of each sequence set in the virtual calligraphy writing system based on the first partition details, for each virtual calligraphy writing system, at the single stroke level, network training can be performed on the divided first, second, and third sequences to obtain M first feature networks; after dividing the first, second, and third sequences of each sequence set in the virtual calligraphy writing system based on the second partition details, for each virtual calligraphy writing system, at the important part of the stroke level, network training can be performed on the divided first, second, and third sequences to obtain P second feature networks, so as to obtain the feature network set (including M first feature networks and P second feature networks) corresponding to the target text for each virtual calligraphy writing system.

[0098] That is, after all three types of sequences are divided, a deep learning network can be used for network training to obtain a feature network and achieve feature extraction. Since the writing controller state sequence (first sequence), hand posture sequence (second sequence), and writing trajectory sequence (third sequence) are all time-varying sequences of high-dimensional vectors, it is more suitable to adopt an overall network structure that combines a convolutional neural network (CNN) structure such as SwinTransformer with a Transformer network structure. This allows for the implicit extraction of time-varying features of the sequence based on the CNN's ability to extract local features.

[0099] For the segmented single strokes and important parts of strokes, their training is relatively independent due to significant differences in their inherent characteristics and subsequent application scenarios. For either type, the three information sequences they contain also differ significantly in their characteristics and application forms, thus requiring independent training as well. Network training is primarily based on the "network training module," which can be developed using open-source deep learning platforms such as PyTorch or Tensorflow. The hardware platform on which it runs can be flexibly selected, choosing either a local server or a cloud-based deep learning platform, depending on the requirements.

[0100] Specifically, when training a network to obtain M first feature networks for the first sequence, second sequence, and third sequence after the first segmentation result set at the single stroke level, the process includes:

[0101] Using a single stroke as the training unit, the first sequence, second sequence, and third sequence corresponding to each sequence set included in the first division result set are used as training data in three dimensions to perform network training in three dimensions, thereby obtaining the first writing controller feature network, the first hand posture feature network, and the first writing trajectory feature network associated with all strokes of the target character;

[0102] When training P second feature networks on the first, second, and third sequences after segmentation in the second segmentation result set at the level of stroke importance, the process includes:

[0103] Using the important parts of strokes as training units, the first, second, and third sequences corresponding to the sequence sets included in the second division result set are used as training data in three dimensions to perform network training in three dimensions, thereby obtaining the second writing controller feature network, the second hand posture feature network, and the second writing trajectory feature network associated with all the important parts of the target character.

[0104] After performing sequence segmentation at the single stroke level, for a virtual calligraphy writing system, the first sequence corresponding to the first number of sequence sets included in the current virtual calligraphy writing system is aggregated to obtain the first posture information segmentation result set; the second sequence corresponding to the first number of sequence sets is aggregated to obtain the first hand posture segmentation result set; and the third sequence corresponding to the first number of sequence sets is aggregated to obtain the first text trajectory segmentation result set, thereby obtaining the first segmentation result set.

[0105] When training the network at the single stroke level, a single stroke is used as the training unit. The first pose information segmentation result set, the first hand pose segmentation result set, and the first character trajectory segmentation result set are used as training data respectively. The network is trained in three dimensions to obtain the first writing controller feature network, the first hand pose feature network, and the first writing trajectory feature network that are associated with all strokes of the target character under the current virtual brush calligraphy writing system.

[0106] After performing sequence division at the level of important strokes, for a virtual calligraphy writing system, the first sequence corresponding to the first number of sequence sets included in the current virtual calligraphy writing system is aggregated to obtain a second pose information division result set; the second sequence corresponding to the first number of sequence sets is aggregated to obtain a second hand pose division result set; the third sequence corresponding to the first number of sequence sets is aggregated to obtain a second text trajectory division result set, and then a second division result set is obtained.

[0107] When training the network at the level of important strokes, the important strokes are used as training units. The second pose information segmentation result set, the second hand pose segmentation result set, and the second character trajectory segmentation result set are used as training data respectively. The network is trained in three dimensions to obtain the second writing controller feature network, the second hand pose feature network, and the second writing trajectory feature network associated with all the important strokes of the target character in the current virtual brush calligraphy writing system.

[0108] In other words, for each virtual calligraphy writing system, by training the network using single strokes as the basic training unit (including all strokes), three feature networks can be determined for the writing controller state sequence, hand posture sequence, and writing trajectory sequence. By training the network using important parts of strokes (including all important parts of strokes) as the basic training unit, three corresponding feature networks can be obtained. Thus, for a single virtual calligraphy writing system, six feature networks (i.e., a set of feature networks) are ultimately obtained. By aggregating the feature network sets corresponding to N virtual calligraphy writing systems, N feature network sets corresponding to the target text can be obtained. After aggregating the N feature network sets corresponding to each target text in the text set, a target feature library is generated.

[0109] The above describes the process of constructing a feature network. By extracting deep features using deep learning methods, and constructing a matching feature network for each virtual calligraphy writing system, a target feature library corresponding to multiple dimensions and associated with multiple virtual calligraphy writing systems can be obtained.

[0110] Through the processing in the first and second stages, for each virtual calligraphy writing system, its own characteristics can be comprehensively considered, and three dimensions of feature representation can be determined at the level of single strokes and the level of important parts of strokes, respectively. This provides an important foundation and guarantee for various subsequent application forms (such as virtual writing process demonstration, adaptive beautification of user writing content, analysis and evaluation of the advantages and disadvantages of writing).

[0111] As an optional embodiment, the method further includes:

[0112] Obtain a video of the first character written by the user in the first virtual calligraphy writing system.

[0113] A first feature network set matching the first character is determined in the target feature library, and the first feature network set matches the first virtual brush calligraphy writing system.

[0114] Based on the first feature network set and the writing video corresponding to the first character, writing guidance information for the first character is generated and output.

[0115] After constructing the target feature library, when the experiencer writes the first character in the first virtual brush calligraphy writing system, which is one of the N virtual brush calligraphy writing systems, the writing video corresponding to the experiencer's writing of the first character is obtained. By obtaining the writing video corresponding to the experiencer's writing of the first character, the character writing process and the character itself can be obtained. After obtaining the writing video corresponding to the first character, the first feature network set that matches the first character and the first virtual brush calligraphy writing system is determined in the target feature library. That is, first, the first target character that matches the first character is determined in the character set, and then the first feature network set that is adapted to the first target character and the first virtual brush calligraphy writing system is searched for in the target feature library. Searching for the first feature network set that is adapted to the first target character and the first virtual brush calligraphy writing system can be understood as: searching for the first feature network set that is adapted to the first virtual brush calligraphy writing system in the N feature network sets corresponding to the first target character.

[0116] After determining the first feature network set, based on the first feature network set and the writing video corresponding to the first character, writing guidance information for guiding the writing of the first character written by the current experiencer is generated and output, so that the experiencer can rewrite the high-quality first character based on the writing guidance information. After generating the writing guidance information, the first character written by the experiencer can also be beautified based on the writing guidance information; or the writing situation of the experiencer can be evaluated based on the writing guidance information. Among them, when outputting the writing guidance information, it can be output in the form of images, animations or text.

[0117] As an example, as Figure 5 shown, after an experiencer with little writing experience writes the target character "yong" in a certain virtual brush calligraphy writing system, the matching feature network set can be determined in the target feature library, and the feature network set is used to perform a differential analysis and comparison of the features corresponding to the writing process with the reference features, so as to show the experiencer, in the form of images, 3D animations, etc., which parts of the writing process can be improved in terms of touch pen control, hand control and trajectory control.

[0118] By providing guidance on the writing situation of the experiencer, the experiencer can write high-quality characters based on the guidance, improving the experiencer's writing level and writing experience.

[0119] Next, the process of constructing the feature library corresponding to a virtual brush calligraphy writing system is introduced through an example, as Figure 6 shown:

[0120] In the first stage: A first number of calligraphers hold the writing controller and write target characters within the current virtual calligraphy writing system. The character set includes a preset number of target characters, and the calligraphers need to write each target character sequentially. During each calligrapher's writing of a target character, the system obtains the position of the writing controller based on the writing controller position acquisition module, the rotation angle of the writing controller based on the writing controller rotation information acquisition module, the hand posture recognition module to obtain the calligrapher's hand grip posture, and the writing trajectory acquisition module to obtain the writing trajectory on the virtual paper.

[0121] During the writing of the target text, the information acquired by the writing controller position acquisition module and the information acquired by the writing controller rotation information acquisition module are stored in the writing controller information storage module to obtain a complete first sequence; during the writing of the target text, the information acquired by the hand posture recognition module is stored in the hand posture information storage module to obtain a complete second sequence; during the writing of the target text, the information acquired by the writing trajectory acquisition module is stored in the writing trajectory storage module to obtain a complete third sequence. For a target text, there are a first number of sequence sets, which include the first sequence, the second sequence, and the third sequence.

[0122] In the second stage: For each target character being written, the virtual character segmentation module sequentially performs single-stroke segmentation and stroke importance segmentation on the target character within the current virtual brush calligraphy writing system, obtaining the first segmentation details associated with the single stroke and the second segmentation details associated with the stroke importance. Based on the first segmentation details, the writing trajectory segmentation module is used to segment each of the third sequences in the first set of sequences corresponding to the target character, obtaining the first character trajectory segmentation result corresponding to the third sequence. Then, based on the first character trajectory segmentation result, the writing controller and hand posture segmentation module are used to segment the matched first and second sequences, obtaining the first position information segmentation result corresponding to the first sequence and the first hand posture segmentation result corresponding to the second sequence. Based on the second segmentation details, the writing trajectory segmentation module is used to segment each of the third sequences in the first set of sequences corresponding to the target character, obtaining the second character trajectory segmentation result corresponding to the third sequence. Then, based on the second character trajectory segmentation result, the writing controller and hand posture segmentation module are used to segment the matched first and second sequences, obtaining the second position information segmentation result corresponding to the first sequence and the second hand posture segmentation result corresponding to the second sequence.

[0123] For the target text, using single strokes as training units, network training is performed on the first sequence, second sequence, and third sequence after being divided based on single strokes in the first number of sequence sets, to obtain the first writing controller feature network, the first hand posture feature network, and the first writing trajectory feature network associated with all strokes of the target text.

[0124] For the target text, the important parts of the strokes are used as training units. The network is trained on the first sequence, the second sequence and the third sequence after being divided based on the important parts of the strokes in the first number of sequence sets. The network obtains the second writing controller feature network, the second hand posture feature network and the second writing trajectory feature network associated with all the important parts of the strokes of the target text.

[0125] After obtaining the feature network set (including the above 6 feature networks) corresponding to each of the preset number of target characters, the feature library corresponding to the current virtual brush calligraphy writing system is determined based on the feature network set corresponding to each of the preset number of target characters.

[0126] The overall processing flow of this application can be divided into a multi-dimensional information acquisition stage (first stage) and a multi-dimensional information extraction stage (second stage). In the first stage, the main objective is to fully acquire the changes in several dimensions of the writing process for each selected writer as they write each character in the virtual calligraphy writing system. In the second stage, the main objective is to further extract deep features from the multi-dimensional change information obtained in the first stage using deep learning methods, thereby obtaining a multi-dimensional, multi-level feature library of the virtual writing process.

[0127] The above is the overall implementation plan of the feature library generation method provided in this application embodiment. During the process of a calligrapher writing text based on a virtual calligraphy writing system, the complete state information of the writing controller, the complete state information of the calligrapher's hand, and the complete writing trajectory information are obtained. Based on the characteristics of calligraphy writing, deep features are extracted in the dimensions of character shape, hand posture, and writing controller state. A multi-dimensional and highly complete feature library that is truly suitable for the characteristics of the virtual calligraphy writing system is established for the written text. Based on this feature library, intelligent guidance can be provided for the subsequent writing process, and an important foundation can be provided for specific applications such as intelligent beautification of the written content.

[0128] Furthermore, by having calligraphers write target text on different virtual calligraphy writing systems, a multi-dimensional information set adapted to each virtual calligraphy writing system can be obtained based on the system's own characteristics, enabling targeted acquisition of multi-dimensional information. By extracting deep features using deep learning methods and constructing matching feature networks for each virtual calligraphy writing system, a target feature library corresponding to multiple dimensions and associated with multiple virtual calligraphy writing systems can be obtained. By providing guidance on the user's writing based on the matching feature network set, the user can write high-quality text based on the guidance, improving their writing level and writing experience.

[0129] This application provides a feature library generation apparatus, such as... Figure 7 As shown, it includes:

[0130] The first acquisition module 701 is used to acquire a multi-dimensional information set corresponding to each target character in the text set in N virtual calligraphy writing systems. The multi-dimensional information set includes a sequence set corresponding to a first number of calligraphy writers. The sequence set includes a first sequence representing the pose information of the writing controller, a second sequence representing the hand posture of the writer, and a third sequence representing the trajectory information of the text. N is an integer greater than or equal to 1, and the text set includes a preset number of target characters.

[0131] The extraction module 702 is used to extract information from the N multi-dimensional information sets corresponding to each target character, and to obtain the feature network sets corresponding to the target character in the N virtual calligraphy writing systems. The feature network sets include M first feature networks associated with all strokes of the target character and P second feature networks associated with important parts of the strokes of the target character.

[0132] The generation module 703 is used to generate a target feature library based on the N feature network sets corresponding to each target character in the text set.

[0133] Optionally, the first acquisition module includes:

[0134] The first processing submodule is configured to, in each virtual calligraphy writing system, for each target character in the character set, obtain a sequence set corresponding to the writing of the target character by a first number of calligraphy writers, determine the multi-dimensional information set corresponding to the target character in the virtual calligraphy writing system based on the first number of sequence sets, and aggregate the multi-dimensional information sets corresponding to a preset number of target characters in N virtual calligraphy writing systems to obtain the multi-dimensional information set corresponding to the character set; or

[0135] The second processing submodule is used to obtain, for each target character in the text set, N multi-dimensional information sets corresponding to the first number of calligraphy writers writing the target characters in N virtual calligraphy writing systems, and obtain the multi-dimensional information set corresponding to the text set based on the N multi-dimensional information sets corresponding to the preset number of target characters respectively.

[0136] Optionally, the first processing submodule includes:

[0137] The acquisition unit is used to acquire, during the first time period when each calligrapher writes the target text, the pose vector of the writing controller at each time point, the hand posture vector of the calligrapher at each time point, and the writing trajectory vector of the target text at each time point, wherein the pose vector represents the position and rotation angle of the writing controller.

[0138] The generation unit is configured to generate, for each calligrapher, a first sequence based on multiple pose vectors corresponding to the writing controller in the first time period, a second sequence based on multiple hand posture vectors corresponding to the calligrapher in the first time period, a third sequence based on multiple writing trajectory vectors corresponding to the target text in the first time period, and generate the sequence set based on the first sequence, the second sequence, and the third sequence.

[0139] Optionally, the extraction module includes:

[0140] The first division submodule is used to divide the target text into single strokes and important parts of strokes in each virtual calligraphy writing system, and obtain the first division details corresponding to the single stroke and the second division details corresponding to the important parts of strokes. The important parts of strokes include at least one of the following: the starting part, the ending part, the turning part of a single stroke, and the connecting segment of a stroke.

[0141] The second partitioning submodule is used to partition the first sequence, second sequence and third sequence corresponding to each sequence set in the matched multi-dimensional information set according to the first partitioning details for each virtual calligraphy writing system, and obtain a first partitioning result set; and to partition the first sequence, second sequence and third sequence corresponding to each sequence set in the matched multi-dimensional information set according to the second partitioning details, and obtain a second partitioning result set.

[0142] The third processing submodule is used to perform network training on the first sequence, second sequence and third sequence after division in the first division result set at the single stroke level for each virtual calligraphy writing system to obtain M first feature networks, and to perform network training on the first sequence, second sequence and third sequence after division in the second division result set at the stroke importance part level to obtain P second feature networks, so as to obtain the feature network set corresponding to the target character in the virtual calligraphy writing system.

[0143] Optionally, the second partitioning submodule includes:

[0144] The first segmentation unit is used to segment the third sequence of each sequence set in the matched multi-dimensional information set according to the first segmentation details, and obtain the first text trajectory segmentation result;

[0145] The second division unit is used to divide the matched first sequence and second sequence according to the first text trajectory division result corresponding to the sequence set for each sequence set, and obtain the first pose information division result and the first hand pose division result.

[0146] Optionally, the second partitioning submodule includes:

[0147] The third segmentation unit is used to segment the third sequence of each sequence set in the matched multi-dimensional information set according to the second segmentation details, and obtain the second text trajectory segmentation result;

[0148] The fourth segmentation unit is used to segment the matched first sequence and second sequence according to the second text trajectory segmentation result corresponding to the sequence set for each sequence set, and to obtain the second pose information segmentation result and the second hand pose segmentation result.

[0149] Optionally, the third processing submodule is further configured to:

[0150] Using a single stroke as the training unit, the first sequence, second sequence, and third sequence corresponding to each sequence set included in the first division result set are used as training data in three dimensions to perform network training in three dimensions, thereby obtaining the first writing controller feature network, the first hand posture feature network, and the first writing trajectory feature network associated with all strokes of the target character;

[0151] Using the important parts of strokes as training units, the first, second, and third sequences corresponding to the sequence sets included in the second division result set are used as training data in three dimensions to perform network training in three dimensions, thereby obtaining the second writing controller feature network, the second hand posture feature network, and the second writing trajectory feature network associated with all the important parts of the target character.

[0152] Optionally, the device further includes:

[0153] The second acquisition module is used to acquire the writing video of the user writing the first character in the first virtual calligraphy writing system.

[0154] The determination module is used to determine a first feature network set that matches the first character in the target feature library, wherein the first feature network set matches the first virtual brush calligraphy writing system;

[0155] The output generation module is used to generate and output writing guidance information for the first character based on the first feature network set and the writing video corresponding to the first character.

[0156] As the device embodiment is basically similar to the method embodiment, the description is relatively simple. For relevant details, please refer to the description of the method embodiment.

[0157] This application also provides an electronic device, including: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the various processes of the above-described feature library generation method embodiments and achieves the same technical effects. To avoid repetition, it will not be described again here.

[0158] For example, Figure 8 A schematic diagram of the physical structure of an electronic device is shown. (For example...) Figure 8 As shown, the electronic device may include a processor 810, a communications interface 820, a memory 830, and a communication bus 840. The processor 810, communications interface 820, and memory 830 communicate with each other via the communication bus 840. The processor 810 can call logical instructions stored in the memory 830. The processor 810 is used to execute various processes of the feature library generation method of this application embodiment, which will not be further elaborated here.

[0159] Furthermore, the logical instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.

[0160] This application also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the various processes of the above-described feature library generation method embodiments and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0161] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0162] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0163] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

[0164] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed in this application can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0165] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0166] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0167] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0168] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0169] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0170] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for generating a feature library, characterized in that, include: Obtain the multi-dimensional information set corresponding to each target character in the text set in N virtual calligraphy writing systems. The multi-dimensional information set includes a sequence set corresponding to a first number of calligraphy writers. The sequence set includes a first sequence representing the pose information of the writing controller, a second sequence representing the hand posture of the writer, and a third sequence representing the trajectory information of the text. For each target character, information is extracted from the N multi-dimensional information sets corresponding to the target character to obtain the feature network sets corresponding to the target character in the N virtual calligraphy writing systems. The feature network sets include M first feature networks associated with all strokes of the target character and P second feature networks associated with important parts of the strokes of the target character. A target feature library is generated based on the N feature network sets corresponding to each target character in the text set; Wherein, N is an integer greater than or equal to 1, and the text set includes a preset number of target texts; The step of extracting information from the N multi-dimensional information sets corresponding to the target text to obtain the feature network sets corresponding to the target text in the N virtual calligraphy writing systems includes: In each virtual calligraphy writing system, the target character is divided into single strokes and important parts of strokes in sequence, and the first division details corresponding to the single strokes and the second division details corresponding to the important parts of strokes are obtained. The important parts of strokes include at least one of the following: the starting part, the ending part, the turning part of the single stroke, and the connecting segment of the stroke. For each virtual calligraphy writing system, the first sequence, second sequence, and third sequence corresponding to each sequence set in the matched multi-dimensional information set are divided according to the first division details to obtain a first division result set. The second sequence and third sequence corresponding to each sequence set in the matched multi-dimensional information set are divided according to the second division details to obtain a second division result set. For each virtual calligraphy writing system, at the single stroke level, network training is performed on the first sequence, second sequence, and third sequence after division in the first division result set to obtain M first feature networks. At the important part of the stroke level, network training is performed on the first sequence, second sequence, and third sequence after division in the second division result set to obtain P second feature networks, so as to obtain the feature network set corresponding to the target character in the virtual calligraphy writing system.

2. The method according to claim 1, characterized in that, The acquisition of the multi-dimensional information set corresponding to each target character in the text set in N virtual calligraphy writing systems includes: In each virtual calligraphy writing system, for each target character in the aforementioned character set... Obtain the sequence set corresponding to the first number of calligraphy writers writing the target text respectively, determine the multi-dimensional information set corresponding to the target text in the virtual calligraphy writing system based on the first number of sequence sets, aggregate the multi-dimensional information sets corresponding to the preset number of target texts in N virtual calligraphy writing systems respectively, and obtain the multi-dimensional information set corresponding to the text set. or For each target character in the text set, obtain N multi-dimensional information sets corresponding to the target characters written by the first number of calligraphy writers in N virtual calligraphy writing systems. Based on the N multi-dimensional information sets corresponding to the preset number of target characters, obtain the multi-dimensional information set corresponding to the text set.

3. The method according to claim 2, characterized in that, The step of obtaining the sequence set corresponding to the first number of calligraphy writers writing the target character includes: During the first time period when each calligrapher writes the target text, the pose vector of the writing controller at each time point, the hand posture vector of the calligrapher at each time point, and the writing trajectory vector of the target text at each time point are obtained. The pose vector represents the position and rotation angle of the writing controller. For each calligrapher, a first sequence is generated based on multiple pose vectors corresponding to the writing controller in the first time period, a second sequence is generated based on multiple hand posture vectors corresponding to the calligrapher in the first time period, a third sequence is generated based on multiple writing trajectory vectors corresponding to the target text in the first time period, and a sequence set is generated based on the first sequence, the second sequence, and the third sequence.

4. The method according to claim 1, characterized in that, The step of dividing the first sequence, second sequence, and third sequence corresponding to each sequence set in the matched multi-dimensional information set according to the first division details, and obtaining the first division result set, includes: Based on the first division details, the third sequence of each sequence set in the matched multi-dimensional information set is divided to obtain the first text trajectory division result; For each sequence set, the first and second sequences are divided according to the first text trajectory division result corresponding to the sequence set, and the first pose information division result and the first hand pose division result are obtained.

5. The method according to claim 1, characterized in that, The step of dividing the first sequence, second sequence, and third sequence corresponding to each sequence set in the matched multi-dimensional information set according to the second division details, and obtaining the second division result set, includes: Based on the second division details, the third sequence of each sequence set in the matched multi-dimensional information set is divided to obtain the second text trajectory division result; For each sequence set, the matching first sequence and second sequence are divided according to the second text trajectory division result corresponding to the sequence set, and the second pose information division result and the second hand pose division result are obtained.

6. The method according to claim 4 or 5, characterized in that, The step of training M first feature networks by performing network training on the first sequence, second sequence, and third sequence after division in the first division result set at the single stroke level includes: Using a single stroke as the training unit, the first sequence, second sequence, and third sequence corresponding to each sequence set included in the first division result set are used as training data in three dimensions to perform network training in three dimensions, thereby obtaining the first writing controller feature network, the first hand posture feature network, and the first writing trajectory feature network associated with all strokes of the target character; The step of training P second feature networks by performing network training on the first, second, and third sequences after segmentation in the second segmentation result set at the level of stroke importance includes: Using the important parts of strokes as training units, the first, second, and third sequences corresponding to the sequence sets included in the second division result set are used as training data in three dimensions to perform network training in three dimensions, thereby obtaining the second writing controller feature network, the second hand posture feature network, and the second writing trajectory feature network associated with all the important parts of the target character.

7. The method according to claim 1, characterized in that, The method further includes: Obtain a video of the user writing the first character in the first virtual calligraphy writing system. A first feature network set matching the first character is determined in the target feature library, and the first feature network set matches the first virtual brush calligraphy writing system. Based on the first feature network set and the writing video corresponding to the first character, writing guidance information for the first character is generated and output.

8. A feature library generation apparatus, characterized in that, include: The first acquisition module is used to acquire the multi-dimensional information set corresponding to each target character in the text set in N virtual calligraphy writing systems. The multi-dimensional information set includes a sequence set corresponding to a first number of calligraphy writers. The sequence set includes a first sequence representing the pose information of the writing controller, a second sequence representing the hand posture of the writer, and a third sequence representing the trajectory information of the text. The extraction module is used to extract information from N multi-dimensional information sets corresponding to each target character, and to obtain the feature network sets corresponding to the target character in the N virtual calligraphy writing systems. The feature network sets include M first feature networks associated with all strokes of the target character and P second feature networks associated with important parts of the strokes of the target character. The generation module is used to generate a target feature library based on the N feature network sets corresponding to each target character in the text set; Wherein, N is an integer greater than or equal to 1, and the text set includes a preset number of target texts; The extraction and acquisition module includes: The first division submodule is used to divide the target text into single strokes and important parts of strokes in each virtual calligraphy writing system, and obtain the first division details corresponding to the single stroke and the second division details corresponding to the important parts of strokes. The important parts of strokes include at least one of the following: the starting part, the ending part, the turning part of a single stroke, and the connecting segment of a stroke. The second partitioning submodule is used to partition the first sequence, second sequence and third sequence corresponding to each sequence set in the matched multi-dimensional information set according to the first partitioning details for each virtual calligraphy writing system, and obtain a first partitioning result set; and to partition the first sequence, second sequence and third sequence corresponding to each sequence set in the matched multi-dimensional information set according to the second partitioning details, and obtain a second partitioning result set. The third processing submodule is used to perform network training on the first sequence, second sequence and third sequence after division in the first division result set at the single stroke level for each virtual calligraphy writing system to obtain M first feature networks, and to perform network training on the first sequence, second sequence and third sequence after division in the second division result set at the stroke importance part level to obtain P second feature networks, so as to obtain the feature network set corresponding to the target character in the virtual calligraphy writing system.

9. An electronic device, characterized in that, It includes a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the feature library generation method as described in any one of claims 1 to 7.

Images