Chinese character virtual learning method and device combining vr and mortise and tenon

Abstract

The application relates to the technical field of computer application, and provides a Chinese character virtual learning method and device combining VR and mortise and tenon. The method obtains user input information through a VR device to determine a target Chinese character, then carries out skeleton extraction and topological analysis on the target Chinese character to obtain a structure unit set of the target Chinese character, matches a target mortise and tenon template based on connection type information in the structure unit set, determines mortise and tenon geometric parameters based on stroke information in the structure unit set, and finally obtains user splicing information in real time through the VR device to determine the user's operation until the user completes the splicing of the target Chinese character. The method realizes automatic mapping of Chinese character structure and three-dimensional mortise and tenon components, improves the learning experience of Chinese character form in a three-dimensional space, improves the interaction efficiency and recognition accuracy in the Chinese character structure learning process through multi-modal information cooperation.

Description

Technical Field

[0001] This application relates to the field of computer application technology, and in particular to a method and device for virtual learning of Chinese characters that combines VR and mortise and tenon joints. Background Technology

[0002] Virtual Reality (VR) technology, as a cutting-edge achievement in the field of human-computer interaction, is changing people's lives and ways of cognition at an unprecedented pace. Leveraging the powerful computing capabilities of computers, VR technology meticulously constructs realistic three-dimensional virtual environments, making users feel as if they are in a completely new world, allowing them to fully immerse themselves and engage in natural and fluid interactions with the virtual environment. VR systems typically consist of hardware devices, software platforms, and content resources, simulating the real world or creating unique virtual scenes to provide users with an immersive experience.

[0003] As a core technology in the field of virtual reality, VR interaction technology allows users to deeply interact with the virtual world through various input devices. Among these, controllers, gloves, and motion capture devices are the most common interaction tools. These devices can capture the user's hand movements and positions, allowing users to grasp objects, manipulate tools, and so on in the virtual environment. Motion capture typically uses technologies such as infrared cameras and laser tracking to accurately identify subtle hand and body movements, enabling users to move freely in virtual space. Haptic interaction uses force feedback gloves, vibration feedback controllers, and other devices to simulate the texture and feel of objects, allowing users to experience the realism of touch and collision in virtual scenes. With advancements in haptic technology, some VR systems can even provide force feedback, allowing users to feel the weight, elasticity, or resistance of virtual objects, enhancing the realism of the interaction. Sound interaction uses spatial audio technology to dynamically adjust sound effects based on the user's location, allowing users to perceive the direction and distance of sounds, simulating a natural auditory environment.

[0004] Mortise and tenon joinery, a treasure of traditional Chinese woodworking, carries profound historical significance and exquisite craftsmanship. However, traditional learning and transmission methods are often limited by factors such as physical demonstrations and operational difficulties, making it difficult to fully stimulate learners' interest and participation. Similarly, Chinese character education faces the challenge of being dull and uninteresting.

[0005] In-depth research has revealed a certain similarity and mapping between mortise and tenon structures and the strokes of Chinese characters. Morphologically, the tenons and mortises in mortise and tenon joints echo the beginning and ending strokes of Chinese characters. In terms of assembly, mortise and tenon structures achieve stable connections between components through the ingenious combination of tenons and mortises of different shapes and sizes, just as Chinese character strokes achieve their connections through different combinations and arrangements. These characteristics provide a solid theoretical foundation and rich imaginative space for the integration and innovation of the two. However, how to specifically combine VR with mortise and tenon technology in practical applications to provide users with a more efficient method for learning Chinese characters is a technical problem that needs to be solved. Summary of the Invention

[0006] In view of this, the embodiments of this application provide a method and apparatus for virtual Chinese character learning that combines VR and mortise and tenon joints, in order to solve the problem in the prior art of how to combine VR and mortise and tenon joint technology to provide users with an efficient method for learning Chinese characters.

[0007] A first aspect of this application provides a method for virtual learning of Chinese characters that combines VR with mortise and tenon joints, including:

[0008] The target Chinese character is determined by obtaining user input information through VR devices.

[0009] Skeleton extraction and topological analysis are performed on the target Chinese character to obtain a set of structural units of the target Chinese character; the set of structural units includes at least the stroke information and connection type information of the target Chinese character;

[0010] The target mortise and tenon template is determined by matching it from a set of preset mortise and tenon templates based on the connection type information.

[0011] The geometric parameters of the mortise and tenon joint are determined based on the stroke information, and mortise and tenon parts are generated based on the target mortise and tenon template and the geometric parameters of the mortise and tenon joint, so that users can perform splicing operations; each mortise and tenon part has a unique identifier, which includes the type identifier, geometric parameters and interface direction information of the mortise and tenon part;

[0012] The system uses VR devices to acquire real-time information about the user's character assembly and judges the user's actions until the user completes the assembly of the target Chinese character.

[0013] A second aspect of this application provides a virtual Chinese character learning device combining VR and mortise and tenon joints, comprising:

[0014] The acquisition module is configured to acquire user input information through the VR device and determine the target Chinese character from the input information.

[0015] The extraction module is configured to perform skeleton extraction and topological analysis on the target Chinese character to obtain a set of structural units of the target Chinese character; the set of structural units includes at least the stroke information and connection type information of the target Chinese character;

[0016] The matching module is configured to match and determine the target mortise and tenon template in a preset set of mortise and tenon templates based on the connection type information;

[0017] The generation module is configured to determine the mortise and tenon geometric parameters based on stroke information, and generate mortise and tenon parts based on the target mortise and tenon template and mortise and tenon geometric parameters for users to perform splicing operations; wherein, each mortise and tenon part has a unique identifier, which includes the type identifier, geometric parameters and interface direction information of the mortise and tenon part;

[0018] The judgment module is configured to acquire the user's splicing information in real time through the VR device, and judge the user's operation until the user completes the splicing of the target Chinese character.

[0019] The beneficial effects of this application embodiment compared with the prior art are as follows: This application embodiment obtains user input information through VR device to determine the target Chinese character, then performs skeleton extraction and topological analysis on the target Chinese character to obtain a set of structural units of the target Chinese character, matches the target mortise and tenon template based on the connection type information in the set of structural units, determines the mortise and tenon geometric parameters based on the stroke information in the set of structural units, and finally obtains the user's splicing information in real time through VR device, judges the user's operation until the user completes the splicing of the target Chinese character. This method realizes the automatic mapping of Chinese character structure and three-dimensional mortise and tenon components, improves the learning experience of Chinese character morphology in three-dimensional space, and improves the interaction efficiency and recognition accuracy in the process of learning Chinese character structure through multimodal information collaboration. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a flowchart illustrating a virtual Chinese character learning method combining VR and mortise and tenon joints, provided in an embodiment of this application.

[0022] Figure 2 This is a real-world demonstration diagram of multi-user learning provided in an embodiment of this application.

[0023] Figure 3 This is a schematic diagram of number pre-matching during the splicing process provided in the embodiments of this application.

[0024] Figure 4 This is a schematic diagram of a splicing process provided in an embodiment of this application.

[0025] Figure 5This is a schematic diagram of another splicing process provided in an embodiment of this application.

[0026] Figure 6 This is a schematic diagram of another splicing process provided in the embodiments of this application.

[0027] Figure 7 This is a comparison chart of the average satisfaction of testers provided in the embodiments of this application.

[0028] Figure 8 This is a schematic diagram of a Chinese character virtual learning device that combines VR and mortise and tenon joints, provided in an embodiment of this application. Detailed Implementation

[0029] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0030] The following will describe in detail, with reference to the accompanying drawings, a method and apparatus for virtual learning of Chinese characters that combines VR and mortise and tenon joints according to an embodiment of this application.

[0031] As mentioned above, how to specifically combine VR with mortise and tenon joint technology in practical applications to provide users with a more efficient method for learning Chinese characters is a technical problem that needs to be solved.

[0032] In related technologies, some solutions use mortise and tenon joints to create Chinese character stroke teaching aids to assist users in learning Chinese characters. However, physical teaching aids take up a lot of space, cannot be updated synchronously as the user's character knowledge increases, and cannot provide prompts during the user's assembly process, which seriously affects the user experience.

[0033] In view of this, this application provides a virtual learning method for Chinese characters that combines VR and mortise and tenon joints. By extracting the skeleton and performing topological analysis on the structure of Chinese characters, the stroke connection relationships are transformed into calculable spatial structural relationships. Based on a parametric mortise and tenon template, automatic mapping and generation between the Chinese character structure and mortise and tenon components are achieved, improving the efficiency and consistency of component generation. Through a splicing verification mechanism of "number pre-matching + spatial geometric constraint judgment", real-time judgment of insertion direction, position alignment, and interference detection is achieved, improving splicing accuracy and structural stability.

[0034] Meanwhile, the solution can also be used for multi-user learning. In this case, the multi-user VR interaction module can synchronize pose and operation status to ensure the consistency of collaboration and the stability of system operation in the immersive environment, which is more suitable for more immersive application scenarios based on head-mounted display devices.

[0035] Figure 1 This is a flowchart illustrating a virtual Chinese character learning method combining VR and mortise and tenon joints, provided in an embodiment of this application. For example... Figure 1 As shown, the method includes the following steps:

[0036] In step S101, user input information is obtained through the VR device, and the target Chinese character is determined from the input information.

[0037] In step S102, the skeleton of the target Chinese character is extracted and topological analysis is performed to obtain a set of structural units of the target Chinese character.

[0038] The set of structural units includes at least the stroke information and connection type information of the target Chinese character.

[0039] In step S103, the target mortise and tenon template is determined by matching the connection type information in the preset mortise and tenon template set.

[0040] In step S104, the mortise and tenon geometric parameters are determined based on the stroke information, and the mortise and tenon parts are generated based on the target mortise and tenon template and the mortise and tenon geometric parameters for the user to perform the splicing operation.

[0041] Each mortise and tenon joint has a unique identifier, which includes the type identifier, geometric parameters, and interface direction information of the mortise and tenon joint.

[0042] In step S105, the user's splicing information is obtained in real time through the VR device, and the user's operation is judged until the user completes the splicing of the target Chinese character.

[0043] In some embodiments of this application, the method can be executed by a VR device. The VR device can be a head-mounted device such as a VR headset or VR glasses, or other devices; no limitation is made here.

[0044] In some embodiments of this application, user input information can be obtained first through a VR device. The method of user input is not limited, and includes, but is not limited to, voice input, body language input, and text input via a keyboard or touchscreen connected to the VR device. Furthermore, the user input information can be descriptive.

[0045] After receiving user input, the VR device can identify the target Chinese character. In some examples, the VR device can perform semantic analysis on this descriptive information to extract the target Chinese character for subsequent splicing learning. The target Chinese character can be cached in the VR device.

[0046] Next, the VR device can perform skeleton extraction and topological analysis on the target Chinese character to obtain a set of structural units for the target Chinese character. This set of structural units includes at least the stroke information and connection type information of the target Chinese character.

[0047] In some implementations, a target mortise and tenon template can be matched and determined from a set of preset mortise and tenon templates based on connection type information. Alternatively, the mortise and tenon geometric parameters can be determined based on stroke information, and mortise and tenon parts can be generated based on the target mortise and tenon template and mortise and tenon geometric parameters for users to perform splicing operations. Each mortise and tenon part has a unique identifier, which includes the type identifier, geometric parameters, and interface direction information of the mortise and tenon part.

[0048] VR devices can display specific mortise and tenon parts in the current scene, or provide access links in the current scene. Users can perform operations such as moving and splicing on the displayed mortise and tenon parts by clicking on the displayed mortise and tenon parts or the links between them.

[0049] VR devices can acquire the user's splicing information in real time and use this information to judge the user's operation until the user completes the splicing of the target Chinese character.

[0050] According to the technical solution provided in this application, user input information is obtained through a VR device to determine the target Chinese character. Then, skeleton extraction and topological analysis are performed on the target Chinese character to obtain a set of structural units. Based on the connection type information in the structural unit set, a target mortise and tenon template is matched. Based on the stroke information in the structural unit set, the geometric parameters of the mortise and tenon are determined. Finally, user splicing information is obtained in real time through the VR device, and the user's operation is judged until the user completes the splicing of the target Chinese character. This method achieves automatic mapping between Chinese character structure and three-dimensional mortise and tenon components, improves the learning experience of Chinese character morphology in three-dimensional space, and enhances the interactive efficiency and recognition accuracy in the Chinese character structure learning process through multimodal information collaboration.

[0051] In some embodiments of this application, the virtual Chinese character learning method can be used for single-user learning as well as multi-user learning. In multi-user learning, different users can learn independently or through comparative learning. In this case, multi-user synchronous operation in the virtual scene can be achieved through mirroring. Specifically, this includes:

[0052] Automatic connection mechanism: The networkDiscovery class function in the Mirror component is used to find hosts, publish the server, stop the search, and perform other operations.

[0053] Synchronize multi-user information: Helmet, hands, and name need to be updated in real time through communication between the client and server to create local users and track their own devices.

[0054] Synchronize interactive information: Use the [Command(requiresAuthority=false)] command to instruct the server to perform certain operations that can only be performed on the server, and use the [ClientRpc] command to synchronize the server with all clients.

[0055] A live demonstration of multi-user learning, such as Figure 2 As shown.

[0056] In some embodiments of this application, stroke information includes at least stroke type and stroke size. Stroke type includes, for example, horizontal, vertical, left-falling, and right-falling strokes. Stroke size includes at least stroke length and stroke width.

[0057] In some embodiments of this application, the connection types include at least vertical connection types, diagonal connection types, and individual structure types.

[0058] The connection type is determined in the following way:

[0059] Connectivity point detection is performed on the target Chinese character to obtain connectivity point information;

[0060] In response to the determination that the number of connection points is 0, the connection type is determined to be a single structure type;

[0061] In response to the determination that the number of connection points is greater than 1, the angle between the direction vectors corresponding to each connection point is determined; wherein, the direction vectors corresponding to the connection point include the direction vector starting from the connection point and the direction vector ending at the connection point.

[0062] In response to the absolute value of the difference between the angle and 90 degrees being less than a preset threshold, the connection type is determined to be a vertical connection type;

[0063] In response to the absolute value of the difference between the angle and 90 degrees being greater than or equal to a preset threshold, the connection type is determined to be an oblique connection type.

[0064] In other words, the standalone structure type corresponds to an independent mortise and tenon part that has no connection point with other mortise and tenon parts, the vertical connection type corresponds to a mortise and tenon part that is almost perpendicular to other mortise and tenon parts, and the oblique connection type corresponds to a mortise and tenon part that is obliquely connected with other mortise and tenon parts.

[0065] The preset threshold value can be set according to actual needs, for example, 10 degrees. At this time, the directional vector angle between two matching vertical connection type mortise and tenon parts is [80 degrees, 100 degrees], and the directional vector angle between two matching oblique connection type mortise and tenon parts is greater than (0 degrees, 80 degrees) and (100 degrees, 180 degrees).

[0066] In some embodiments of this application, the vertical connection type may include a T-shaped intersection type, a vertical intersection type, and a corner connection type. Specifically, in the T-shaped intersection type, one direction vector originates from the connection point, and the other direction vector originates from the connection point; in the vertical intersection type, both direction vectors originate from the connection point; and in the corner connection type, both direction vectors originate from the connection point.

[0067] Diagonal connection types can include diagonal T-shaped intersections, non-perpendicular intersections, and diagonal corner intersections. Specifically, in a diagonal T-shaped intersection, one direction vector originates at the connection point, and the other direction vector uses the connection point as its midpoint; in a non-perpendicular intersection, both direction vectors use the connection point as their midpoint; and in a diagonal corner intersection, both direction vectors originate at the connection point.

[0068] In other words, during the structural analysis phase, the target Chinese character can be extracted and its topology analyzed to generate a set of stroke segments and a set of connection points. The connection type can be classified according to the angle between the direction vectors of the segments at the connection points and the position of the connection points in the segments, thereby forming a set of structural units and their topological relationship data.

[0069] In some embodiments of this application, determining the geometric parameters of the tenon and mortise based on stroke information may include determining the tenon width, tenon length, mortise depth, and assembly tolerance based on stroke information.

[0070] That is, in the mortise and tenon generation stage, the corresponding template can first be selected from the pre-set parametric mortise and tenon template set according to the connection type; then, the mortise and tenon geometric parameters are calculated according to the stroke width, component thickness and assembly tolerance, the two-dimensional stroke skeleton is stretched into a three-dimensional basic component, and the tenon or mortise structure is generated at the corresponding position. Each mortise and tenon component is assigned a unique number when it is generated. This number includes at least the mortise and tenon type identifier, the size parameter code and the interface direction code, and is bound and stored with the three-dimensional geometric data of the component.

[0071] Users can then use the generated mortise and tenon parts to assemble Chinese characters.

[0072] In some embodiments of this application, providing feedback on the user's current operation information may include:

[0073] The unique identifiers of the two mortise and tenon parts operated on are obtained from the user's current operation information;

[0074] In response to determining the interface type matching of two mortise and tenon parts, spatial matching calculations are performed on the two mortise and tenon parts;

[0075] In response to the determination that the spatial matching calculation result is true, the connection relationship data of the two mortise and tenon parts is generated, and the user splicing topology is updated based on the connection relationship data; the user splicing topology uses components as nodes and the connection relationship of components as edges;

[0076] In response to the determination that the updated user splicing topology satisfies the preset connectivity, the updated user splicing topology is determined to be stable;

[0077] Upon confirming that the updated user splicing topology satisfies the closure condition, the splicing is deemed successful.

[0078] In other words, when a user performs a splicing operation, the VR device can obtain the unique identifiers of the two mortise and tenon parts being manipulated in real time, and then determine whether the interface types of the two mortise and tenon parts match. For example, if the interface type of the first mortise and tenon part is a vertical intersection type, and the interface type of the second mortise and tenon part is also a vertical intersection type, then the two mortise and tenon parts match in terms of interface type. However, if the interface type of the first mortise and tenon part is a T-shaped intersection type, and the interface type of the second mortise and tenon part is a diagonal T-shaped intersection type, then the two mortise and tenon parts do not match in terms of interface type.

[0079] For two mortise and tenon parts that are determined to have a matching interface type, it can be further determined whether they match spatially. Spatial matching calculations for the two mortise and tenon parts can include: determining the insertion direction matching based on the interface direction information of the two mortise and tenon parts; determining the position alignment of the two mortise and tenon parts; and determining interference detection between the two mortise and tenon parts.

[0080] The insertion direction matching determination can be performed in the following way:

[0081] Obtain the insertion direction vector of the tenon part and the insertion direction vector of the mortise part. ;

[0082] calculate and The angle between ;

[0083] Response to determination If the angle is less than the preset threshold, the insertion direction is determined to be compatible.

[0084] The preset angle threshold can be set from 0 degrees to 5 degrees.

[0085] Position alignment can be determined in the following way:

[0086] Obtain the center point coordinates of the tenon part and the center point coordinates of the mortise groove part ;

[0087] calculate and Euclidean distance ;in, for The three-dimensional coordinate components, for The three-dimensional coordinate components;

[0088] Response to determination If the value is less than the preset alignment threshold, the position is determined to be aligned.

[0089] The preset alignment threshold can be set to 0 to 0.5.

[0090] Interference detection and determination can be performed in the following ways:

[0091] Collision testing is performed on the component surfaces, excluding the mortise and tenon mating surfaces of the two mortise and tenon parts.

[0092] In response to the determination that no illegal geometric interference exists, the splicing method is determined.

[0093] One method to determine whether there is illegal geometric interference is to set up multiple collision boxes for each mortise and tenon part, and perform collision detection between the mortise and tenon part and its corresponding collision box to determine whether there is illegal geometric interference.

[0094] Taking the 3D development engine Unity as an example, its collision detection is based on the physics engine. When two collision volumes intersect, the physics engine considers a collision to have occurred. This application's embodiment can determine the absence of illegal geometric interference by judging whether clipping occurs when two mortise and tenon parts collide. The absence of clipping can be implemented using Unity itself.

[0095] In some embodiments of this application, if all the above judgments are true, i.e., the spatial matching calculation result is true, then the two mortise and tenon parts can be considered to be perfectly matched, and the user's splicing operation is correct. At this time, connection relationship data of the two mortise and tenon parts can be generated, and the user splicing topology with components as nodes and component connection relationships as edges can be updated based on the connection relationship data. Here, a component refers to a mortise and tenon part that meets the splicing requirements, i.e., a mortise and tenon part that has been spliced.

[0096] Furthermore, connectivity can be determined for the user-assembled topology. When a Chinese character is assembled, theoretically the topology is connected, or connected through hidden structures that are not yet assembled. If the user-assembled topology satisfies the preset connectivity, then the updated user-assembled topology can be considered stable.

[0097] Furthermore, once the target Chinese characters are assembled as a whole, their topology can be considered closed. Therefore, after confirming that the user's assembled topology is stable, it is possible to further determine whether it meets the closure condition; if it does, then the assembly can be considered successful.

[0098] In other words, VR devices can construct a connection topology based on existing connections and determine the structural connectivity and closed-loop conditions. When preset stability conditions are met, the target Chinese character structure is considered complete. After the connection is completed, an activation certificate can be issued to the user, and the virtual model associated with the target Chinese character can be switched from an inactive state to an active state, giving it interactive attributes.

[0099] This application's embodiments achieve automatic conversion between Chinese character structures and three-dimensional mortise and tenon components by performing topological analysis and parametric mortise and tenon mapping on the Chinese character structure; improve the accuracy of splicing verification and structural stability through number pre-matching and spatial geometric constraint judgment mechanisms; and ensure the consistency of collaborative operation and system operation stability in an immersive virtual environment through a multi-user pose synchronization mechanism.

[0100] The following example, using multi-user operation, details the virtual Chinese character learning method combining VR and mortise and tenon joints provided in this application. The development system environment for the method provided in this application is a Windows 11 64-bit operating system; the computer used for development is equipped with an AMD Ryzen 5 5600H processor. For core development software, Unity 2022.3.16f1c1 is used, employing a general rendering pipeline, with C# as the programming language and Visual Studio 2022 as the integrated development environment / editor. Supported plugin versions include PICO platform Unity Integrated Software Development Kit version 3.2.0, Unity Real-Time Preview plugin version 1.0.5, Unity extension plugin tripoforunity version 1.0.1, and network synchronization plugin Mirror version 96.0.1. The build output is an APK installation package. Hardware testing uses PICO Neo3 and PICO 4 devices. A local area network connection is required.

[0101] Step 1: Establishing a multi-user virtual learning environment

[0102] Users enter a virtual learning environment built on a real-time development platform through a head-mounted display device. The VR device system automatically connects multiple users through network communication mechanisms and synchronously updates the pose information and interaction status of each user, thereby ensuring the consistency of multi-user operations in the virtual scene.

[0103] Step 2: Determine the target Chinese character

[0104] Users provide target description information to the system via voice or text input. The system performs semantic processing on the input information, extracts the target Chinese characters, and stores the target Chinese characters as core data.

[0105] Meanwhile, the system can generate visual reference information associated with the target Chinese character to assist in subsequent structure construction.

[0106] Step 3: Analysis of Chinese Character Structure

[0107] The system calls the Chinese character structure analysis module to extract the skeleton of the target Chinese character and generate a set of stroke segments. Each segment records its start point, end point, and direction vector.

[0108] The system performs connection point detection on the set of line segments: if the distance between the endpoints of two line segments is less than a preset threshold, or if the two line segments have a geometric intersection, they are determined to be connection points. At the connection point, the angle between the direction vectors of adjacent line segments is calculated, and the connection type is classified according to the angle, thereby generating a set of structural units and their topological relationship data.

[0109] Step 4: Matching the mortise and tenon templates

[0110] As shown in Table 1, the system has a pre-set set of Chinese characters and mortise and tenon templates, including straight tenon templates, half-tenon templates, and corner tenon templates. Based on the connection type determined in step 3, the system automatically matches the corresponding mortise and tenon template and determines the interface attributes, including: tenon or mortise type and insertion direction.

[0111] Table 1. Comparison of Selected Chinese Characters and Mortise and Tenon Templates

[0112]

[0113] Step 5: Generating and assigning numbers to mortise and tenon components

[0114] The mortise and tenon component generation module generates components based on a pre-established standard mortise and tenon template library. It calculates the size mapping ratio according to the stroke width and thickness, and performs scaling transformation on the mortise and tenon template. Then, it calculates the rotation matrix according to the direction vector at the target connection point to make the template insertion direction consistent with the stroke direction. Finally, it aligns the template to the corresponding connection point position through translation transformation, thereby completing the generation of the mortise and tenon component.

[0115] After generating each mortise and tenon component, the system assigns it a unique number, which is then bound to and stored in conjunction with the component's three-dimensional geometric data. The generated set of mortise and tenon components is then loaded into the virtual splicing area.

[0116] Step 6: Assembly Determination

[0117] like Figure 3As shown, during the assembly process, after the interface type matches successfully, the system can also perform pre-matching of numbers based on the numbering information of the mortise and tenon parts. By determining whether there is an equivalent substitution relationship between the parts, the system queries the assembly table to determine the assembly relationship. If the numbering pre-matching passes, spatial geometric constraint calculations are further performed. When the above conditions are met simultaneously, connection relationship data is generated. The numbering information can be, for example, a unique identifier for the mortise and tenon parts.

[0118] For example, mortise and tenon parts C_2, C_3, B_2, and B_3 all belong to equivalence group A. For splicing group 1, it can include mortise and tenon part C_1 and two mortise and tenon parts from equivalence group A. Mortise and tenon part C_1 can be spliced ​​with one mortise and tenon part from equivalence group A, and the spliced ​​topology can be further spliced ​​with another mortise and tenon part from equivalence group A. That is, if we select mortise and tenon part B_2 from equivalence group A, C_1 + B_2 = C002. Furthermore, if we select mortise and tenon part C_3 from equivalence group A, then C_1 + B_2 + C_3 = C002 + C_3 = C003.

[0119] The mortise and tenon structure used in the system is abstracted into standardized digital components and mapped to the stroke structure of Chinese characters to support diverse Chinese character splicing learning scenarios. The specific splicing process is as follows: Figure 4 , Figure 5 and Figure 6 As shown.

[0120] The embodiments of this application can be applied to the construction of a multi-person virtual learning system for Chinese characters in the field of education and entertainment. By applying consumer-grade head-mounted display devices, it innovatively solves the problems of single interactive forms and insufficient scene immersion in traditional Chinese character learning.

[0121] In the test, 16 users were divided into four groups: A, B, C, and D. They first used the system to learn Chinese characters, and then learned the same content using other methods. After the test, users rated the two learning methods on four dimensions: immersion, convenience, fun, and information acquisition efficiency. Each dimension used a scale of 1-5, with 1 indicating very dissatisfied and 5 indicating very satisfied. The comparison of group A's scores is shown in Tables 2 and 3.

[0122] Table 2. Ratings of this application by Group A users.

[0123]

[0124] Table 3. Ratings of other options by Group A users.

[0125]

[0126] By calculating the average satisfaction level of all test participants and conducting a comparative analysis, the results are as follows: Figure 7 As shown, it can be concluded that users generally rate this application higher when using the system for learning Chinese characters. User feedback indicates that the system performs well in terms of immersion and engagement, mainly due to its immersive environment and gamified learning methods.

[0127] Furthermore, the method provided in this application embodiment also has strong interactivity and driving force that traditional learning methods do not possess, making it more suitable for immersive application scenarios based on head-mounted display devices.

[0128] All of the above-mentioned optional technical solutions can be combined in any way to form the optional embodiments of this application, and will not be described in detail here.

[0129] The following are embodiments of the apparatus described in this application, which can be used to execute the embodiments of the method described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method described in this application.

[0130] Figure 8 This is a schematic diagram of a Chinese character virtual learning device combining VR and mortise and tenon joints, provided in an embodiment of this application. For example... Figure 8 As shown, the device includes:

[0131] The acquisition module 801 is configured to acquire user input information through a VR device and determine the target Chinese character from the input information.

[0132] The extraction module 802 is configured to perform skeleton extraction and topological analysis on the target Chinese character to obtain a set of structural units of the target Chinese character; the set of structural units includes at least the stroke information and connection type information of the target Chinese character.

[0133] Matching module 803 is configured to match and determine the target mortise and tenon template in a preset mortise and tenon template set based on connection type information.

[0134] The generation module 804 is configured to determine the mortise and tenon geometric parameters based on stroke information, and generate mortise and tenon parts based on the target mortise and tenon template and mortise and tenon geometric parameters for users to perform splicing operations; wherein, each mortise and tenon part has a unique identifier, which includes the type identifier, geometric parameters and interface direction information of the mortise and tenon part.

[0135] The judgment module 805 is configured to acquire the user's splicing information in real time through the VR device, and to judge the user's operation until the user completes the splicing of the target Chinese character.

[0136] According to the technical solution provided in this application, user input information is obtained through a VR device to determine the target Chinese character. Then, skeleton extraction and topological analysis are performed on the target Chinese character to obtain a set of structural units. Based on the connection type information in the structural unit set, a target mortise and tenon template is matched. Based on the stroke information in the structural unit set, the geometric parameters of the mortise and tenon are determined. Finally, user splicing information is obtained in real time through the VR device, and the user's operation is judged until the user completes the splicing of the target Chinese character. This method achieves automatic mapping between Chinese character structure and three-dimensional mortise and tenon components, improves the learning experience of Chinese character morphology in three-dimensional space, and enhances the interactive efficiency and recognition accuracy in the Chinese character structure learning process through multimodal information collaboration.

[0137] In some implementations, the stroke information includes at least stroke type and stroke size; the connection type includes at least vertical connection type, diagonal connection type, and single structure type; the connection type is determined as follows: connection point detection is performed on the target Chinese character to obtain connection point information; in response to determining that the number of connection points is 0, the connection type is determined to be a single structure type; in response to determining that the number of connection points is greater than 1, the angle of the direction vector corresponding to each connection point is determined; wherein, the direction vector corresponding to the connection point includes a direction vector starting from the connection point and a direction vector with the connection point as the intermediate point; in response to determining that the absolute value of the difference between the angle and 90 degrees is less than a preset threshold, the connection type is determined to be a vertical connection type; in response to determining that the absolute value of the difference between the angle and 90 degrees is greater than or equal to a preset threshold, the connection type is determined to be a diagonal connection type.

[0138] In some implementations, the vertical connection types include T-shaped intersection, perpendicular intersection, and corner connection; wherein, in the T-shaped intersection type, one direction vector starts from the connection point and the other direction vector ends at the connection point; in the perpendicular intersection type, both direction vectors end at the connection point; and in the corner connection type, both direction vectors start from the connection point. The diagonal connection types include diagonal T-shaped intersection, non-perpendicular intersection, and diagonal corner connection; wherein, in the diagonal T-shaped intersection type, one direction vector starts from the connection point and the other direction vector ends at the connection point; in the non-perpendicular intersection type, both direction vectors end at the connection point; and in the diagonal corner connection type, both direction vectors start from the connection point.

[0139] In some implementations, the geometric parameters of the tenon and mortise are determined based on stroke information, including: determining the tenon width, tenon length, mortise depth, and assembly tolerance based on stroke information.

[0140] In some implementations, feedback is provided on the user's current operation information, including: obtaining the unique identifiers of the two mortise and tenon parts operated on from the user's current operation information; performing spatial matching calculations on the two mortise and tenon parts in response to determining that the interface types of the two mortise and tenon parts match; generating connection relationship data for the two mortise and tenon parts in response to determining that the spatial matching calculation result is true, and updating the user splicing topology based on the connection relationship data; the user splicing topology uses components as nodes and the connection relationships of components as edges; determining that the updated user splicing topology is stable in response to determining that it satisfies the preset connectivity; and determining that the splicing is successful in response to determining that the updated user splicing topology satisfies the closure condition.

[0141] In some implementations, spatial matching calculations are performed on the two mortise and tenon parts, including: determining the insertion direction matching based on the interface direction information of the two mortise and tenon parts; determining the position alignment of the two mortise and tenon parts; and determining the interference detection between the two mortise and tenon parts.

[0142] In some implementations, the insertion direction matching is determined as follows: the insertion direction vector of the tenon part is obtained. and the insertion direction vector of the mortise part. ;calculate and The angle between ; Response to determination If the angle is less than the preset threshold, the insertion direction is determined to be compatible.

[0143] In some implementations, the position alignment determination is performed as follows: obtaining the coordinates of the center point of the tenon part. and the center point coordinates of the mortise groove part ;calculate and Euclidean distance ;in, for The three-dimensional coordinate components, for The three-dimensional coordinate components; in response to determining If the value is less than the preset alignment threshold, the position is determined to be aligned.

[0144] In some implementations, interference detection is performed by: performing collision detection on the component surfaces other than the mortise and tenon mating surfaces of the two mortise and tenon parts; and determining the splicing method in response to the determination that no illegal geometric interference exists.

[0145] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0146] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A virtual learning method for Chinese characters combining VR and mortise and tenon joints, characterized in that, include: The target Chinese character is determined from the user input information obtained through VR devices. Skeleton extraction and topological analysis are performed on the target Chinese character to obtain a set of structural units of the target Chinese character; the set of structural units includes at least the stroke information and connection type information of the target Chinese character; The target mortise and tenon template is determined by matching it from a set of preset mortise and tenon templates based on the connection type information. The geometric parameters of the mortise and tenon joint are determined based on the stroke information, and mortise and tenon parts are generated based on the target mortise and tenon template and the geometric parameters of the mortise and tenon joint, so that users can perform splicing operations; wherein, each mortise and tenon part has a unique identifier, the unique identifier including the type identifier, geometric parameters and interface direction information of the mortise and tenon part; The system uses VR devices to acquire real-time information about the user's character assembly and judges the user's actions until the user completes the assembly of the target Chinese character.

2. The method according to claim 1, characterized in that, The stroke information includes at least the stroke type and stroke size; The connection types include at least vertical connection types, diagonal connection types, and individual structure types; The connection type is determined in the following manner: Connectivity point detection is performed on the target Chinese character to obtain connectivity point information; In response to determining that the number of connection points is 0, the connection type is determined to be a single structure type; In response to the determination that the number of connection points is greater than 1, the angle between the direction vectors corresponding to each connection point is determined; wherein, the direction vectors corresponding to the connection point include the direction vector starting from the connection point and the direction vector ending at the connection point. In response to determining that the absolute value of the difference between the angle and 90 degrees is less than a preset threshold, the connection type is determined to be a vertical connection type; In response to determining that the absolute value of the difference between the angle and 90 degrees is greater than or equal to a preset threshold, the connection type is determined to be an oblique connection type.

3. The method according to claim 2, characterized in that, The vertical connection types include T-shaped intersection type, vertical intersection type, and corner connection type; In the T-shaped intersection type, one direction vector starts from the connection point, and the other direction vector starts from the connection point. In the vertical intersection type, both directional vectors take the connection point as the midpoint; In the aforementioned corner joint type, both directional vectors originate from the connection point; The oblique connection types include oblique T-shaped intersection type, non-perpendicular intersection type, and oblique angle type; In the aforementioned oblique T-shaped intersection type, one direction vector starts from the connection point, and the other direction vector starts from the connection point; In the non-perpendicular intersection type, both directional vectors take the connection point as the midpoint; In the oblique angle connection type, both directional vectors originate from the connection point.

4. The method according to claim 1, characterized in that, Determining the geometric parameters of the mortise and tenon joint based on stroke information includes: The tenon width, tenon length, mortise depth, and assembly tolerances are determined based on the stroke information.

5. The method according to claim 1, characterized in that, Provide feedback on the user's action, including: The unique identifiers of the two mortise and tenon parts operated on are obtained from the user's current operation information; In response to determining the interface type matching of two mortise and tenon parts, spatial matching calculations are performed on the two mortise and tenon parts; In response to the determination that the spatial matching calculation result is true, the connection relationship data of the two mortise and tenon parts is generated, and the user splicing topology is updated based on the connection relationship data; the user splicing topology uses components as nodes and the connection relationship of components as edges; In response to the determination that the updated user splicing topology satisfies the preset connectivity, the updated user splicing topology is determined to be stable; Upon confirming that the updated user splicing topology satisfies the closure condition, the splicing is deemed successful.

6. The method according to claim 5, characterized in that, Spatial matching calculations are performed for the two mortise and tenon parts, including: The insertion direction is determined based on the interface direction information of the two mortise and tenon parts. Determine the alignment of the two mortise and tenon parts; Interference detection was performed on the two mortise and tenon parts to determine their integrity.

7. The method according to claim 6, characterized in that, The insertion direction matching determination is performed in the following manner: Obtain the insertion direction vector of the tenon part and the insertion direction vector of the mortise part. ; calculate and The angle between ; Response to determination If the angle is less than the preset threshold, the insertion direction is determined to be compatible.

8. The method according to claim 6, characterized in that, The position alignment determination is performed in the following manner: Obtain the center point coordinates of the tenon part and the center point coordinates of the mortise groove part ; calculate and Euclidean distance ;in, for The three-dimensional coordinate components, for The three-dimensional coordinate components; Response to determination If the value is less than the preset alignment threshold, the position is determined to be aligned.

9. The method according to claim 6, characterized in that, The interference detection and determination are performed in the following manner: Collision testing is performed on the component surfaces, excluding the mortise and tenon mating surfaces of the two mortise and tenon parts. In response to the determination that no illegal geometric interference exists, the splicing method is determined.

10. A virtual learning device for Chinese characters combining VR and mortise and tenon joints, characterized in that, include: The acquisition module is configured to acquire user input information through a VR device and determine the target Chinese character from the input information; The extraction module is configured to perform skeleton extraction and topological analysis on the target Chinese character to obtain a set of structural units of the target Chinese character. The set of structural units includes at least the stroke information and connection type information of the target Chinese character; The matching module is configured to match and determine the target mortise and tenon template in a preset set of mortise and tenon templates based on the connection type information; The generation module is configured to determine the mortise and tenon geometric parameters based on stroke information, and generate mortise and tenon parts based on the target mortise and tenon template and the mortise and tenon geometric parameters for users to perform splicing operations; wherein, each mortise and tenon part has a unique identifier, the unique identifier including the type identifier, geometric parameters and interface direction information of the mortise and tenon part; The judgment module is configured to acquire the user's splicing information in real time through the VR device, and judge the user's operation until the user completes the splicing of the target Chinese character.

Images