Immersive collaborative cultivation situation simulation system and method based on augmented reality

Abstract

The invention provides an immersive cooperative care situation simulation system and method based on augmented reality, and the method comprises the steps: generating an interaction path of a corresponding user in an interactive virtual scene through the action interaction data and spatial positioning information of each user, and determining the behavior conflict trend of each user in the interactive virtual scene; based on all the behavior conflict trends, determining a behavior constraint area of the action of each user in the interaction process; when it is detected that the interaction path of the user and the behavior constraint area are spatially overlapped, monitoring current physiological data of the user, and determining a collaborative correction path of the user in the interactive virtual scene in combination with action interaction data of the user; and indicating a user to perform interaction correction according to the interaction scene picture according to each collaborative correction path. By adopting the scheme of the invention, behavior conflicts during multi-person immersive collaborative interaction in a virtual scene can be avoided.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of scenario simulation, and more particularly, to an immersive collaborative co-creation scenario simulation system and method based on extended reality. BACKGROUND

[0002] With the rapid development of extended reality (including virtual reality, augmented reality, and mixed reality) technology, immersive collaborative co-creation systems have shown great potential in education, training, rehabilitation, and social interaction. These systems allow multiple users to enter a shared, highly realistic virtual scene (such as a collaborative learning environment, a simulation training field, or an interactive game space) by wearing devices and control their virtual characters to interact and cooperate ("co-creation").

[0003] However, during multi-user immersive collaborative interaction, the naturalness of user behavior and the limitations of spatial perception often lead to behavior conflicts, which mainly manifest as physical space conflicts, task goal conflicts, and attention / focus conflicts. Physical space conflicts: when users' virtual characters move and operate in the shared virtual space, due to differences in physical space perception, operation delays, or misunderstandings of intentions, spatial interference phenomena such as path crossing, collision, and overlapping operation areas may occur. Task goal conflicts: when collaborating to complete tasks, users may have conflicts in action due to different strategies, resource competition, or poor communication, such as competing for the same virtual object or blocking the operation path of the other party. Attention / focus conflicts: users' attention focus may not be consistent, causing one party to interfere with the other party's ongoing operation. Therefore, how to avoid behavior conflicts during multi-user immersive collaborative interaction in virtual scenarios has become a problem faced by the industry. SUMMARY

[0004] The present application provides an immersive collaborative co-creation scenario simulation system and method based on extended reality, which can avoid behavior conflicts during multi-user immersive collaborative interaction in virtual scenarios.

[0005] In a first aspect, the present application provides a behavior conflict regulation method for immersive collaborative co-creation, applied to an immersive collaborative co-creation scenario simulation system based on extended reality. The immersive user-interactive virtual scene is created through an extended reality device, and each user corresponds to a virtual character and an interactive scene picture in the interactive virtual scene. The method includes the following steps: Collecting action interaction data and spatial positioning information of each user based on the extended reality device; Generating an interaction path of the corresponding user in the interactive virtual scene based on the action interaction data and spatial positioning information of each user, and determining the behavior conflict trend of each user in the interactive virtual scene according to the spatial coupling characteristics between each interaction path; reconstruct paths of virtual roles corresponding to each user in the interactive virtual scene based on all behavior conflict tendencies, to generate behavior constraint regions of actions of each user in the interaction process; When it is detected that the interaction path of the user spatially overlaps with the behavior constraint region, monitor current physiological data of the user, and generate a collaborative correction path of the user in the interactive virtual scene based on dynamic change characteristics of the physiological data and action interaction data of the user, to further obtain collaborative correction paths of each user in the interactive virtual scene; convert each collaborative correction path into a visual instruction and input the visual instruction into an interaction scene picture of the corresponding user, to instruct the user to perform interaction correction.

[0006] In some embodiments, generating the interaction path of the corresponding user in the interactive virtual scene based on the action interaction data and the spatial positioning information of each user specifically includes: select a user as a selected user, perform data alignment on the action interaction data and the spatial positioning information of the selected user, to obtain action-spatial alignment data of the selected user; determine the interaction path of the selected user in the interactive virtual scene according to the action-spatial alignment data of the selected user; continue to determine the interaction path of each remaining user in the interactive virtual scene.

[0007] In some embodiments, determining the behavior conflict tendency of each user in the interactive virtual scene according to spatial coupling characteristics between each interaction path specifically includes: determine spatial coupling characteristics between each interaction path; identify a potential conflict region according to the spatial coupling characteristics between each interaction path; determine the behavior conflict tendency of each user in the interactive virtual scene according to the potential conflict region.

[0008] In some embodiments, reconstructing paths of virtual roles corresponding to each user in the interactive virtual scene based on all behavior conflict tendencies, to generate behavior constraint regions of actions of each user in the interaction process specifically includes: arrange all behavior conflict tendencies according to corresponding user levels, to obtain a behavior conflict tendency sequence; reconstruct paths of virtual roles corresponding to each user in the interactive virtual scene based on the behavior conflict tendency sequence, to obtain reconstructed paths of virtual roles corresponding to each user; determine behavior constraint regions of actions of each user in the interaction process through all reconstructed paths.

[0009] In some embodiments, generating the collaborative correction path of the user in the interactive virtual scene based on the dynamic change feature of the physiological data and the action interaction data of the user specifically comprises: determining the dynamic change feature of the physiological data; determining the multi-modal feature vector of the user according to the dynamic change feature and the action interaction data of the user; predicting the action trend of the user based on the multi-modal feature vector of the user; generating the collaborative correction path of the user in the interactive virtual scene according to the action trend of the user and the behavior constraint region.

[0010] In some embodiments, converting each collaborative correction path into a visual instruction and inputting the visual instruction into the interactive scene picture of the corresponding user to instruct the user to perform interactive correction specifically comprises: obtaining the current action state of the user; parsing each collaborative correction path into an instruction sequence of the corresponding user; determining a visual instruction according to the instruction sequence and inputting the visual instruction into the interactive scene picture of the corresponding user to instruct the user to perform interactive correction according to the interactive scene picture in combination with the current action state of the user.

[0011] In some embodiments, the action interaction data includes joint motion data, gesture operation, and button click of the user.

[0012] In some embodiments, the spatial positioning information includes three-dimensional coordinate position and orientation information of the user.

[0013] In some embodiments, the current physiological data of the user is monitored through a physiological sensor built in an extended reality device.

[0014] In a second aspect, the present application provides an immersive collaborative breeding scenario simulation system based on extended reality, which comprises a behavior conflict regulation unit, and the behavior conflict regulation unit comprises: a collection module configured to collect action interaction data and spatial positioning information of each user based on an extended reality device; a processing module configured to generate an interaction path of the corresponding user in an interactive virtual scene through the action interaction data and spatial positioning information of each user, and determine a behavior conflict trend of each user in the interactive virtual scene according to the spatial coupling feature between each interaction path; the processing module is further configured to reconstruct the path of the corresponding virtual role of each user in the interactive virtual scene based on all the behavior conflict trends, and generate a behavior constraint region of the action of each user in the interactive process; The processing module is further configured to monitor current physiological data of the user when detecting that the interaction path of the user spatially overlaps with the behavior constraint region, and generate a collaborative correction path of the user in the interactive virtual scene based on dynamic change characteristics of the physiological data and action interaction data of the user, thereby obtaining collaborative correction paths of the users in the interactive virtual scene. The execution module is configured to convert each collaborative correction path into a visual instruction and input the visual instruction into an interaction scene picture of a corresponding user to instruct the user to perform interaction correction.

[0015] The technical scheme provided by the embodiments disclosed in the present application has the following beneficial effects: In the immersive collaborative co-cultivation scenario simulation system and method based on extended reality provided in the present application, action interaction data and spatial positioning information of each user are first collected based on an extended reality device; an interaction path of a corresponding user in the interactive virtual scene is generated based on the action interaction data and spatial positioning information of each user, a behavior conflict trend of each user in the interactive virtual scene is determined according to spatial coupling characteristics between each interaction path; a path of a corresponding virtual role of each user in the interactive virtual scene is path-reconstructed based on all behavior conflict trends, a behavior constraint region of an action of each user in an interaction process is generated; when detecting that an interaction path of a user spatially overlaps with the behavior constraint region, current physiological data of the user is monitored, and a collaborative correction path of the user in the interactive virtual scene is generated based on dynamic change characteristics of the physiological data and action interaction data of the user, thereby obtaining collaborative correction paths of the users in the interactive virtual scene; each collaborative correction path is converted into a visual instruction and input into an interaction scene picture of a corresponding user to instruct the user to perform interaction correction.

[0016] It can be seen that, in the collaborative co-cultivation scenario simulation process, the present application collects user actions and positioning data in real time through an extended reality device, provides an accurate basis for conflict analysis; interaction paths are generated and spatial coupling characteristics are analyzed to realize early prediction of conflict trends; paths are reconstructed based on trends and behavior constraint regions are defined to actively isolate high-risk areas; physiological data is monitored when paths overlap, and personalized collaborative correction paths are generated in combination with action information to adapt to user states; finally, visual instructions are converted to guide correction to ensure real-time feedback. Overall, the scheme effectively reduces the behavior conflict risk in multi-user virtual collaboration, improves the safety and collaboration efficiency of interaction through data-driven prediction, personalized adjustment and intuitive guidance, and avoids behavior conflicts in multi-user immersive collaborative interaction in a virtual scene. BRIEF DESCRIPTION OF DRAWINGS

[0017] Figure 1is an exemplary flowchart of an extended reality-based immersive co-creation scenario simulation method according to some embodiments of the present application; Figure 2 is a core framework diagram of interaction logic according to some embodiments of the present application; Figure 3 is an exemplary flowchart of determining a behavior constraint region according to some embodiments of the present application; Figure 4 is a structural schematic diagram of a behavior conflict adjustment unit according to some embodiments of the present application; Figure 5 is a structural schematic diagram of a computer device for implementing an extended reality-based immersive co-creation scenario simulation method according to some embodiments of the present application. DETAILED DESCRIPTION

[0018] In order to better understand the technical solutions of the present application, the technical solutions of the present application will be described in detail below in combination with the drawings in the specification and specific embodiments.

[0019] Reference Figure 1 The figure is an exemplary flowchart of an extended reality-based immersive co-creation scenario simulation method according to some embodiments of the present application. In this embodiment, an immersive and interactive virtual scene needs to be created through an extended reality device first, and virtual roles of each user in the interactive virtual scene are generated based on user interaction instructions. In this embodiment, the immersive and interactive virtual scene created through the extended reality device can be constructed by using a three-dimensional modeling technology to build an immersive and interactive virtual scene including a kindergarten, a family and a community, which restores the spatial layout and item arrangement of each place to enhance the sense of immersion. Wherein, generating virtual roles of each user in the interactive virtual scene based on user interaction instructions is to create virtual roles of each user in the interactive virtual scene based on personal information (such as identity, age) and interaction instructions (such as appearance and occupation instructions of creating a role) input by the user after the user wears the extended reality device and enters the virtual scene. Each user corresponds to an interactive scene picture in the interactive virtual scene, and each virtual role has a unique identifier for subsequent data processing. In other embodiments, it can also be implemented in this way, which is not limited here, such as Figure 1 The extended reality-based immersive co-creation scenario simulation method according to some embodiments of the present application includes the following steps: In step 101, action interaction data and spatial positioning information of each user are collected based on the extended reality device.

[0020] In a specific implementation, the joint motion data, gesture operation, and button click motion interaction data of the user are collected in real time by the inertial measurement sensor, camera, and motion capture module built in the extended reality device, and the three-dimensional coordinate position and orientation information of the user in the physical space are obtained by using an external positioning base station, the three-dimensional coordinate position and orientation information are taken as the spatial positioning information of the corresponding user, the spatial positioning information of each user is obtained, and the three-dimensional coordinate position and orientation information are mapped to the coordinate system of the interactive virtual scene. In other embodiments, other manners can also be used for collection, which is not limited here.

[0021] It should be noted that the motion interaction data in the present application represents the data of the user's motion interaction when operating the corresponding virtual character, and the spatial positioning information represents the information of the user's positioning position in space when operating the corresponding virtual character, which can be used for analyzing the interaction of the user.

[0022] In some embodiments, with reference to Figure 2 As shown in the figure, it is the core framework diagram of the interaction logic in some embodiments of the present application, as Figure 2 described, the bidirectional data flow is that the user inputs the motion interaction data and spatial position data to the virtual scene through the extended reality device (such as a virtual reality headset, a motion capture device), and the system outputs the visual instruction, conflict warning, and collaborative path guidance to the user through the virtual scene; the conflict-adjustment closed loop is based on the spatial coupling analysis of the multi-user interaction path, and the potential behavior conflict is identified in real time, the “behavior constraint area” is generated through path reconstruction, the collaborative path is dynamically adjusted combined with the physiological data, and the closed loop of “detection-intervention-feedback” is formed; the collaborative conjugate mechanism is that the system generates the personalized collaborative path according to the motion characteristics and physiological state of each user, and ensures that the multi-user behavior avoids conflict and maintains the consistency of the collaborative goal in the virtual scene.

[0023] In step 102, the interaction path of the corresponding user in the interactive virtual scene is generated through the motion interaction data and spatial positioning information of each user, and the behavior conflict trend of each user in the interactive virtual scene is determined according to the spatial coupling characteristics between each interaction path.

[0024] In some embodiments, the generation of the interaction path of the corresponding user in the interactive virtual scene through the motion interaction data and spatial positioning information of each user can be realized by the following steps: A user is selected as a selected user, the motion interaction data and spatial positioning information of the selected user are data-aligned, and the motion-space alignment data of the selected user is obtained; The interaction path of the selected user in the interactive virtual scene is determined according to the motion-space alignment data of the selected user; continue to determine an interaction path of the remaining users in the interactive virtual scene.

[0025] In a specific implementation, first, the action interaction data and the spatial positioning information of the selected user are aligned to a unified time coordinate system by using a time stamp synchronization algorithm (such as the network time protocol), to obtain aligned data. The missing data in the aligned data within 20 ms is filled by using a linear interpolation method. The filled data is taken as the action-spatial aligned data of the selected user, wherein the action-spatial aligned data represents the aligned data of the user in the spatial position and the action interaction. Then, repetitive action patterns (such as grabbing and walking) are identified from the action-spatial aligned data based on a dynamic time warping algorithm. The selected user is identified in an action state (such as “reaching out → grabbing → moving an object”) by using a hidden Markov model in combination with the action-spatial aligned data. The selected user in the interactive virtual scene is generated in the interaction path by using a Bezier curve to fit key points (such as action start / stop points) in combination with the repetitive action patterns and the action state. In other embodiments, other ways can also be used to determine the interaction path, which is not limited here.

[0026] It should be noted that the interaction path in the present application represents the path of the user when interacting in the interactive virtual scene, and can be used to analyze the interaction of the user in the interactive virtual scene, to facilitate subsequent conflict detection and path planning.

[0027] In some embodiments, the behavior conflict trend of each user in the interactive virtual scene can be determined according to the spatial coupling characteristics between the interaction paths by using the following steps: determine the spatial coupling characteristics between the interaction paths; identify a potential conflict area according to the spatial coupling characteristics between the interaction paths; determine the behavior conflict trend of each user in the interactive virtual scene according to the potential conflict area.

[0028] In specific implementation, a dynamic time warping algorithm is used to calculate the temporal similarity sequence between interaction paths, and Euclidean distance is used to measure the spatial proximity between interaction paths. Based on a spatiotemporal cube, the spatial coupling characteristics between each interaction path are determined by combining the temporal similarity sequence and the spatial proximity. For example, if the spatial distance between paths is less than 0.5 meters and the directional angle is less than 30 degrees within 10 seconds, it is marked as strong coupling. Here, the spatial coupling characteristics represent the temporal and spatial proximity of each interaction path. Then, a density clustering algorithm is used to cluster the regions corresponding to strong coupling in the spatial coupling characteristics to identify potential... In the conflict zone, the potential conflict zone refers to the potential conflict area between various interaction paths. The core point of the potential conflict zone is defined as a spatiotemporal point containing at least two paths within a radius of 0.3 meters. Finally, the spatiotemporal density, duration, and action state of the potential conflict zone are combined to use a weighted decision tree model to evaluate the severity of the conflict. The output includes the conflict type (path intersection, resource contention), risk level (low / medium / high), and scope of impact for each user, thus obtaining the behavioral conflict trend of each user in the interactive virtual scene. Other methods can be used to determine the conflict in other embodiments, which are not limited here.

[0029] It should be noted that the behavioral conflict trend in this application represents the trend of conflicting user behaviors in an interactive virtual scene, which can be used to analyze the user interaction in the interactive virtual scene and provide a basis for subsequent path reconstruction.

[0030] In step 103, the paths of the virtual characters corresponding to each user in the interactive virtual scene are reconstructed based on all behavioral conflict trends, generating behavioral constraint areas for each user's actions during the interaction process.

[0031] In some embodiments, reference Figure 3 As shown, this diagram is an exemplary flowchart for determining behavioral constraint regions in some embodiments of this application. In this embodiment, the paths of the virtual characters corresponding to each user in the interactive virtual scene are reconstructed based on all behavioral conflict trends to generate the behavioral constraint regions for each user's actions during the interaction process. This can be achieved through the following steps: First, in step 1031, all behavioral conflict trends are arranged according to their corresponding user levels to obtain a behavioral conflict trend sequence; Secondly, in step 1032, the paths of the virtual characters corresponding to each user in the interactive virtual scene are reconstructed based on the behavioral conflict trend sequence to obtain the reconstructed paths of the virtual characters corresponding to each user. Finally, in step 1033, the behavioral constraint areas of each user's actions during the interaction process are determined through all the reconstruction paths.

[0032] In a specific implementation, first, the user level (e.g., the teacher role is higher than the student, and the administrator role is higher than the ordinary participant) is obtained from the database corresponding to the interactive virtual scene, a priority queue algorithm (e.g., the priority queue in Java) is used to arrange all behavior conflict trends from high to low according to the user level, a behavior conflict trend sequence is formed, and it is ensured that the path of the high-level user remains unchanged. Then, based on the behavior conflict trend sequence, the paths of low-level users are reconstructed using the reciprocity speed obstacle algorithm, and the paths of the virtual roles of each user are obtained as the constraint condition of the paths of the high-level users. Meanwhile, the curvature continuity of each reconstructed path is optimized using the Bezier curve fitting technology. The reconstructed path represents the path after the path with conflict is reconstructed. Finally, a Gaussian kernel function is used to generate a behavior constraint region in a three-dimensional space with the conflict points of each reconstructed path as the center, the range of the behavior constraint region is controlled by adjusting the standard deviation of the kernel function (e.g., the field strength range of the serious conflict region is set to a radius of 2 meters), and the behavior constraint region of the action of each user in the interaction process is obtained. In other embodiments, other methods can also be used to determine the behavior constraint region, which is not limited here.

[0033] It should be noted that the behavior constraint region in the present application represents a constraint region in which each user avoids action interference during the interaction process. The behavior constraint region can be bound to the virtual role of the user to achieve dynamic and real-time behavior constraint and prompting, and to provide a basis for subsequent interaction correction.

[0034] In step 104, when it is detected that the interaction path of the user overlaps with the behavior constraint region, the current physiological data of the user is monitored, and a collaborative correction path of the user in the interactive virtual scene is generated based on the dynamic change characteristics of the physiological data and the action interaction data of the user, and then the collaborative correction paths of each user in the interactive virtual scene are obtained.

[0035] In a specific implementation, when it is detected that the interaction path of the user overlaps with the behavior constraint region, the current physiological data of the user can be monitored in the following manner: when it is detected that the interaction path of the user overlaps with the behavior constraint region, the physiological data acquisition process is immediately triggered by the extended reality device: the multi-modal physiological signals of the heart rate, skin conductance, and respiratory rate of the user are synchronously collected by the physiological sensor (e.g., a heart rate band or an electromyography patch) built in the extended reality device at a sampling rate of 100 Hz or higher; at the same time, the event timestamp is recorded to ensure that the time accuracy error of the action data is less than 10 ms; after the collected multi-modal physiological signals are preprocessed by band-pass filtering and baseline correction, the preprocessed multi-modal physiological signals are used as the current physiological data of the user. The physiological data represents the physiological data of the user after the behavior interference. In other embodiments, other methods can also be used to monitor the physiological data, which is not limited here.

[0036] In some embodiments, generating a collaborative correction path for a user in the interactive virtual scene based on the dynamic change characteristics of the physiological data and the user's action interaction data can be achieved through the following steps: Determine the dynamic change characteristics of the physiological data; The user's multimodal feature vector is determined based on the dynamic change characteristics and the user's action interaction data; Predict the user's action trends based on the user's multimodal feature vectors; Based on the user's action trends and the behavioral constraint area, a collaborative correction path is generated for that user in the interactive virtual scene.

[0037] In practice, firstly, a sliding window method (e.g., window size of 5 seconds, step size of 1 second) is used to extract dynamic change features from physiological data. These dynamic change features represent the user's dynamic physiological changes, including the standard deviation of heart rate variability, the rising slope of skin conductance response, and the fluctuation amplitude of respiratory rate. Next, the dynamic change features and action interaction data are normalized and then concatenated according to timestamps to form a feature vector containing multimodal information. This feature vector is used as the user's multimodal feature vector. Then, a Long Short-Term Memory (LSTM) network is used to train the multimodal feature vector, predicting the user's behavior within the next 3 seconds through forward propagation. The action trend (such as movement direction, probability distribution of operation intention) is defined as the user's trend in action. Finally, based on the constraints of the behavior constraint area and combined with the predicted action trend, the A* algorithm is used to search for the optimal path in the navigation grid of the virtual scene. At the same time, a reinforcement learning framework is introduced, using physiological comfort (such as avoiding drastic fluctuations in physiological data) and task efficiency (such as the shortest path) as reward functions to iteratively optimize the optimal path. Finally, a corrected path that meets the collaborative requirements is generated, and this corrected path is used as the user's collaborative corrected path in the interactive virtual scene. Other methods can be used to determine this path in other embodiments, which are not limited here.

[0038] It should be noted that the collaborative correction path in this application refers to a correction path that does not conflict when users interact collaboratively in an interactive virtual scene. It can be used to correct the user's path to avoid conflicts between users.

[0039] In step 105, each collaborative correction path is converted into a visual instruction and input into the corresponding user's interactive scene screen to instruct the user to perform interactive correction.

[0040] In some embodiments, converting each collaborative correction path into a visual instruction input to the corresponding user's interactive scene screen to instruct the user to perform interactive correction can be achieved through the following steps: Acquire a current action state of a user; Parse each collaborative correction path into an instruction sequence corresponding to the user; Determine a visualization instruction according to the instruction sequence, and input the visualization instruction into an interactive scene picture of the corresponding user to instruct the user to interactively correct according to the interactive scene picture in combination with the current action state of the user.

[0041] In a specific implementation, first, the joint angle and gesture action data of the user are collected in real time by the sensors and cameras built in the extended reality device, the collected data are subjected to feature extraction by a convolutional neural network, and the current action state (such as walking, grabbing, or waiting) of the user is recognized in combination with the feature extraction result by a hidden Markov model; then, the collaborative correction path is decomposed into an instruction sequence containing a timestamp, a spatial coordinate, and an action parameter (speed or direction), for example, structured in the format of “arrive at coordinate (x, y, z) at t time with a speed of v”; then, the visualization instruction is dynamically generated according to the current action state of the user and the instruction sequence by a rule engine: if the user action lags behind the path planning, a highlighted arrow is generated to indicate the forward direction, if the user approaches a path turning point, a particle special effect is used to mark the turning target, and if the user is in a resource interaction area, a semi-transparent light column is used to prompt the operation range, wherein the visualization instruction represents the instruction after the path is visualized; finally, the visualization instruction is superimposed into the interactive scene picture by using the graphic rendering technology of the open graphics library, real-time rendering of 60 frames per second is realized by using the web image library, and adaptive scaling and projection transformation are performed according to the screen parameters (resolution and field of view) of the user device, so that the visualization instruction is clear and does not block key scene information, effectively guiding the user to interactively correct.

[0042] In addition, another aspect of the present application, in some embodiments, the present application provides an extended reality-based immersive collaborative co-cultivation scenario simulation system, which comprises a behavior conflict regulation unit, which is described with reference to Figure 4 The figure is a structural schematic diagram of a behavior conflict regulation unit according to some embodiments of the present application. The behavior conflict regulation unit 400 comprises a collection module 401, a processing module 402, and an execution module 403, which are described as follows: The collection module 401 is mainly used for collecting the action interaction data and spatial positioning information of each user based on the extended reality device in the present application; The processing module 402 is used for generating an interactive path of the corresponding user in the interactive virtual scene according to the action interaction data and spatial positioning information of each user, and determining the behavior conflict trend of each user in the interactive virtual scene according to the spatial coupling characteristics between each interactive path in the present application; It should be noted that the processing module 402 is further configured to reconstruct paths of virtual roles corresponding to each user in the interactive virtual scene based on all behavior conflict trends, and generate behavior constraint regions of actions of each user in the interactive process. In addition, it should be noted that the processing module 402 is further configured to monitor current physiological data of the user when detecting that the interaction path of the user overlaps with the behavior constraint region in space, and generate a collaborative correction path of the user in the interactive virtual scene based on dynamic change characteristics of the physiological data and action interaction data of the user, thereby obtaining collaborative correction paths of each user in the interactive virtual scene. The execution module 403 is mainly configured to convert each collaborative correction path into a visual instruction and input the visual instruction into an interaction scene picture of a corresponding user to instruct the user to perform interaction correction.

[0043] In addition, the present application further provides a computer device, which comprises a memory and a processor, the memory stores code, and the processor is configured to acquire the code and execute the above-mentioned immersive collaborative co-creation scenario simulation method based on extended reality.

[0044] In some embodiments, with reference to Figure 5 The figure is a structural schematic diagram of a computer device for implementing the immersive collaborative co-creation scenario simulation method based on extended reality according to some embodiments of the present application. The immersive collaborative co-creation scenario simulation method based on extended reality in the above-mentioned embodiments can be implemented by the computer device shown in the figure. The computer device 500 comprises at least one processor 501, a communication bus 502, a memory 503, and at least one communication interface 504. Figure 5 The processor 501 can be a general central processing unit (CPU) or an application-specific integrated circuit (ASIC).

[0045] The processor 501 can be a general central processing unit (CPU) or an application-specific integrated circuit (ASIC).

[0046] The communication bus 502 can be used to transmit information between the above-mentioned components.

[0047] The memory 503 can be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM), or other type of dynamic storage device that can store information and instructions, an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disk storage, a magnetic disk storage or other magnetic storage devices, or any other medium capable of storing desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited to this. The memory 503 can exist independently, and is connected to the processor 501 through the communication bus 502. The memory 503 can also be integrated with the processor 501.

[0048] The memory 503 is configured to store program codes for implementing the solutions of the present application, and the processor 501 is configured to control the execution of the program codes. The processor 501 is configured to execute the program codes stored in the memory 503. The program codes can include one or more software modules. The methods used in the above embodiments can be implemented by the processor 501 and one or more software modules in the program codes in the memory 503.

[0049] The communication interface 504 is configured to communicate with other devices or communication networks, such as an Ethernet, a radio access network (RAN), a wireless local area network (WLAN), etc., using any transceiver-like device.

[0050] In specific implementations, as an example, the computer device can include multiple processors, each of which can be a single-CPU processor or a multi-CPU processor. The processor herein can refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0051] The computer device described above can be a general-purpose computer device or a special-purpose computer device. In a specific implementation, the computer device can be a desktop computer, a laptop computer, a network server, a personal digital assistant (PDA), a mobile phone, a tablet computer, a wireless terminal device, a communication device, or an embedded device. The embodiments of the present application do not limit the type of the computer device.

[0052] In addition, the present application also provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the above-mentioned immersive co-creation scenario simulation method based on extended reality.

[0053] Although the preferred embodiments of the present application have been described, those skilled in the art who are familiar with the basic inventive concept can make further changes and modifications to the embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0054] Obviously, those skilled in the art can make various modifications and variations to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application also intends to include these modifications and variations.

Claims

1. A method for regulating behavioral conflicts in immersive collaborative education, applied to an extended reality-based immersive collaborative education scenario simulation system, wherein, The method involves creating an immersive, user-interactive virtual scene using extended reality devices, where each user corresponds to a virtual character and an interactive scene screen within the scene. The method is characterized by the following steps: Based on the extended reality device, the action interaction data and spatial positioning information of each user are collected; The interaction path of the corresponding user in the interactive virtual scene is generated by the action interaction data and spatial positioning information of each user, and the behavioral conflict trend of each user in the interactive virtual scene is determined according to the spatial coupling characteristics between each interaction path. Based on all behavioral conflict trends, the paths of the virtual characters corresponding to each user in the interactive virtual scene are reconstructed to generate behavioral constraint areas for each user's actions during the interaction process. When a user's interaction path is detected to overlap with the behavior constraint area, the user's current physiological data is monitored, and a collaborative correction path for the user in the interactive virtual scene is generated based on the dynamic change characteristics of the physiological data and the user's action interaction data, thereby obtaining the collaborative correction paths for each user in the interactive virtual scene. Each collaborative correction path is converted into a visual instruction and input into the corresponding user's interactive scene screen to instruct the user to perform interactive correction.

2. The method as described in claim 1, characterized in that, Generating the corresponding user's interaction path in the interactive virtual scene based on the user's action interaction data and spatial positioning information specifically includes: Select a user as the selected user, and align the selected user's action interaction data and spatial positioning information to obtain the selected user's action-space aligned data. The interaction path of the selected user in the interactive virtual scene is determined based on the selected user's action-space alignment data. Continue to determine the interaction paths of the remaining users in the interactive virtual scene.

3. The method as described in claim 1, characterized in that, Determining the behavioral conflict trends of each user in the interactive virtual scene based on the spatial coupling characteristics between various interaction paths specifically includes: Determine the spatial coupling characteristics between each interaction path; Potential conflict areas are identified based on the spatial coupling characteristics between the various interaction paths; Based on the potential conflict areas, determine the behavioral conflict trends of each user in the interactive virtual scene.

4. The method as described in claim 1, characterized in that, Based on all behavioral conflict trends, the paths of the virtual characters corresponding to each user in the interactive virtual scene are reconstructed to generate behavioral constraint regions for each user's actions during the interaction process, specifically including: Arrange all behavioral conflict trends according to their corresponding user levels to obtain a behavioral conflict trend sequence; Based on the behavioral conflict trend sequence, the paths of each user's corresponding virtual character in the interactive virtual scene are reconstructed to obtain the reconstructed paths of each user's corresponding virtual character. The behavioral constraints of each user's actions during the interaction process are determined by all the reconstruction paths.

5. The method as described in claim 1, characterized in that, The collaborative correction path for the user in the interactive virtual scene is generated based on the dynamic change characteristics of the physiological data and the user's action interaction data, specifically including: Determine the dynamic change characteristics of the physiological data; The user's multimodal feature vector is determined based on the dynamic change characteristics and the user's action interaction data; Predict the user's action trends based on the user's multimodal feature vectors; Based on the user's action trends and the behavioral constraint area, a collaborative correction path is generated for that user in the interactive virtual scene.

6. The method as described in claim 1, characterized in that, Each collaborative correction path is converted into a visual instruction and input into the corresponding user's interactive scene screen to instruct the user to perform interactive correction. Specifically, this includes: Get the user's current action state; Each collaborative correction path is parsed into a sequence of instructions for the corresponding user; The visualization instructions are determined based on the instruction sequence and input into the corresponding user's interactive scene screen, so as to instruct the user to perform interactive correction based on the interactive scene screen in combination with the user's current action state.

7. The method as described in claim 1, characterized in that, The motion interaction data includes the user's joint movement data, gesture operations, and button clicks.

8. The method as described in claim 1, characterized in that, The spatial positioning information includes the user's three-dimensional coordinates and orientation information.

9. The method as described in claim 1, characterized in that, The device monitors the user's current physiological data using built-in physiological sensors.

10. An immersive collaborative education scenario simulation system based on extended reality, the system comprising a behavioral conflict mediation unit, characterized in that, The behavioral conflict adjustment unit includes: The data acquisition module is used to collect action interaction data and spatial positioning information of each user based on the extended reality device; The processing module is used to generate the corresponding user's interaction path in the interactive virtual scene through the action interaction data and spatial positioning information of each user, and to determine the behavioral conflict trend of each user in the interactive virtual scene based on the spatial coupling characteristics between each interaction path. The processing module is also used to reconstruct the paths of the virtual characters corresponding to each user in the interactive virtual scene based on all behavioral conflict trends, and generate behavioral constraint areas for each user's actions during the interaction process. The processing module is also used to monitor the current physiological data of the user when it is detected that the user's interaction path and the behavior constraint area overlap spatially, and generate the user's collaborative correction path in the interactive virtual scene based on the dynamic change characteristics of the physiological data and the user's action interaction data, thereby obtaining the collaborative correction path of each user in the interactive virtual scene. The execution module is used to convert each collaborative correction path into visual instructions and input them into the corresponding user's interactive scene screen to instruct the user to perform interactive correction.

Images