A BCI-driven VR environment generation system and device

Abstract

The present invention discloses a BCI-driven VR environment generation system and device, which relates to the field of virtual reality technology. The method includes: a client uploads a target person portrait label; a VR environment playback device plays the VR environment to the wearer; a brain-computer interface BCI collects brain wave signals when playing the VR environment; a first VR environment construction module of the server generates an initial VR environment according to the portrait label, a brain wave signal acquisition module stores the collected brain wave signal sequence, a brain wave signal analysis module analyzes and obtains a first emotion value coefficient, and a second VR environment construction module obtains a target VR environment according to the coefficient sorting and sends it to the VR environment playback device for playback. The technical problem that the existing VR environment is difficult to effectively provide a personalized and accurate auxiliary treatment environment according to the characteristics and real-time emotional response of the target user is solved, and the technical effect of improving the psychological condition of the target user and enhancing the treatment effect is achieved.

Description

Technical Field

[0001] The present application relates to the field of virtual reality technology, and in particular to a BCI-driven VR environment generation system and device. Background Art

[0002] In the field of mental health auxiliary treatment, with the intensification of social competition and the increase of life pressure, people's psychological problems are becoming increasingly prominent, and the demand for efficient and personalized auxiliary treatment methods that can provide positive emotions and pleasant scenes to improve mood is becoming more and more urgent. Traditional treatment methods, such as drug therapy and psychotherapy, are insufficient in meeting the personalized needs of target users and real-time monitoring of treatment effects. Although virtual reality (VR) technology has been applied to related treatments, it is difficult for VR technology alone to accurately create scenes that can provide positive emotions and pleasant scenes, and it is difficult to fully reflect the target users' interest in the content and the treatment effect. Although brain-computer interface (BCI) technology can monitor the emotions of target users in real time, it still needs further exploration to effectively combine it with VR technology and apply it to create scenes that provide positive emotions. In addition, the current VR construction of the environment needs to be improved in terms of authenticity and accuracy, and it is difficult to accurately construct pleasant scenes to meet the personalized needs of target users. At the same time, medical institutions also need more effective means to monitor the mental changes of target users in real time in order to adjust the treatment plan so that it can better guide the target users into a positive emotional state.

[0003] At the current stage, there is a technical problem in the relevant technologies that the VR environment cannot effectively provide a personalized and accurate auxiliary treatment environment based on the characteristics and real-time emotional responses of the target users. Summary of the invention

[0004] The present application provides a BCI-driven VR environment generation system and device, which adopts a client to upload a target person portrait tag; a VR environment playback device plays the VR environment to the wearer; the brain-computer interface BCI collects brain wave signals when playing the VR environment; a first VR environment construction module of the server generates an initial VR environment according to the portrait tag, a brain wave signal acquisition module stores the collected brain wave signal sequence, a brain wave signal analysis module analyzes and obtains a first emotion value coefficient, and a second VR environment construction module obtains a target VR environment according to the coefficient sorting and sends it to the VR environment playback device for playback, thereby achieving the technical effect of providing a personalized and accurate auxiliary treatment environment for the target user, generating and playing the most suitable VR environment according to the patient's characteristics and real-time emotional response, and improving the psychological condition of the target user and enhancing the treatment effect.

[0005] The present application provides a BCI-driven VR environment generation system, comprising:

[0006] A client is used to upload a target person portrait label; a VR environment playback device, which is worn by the target person and is used to play the VR environment to the target person; a brain-computer interface BCI, whose front-end electrode contacts the scalp of the target person and is used to collect target task brain wave signals when playing the VR environment to the target person; a server: a first VR environment construction module, which is used to perform positive emotional scene analysis based on the target person portrait label and generate a number of initial VR environments; a brain wave signal acquisition module, which is used to store a number of groups of brain wave signal sequences collected by the brain-computer interface BCI when the several initial VR environments are played; a brain wave signal analysis module, which is used to traverse the several groups of brain wave signal sequences to perform emotional analysis and obtain a first emotional value coefficient; a second VR environment construction module, which is used to perform maximum value sorting on the several initial VR environments based on the first emotional value coefficient, obtain a target VR environment for modeling, and send it to the VR environment playback device for playback.

[0007] The present application also provides a BCI-driven VR environment generation device, wherein the device is deployed to perform the following steps:

[0008] A client is used to upload a target person portrait label; a VR environment playback device, which is worn by the target person and is used to play the VR environment to the target person; a brain-computer interface BCI, whose front-end electrode contacts the scalp and is used to collect brain wave signals when playing the VR environment to the target person; a server: a first VR environment construction module, which is used to perform positive emotional scene analysis based on the target person portrait label and generate a number of initial VR environments; a brain wave signal acquisition module, which is used to store a number of groups of brain wave signal sequences collected by the brain-computer interface BCI when the several initial VR environments are played; a brain wave signal analysis module, which is used to traverse the several groups of brain wave signal sequences to perform emotional analysis and obtain a first emotional value coefficient; a second VR environment construction module, which is used to perform maximum value sorting on the several initial VR environments according to the first emotional value coefficient, obtain a target VR environment for modeling, and send it to the VR environment playback device for playback.

[0009] A BCI-driven VR environment generation system and device proposed in this application, first, the client uploads the target person portrait label; the VR environment playback device plays the VR environment to the wearer; the brain-computer interface BCI collects brain wave signals when playing the VR environment; the first VR environment construction module of the server generates an initial VR environment according to the portrait label, the brain wave signal acquisition module stores the collected brain wave signal sequence, the brain wave signal analysis module analyzes to obtain the first emotion value coefficient, and the second VR environment construction module obtains the target VR environment according to the coefficient sorting and sends it to the VR environment playback device for playback. By providing the target user with a personalized and accurate auxiliary treatment environment, the most suitable VR environment is generated and played according to the patient's characteristics and real-time emotional response, so as to achieve the technical effect of improving the psychological condition of the target user and enhancing the treatment effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solution of the embodiment of the present invention, the accompanying drawings of the embodiment of the present invention will be briefly introduced below. A flow chart is used in the present application to illustrate the operations performed by the system according to the embodiment of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, various steps can be processed in reverse order or simultaneously as needed. At the same time, other operations can also be added to these processes, or one or more operations can be removed from these processes.

[0011] Figure 1 A structural diagram of a BCI-driven VR environment generation system provided in an embodiment of the present application;

[0012] Figure 2 A schematic diagram of the structure of a brain wave signal analysis module of a BCI-driven VR environment generation system provided in an embodiment of the present application.

[0013] Explanation of the accompanying drawings: client 10, VR environment playback device 20, brain-computer interface BCI 30, first VR environment construction module 40, brain wave signal acquisition module 50, brain wave signal analysis module 60, second VR environment construction module 70. DETAILED DESCRIPTION

[0014] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below.

[0015] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0016] In the following description, reference is made to "some embodiments", which describe a subset of all possible embodiments, but it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict, and the terms "first\second" involved are merely to distinguish similar objects and do not represent a specific ordering of objects. The terms "including" and "having" and any variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or modules that are not clearly listed or inherent to these processes, methods, products, or devices. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those generally understood by technicians in the technical field of this application. The terms used herein are for the purpose of describing the embodiments of the present application only.

[0017] The present application embodiment provides a BCI-driven VR environment generation system, such as Figure 1 As shown, the method includes:

[0018] The client 10 is used to upload a target person portrait tag. Specifically, the client first interacts with the target person to obtain various information related to the person, including the target person's interests, personality traits, psychological state, past experiences, etc. The client organizes and analyzes the obtained information, extracts key elements that can reflect the characteristics of the target person, and converts the key elements into the form of a portrait tag. The portrait tag is a concise and accurate description of the characteristics of the target person, represented by a set of keywords, data or identifiers. The client uploads the generated portrait tag to the server through a network connection, and the server receives and stores the portrait tag for subsequent VR environment construction and optimization.

[0019] A VR environment playback device 20 is worn by the target person and is used to play the VR environment to the target person. Specifically, the VR environment playback device 20 is designed as a wearable device, and the target person wears it on the appropriate position of the head or body. During the wearing process, the device needs to fit closely with the target person's body to ensure a comfortable and stable wearing experience. The device establishes a connection with a server or other signal source to receive relevant data and instructions for playing the VR environment. The data includes information such as images, sounds, and tactile feedback of the virtual scene. After receiving the data, the VR environment playback device 20 uses its own hardware and software systems to convert the data into a realistic VR environment, and presents an immersive virtual scene to the target person through output devices such as display screens and speakers. After wearing the device, the target person can interact with the VR environment by turning his head, performing gestures, etc., and the device will capture the real-time information. The device captures the movements and instructions of the target person and adjusts the presentation of the virtual scene accordingly to provide a more natural and intuitive interactive experience. The device continuously plays the constructed VR environment to the target person, making the target person feel as if they are in a new virtual world. During the playback process, the device will try its best to ensure the clarity, smoothness and stability of the image, as well as the realism and three-dimensional sense of the sound to enhance the sense of immersion. The device collects feedback information from the target person during the experience, such as visual focus, emotional response, etc. The feedback information can be used to optimize the playback effect of the VR environment, or provide a basis for subsequent data analysis and improvement. The VR environment playback device 20 provides an immersive VR environment experience for the target person through the steps of wearing, receiving signals, generating an environment, interactive experience, playback and feedback adjustment.

[0020] Brain-computer interface BCI30, the front-end electrode of the brain-computer interface BCI30 is in contact with the scalp, and is used to collect brain wave signals when playing the VR environment to the target person. Specifically, the front-end electrode of the brain-computer interface BCI30 is gently placed on the scalp of the target person to ensure that the electrode is in full contact with the scalp, and a connection is established between the brain-computer interface BCI30 and the relevant data acquisition and processing system so that the collected brain wave signals can be transmitted to the subsequent processing module, and the VR environment playback device is started to play the VR environment to the target person. During the process of the target person experiencing the VR environment, the front-end electrode of the brain-computer interface BCI30 senses the electrophysiological signals on the scalp in real time. The electrophysiological signals contain the brain wave information generated by brain activity. The brain-computer interface BCI30 collects these signals, and the collected brain wave signals are transmitted through connecting lines or wireless The brain-computer interface BCI30 collects brain wave signals in real time when the target person plays the VR environment, and transmits and processes the brain wave signals in real time through contact with the scalp, and transmits and stores the signals for analysis, providing data support for further applications.

[0021] Server: The first VR environment construction module 40 is used to perform positive emotion scene analysis based on the target person portrait label and generate a number of initial VR environments. Specifically, the first VR environment construction module 40 first receives the target person portrait label uploaded from the client. The portrait label contains various characteristics and information of the target person. The module conducts an in-depth analysis of the portrait label and interprets key information about the target person's interests, hobbies, psychological state, etc. Based on the analysis results, the module determines which scenes may trigger the target person's positive emotions. For example, if the target person likes nature, then beautiful natural scenery scenes may be considered positive emotion scenes. Based on the determined positive emotion scenes, the module uses relevant technologies and data to generate a number of initial VR environments. The environment includes a combination of specific virtual scenes, objects, sounds and other elements. When generating the initial VR environments, the module generates a number of initial VR environments. In the process of generating the initial VR environment, the module will also optimize the details of the scene to ensure the authenticity and attractiveness of the environment, for example, adjusting the light, color, texture, etc. of the scene to make it more in line with the preferences of the target person. In order to meet the different needs and emotional responses of the target person, the module will generate multiple initial VR environments with certain differences to provide more choices. After generating the initial VR environment, the module will evaluate and adjust the environment according to some preset standards or algorithms to ensure that they can effectively induce positive emotions of the target person. The first VR environment construction module 40 can generate several suitable initial VR environments according to the target person portrait label, providing a basis for subsequent selection and optimization.

[0022] In a possible implementation, the server: the first VR environment construction module 40 further includes: a historical sample data set acquisition unit, which is used to obtain a historical sample data set by performing hierarchical neighborhood sample collection on the target person portrait label, wherein the historical sample data set includes a VR environment use case set and a sample person brain wave signal sequence record data set. Specifically, the historical sample data set acquisition unit receives a target person portrait label, which is an abstract description of the target person's characteristics and does not contain any specific privacy information. The portrait label is used to collect hierarchical neighborhood samples. In the large amount of existing data, sample data with similar characteristics to the target person are gradually screened out according to the characteristics of the portrait label. For example, if the target person's portrait label shows that he is interested in natural scenery, then the historical sample data set acquisition unit will look for other samples with similar interest characteristics in the historical sample data, and finally obtain the historical sample data set. The historical sample data set includes a VR environment use case set and a sample person brain wave signal sequence record data set. The VR environment use case set can be understood as examples of various types of virtual scenes, and the sample person brain wave signal sequence record data set is a record of brain wave signals of other people in the past when they experienced different VR environments. All data are processed to conceal the privacy information of each user, ensuring that no specific identity is involved or data that indirectly indicates an identity is data.

[0023] The second emotion value coefficient acquisition unit is used to interact with the brain wave signal analysis module, and schedule the brain wave signal analysis module to traverse the sample personnel brain wave signal sequence record data set for analysis to obtain the second emotion value coefficient. Specifically, the second emotion value coefficient acquisition unit calls the brain wave signal analysis module to process the sample personnel brain wave signal sequence record data set. The brain wave signal analysis module will use specific algorithms and models to interpret and analyze the brain wave signals. Specifically, it will traverse the entire sample personnel brain wave signal sequence record data set, analyze each sample brain wave signal one by one, and analyze the characteristics of the brain wave signal, such as frequency, amplitude, etc., to infer the emotional state of the sample personnel when experiencing the VR environment. Based on the analysis of a large number of sample brain wave signals, a second emotion value coefficient is finally obtained. The coefficient can be used to measure the degree of emotional impact that a specific VR environment may have on people with similar characteristics. During the entire analysis process, all sample data involved do not contain any information that may leak user privacy.

[0024] An initial VR environment construction unit, the initial VR environment construction unit is used to sort the positive emotional scenes of the VR environment use case set according to the second emotional value coefficient, and construct a number of the initial VR environments. Specifically, the initial VR environment construction unit processes the VR environment use case set based on the second emotional value coefficient. The VR environment use case set contains various types of virtual scene examples, and the examples may have different themes, styles and elements. According to the second emotional value coefficient, that is, those indicators of the degree that can induce positive emotions, each use case in the VR environment use case set is sorted. For example, use cases with higher emotional value coefficients are selected, and it is believed that the use cases are more likely to induce positive emotions of the target person. Through the sorting process, several initial VR environments are constructed. The initial VR environment is tailored for the target person and aims to provide a virtual experience that is most likely to induce their positive emotions. During the construction process, the privacy protection principle is strictly observed to ensure that no user privacy information is involved.

[0025] In a possible implementation, the historical sample data set acquisition unit further includes: a portrait label data acquisition subunit, and the portrait label data acquisition subunit is used to obtain the portrait label weight distribution information and the portrait label deviation normalization tolerance threshold of the target person's portrait label. Specifically, the portrait label data acquisition subunit first processes the portrait label of the target person. The target person's portrait label includes information such as age, gender, growth environment, and causes of depression. This information is very important for understanding the characteristics and needs of the target person. The portrait label weight distribution information refers to the relative importance assigned to each portrait label. For example, it may be believed that age affects the demand for VR environment to a certain extent, but gender may have a greater impact, so there will be different weight distributions. The portrait label deviation normalization tolerance threshold is a parameter used to measure the acceptable range of portrait label deviation, which determines the degree and suitability of the portrait label to provide positive emotions in the subsequent screening process.

[0026] The first screening condition construction subunit is used to construct the first screening condition, wherein the first screening condition is used to count the proportion of label attributes whose portrait label deviation normalization parameter is less than or equal to the portrait label deviation normalization fault tolerance threshold, and the first screening condition has a first screening threshold. Specifically, the first screening condition construction subunit constructs the first screening condition, and the screening condition mainly focuses on the relationship between the portrait label deviation normalization parameter and the fault tolerance threshold. For the portrait label of the target person, its deviation normalization parameter will be calculated, and the parameter reflects the degree of difference between the portrait label of the target person and the portrait label in the historical sample. If this deviation normalization parameter is less than or equal to the portrait label deviation normalization fault tolerance threshold, it is considered that the label attribute meets the first screening condition, and the proportion of label attributes that meet the condition is counted. The first screening condition also has a first screening threshold. When this proportion is greater than or equal to the first screening threshold, the first screening condition is met.

[0027] The second screening condition construction subunit is used to construct the second screening condition, wherein the second screening condition is used to calculate the weighted mean of the portrait tag deviation normalization parameter based on the portrait tag weight distribution information, and the second screening condition has a second screening threshold, and the second screening threshold is the maximum screening threshold. Specifically, the second screening condition construction subunit constructs the second screening condition, and the condition is based on the portrait tag weight distribution information. The weight distribution information is used to calculate the weighted average of the portrait tag deviation normalization parameter to more accurately reflect the importance differences of different portrait tags. The second screening condition has a second screening threshold, and the threshold is set to the maximum screening threshold. When the weighted mean is less than or equal to the maximum screening threshold, the second screening condition is met.

[0028] The first-level neighborhood historical sample data set adding subunit is used to add samples to the first-level neighborhood historical sample data set when the first screening condition and the second screening condition are met at the same time, wherein the first screening condition is met when the output value of the first screening condition is greater than or equal to the first screening threshold, and the second screening condition is met when the output value of the second screening condition is less than or equal to the second screening threshold. Specifically, the first-level neighborhood historical sample data set adding subunit is responsible for adding qualified samples to the first-level neighborhood historical sample data set, and samples will only be added when the first screening condition and the second screening condition are met at the same time. Specifically, when the output value of the first screening condition is greater than or equal to the first screening threshold, and the output value of the second screening condition is less than or equal to the second screening threshold, the sample is considered to meet the requirements, ensuring that the samples added to the first-level neighborhood historical sample data set have a high similarity with the portrait label of the target person.

[0029] The secondary neighborhood historical sample data set collection subunit is used to traverse the primary neighborhood historical sample data set when the primary neighborhood historical sample data set meets the preset sample quantity, and collect the secondary neighborhood historical sample data set in combination with the first screening condition and the second screening condition. Specifically, the secondary neighborhood historical sample data set collection subunit starts working when the primary neighborhood historical sample data set meets the preset sample quantity, traverses the primary neighborhood historical sample data set, and again screens in combination with the first screening condition and the second screening condition, with the purpose of further mining samples that are more similar to the target person's portrait label from the primary neighborhood historical sample data set to form a secondary neighborhood historical sample data set.

[0030] The historical sample data set adding subunit is used to add the primary neighborhood historical sample data set and the secondary neighborhood historical sample data set into the historical sample data set. Specifically, the historical sample data set adding subunit adds the primary neighborhood historical sample data set and the secondary neighborhood historical sample data set into the historical sample data set, enriching the historical sample data set and providing more reference data for subsequent analysis and construction of the initial VR environment.

[0031] In a possible implementation, the initial VR environment construction unit further includes: a constituent element repetition acquisition subunit, the constituent element repetition acquisition subunit is used to perform constituent element repetition analysis on the VR environment use case set to obtain multiple constituent element repetitions, wherein the constituent element repetition is equal to the ratio of the number of intersections of constituent elements of any two VR environment use cases to the number of unions of constituent elements, and the constituent elements represent the physical description quantity of the environment. Specifically, the constituent element repetition acquisition subunit first analyzes the VR environment use case set, which contains many different virtual environment examples, and measures the similarity between different VR environment use cases by calculating the constituent element repetition. The constituent element refers to the physical description quantity of the virtual environment, such as specific objects, colors, textures, etc. For any two VR environment use cases, the ratio of the number of intersections of their constituent elements (i.e., the number of constituent elements they have in common) to the number of unions of their constituent elements (i.e., the sum of the number of all different constituent elements) is calculated to obtain the constituent element repetition. In this way, the similarity between different use cases can be quantified, providing a basis for subsequent classification and screening.

[0032] A clustering subunit, the clustering subunit is used to cluster VR environment use cases greater than or equal to the component repetition threshold into the same category, and cluster VR environment use cases less than the component repetition threshold into different categories according to the repetition of the multiple components, so as to obtain multiple clusters of VR environment use cases. Specifically, the clustering subunit performs classification operations based on the component repetition obtained above, sets a component repetition threshold, and classifies VR environment use cases with repetition greater than or equal to the threshold into the same category. These use cases have high similarity in the components, while use cases with repetition less than the threshold are clustered into different categories, and the entire VR environment use case set is divided into multiple different clusters, each cluster representing a class of VR environment use cases with similar components.

[0033] A VR environment use case acquisition subunit, the VR environment use case acquisition subunit is used to traverse the multiple clusters of VR environment use cases to sort the maximum value of the second emotion value coefficient, and obtain multiple VR environment use cases. Specifically, the VR environment use case acquisition subunit performs further screening on the basis of multiple clusters of VR environment use cases, traverses the VR environment use cases in each cluster, and checks their second emotion value coefficients. The second emotion value coefficient is obtained by analyzing the sample personnel's brain wave signal sequence record data set, and is used to measure the degree to which the VR environment use case induces positive emotions. By comparing the second emotion value coefficient of each use case, the use case with the maximum value is selected, thereby obtaining multiple VR environment use cases, and the use cases have a high potential in inducing positive emotions.

[0034] The initial VR environment extraction subunit is used to extract the several initial VR environments that are greater than or equal to the threshold of the emotion value coefficient from the multiple VR environment use cases. Specifically, the initial VR environment extraction subunit extracts the initial VR environment that meets the requirements from the multiple VR environment use cases obtained above, sets an emotion value coefficient threshold, and only when the emotion value coefficient of the VR environment use case is greater than or equal to the threshold will it be extracted as the initial VR environment, ensuring that the extracted initial VR environment has a high potential to induce positive emotions, and is ready for subsequent provision to the target personnel.

[0035] In a possible implementation, the initial VR environment construction unit further includes: a first VR environment use case acquisition subunit to be modeled, the first VR environment use case acquisition subunit to be modeled is used to obtain a first VR environment use case to be modeled, wherein the first VR environment use case to be modeled has an environment color image set, an environment depth image set, an image map coordinate label set, an environment three-dimensional laser point cloud, and a point cloud map coordinate label set, the environment color image set and the environment depth image set correspond one-to-one, and the image map coordinates and the point cloud map coordinates are in the same coordinate space. Specifically, the task of the first VR environment use case acquisition subunit to be modeled is to obtain the first VR environment use case to be modeled. The first VR environment use case to be modeled includes multiple important components. The environmental color image set provides color visual information of the virtual environment; the environmental depth image set gives the depth information of the object, which helps to build a more realistic three-dimensional scene; the image map coordinate label set provides location information for the color image and the depth image, as well as the environmental three-dimensional laser point cloud, which obtains more accurate object surface information through laser scanning, and the point cloud map coordinate label set is used to determine the position of the point cloud. The environmental color image set and the environmental depth image set correspond one-to-one, which means that for each color image there is a corresponding depth image, which helps to build a three-dimensional scene more accurately. The image map coordinates and the point cloud map coordinates are in the same coordinate space, ensuring that information from different data sources can be integrated in a unified space.

[0036] A set fusion subunit, the set fusion subunit is used to fuse the environment color image set, the environment depth image set and the image map coordinate label set through the three-dimensional modeling channel to obtain a first environment modeling point cloud. Specifically, the set fusion subunit uses the three-dimensional modeling channel to perform a fusion operation on multiple sets, and inputs the environment color image set, the environment depth image set and the image map coordinate label set into the three-dimensional modeling channel. The channel may use specific algorithms and techniques to integrate the color image, the depth image and the coordinate label to generate a first environment modeling point cloud. The point cloud is a data set composed of a large number of three-dimensional points, which can represent the surface of objects in the virtual environment. By fusing different types of information, a richer and more accurate three-dimensional model can be constructed.

[0037] A depth correction subunit, the depth correction subunit is used to perform depth correction on the first environment modeling point cloud according to the environment three-dimensional laser point cloud and the point cloud map coordinate label set, obtain a first initial VR environment, and add it to the several initial VR environments. Specifically, the depth correction subunit performs depth correction on the first environment modeling point cloud, using the environment three-dimensional laser point cloud and the point cloud map coordinate label set as a reference, and compares it with the first environment modeling point cloud. If it is found that the depth information of the first environment modeling point cloud is inconsistent with the environment three-dimensional laser point cloud, depth correction is performed to improve the accuracy and authenticity of the point cloud. After depth correction, the first initial VR environment is obtained, and it is added to several initial VR environments to provide more options for subsequent screening and optimization.

[0038] The depth mean acquisition subunit is used in which the depth correction is to take the depth mean when the first environment modeling point cloud is inconsistent with the environment three-dimensional laser point cloud. Specifically, the depth mean acquisition subunit further explains the specific method of depth correction. When the first environment modeling point cloud is inconsistent with the environment three-dimensional laser point cloud, the depth mean of the two is taken as the corrected depth value, which can balance the differences between different data sources to a certain extent, improve the quality and accuracy of the point cloud, and ensure that the generated first initial VR environment is more realistic and reliable, and can better meet the needs of users.

[0039] In a possible implementation, the set fusion subunit further includes: a three-dimensional modeling channel composition micro-unit, the three-dimensional modeling channel composition micro-unit is used for the three-dimensional modeling channel to include a point color feature extraction unit, a point depth feature extraction unit, and a point fully connected layer, wherein the point color feature extraction unit and the point depth feature extraction unit are connected in parallel, and the point fully connected layer is used to fully connect the output layers of the point color feature extraction unit and the point depth feature extraction unit. Specifically, the three-dimensional modeling channel composition micro-unit first clarifies the composition structure of the three-dimensional modeling channel. The three-dimensional modeling channel consists of three main parts: a point color feature extraction unit, a point depth feature extraction unit, and a point fully connected layer. The point color feature extraction unit and the point depth feature extraction unit are connected in parallel, and can process the input data at the same time, respectively responsible for extracting the color features and depth features in the virtual environment, and the point fully connected layer is used to fully connect the output layers of the point color feature extraction unit and the point depth feature extraction unit to achieve the fusion of color and depth information.

[0040] The point color feature extraction unit supervised training micro-unit is used for the point color feature extraction unit to perform supervised training generation through color image recording data, image map coordinate label recording data and point color feature identification true value. Specifically, the point color feature extraction unit supervises the training micro-unit to collect a large amount of color image recording data. The images should cover a variety of different scenes, objects and lighting conditions to ensure that the trained units can adapt to different virtual environments. Image map coordinate labels are assigned to each color image to record data, and the location information of each pixel in the image is clarified. Color features of specific points in the color image are identified by professionals or through automated annotation tools to determine the true value of the point color feature identification. The true value can include specific feature descriptions such as the RGB value, hue, and saturation of the color. The convolutional neural network (CNN) in deep learning is selected as the basic architecture of the training algorithm. CNN performs well in image feature extraction and can effectively capture color patterns and features in images. A CNN structure suitable for point color feature extraction is designed, including multiple convolutional layers, pooling layers and fully connected layers. The convolutional layer is used to extract local color features in the image, the pooling layer is used to reduce the feature dimension, and the fully connected layer integrates and outputs the extracted features to convert the color The image record data and the corresponding image map coordinate label record data are input into CNN. CNN processes the input image and gradually extracts the color features in the image through layers of convolution and pooling operations. In each training batch, the difference between the color features output by CNN and the true value of the point color feature identification is calculated. The loss function (such as mean square error, cross entropy, etc.) is usually used to measure this difference. According to the value of the loss function, the weight and bias parameters of CNN are adjusted through the back propagation algorithm so that the output color features gradually approach the true value. Repeat the above steps and continuously iterate the training until the loss function converges to a smaller value or reaches the preset number of training times. During the training process, the validation set can be used regularly to verify the trained point color feature extraction unit. The validation set is a set of color image data independent of the training set, which is used to evaluate the performance of the model on unseen data. According to the results of the validation set, the training parameters (such as learning rate, batch size, etc.) or network structure are adjusted to improve the generalization ability and accuracy of the model.

[0041] A point depth feature extraction unit supervised training micro-unit, wherein the point depth feature extraction unit supervised training micro-unit is used for the point depth feature extraction unit to generate supervised training through depth image recording data, image map coordinate label recording data and point depth feature identification true value. Specifically, the point depth feature extraction unit supervised training micro-unit collects depth image recording data corresponding to the color image, and the depth image can be obtained by a depth camera or other depth measurement equipment, provides distance information of objects in the virtual environment, assigns image map coordinate label recording data to the depth image, determines the position of each pixel, and annotates the point depth feature identification true value in the depth image, such as the actual depth value or depth range of the object surface, adopts the same CNN architecture as the point color feature extraction unit, adopts a CNN architecture similar to the point color feature extraction unit, or selects a network structure specifically used for depth estimation according to the characteristics of the depth image, such as a depth estimation network based on deep learning, including different numbers and types of convolutional layers, pooling layers and fully connected layers, taking into account the depth image Due to the particularity of the network, some modules specifically used to process depth information can be added, such as depth regularization layer, depth residual connection, etc. The depth image record data and the corresponding image map coordinate label record data are input into the network. The network processes the depth image, extracts the depth features therein, and calculates the loss function between the depth features output by the network and the true value of the point depth feature identification, such as the mean square error loss. The network parameters are adjusted through the back propagation algorithm to make the output depth features gradually approach the true value. The training is continuously iterated until the loss function converges or the preset number of training times is reached. The trained point depth feature extraction unit is verified using the verification set to evaluate its performance on unseen depth images. The training parameters and network structure are adjusted according to the verification results to improve the accuracy and generalization ability of the model.

[0042] The environment modeling point cloud output micro-unit is used by the point fully connected layer to fuse the color and depth of the point and output the environment modeling point cloud. Specifically, prepare the color image and depth image feature data that have been processed by the point color feature extraction unit and the point depth feature extraction unit, ensure that the feature data matches the corresponding image map coordinate label record data, the goal of the point fully connected layer is to fuse the color features and the depth features, and output the environment modeling point cloud, the training goal is to enable the fused point cloud to accurately represent the color and depth information of objects in the virtual environment, input the color features and the depth features into the point fully connected layer, the fully connected layer fuses the two features by adjusting the weights and bias parameters, calculate the loss function between the fused point cloud and the real environment modeling point cloud (which can be obtained by other accurate modeling methods or labeled data), such as the point cloud distance loss, adjust the parameters of the fully connected layer through the back propagation algorithm, so that the output point cloud gradually approaches the real point cloud, repeat the training process until the loss function converges or reaches the preset number of training times, effectively train the point color feature extraction unit, the point depth feature extraction unit and the point fully connected layer, thereby achieving high-quality environment modeling point cloud output, and providing strong support for building a more realistic virtual environment.

[0043] In a possible implementation, the first VR environment use case acquisition subunit to be modeled further includes: an environment color image set acquisition micro-unit, and the environment color image set acquisition micro-unit is used to acquire the environment color image set for a preset scene through an RGB camera under the constraint of the image map coordinate label set. Specifically, the environment color image set acquisition micro-unit acquires the environment color image set, using an RGB camera as an acquisition device. The RGB camera can capture color image information in a preset scene. During the acquisition process, it is constrained by the image map coordinate label set. The image map coordinate label set provides location information constraints for the acquired color images, ensuring that the position of each image can be accurately determined and recorded. In the subsequent processing process, different color images can be integrated and aligned according to their coordinate labels to build a more complete and accurate virtual environment.

[0044] An environmental depth image set acquisition micro-unit is used to acquire the environmental depth image set for a preset scene through a depth camera under the constraints of the image map coordinate label set. Specifically, the environmental depth image set acquisition micro-unit is responsible for acquiring the environmental depth image set, and the acquisition is performed using a depth camera. The depth camera can measure the distance between the object and the camera in the preset scene, thereby generating a depth image. It is subject to the constraints of the image map coordinate label set, which means that when acquiring the depth image, the position of each depth image can also be determined according to the coordinate label, so as to correspond and fuse with the color image set. The depth image set provides the depth information of objects in the virtual environment, and combined with the color image set, a more realistic three-dimensional scene can be constructed.

[0045] A three-dimensional laser point cloud acquisition micro-unit is used to acquire the three-dimensional laser point cloud of the environment under the constraints of the point cloud map coordinate label set for a preset scene through a laser radar. Specifically, the three-dimensional laser point cloud acquisition micro-unit is used to acquire the three-dimensional laser point cloud of the environment, using a laser radar as a collection tool. The laser radar acquires the three-dimensional position information of the object by emitting a laser beam and measuring the time of reflected light, thereby generating a three-dimensional laser point cloud. The collection process is constrained by the point cloud map coordinate label set. The point cloud map coordinate label set provides coordinate information for the three-dimensional laser point cloud, so that the position of each point can be accurately recorded. In subsequent processing, the three-dimensional laser point cloud is accurately fused with the color image set and the depth image set to build a more accurate and detailed virtual environment model.

[0046] In a possible implementation, the depth correction subunit further includes: a VR environment use case composition micro-unit, and the VR environment use case composition micro-unit is used for the first VR environment use case to be modeled and also includes a 3D audio source set and an audio source map coordinate label set. Specifically, the VR environment use case composition micro-unit clarifies the composition content of the first VR environment use case to be modeled. In addition to the previously mentioned environment color image set, environment depth image set, image map coordinate label set, environment three-dimensional laser point cloud, point cloud map coordinate label set, etc., the first VR environment use case to be modeled also includes a 3D audio source set and an audio source map coordinate label set. The 3D audio source set provides audio information in the virtual environment, making the VR experience richer and more realistic, and the audio source map coordinate label set provides location information for these audio sources, so as to accurately locate and play audio in the virtual environment.

[0047] The micro-unit for updating the initial VR environment is used to add the 3D audio source set to the first initial VR environment according to the audio source map coordinate label set, and to add the first updated initial VR environment to the several initial VR environments. Specifically, the micro-unit for updating the initial VR environment is responsible for updating the initial VR environment, adding the 3D audio source set to the first initial VR environment, and adding the audio element to the first initial VR environment which originally only has visual information, making it a more complete and immersive experience. After obtaining the updated first updated initial VR environment, it is added to several initial VR environments. In the subsequent screening and optimization process, this initial VR environment with audio also has the opportunity to be selected, providing a richer VR experience for the target personnel.

[0048] The brain wave signal acquisition module 50 is used to store several groups of brain wave signal sequences collected by the brain-computer interface BCI when the several initial VR environments are played. Specifically, the brain wave signal acquisition module 50 is in a standby state, waiting to receive the data collected by the brain-computer interface BCI when the target person experiences several initial VR environments. When the target person wears the VR environment playback device and begins to experience each initial VR environment, the front-end electrode of the brain-computer interface BCI contacts the scalp, monitors the electrophysiological signals generated by brain activity in real time, and continuously collects brain wave signals. Subsequently, the BCI sends the collected several groups of brain wave signal sequences to the acquisition module, which stores them, performs data verification and error detection to ensure that the data is accurate and reliable, and attempts to re-receive or request to collect again if there is a problem. The stored brain wave signal sequence will serve as an important basis for the brain wave signal analysis module to perform emotional analysis, providing key neural feedback information for the system to screen out the target VR environment that can best induce positive emotions in the target person.

[0049] The brain wave signal analysis module 60 is used to traverse the plurality of groups of brain wave signal sequences to perform emotion analysis and obtain a first emotion value coefficient. Specifically, the brain wave signal analysis module 60 is responsible for conducting in-depth analysis on the collected groups of brain wave signal sequences to determine the emotional state of the target person reflected by these signals, and finally obtain the first emotion value coefficient. After the brain wave signal acquisition module 50 stores the groups of brain wave signal sequences collected by the brain-computer interface BCI when the target person watches several initial VR environments, the brain wave signal analysis module 60 starts to work, obtains the brain wave signal sequence, and prepares for traversal analysis. During the analysis process, the brain wave signal sequence is encoded based on the time attribute and spatial attribute of the brain wave, and a brain wave signal encoding matrix sequence is generated to convert the original brain wave signal into a form that is easier to analyze. The brain wave signal encoding matrix sequence is processed through a specific neural network model, such as a CNN-LSTM emotion analysis network. The construction of the network includes multiple steps. First, the brain wave signal encoding matrix sequence is recorded as input, and the brain wave signal time feature identification data, the brain wave signal frequency feature identification data and the brain wave signal space feature identification data are used to identify the brain wave signal. The feature identification data is used as supervision to train the CNN neural network and construct the CNN feature extraction layer. The time feature identification data of the EEG signal, the frequency feature identification data of the EEG signal, the spatial feature identification data of the EEG signal and the sequence record data of the EEG signal encoding matrix are used as input. The emotional pleasure identification data is used as supervision to train the LSTM neural network and generate the emotional value evaluation layer. The CNN feature extraction layer and the emotional value evaluation layer are integrated to generate the CNN-LSTM emotional analysis network. After processing the EEG signal using the emotional analysis network, the emotional value coefficient corresponding to each initial VR environment can be output. By traversing all the brain wave signal sequences and the corresponding initial VR environments, the emotional reactions of the target person in different environments are comprehensively analyzed, and finally a first emotional value coefficient that can reflect the overall emotional state is determined. The first emotional value coefficient will serve as an important basis for subsequent screening of the target VR environment, helping the system find the virtual environment that can best induce the target person's positive emotions, and providing the target person with a more personalized and effective VR experience-assisted treatment.

[0050] In one possible implementation, Figure 2As shown, the brain wave signal analysis module further includes: a first group of brain wave signal sequence acquisition unit, the first group of brain wave signal sequence acquisition unit is used to obtain the first group of brain wave signal sequences of the plurality of groups of brain wave signal sequences. Specifically, the first group of brain wave signal sequence acquisition unit needs to interact with the module storing the brain wave signal sequence, accurately identify and extract the first group of brain wave signal sequences from the plurality of groups of brain wave signal sequences, and needs to ensure that the specific first group of signals is accurately acquired, and the position of the first group of signals will be determined according to criteria such as time sequence, and then extracted through the data transmission channel, ready for the next step of processing.

[0051] The first group of brain wave signal sequence encoding units, the first group of brain wave signal sequence encoding units is used to encode the first group of brain wave signal sequences based on the brain wave time attributes and the brain wave space attributes, and generate a brain wave signal encoding matrix sequence. Specifically, once the first group of brain wave signal sequence encoding units obtains the first group of brain wave signal sequences, the brain wave signal has time attributes and space attributes. The time attribute reflects the change of the signal over time, while the space attribute is related to the position of different electrodes on the scalp and the difference in the collected signals. For the time attribute, the unit will analyze the changes in the intensity, frequency and other characteristics of the signal at different time points. For example, to observe the fluctuation of brain waves in a specific time period and determine whether there are periodic patterns or sudden changes, it is necessary to use time series analysis technology to segment, filter and process the signal to extract meaningful time features. For the space attribute, the difference in signals collected at different electrode positions will be considered. ,Brain activities in different areas may produce brain wave signals of different intensities and patterns on different electrodes.,By comparing the signals collected by each electrode, the spatial distribution of brain activity,can be determined, which involves spatial filtering, electrode position calibration and other technologies,to ensure accurate capture of spatial features, taking into account both time and space attributes, the unit will use a specific encoding algorithm to convert the first set of brain wave signal sequences into brain wave signal encoding matrix sequences.,The matrix sequence can more effectively represent the characteristics of brain wave signals,,providing a more convenient data form for subsequent analysis.,The encoding process involves complex operations such as mathematical transformation,,feature extraction, to ensure that the generated encoding matrix sequence can accurately reflect the essential characteristics of brain wave signals.

[0052] The coding matrix sequence processing unit is used to process the brain wave signal coding matrix sequence through the CNN-LSTM emotion analysis network, output the first initial VR environment emotion value coefficient, and add it into the first emotion value coefficient. Specifically, with the encoded brain wave signal coding matrix sequence, the coding matrix sequence processing unit uses the powerful CNN-LSTM emotion analysis network for processing. The CNN (convolutional neural network) part can effectively extract local features and spatial patterns in the coding matrix sequence. Through the combination of multiple convolutional layers and pooling layers, it automatically learns features of different scales. For example, in the lower-level convolutional layers, local features such as edges and textures of the signal are extracted; while in the higher-level convolutional layers, more complex patterns and structures can be captured. The pooling layer reduces the feature dimension, improves computational efficiency, and enhances the network's robustness to input changes. The LSTM (long short-term memory network) part is good at processing time series data, can capture the temporal dynamic information in the brain wave signal, remember past information, and update the current state according to the current input and past state. It is very important for analyzing the changes of brain wave signals over time and the dynamic evolution of emotions. , LSTM units can control the flow and forgetting of information through gating mechanisms to ensure that the network can effectively learn long-term dependencies. Through the synergy of CNN and LSTM, the emotion analysis network can conduct in-depth analysis of the brain wave signal encoding matrix sequence and extract emotion-related features. The network will output the first initial VR environment emotion value coefficient. The coefficient reflects the emotional state of the target person in a specific initial VR environment. For example, a higher coefficient indicates that the target person experiences positive emotions in this environment, while a lower coefficient may indicate that the emotion is more negative or neutral. The output emotion value coefficient will be added to the first emotion value coefficient. Taking into account the emotional responses in multiple initial VR environments, a comprehensive emotion value evaluation system is gradually constructed. By continuously processing different brain wave signal sequences and corresponding VR environments, the first emotion value coefficient will more and more accurately reflect the target person's emotional response to the entire VR experience.

[0053] CNN feature extraction layer construction unit, the CNN feature extraction layer construction unit is used in it, the CNN-LSTM emotion analysis network construction step includes: by taking the brain wave signal coding matrix sequence recording data as input, taking the brain wave signal time feature identification data, the brain wave signal frequency feature identification data and the brain wave signal space feature identification data as supervision, training the CNN neural network, and constructing the CNN feature extraction layer. Specifically, the CNN feature extraction layer construction unit focuses on constructing the CNN feature extraction layer in the CNN-LSTM emotion analysis network, an important part of the entire emotion analysis network, responsible for extracting key features from the input brain wave signal coding matrix sequence, taking the brain wave signal coding matrix sequence recording data as input, the brain wave signal data encoded in the previous steps contains rich time and space information, taking the brain wave signal time feature identification data, frequency feature identification data and space feature identification data as supervision information, the identification data is pre-labeled or determined by other means, and is used to guide the CNN neural network to learn the correct features, for example, the time feature identification data can indicate important events or emotional changes within a specific time period, and the frequency feature identification data can indicate important events or emotional changes within a specific time period. To represent the relationship between brain wave activity and emotions in different frequency ranges, spatial feature identification data can help the network determine the association between activities in different brain regions and emotions. By using a large amount of labeled data for training, the CNN neural network will automatically adjust its weights and parameters to minimize the difference between the prediction results and the supervision information. During the training process, the network will gradually learn how to extract emotion-related features from the input encoding matrix sequence. For example, it will learn specific patterns, textures or structures that correspond to different emotional states. After multiple iterative training, when the performance of the network reaches a certain standard or converges, the CNN feature extraction layer is constructed. The feature extraction layer can effectively extract important features in brain wave signals and provide strong support for subsequent emotion analysis.

[0054] An emotion value assessment layer generating unit, wherein the emotion value assessment layer is used to train an LSTM neural network by taking brain wave signal time feature identification data, brain wave signal frequency feature identification data, brain wave signal space feature identification data and brain wave signal encoding matrix sequence recording data as input and taking emotion pleasure identification data as supervision to generate an emotion value assessment layer. Specifically, the emotion value assessment layer generation unit generates the emotion value assessment layer, which is the part of the CNN-LSTM emotion analysis network responsible for assessing the emotion value. It takes the time, frequency, and spatial feature identification data of the brain wave signal and the sequence record data of the brain wave signal encoding matrix as input. The input information provides rich brain wave signal features and context information, which helps the LSTM neural network to better understand the relationship between the input data and the emotion value. It takes the emotion pleasure identification data as supervision, and provides specific quantitative indicators of the emotion value. For example, different degrees of emotion pleasure can be represented by numerical values, and graded from very unpleasant to very pleasant. The LSTM neural network will adjust its parameters according to these supervisory information to learn how to accurately assess the emotion value reflected by the brain wave signal. During the training process, the LSTM neural network will use its memory unit and gating mechanism to perform sequence modeling on the input data, remember past information, and combine the current input to predict the emotion pleasure. By continuously adjusting the weights and parameters, the network will gradually improve the accuracy of the assessment of the emotion value. After sufficient training, the emotion value assessment layer is generated, and the assessment result of the emotion value is output according to the input brain wave signal data, which provides an important basis for determining the impact of the VR environment on the target person's emotions.

[0055] The emotion analysis network generation unit is used to fuse the CNN feature extraction layer and the emotion value assessment layer to generate the CNN-LSTM emotion analysis network. Specifically, the emotion analysis network generation unit fuses the previously constructed CNN feature extraction layer and emotion value assessment layer to form a complete CNN-LSTM emotion analysis network. The fusion process needs to consider the information interaction and collaborative work between the two levels. On the one hand, the features extracted by the CNN feature extraction layer need to be effectively transmitted to the emotion value assessment layer so that the latter can accurately assess the emotion value based on these features. Technical means such as feature mapping, connection or shared weights are involved to ensure smooth information flow between the two levels. On the other hand, the feedback of the emotion value assessment layer can also be back-propagated to the CNN feature extraction layer to help it further optimize the feature extraction process. For example, if the emotion value assessment result is inaccurate, it may mean that the CNN feature extraction layer has not extracted enough effective features. At this time, the parameters of CNN can be adjusted to improve the quality and relevance of the features. Through a carefully designed fusion method, the two-level network is combined to form a powerful emotion analysis network. The network can comprehensively utilize the advantages of CNN and LSTM to perform comprehensive and accurate emotion analysis on EEG signals, while processing the spatial features and temporal dynamic information of EEG signals, so as to more deeply understand the emotional state of the target person and provide strong support for the optimization and personalization of the VR environment.

[0056] The second VR environment construction module 70 is used to perform maximum sorting on the several initial VR environments according to the first emotion value coefficient, obtain the target VR environment for modeling, and send it to the VR environment playback device for playback. Specifically, the second VR environment construction module 70 first receives the first emotion value coefficient obtained by the brain wave signal analysis module, performs maximum sorting on several initial VR environments according to the coefficient, determines the initial VR environment with the highest emotion value coefficient as the target VR environment by comparing the coefficient size, and then models the target VR environment, including optimizing graphics rendering, adjusting audio, optimizing interaction design, and optimizing stability and performance, etc. After the modeling is completed, the target VR environment is sent to the VR environment playback device to ensure stable and secure data transmission, so that the target person can experience the virtual scene that can best induce positive emotions by wearing the playback device and immersing himself in the carefully screened and optimized target VR environment.

[0057] The embodiment of the present application adopts the method of uploading a portrait tag of a target person by a client; a VR environment playback device plays the VR environment to the wearer; a brain-computer interface BCI collects brain wave signals when playing the VR environment; a first VR environment construction module of the server generates an initial VR environment according to the portrait tag, a brain wave signal acquisition module stores the collected brain wave signal sequence, a brain wave signal analysis module analyzes and obtains a first emotion value coefficient, and a second VR environment construction module obtains the target VR environment according to the coefficient sorting and sends it to the VR environment playback device for playback, thereby achieving the technical effect of providing a personalized and accurate auxiliary treatment environment for the target user, generating and playing the most suitable VR environment according to the characteristics of the patient and the real-time emotional response, thereby improving the psychological condition of the target user and enhancing the treatment effect.

[0058] In one embodiment, a BCI-driven VR environment generation device is provided, which is used to deploy a BCI-driven VR environment generation system, and the device performs the following steps: a client, which is used to upload a target person portrait label; a VR environment playback device, which is worn by the target person and is used to play the VR environment to the target person; a brain-computer interface BCI, wherein the front-end electrode of the brain-computer interface BCI contacts the scalp and is used to collect brain wave signals when playing the VR environment to the target person; a server: a first VR environment construction module, which is used to perform positive emotional scene analysis based on the target person portrait label and generate a plurality of initial VR environments; a brain wave signal acquisition module, which is used to store a plurality of groups of brain wave signal sequences collected by the brain-computer interface BCI when the plurality of initial VR environments are played; a brain wave signal analysis module, which is used to traverse the plurality of groups of brain wave signal sequences to perform emotional analysis and obtain a first emotional value coefficient; a second VR environment construction module, which is used to perform maximum value sorting on the plurality of initial VR environments based on the first emotional value coefficient, obtain a target VR environment for modeling, and send it to the VR environment playback device for playback.

[0059] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, any number of different modules may be used and run on the user terminal and / or server, and the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0060] The above specific implementation manner does not constitute a limitation to the protection scope of the present application. It should be understood by those skilled in the art that various modifications, combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application should be included in the protection scope of the present application. In some cases, the actions or steps recorded in the present application can be performed in an order different from that in the embodiment and can still achieve the desired results. In addition, the process depicted in the accompanying drawings does not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

Claims

1. A BCI-driven VR environment generation system, characterized in that: include: Client, used to upload target person portrait labels; A VR environment playback device, which is worn by a target person and is used to play a VR environment to the target person; A brain-computer interface (BCI), wherein the front-end electrode of the brain-computer interface (BCI) contacts the scalp of the target person and is used to collect brain wave signals of the target person when playing the VR environment to the target person; server: A first VR environment construction module is used to perform positive emotion scene analysis based on the target person portrait label to generate a plurality of initial VR environments; A brain wave signal acquisition module, used to store a plurality of groups of brain wave signal sequences acquired when the brain-computer interface BCI is played in the plurality of initial VR environments; A brain wave signal analysis module, used for traversing the plurality of groups of brain wave signal sequences to perform emotion analysis and obtain a first emotion value coefficient; The second VR environment construction module is used to perform maximum value sorting on the multiple initial VR environments according to the first emotion value coefficient, obtain a target VR environment for modeling, and send it to the VR environment playback device for playback.

2. The system according to claim 1, characterized in that The first VR environment building module includes: A historical sample data set acquisition unit, the historical sample data set acquisition unit is used to perform hierarchical neighborhood sample collection through the target person portrait label to obtain a historical sample data set, wherein the historical sample data set includes a VR environment use case set and a sample person brain wave signal sequence record data set; A second emotion value coefficient acquisition unit, the second emotion value coefficient acquisition unit is used to interact with the brain wave signal analysis module, schedule the brain wave signal analysis module to traverse the brain wave signal sequence record data set of the sample personnel for analysis, and obtain a second emotion value coefficient; An initial VR environment construction unit, wherein the initial VR environment construction unit is used to sort the positive emotional scenes of the VR environment use case set according to the second emotional value coefficient, and construct the plurality of initial VR environments.

3. The system according to claim 2, characterized in that The first VR environment construction module executes the steps of: acquiring a historical sample data set by performing hierarchical neighborhood sample collection on the target person portrait label, including: A portrait label data acquisition subunit, the portrait label data acquisition subunit is used to obtain the portrait label weight distribution information and the portrait label deviation normalization fault tolerance threshold of the target person portrait label; A first screening condition construction subunit, wherein the first screening condition construction subunit is used to construct a first screening condition, wherein the first screening condition is used to count the proportion of label attributes whose portrait label deviation normalization parameter is less than or equal to the portrait label deviation normalization tolerance threshold, and the first screening condition has a first screening threshold; A second screening condition constructing subunit, the second screening condition constructing subunit is used to construct a second screening condition, wherein the second screening condition is used to calculate a weighted mean of the portrait label deviation normalization parameter based on the portrait label weight distribution information, and the second screening condition has a second screening threshold, and the second screening threshold is a maximum screening threshold; A primary neighborhood historical sample data set adding subunit, the historical sample data set adding subunit is used to add the primary neighborhood historical sample data set into the primary neighborhood historical sample data set when the first screening condition and the second screening condition are simultaneously met, wherein the first screening condition is met when the output value of the first screening condition is greater than or equal to the first screening threshold, and the second screening condition is met when the output value of the second screening condition is less than or equal to the second screening threshold; A secondary neighborhood historical sample data set collection subunit, wherein the secondary neighborhood historical sample data set collection subunit is used to traverse the primary neighborhood historical sample data set when the primary neighborhood historical sample data set meets a preset sample quantity, and collect the secondary neighborhood historical sample data set in combination with the first screening condition and the second screening condition; The historical sample data set adding subunit is used to add the primary neighborhood historical sample data set and the secondary neighborhood historical sample data set into the historical sample data set.

4. The system according to claim 2, characterized in that The first VR environment construction module executes the step of sorting the VR environment use case set into positive emotional scenes according to the second emotion value coefficient to obtain the plurality of initial VR environments, including: A component repetition acquisition subunit, the component repetition acquisition subunit is used to perform component repetition analysis on the VR environment use case set to obtain multiple component repetitions, wherein the component repetition is equal to the ratio of the number of intersections of any two VR environment use case components to the number of unions of the components, and the components represent the physical description quantity of the environment; A clustering subunit, the clustering subunit is used to cluster VR environment use cases that are greater than or equal to a component repetition threshold as the same type, and cluster VR environment use cases that are less than the component repetition threshold as different types, according to the multiple component repetitions, to obtain multiple clusters of VR environment use cases; A VR environment use case acquisition subunit, the VR environment use case acquisition subunit is used to traverse the multiple clusters of VR environment use cases to sort the maximum value of the second emotion value coefficient, and obtain multiple VR environment use cases; The initial VR environment extraction subunit is used to extract a plurality of the initial VR environments that are greater than or equal to an emotion value coefficient threshold from the plurality of VR environment use cases.

5. The system according to claim 2, characterized in that Constructing the several initial VR environments, including: A first VR environment use case acquisition subunit to be modeled, the first VR environment use case acquisition subunit to be modeled is used to obtain a first VR environment use case to be modeled, wherein the first VR environment use case to be modeled has an environment color image set, an environment depth image set, an image map coordinate label set, an environment three-dimensional laser point cloud, and a point cloud map coordinate label set, the environment color image set and the environment depth image set correspond one to one, and the image map coordinates and the point cloud map coordinates are in the same coordinate space; A set fusion subunit, the set fusion subunit is used to fuse the environment color image set, the environment depth image set and the image map coordinate label set through a three-dimensional modeling channel to obtain a first environment modeling point cloud; A depth correction subunit, the depth correction subunit is used to perform depth correction on the first environment modeling point cloud according to the environment three-dimensional laser point cloud and the point cloud map coordinate label set, obtain a first initial VR environment, and add it into the plurality of initial VR environments; The depth mean acquisition subunit is used in which the depth correction is to take the depth mean when the first environment modeling point cloud is inconsistent with the environment three-dimensional laser point cloud.

6. The system according to claim 5, characterized in that The three-dimensional modeling channel construction step includes: The three-dimensional modeling channel composition micro-unit is used for the three-dimensional modeling channel to include a point color feature extraction unit, a point depth feature extraction unit, and a point fully connected layer, wherein the point color feature extraction unit and the point depth feature extraction unit are connected in parallel, and the point fully connected layer is used to fully connect the output layer of the point color feature extraction unit and the point depth feature extraction unit; A point color feature extraction unit supervised training micro-unit, wherein the point color feature extraction unit supervised training micro-unit is used for the point color feature extraction unit to generate supervised training through color image recording data, image map coordinate label recording data and point color feature identification true value; A point depth feature extraction unit supervised training micro-unit, wherein the point depth feature extraction unit supervised training micro-unit is used for the point depth feature extraction unit to generate supervised training through depth image recording data, image map coordinate label recording data and point depth feature identification true value; The environment modeling point cloud output micro-unit is used by the point fully connected layer to fuse the color and depth of the point and output the environment modeling point cloud.

7. The system according to claim 5, characterized in that The first step of constructing a VR environment use case to be modeled includes: An environmental color image set acquisition micro-unit, the environmental color image set acquisition micro-unit is used to acquire the environmental color image set for a preset scene under the constraints of the image map coordinate label set through an RGB camera; An environment depth image set acquisition micro unit, the environment depth image set acquisition micro unit is used to acquire the environment depth image set for a preset scene through a depth camera under the constraint of the image map coordinate label set; A three-dimensional laser point cloud acquisition micro unit is used to acquire the three-dimensional laser point cloud of the environment through a laser radar for a preset scene under the constraints of the point cloud map coordinate label set.

8. The system according to claim 5, characterized in that Performing depth correction on the first environment modeling point cloud according to the environment three-dimensional laser point cloud and the point cloud map coordinate label set to obtain a first initial VR environment, including: A VR environment use case composition micro-unit, wherein the VR environment use case composition micro-unit is used for the first VR environment use case to be modeled and further includes a 3D audio source set and an audio source map coordinate label set; Update the initial VR environment micro-unit, the updated initial VR environment micro-unit is used to add the 3D audio source set in the first initial VR environment according to the audio source map coordinate label set, and the first updated initial VR environment will be added to the several initial VR environments.

9. The system according to claim 1, characterized in that The steps of executing the brain wave signal analysis module include: a first group of brain wave signal sequence acquisition unit, the first group of brain wave signal sequence acquisition unit being used to obtain a first group of brain wave signal sequences among the plurality of groups of brain wave signal sequences; A first group of brain wave signal sequence encoding unit, the first group of brain wave signal sequence encoding unit is used to encode the first group of brain wave signal sequences based on brain wave time attributes and brain wave space attributes to generate a brain wave signal encoding matrix sequence; A coding matrix sequence processing unit, the coding matrix sequence processing unit is used to process the brain wave signal coding matrix sequence through a CNN-LSTM emotion analysis network, output a first initial VR environment emotion value coefficient, and add it to the first emotion value coefficient; A CNN feature extraction layer construction unit, wherein the CNN feature extraction layer construction unit is used, and the CNN-LSTM sentiment analysis network construction steps include: By taking the EEG signal coding matrix sequence record data as input, and taking the EEG signal time feature identification data, EEG signal frequency feature identification data and EEG signal space feature identification data as supervision, the CNN neural network is trained to construct the CNN feature extraction layer; An emotion value assessment layer generating unit, wherein the emotion value assessment layer is used to train an LSTM neural network by taking the brain wave signal time feature identification data, the brain wave signal frequency feature identification data, the brain wave signal space feature identification data and the brain wave signal encoding matrix sequence recording data as input and taking the emotion pleasure identification data as supervision to generate an emotion value assessment layer; The emotion analysis network generation unit is used to fuse the CNN feature extraction layer and the emotion value assessment layer to generate the CNN-LSTM emotion analysis network.

10. A BCI-driven VR environment generation device, characterized in that: Used to deploy a BCI-driven VR environment generation system as described in any one of claims 1 to 9.

Images