Virtual reality psychological disease cognition experience method and system

Abstract

The invention discloses a virtual reality psychological disease cognition experience method and system, and relates to the field of virtual reality psychological disease cognition experience.In actual use, cognition experience is achieved through the virtual reality technology, and the best VR scene and intervention strategy are automatically recommended according to information data of a patient through the big data analysis technology; according to the invention, automatic personalized treatment design is carried out according to the specific condition and demand of each psychological disease patient, and specific problems of the psychological disease patients are more accurately solved, so that effective treatment is provided for each psychological disease patient, and the problem that drug dependence may be caused to the psychological disease patients due to long-term medication is avoided.

Description

Technical Field

[0001] The present invention relates to the field of virtual reality mental illness cognitive experience, and in particular to a virtual reality mental illness cognitive experience method and system. Background Art

[0002] Mental illness is a syndrome of comprehensive impairment of cognitive function caused by impairment of cognitive function. With the rapid increase in the number of adolescent patients, the shortage of human resources for care services has become a hot issue of social and medical concern. In recent years, the accelerated pace of life, increased life pressure, and study pressure have led to an increasing number of psychological-related diseases in different age groups. Existing mental illness treatment is generally carried out by psychotherapists. The level and experience of psychotherapists have a significant impact on the treatment effect of mental illness. Some therapists may lack sufficient training and practical experience and cannot provide effective treatment for patients. In addition, during psychotherapy, you may need to take medications, including antidepressants, antianxiety drugs, etc., such as fluoxetine hydrochloride tablets, paroxetine hydrochloride tablets, diazepam tablets, etc. Long-term use of medications may cause drug dependence. Summary of the invention

[0003] In view of the problems in the related art, the present invention proposes a virtual reality mental illness cognitive experience method and system to overcome the above technical problems existing in the existing related technology.

[0004] To this end, the specific technical solution adopted by the present invention is as follows:

[0005] A virtual reality mental illness cognitive experience method, comprising the following steps:

[0006] S1: Collect patient information data, including age, gender, occupation and educational background, medical history, family history and medication history, collect patient symptom information, previous treatment plans, treatment effects and patient feedback through standardized psychological assessment tools;

[0007] S2: Use big data analysis technology to automatically recommend the best VR scenarios and intervention strategies based on the patient’s information data;

[0008] S3: Uses head-mounted displays to provide high-resolution, low-latency visual experiences, uses panoramic videos recorded by 360-degree cameras to provide a full range of visual experiences, uses high-performance GPUs and advanced rendering technology to provide realistic visual effects, simulates real lighting and shadow effects, and enhances the realism of the scene;

[0009] S4: While using the head-mounted display, it generates corresponding sound effects in real time according to the user's movements and environmental changes, and uses tactile gloves to provide tactile feedback to the fingers to simulate the feeling of touching an object;

[0010] S5: Establish an emergency stop mechanism to allow patients to immediately exit the VR experience if they feel uncomfortable;

[0011] S6: Establish a long-term follow-up mechanism to regularly evaluate the patient's condition and treatment effects, collect and analyze patient feedback, and continuously improve treatment plans and technical means;

[0012] S7: Implement strict data encryption and access control measures to protect patients’ personal information and treatment data.

[0013] Furthermore, collecting patient information data in S1 includes the following steps:

[0014] S11: Determine data needs, clarify project goals and requirements, and determine what types of data need to be collected;

[0015] S12: Design data model, design data structure and database mode according to requirements, and determine data fields and data types;

[0016] S13: Perform data conversion, code conversion and format unification, and store the code conversion and format unification data in a database;

[0017] S14: Ensure compliance with relevant data protection laws and regulations during data collection and implement security measures such as data encryption and access control;

[0018] S15: Regularly check the accuracy, completeness and consistency of data, and establish data quality assessment standards and monitoring mechanisms.

[0019] Furthermore, the big data analysis technology in S2 includes the following steps:

[0020] S21: Perform data cleaning, fill in missing patient information data, identify and process outliers, and ensure data uniqueness;

[0021] S22: Extract useful features from the original patient information data, including symptom severity, treatment response speed, and physiological index change characteristics, select features that are highly correlated with treatment effects, reduce model complexity, and standardize data from different sources to facilitate model training;

[0022] S23: Use deep learning models according to task requirements, train models using training datasets, evaluate model performance using cross-validation and performance indicators, adjust model parameters, and optimize model performance;

[0023] S24: Based on the patient’s characteristics and historical data, a user profile is constructed, and VR scene recommendations are made by combining collaborative filtering and content filtering;

[0024] S25: Real-time monitoring of patients’ physiological and behavioral data, dynamic adjustment of recommendation strategies, collection of patient feedback, and continuous optimization of the recommendation system;

[0025] S26: Use automatic emotion recognition technology to collect patients’ facial expressions, conversation language and physiological data in real time in virtual reality scenes, monitor patients’ emotional changes, and assist in diagnosis and treatment.

[0026] Furthermore, Min-Max normalization is used in S22 to convert features of different scales to the same scale, including the following formula:

[0027] x′=x-min(x) / max(x)-min(x)

[0028] Where x is the original eigenvalue, min(x) and max(x) are the minimum and maximum values ​​of the feature, respectively.

[0029] Furthermore, the deep learning model in S23 uses a deep neural network, including the following algorithms:

[0030] import pandas as pd

[0031] from

[0032] sklearn.model_selection import

[0033] train_test_split

[0034] from sklearn.preprocessing import StandardScaler

[0035] import tensorflow as tf

[0036] from

[0037] tensorflow.keras.models import Sequential

[0038] from tensorflow.keras.layers import Dense,Dropout

[0039] # Load data

[0040] data=pd.read_csv('mental_health_data.csv')

[0041] #Separate features and labels

[0042] X=data.drop(columns=['label'])

[0043] y = data['label']

[0044] # Divide into training set and test set

[0045] X_train,X_test,y_train,y_test=train_test_split(X,y,test_size=0.2,random_state=42)

[0046] #Standardize features

[0047] scaler = StandardScaler()

[0048] X_train=scaler.fit_transform(X_train)

[0049] X_test=scaler.transform(X_test)

[0050] #Build the model

[0051] model=Sequential([

[0052] Dense(128,activation='relu',input_shape=(X_train.shape[1],)),

[0053] Dropout(0.5),

[0054] Dense(64,activation='relu'),

[0055] Dropout(0.5),

[0056] Dense(1,activation='sigmoid')#Use sigmoid activation function for binary classification problem])

[0057] # Compile the model

[0058] model.compile(optimizer='adam',loss='binary_crossentropy',metrics=['acc uracy'])

[0059] #Train model

[0060] history=model.fit(X_train,y_train,batch_size=32,epochs=26,validation_sp lit=0.1)

[0061] #Evaluate the model

[0062] loss,accuracy=model.evaluate(X_test,y_test)

[0063] print(f'TestLoss:{loss}')

[0064] print(f'TestAccuracy:{accuracy}')

[0065] # Generate predictions

[0066] y_pred=model.predict(X_test)

[0067] y_pred = (y_pred>0.5).astype(int) #Convert probability values ​​to category labels

[0068] #Generate classification report

[0069] fromsklearn.metricsimportclassification_report

[0070] print(classification_report(y_test,y_pred)).

[0071] Furthermore, the emotion automatic recognition technology in S26 includes the following steps:

[0072] S261: Capture facial images and video streams through cameras to analyze facial expressions, record sounds through microphones to analyze voice features, and collect physiological data such as heart rate, skin conductance, and skin temperature through wearable devices to assist in emotion recognition;

[0073] S262: Clean, convert and standardize the collected data to ensure that the data quality meets the requirements of subsequent analysis;

[0074] S263: Use computer vision technology to extract key feature points of facial expressions, extract acoustic features of pitch, intensity, and speech rate, and analyze physiological indicators such as heart rate variability and skin temperature;

[0075] S264: Build a deep learning model to map the extracted features to specific emotion categories;

[0076] S265: Based on the output of the model, the emotions are classified into several common emotional states, including happiness, sadness, anger, surprise, fear, disgust, and calmness;

[0077] S266: Smooth the output results of the model to remove noise and outliers and improve the stability and accuracy of the system.

[0078] Further, S262 includes the following detailed steps:

[0079] S2621: Remove invalid or erroneous data, confirm that all necessary file names and tags are correct, and detect and remove duplicate images and video clips;

[0080] S2622: Unify the file format, convert all images into the same format, and unify the video encoding format, resolution, and frame rate parameters;

[0081] S2623: Scale all images to the same size, ensure that all images are in the same color mode, and normalize pixel values;

[0082] S2624: extract frames from the video at fixed intervals to form a series of static images. For videos containing fast motion, motion compensation is performed to reduce blur caused by camera movement or fast object movement.

[0083] S2625: Increase data diversity and improve the generalization ability of the model through operations such as rotation, flipping, cropping, and brightness adjustment;

[0084] S2626: Establish a data feedback mechanism to trace poorly performing models back to data-level problems and make targeted improvements.

[0085] Furthermore, the deep learning model in S264 uses a convolutional neural network, including the following steps:

[0086] import torch

[0087] import torch.nn as nn

[0088] import torch.optim as optim

[0089] from torch.utils.data import DataLoader,Dataset

[0090] from torchvision import datasets,transforms

[0091] #Define dataset and data loader

[0092] transform=transforms.Compose([

[0093] transforms.Resize((32,32)),#Adjust image size

[0094] transforms.ToTensor(),#Convert to Tensor

[0095] transforms.Normalize((0.5,),(0.5,))#Normalization ])

[0097] train_dataset=datasets.MNIST(root='. / data', train=True, download=True, tra nsform=transform)

[0098] test_dataset=datasets.MNIST(root='. / data', train=False, download=True, tra nsform=transform)

[0099] train_loader=DataLoader(train_dataset,batch_size=64,shuffle=True)

[0100] test_loader=DataLoader(test_dataset,batch_size=64,shuffle=False)

[0101] #Define convolutional neural network model

[0102] classSimpleCNN(nn.Module):

[0103] def __init__(self):

[0104] super(SimpleCNN,self).__init__()

[0105] self.conv1=nn.Conv2d(1,32,kernel_size=3,padding=1)self.conv2=nn.Conv2d(32,64,kernel_size=3,padding=1)self.pool=nn.MaxPool2d(kernel_size=2,stride=2)self.fc1=nn.Linear(64*7*7,128)

[0106] self.fc2 = nn.Linear(128,10)

[0107] self.dropout=nn.Dropout(p=0.5)

[0108] defforward(self,x):

[0109] x=self.pool(torch.relu(self.conv1(x)))

[0110] x=self.pool(torch.relu(self.conv2(x)))

[0111] x=x.view(-1,64*7*7)

[0112] x = torch.relu(self.fc1(x))

[0113] x = self.dropout(x)

[0114] x = self.fc2(x)

[0115] returnx

[0116] # Initialize the model, loss function and optimizer

[0117] model = SimpleCNN()

[0118] criterion=nn.CrossEntropyLoss()

[0119] optimizer = optim.Adam (model.parameters (), lr = 0.001) # training model

[0120] num_epochs=10

[0121] for epoch in range(num_epochs):

[0122] model.train()

[0123] running_loss = 0.0

[0124] for images, labels in train_loader:

[0125] optimizer.zero_grad()

[0126] outputs = model(images)

[0127] loss = criterion(outputs, labels)

[0128] loss.backward()

[0129] optimizer.step()

[0130] running_loss += loss.item()

[0131] print(f'Epoch[{epoch + 1} / {num_epochs}], Loss: {running_loss / len(train_loader):.4f}')

[0132] # Test the model

[0133] model.eval()

[0134] correct = 0

[0135] total = 0

[0136] with torch.no_grad():

[0137] for images, labels in test_loader:

[0138] outputs = model(images)

[0139] _, predicted = torch.max(outputs.data, 1)

[0140] total += labels.size(0)

[0141] correct += (predicted == labels).sum().item()

[0142] print(f'Accuracyonthetestset:{100*correct / total:.2f}%').

[0143] Furthermore, a sliding window average algorithm is used in S266, including the following algorithms:

[0144] import

[0145] numpy as np

[0146] def moving_average(data,window_size):

[0147] return np.convolve(data,np.ones(window_size) / window_size,mode='same')

[0148] #Example

[0149] output_data=[1.0,1.2,1.1,1.3,1.4,1.5,1.6,1.7,1.8,1.9]

[0150] window_size=3

[0151] smoothed_data=moving_average(output_data,window_size)

[0152] print(smoothed_data).

[0153] The facial expression extraction of S263 includes the following steps:

[0154] S2631: Use big data analysis technology to collect patient symptom information, previous treatment plans, treatment effects and patient feedback information based on age, gender, occupation and educational background, past medical history, family medical history and drug use history through standardized psychological assessment tools, display treatment scenarios through head-mounted displays, and judge patient emotions based on acoustic characteristics such as facial expressions, pitch, intensity and speech speed, and physiological changes in physiological indicators such as heart rate variability and skin temperature;

[0155] S2632: When switching colors and adding items in the same treatment scene, it is possible to manually collect whether the patient still maintains the corresponding emotions and determine whether the treatment scene has a stimulating effect. For example, after switching colors and adding items, the patient will have a strong stimulation to a certain color or item;

[0156] S2633: After determining that the treatment scene has a stimulating effect, the treatment scene can be manually marked, and the stimulating effects of the treatment scene are divided into high, medium, and low levels. Different levels of stimulating effects are convenient for priority use in subsequent treatments;

[0157] S2634: After determining that the treatment scenario does not have a treatment effect, the treatment scenario is switched through the head mounted display according to the collected information until different emotions of the patient are recorded.

[0158] According to another aspect of the present invention, there is also provided a virtual reality mental illness cognitive experience system, comprising:

[0159] Psychological illness patient information collection module: collects patient information data, including age, gender, occupation and educational background, past medical history, family medical history and drug use history, and collects patient symptom information, past treatment plans, treatment effects and patient feedback information through standardized psychological assessment tools;

[0160] VR scene automatic recommendation module: uses big data analysis technology to automatically recommend the best VR scene and intervention strategy based on the patient's information data;

[0161] Virtual reality scene module: Uses head-mounted displays to provide high-resolution, low-latency visual experiences, uses panoramic videos recorded by 360-degree cameras to provide all-round visual experiences, uses high-performance GPUs and advanced rendering technology to provide realistic visual effects, simulates real lighting and shadow effects, and enhances the realism of the scene;

[0162] Multi-sensory feedback module: While using the head-mounted display, it generates corresponding sound effects in real time according to the user's movements and environmental changes, and uses tactile gloves to provide tactile feedback to the fingers to simulate the feeling of touching an object;

[0163] Automatic emotion recognition module: Uses automatic emotion recognition technology to collect patients' facial expressions, conversation language and physiological data in real time in virtual reality scenes, monitor patients' emotional changes, and assist in diagnosis and treatment;

[0164] Emergency stop module: Set up an emergency stop mechanism to allow patients to immediately exit the VR experience if they feel uncomfortable;

[0165] Condition tracking module: Establish a long-term tracking mechanism, regularly evaluate the patient's condition changes and treatment effects, collect and analyze patient feedback, and continuously improve treatment plans and technical means;

[0166] Data encryption module: Implement strict data encryption and access control measures to protect patients' personal information and treatment data.

[0167] Furthermore, the automatic emotion recognition module includes the following units:

[0168] Emotional data collection unit: captures facial images and video streams through cameras to analyze facial expressions, records sounds through microphones to analyze voice features, and collects physiological data such as heart rate, skin conductance, and skin temperature through wearable devices to assist in emotion recognition;

[0169] Emotional data processing unit: cleans, converts and standardizes the collected data to ensure that the data quality meets the requirements of subsequent analysis;

[0170] Data feature extraction unit: Use computer vision technology to extract key feature points of facial expressions, extract acoustic features of pitch, sound intensity, and speech speed, and analyze physiological indicators such as heart rate variability and skin temperature;

[0171] Deep learning model unit: build a deep learning model to map the extracted features to specific emotion categories;

[0172] Emotion classification unit: classify emotions into several common emotional states, including happy, sad, angry, surprised, fearful, disgusted, and calm emotions;

[0173] Result smoothing unit: Smooths the output results of the model to remove noise and outliers and improve the stability and accuracy of the system.

[0174] The beneficial effects of the present invention are:

[0175] (1) In actual use, the present invention uses virtual reality technology to provide cognitive experience and big data analysis technology to automatically recommend the best VR scene and intervention strategy based on the patient's information data. It also automatically designs personalized treatments based on the specific circumstances and needs of each patient with mental illness, more accurately targeting the specific problems of patients with mental illness, thereby providing effective treatment for each patient with mental illness and avoiding the problem of drug dependence caused by long-term medication.

[0176] (2) In actual use, the present invention can create a realistic virtual environment, allowing patients to feel as if they are in a completely different world. This immersive experience can help patients face and deal with their psychological problems in a safe and controlled environment.

[0177] (3) In actual use, the present invention can use automatic emotion recognition technology to collect the patient's facial expressions, conversation language and physiological data in real time in the virtual reality scene, monitor the patient's emotional changes, and assist in diagnosis and treatment. BRIEF DESCRIPTION OF THE DRAWINGS

[0178] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0179] Figure 1 This is one of the flow charts of a virtual reality mental illness cognitive experience method according to an embodiment of the present invention;

[0180] Figure 2 This is a second stereogram of a virtual reality mental illness cognitive experience method according to an embodiment of the present invention;

[0181] Figure 3 is one of the block diagrams of a virtual reality mental illness cognitive experience system according to an embodiment of the present invention;

[0182] Figure 4 This is the second block diagram of a virtual reality mental illness cognitive experience system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0183] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0184] According to an embodiment of the present invention, a virtual reality mental illness cognitive experience method is provided.

[0185] Embodiment 1

[0186] like Figure 1-Figure 2 As shown, the virtual reality mental illness cognitive experience method according to an embodiment of the present invention includes the following steps:

[0187] S1: Collect patient information data, including age, gender, occupation and educational background, medical history, family history and medication history, collect patient symptom information, previous treatment plans, treatment effects and patient feedback through standardized psychological assessment tools;

[0188] S2: Use big data analysis technology to automatically recommend the best VR scenarios and intervention strategies based on the patient’s information data;

[0189] S3: Uses head-mounted displays to provide high-resolution, low-latency visual experiences, uses panoramic videos recorded by 360-degree cameras to provide a full range of visual experiences, uses high-performance GPUs and advanced rendering technology to provide realistic visual effects, simulates real lighting and shadow effects, and enhances the realism of the scene;

[0190] S4: While using the head-mounted display, it generates corresponding sound effects in real time according to the user's movements and environmental changes, and uses tactile gloves to provide tactile feedback to the fingers to simulate the feeling of touching an object;

[0191] S5: Establish an emergency stop mechanism to allow patients to immediately exit the VR experience if they feel uncomfortable;

[0192] S6: Establish a long-term follow-up mechanism to regularly evaluate the patient's condition and treatment effects, collect and analyze patient feedback, and continuously improve treatment plans and technical means;

[0193] S7: Implement strict data encryption and access control measures to protect patients’ personal information and treatment data.

[0194] Embodiment 2

[0195] According to the virtual reality mental illness cognitive experience method of the embodiment of the present invention, collecting the patient's information data in S1 includes the following steps:

[0196] S11: Determine data needs, clarify project goals and requirements, and determine what types of data need to be collected;

[0197] S12: Design data model, design data structure and database mode according to requirements, and determine data fields and data types;

[0198] S13: performing data conversion, encoding conversion and format unification, and storing the encoding converted and format unified data in a database;

[0199] S14: Ensure compliance with relevant data protection laws and regulations during data collection and implement security measures such as data encryption and access control;

[0200] S15: Regularly check the accuracy, completeness and consistency of data, and establish data quality assessment standards and monitoring mechanisms.

[0201] Embodiment 3

[0202] According to the virtual reality mental illness cognitive experience method of the embodiment of the present invention, the big data analysis technology in S2 includes the following steps:

[0203] S21: Perform data cleaning, fill in missing patient information data, identify and process outliers, and ensure data uniqueness;

[0204] S22: Extract useful features from the original patient information data, including symptom severity, treatment response speed, and physiological index change characteristics, select features that are highly correlated with treatment effects, reduce model complexity, and standardize data from different sources to facilitate model training;

[0205] S23: Use deep learning models according to task requirements, train models using training datasets, evaluate model performance using cross-validation and performance indicators, adjust model parameters, and optimize model performance;

[0206] S24: Based on the patient’s characteristics and historical data, a user profile is constructed, and VR scene recommendations are made by combining collaborative filtering and content filtering;

[0207] S25: Real-time monitoring of patients’ physiological and behavioral data, dynamic adjustment of recommendation strategies, collection of patient feedback, and continuous optimization of the recommendation system;

[0208] S26: Use automatic emotion recognition technology to collect patients’ facial expressions, conversation language and physiological data in real time in virtual reality scenes, monitor patients’ emotional changes, and assist in diagnosis and treatment.

[0209] Embodiment 4

[0210] like Figure 1-Figure 2 As shown, according to the virtual reality mental illness cognitive experience method of the embodiment of the present invention, in S22, Min-Max normalization is used to convert features of different scales to the same scale, including the following formula:

[0211] x′=x-min(x) / max(x)-min(x)

[0212] Where x is the original eigenvalue, min(x) and max(x) are the minimum and maximum values ​​of the feature, respectively.

[0213] Embodiment 5

[0214] According to the virtual reality mental illness cognitive experience method of the embodiment of the present invention, the deep learning model in S23 uses a deep neural network, including the following algorithms:

[0215] import

[0216] pandas as pd

[0217] from sklearn.model_selection

[0218] import

[0219] train_test_split

[0220] from sklearn.preprocessing

[0221] import

[0222] StandardScaler

[0223] import

[0224] tensorflow as tf

[0225] fromtensorflow.keras.modelsimpor tSequential

[0226] from tensorflow.keras.layers import Dense,Dropout

[0227] # Load data

[0228] data=pd.read_csv('mental_health_data.csv')

[0229] #Separate features and labels

[0230] X=data.drop(columns=['label'])

[0231] y = data['label']

[0232] # Divide into training set and test set

[0233] X_train,X_test,y_train,y_test=train_test_split(X,y,test_size=0.2,random_state=42)

[0234] #Standardize features

[0235] scaler = StandardScaler()

[0236] X_train=scaler.fit_transform(X_train)

[0237] X_test=scaler.transform(X_test)

[0238] #Build the model

[0239] model=Sequential([

[0240] Dense(128,activation='relu',input_shape=(X_train.shape[1],)),

[0241] Dropout(0.5),

[0242] Dense(64,activation='relu'),

[0243] Dropout(0.5),

[0244] Dense(1,activation='sigmoid')#Use sigmoid activation function for binary classification problem])

[0245] # Compile the model

[0246] model.compile(optimizer='adam',loss='binary_crossentropy',metrics=['acc uracy'])

[0247] #Train model

[0248] history=model.fit(X_train,y_train,batch_size=32,epochs=26,validation_sp lit=0.1)

[0249] #Evaluate the model

[0250] loss,accuracy=model.evaluate(X_test,y_test)

[0251] print(f'TestLoss:{loss}')

[0252] print(f'TestAccuracy:{accuracy}')

[0253] # Generate predictions

[0254] y_pred=model.predict(X_test)

[0255] y_pred = (y_pred>0.5).astype(int) #Convert probability values ​​to category labels

[0256] #Generate classification report

[0257] fromsklearn.metricsimportclassification_report

[0258] print(classification_report(y_test,y_pred)).

[0259] Embodiment 6

[0260] According to the virtual reality mental illness cognitive experience method of the embodiment of the present invention, the emotion automatic recognition technology in S5 includes the following steps:

[0261] S261: Capture facial images and video streams through cameras to analyze facial expressions, record sounds through microphones to analyze voice features, and collect physiological data such as heart rate, skin conductance, and skin temperature through wearable devices to assist in emotion recognition;

[0262] S262: Clean, convert and standardize the collected data to ensure that the data quality meets the requirements of subsequent analysis;

[0263] S263: Use computer vision technology to extract key feature points of facial expressions, extract acoustic features of pitch, intensity, and speech rate, and analyze physiological indicators such as heart rate variability and skin temperature;

[0264] S264: Build a deep learning model to map the extracted features to specific emotion categories;

[0265] S265: Based on the output of the model, the emotions are classified into several common emotional states, including happiness, sadness, anger, surprise, fear, disgust, and calmness;

[0266] S266: Smooth the output results of the model to remove noise and outliers and improve the stability and accuracy of the system.

[0267] S262 includes the following detailed steps:

[0268] S2621: Remove invalid or erroneous data, confirm that all necessary file names and tags are correct, and detect and remove duplicate images and video clips;

[0269] S2622: Unify the file format, convert all images into the same format, and unify the video encoding format, resolution, and frame rate parameters;

[0270] S2623: Scale all images to the same size, ensure that all images are in the same color mode, and normalize pixel values;

[0271] S2624: extract frames from the video at fixed intervals to form a series of static images. For videos containing fast motion, motion compensation is performed to reduce blur caused by camera movement or fast object movement.

[0272] S2625: Increase data diversity and improve the generalization ability of the model through operations such as rotation, flipping, cropping, and brightness adjustment;

[0273] S2626: Establish a data feedback mechanism to trace poorly performing models back to data-level problems and make targeted improvements.

[0274] like Figure 1-Figure 2 As shown, according to the virtual reality mental illness cognitive experience method of the embodiment of the present invention, the deep learning model in S264 uses a convolutional neural network, including the following steps:

[0275] import

[0276] torch

[0277] import

[0278] torch.nn as nn

[0279] import

[0280] torch.optim as optim

[0281] from torch.utils.data

[0282] import DataLoader,Dataset

[0283] from torchvision import datasets,transforms

[0284] #Define dataset and data loader

[0285] transform=transforms.Compose([

[0286] transforms.Resize((32,32)),#Adjust image size

[0287] transforms.ToTensor(),#Convert to Tensor

[0288] transforms.Normalize((0.5,),(0.5,))#Normalization ])

[0290] train_dataset = datasets.MNIST(root='. / data', train=True, download=True, transform=transform)

[0291] test_dataset = datasets.MNIST(root='. / data', train=False, download=True, transform=transform)

[0292] train_loader = DataLoader(train_dataset, batch_size=64, shuffle=True)

[0293] test_loader = DataLoader(test_dataset, batch_size=64, shuffle=False)

[0294] # Define the convolutional neural network model

[0295] class SimpleCNN(nn.Module):

[0296] def __init__(self):

[0297] super(SimpleCNN, self).__init__()

[0298] self.conv1 = nn.Conv2d(1, 32, kernel_size=3, padding=1)

[0299] self.conv2 = nn.Conv2d(32, 64, kernel_size=3, padding=1)self.pool = nn.MaxPool2d(kernel_size=2, stride=2)self.fc1 = nn.Linear(64 * 7 * 7, 128)

[0300] self.fc2 = nn.Linear(128, 10)

[0301] self.dropout = nn.Dropout(p=0.5)

[0302] def forward(self, x):

[0303] x=self.pool(torch.relu(self.conv1(x)))

[0304] x=self.pool(torch.relu(self.conv2(x)))

[0305] x=x.view(-1,64*7*7)

[0306] x = torch.relu(self.fc1(x))

[0307] x = self.dropout(x)

[0308] x = self.fc2(x)

[0309] returnx

[0310] # Initialize the model, loss function and optimizer

[0311] model = SimpleCNN()

[0312] criterion=nn.CrossEntropyLoss()

[0313] optimizer = optim.Adam (model.parameters (), lr = 0.001) # training model

[0314] num_epochs=10

[0315] for epoch in range(num_epochs):

[0316] model.train()

[0317] running_loss=0.0

[0318] for images,labels in train_loader:

[0319] optimizer.zero_grad()

[0320] outputs = model(images)

[0321] loss=criterion(outputs,labels)

[0322] loss.backward()

[0323] optimizer.step()

[0324] running_loss+=loss.item()

[0325] print(f'Epoch[{epoch+1} / {num_epochs}],Loss:{running_loss / len(train_loader):.4f}')

[0326] #Test model

[0327] model.eval()

[0328] correct=0

[0329] total=0

[0330] with torch.no_grad():

[0331] forimages,labelsintest_loader:

[0332] outputs = model(images)

[0333] _,predicted=torch.max(outputs.data,1)

[0334] total+=labels.size(0)

[0335] correct+=(predicted==labels).sum().item()

[0336] print(f'Accuracyonthetestset:{100*correct / total:.2f}%').

[0337] like Figure 1-Figure 2 As shown, according to the virtual reality mental illness cognitive experience method of the embodiment of the present invention, the sliding window average algorithm is used in S266, including the following algorithms:

[0338] import numpy as np

[0339] def moving_average(data,window_size):

[0340] return np.convolve(data,np.ones(window_size) / window_size,mode='same')

[0341] #Example

[0342] output_data=[1.0,1.2,1.1,1.3,1.4,1.5,1.6,1.7,1.8,1.9]

[0343] window_size=3

[0344] smoothed_data=moving_average(output_data,window_size)

[0345] print(smoothed_data).

[0346] The facial expression extraction of S263 includes the following steps:

[0347] The deep learning model in S264 uses a convolutional neural network, the sliding window average algorithm is used in S266, and the facial expression extraction in S263 includes the following steps:

[0348] S2631: Use big data analysis technology to collect patient symptom information, previous treatment plans, treatment effects and patient feedback information based on age, gender, occupation and educational background, past medical history, family medical history and drug use history through standardized psychological assessment tools, display treatment scenarios through head-mounted displays, and judge patient emotions based on acoustic characteristics such as facial expressions, pitch, intensity and speech speed, and physiological changes in physiological indicators such as heart rate variability and skin temperature;

[0349] S2632: When switching colors and adding items in the same treatment scene, it is possible to manually collect whether the patient still maintains the corresponding emotions and determine whether the treatment scene has a stimulating effect. For example, after switching colors and adding items, the patient will have a strong stimulation to a certain color or item;

[0350] S2633: After determining that the treatment scene has a stimulating effect, the treatment scene can be manually marked, and the stimulating effects of the treatment scene are divided into high, medium, and low levels. Different levels of stimulating effects are convenient for priority use in subsequent treatments;

[0351] S2634: After determining that the treatment scenario does not have a treatment effect, the treatment scenario is switched through the head mounted display according to the collected information until different emotions of the patient are recorded.

[0352] According to another aspect of the present invention, a virtual reality mental illness cognitive experience system is also provided. Figure 3 and Figure 4 As shown, including:

[0353] Psychological illness patient information collection module: collects patient information data, including age, gender, occupation and educational background, past medical history, family medical history and drug use history, and collects patient symptom information, past treatment plans, treatment effects and patient feedback information through standardized psychological assessment tools;

[0354] VR scene automatic recommendation module: uses big data analysis technology to automatically recommend the best VR scene and intervention strategy based on the patient's information data;

[0355] Virtual reality scene module: Uses head-mounted displays to provide high-resolution, low-latency visual experiences, uses panoramic videos recorded by 360-degree cameras to provide all-round visual experiences, uses high-performance GPUs and advanced rendering technology to provide realistic visual effects, simulates real lighting and shadow effects, and enhances the realism of the scene;

[0356] Multi-sensory feedback module: While using the head-mounted display, it generates corresponding sound effects in real time according to the user's movements and environmental changes, and uses tactile gloves to provide tactile feedback to the fingers to simulate the feeling of touching an object;

[0357] Automatic emotion recognition module: Uses automatic emotion recognition technology to collect patients' facial expressions, conversation language and physiological data in real time in virtual reality scenes, monitor patients' emotional changes, and assist in diagnosis and treatment;

[0358] Emergency stop module: Set up an emergency stop mechanism to allow patients to immediately exit the VR experience if they feel uncomfortable;

[0359] Condition tracking module: Establish a long-term tracking mechanism, regularly evaluate the patient's condition changes and treatment effects, collect and analyze patient feedback, and continuously improve treatment plans and technical means;

[0360] Data encryption module: Implement strict data encryption and access control measures to protect patients' personal information and treatment data.

[0361] The automatic emotion recognition module includes the following units:

[0362] Emotional data collection unit: captures facial images and video streams through cameras to analyze facial expressions, records sounds through microphones to analyze voice features, and collects physiological data such as heart rate, skin conductance, and skin temperature through wearable devices to assist in emotion recognition;

[0363] Emotional data processing unit: cleans, converts and standardizes the collected data to ensure that the data quality meets the requirements of subsequent analysis;

[0364] Data feature extraction unit: Use computer vision technology to extract key feature points of facial expressions, extract acoustic features of pitch, sound intensity, and speech speed, and analyze physiological indicators such as heart rate variability and skin temperature;

[0365] Deep learning model unit: build a deep learning model to map the extracted features to specific emotion categories;

[0366] Emotion classification unit: classify emotions into several common emotional states, including happy, sad, angry, surprised, fearful, disgusted, and calm emotions;

[0367] Result smoothing unit: Smooths the output results of the model to remove noise and outliers and improve the stability and accuracy of the system.

[0368] In order to facilitate understanding of the above technical solutions of the present invention, the working principle or operation mode of the present invention in the actual process is described in detail below.

[0369] In summary, with the help of the above technical solutions of the present invention, in actual use, the present invention uses virtual reality technology cognitive experience, uses big data analysis technology to automatically recommend the best VR scene and intervention strategy based on the patient's information data, and automatically personalizes the treatment design according to the specific situation and needs of each patient with mental illness, more accurately targeting the specific problems of patients with mental illness, thereby providing effective treatment for each patient with mental illness, avoiding the problem of drug dependence caused by long-term medication for patients with mental illness, and creating a realistic virtual environment so that patients feel as if they are in a completely different world. This immersive experience can help patients face and deal with their psychological problems in a safe and controlled environment, and can use automatic emotion recognition technology to collect patients' facial expressions, conversation language and physiological data in virtual reality scenes in real time, monitor patients' emotional changes, and assist in diagnosis and treatment.

[0370] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A virtual reality mental illness cognitive experience method, characterized in that: The following steps are involved: S1: Collect patient information data, including age, gender, occupation and educational background, medical history, family history and medication history, collect patient symptom information, previous treatment plans, treatment effects and patient feedback through standardized psychological assessment tools; S2: Use big data analysis technology to automatically recommend the best VR scenarios and intervention strategies based on the patient’s information data; S3: Uses a head-mounted display to provide a high-resolution, low-latency visual experience, uses panoramic video recorded by a 360-degree camera, uses a high-performance GPU and advanced rendering technology to simulate realistic lighting and shadow effects; S4: While using a head-mounted display, it generates corresponding sound effects in real time based on the user's movements and environmental changes, and uses tactile gloves to provide tactile feedback to the fingers; S5: Establish an emergency stop mechanism to allow patients to immediately exit the VR experience if they feel uncomfortable; S6: Establish a long-term follow-up mechanism to regularly evaluate the patient's condition and treatment effects, collect and analyze patient feedback, and continuously improve treatment plans and technical means; S7: Implement strict data encryption and access control measures to protect patients’ personal information and treatment data.

2. A virtual reality psychological disease cognitive experience method according to claim 1, characterized in that: The acquisition of patient information data in S1 includes the following steps: S11: Determine data needs, clarify project goals and requirements, and determine what types of data need to be collected; S12: Design data model, design data structure and database mode according to requirements, and determine data fields and data types; S13: performing data conversion, encoding conversion and format unification, and storing the encoding converted and format unified data in a database; S14: Ensure compliance with relevant data protection laws and regulations during data collection and implement security measures such as data encryption and access control; S15: Regularly check the accuracy, completeness and consistency of data, and establish data quality assessment standards and monitoring mechanisms.

3. A virtual reality psychological disease cognitive experience method according to claim 2, characterized in that: The big data analysis technology in S2 includes the following steps: S21: Perform data cleaning, fill in missing patient information data, identify and process outliers, and ensure data uniqueness; S22: Extract useful features from the original patient information data, including symptom severity, treatment response speed, and physiological index change characteristics, select features that are highly correlated with treatment effects, reduce model complexity, and standardize data from different sources; S23: Use deep learning models according to task requirements, train models using training datasets, evaluate model performance using cross-validation and performance indicators, adjust model parameters, and optimize model performance; S24: Based on the patient’s characteristics and historical data, a user profile is constructed, and VR scene recommendations are made by combining collaborative filtering and content filtering; S25: Real-time monitoring of patients’ physiological and behavioral data, dynamic adjustment of recommendation strategies, collection of patient feedback, and continuous optimization of the recommendation system; S26: Use automatic emotion recognition technology to collect patients’ facial expressions, conversation language and physiological data in real time in virtual reality scenes to monitor patients’ emotional changes.

4. A virtual reality psychological disease cognitive experience method according to claim 3, characterized in that: In S22, Min-Max normalization is used to convert features of different scales to the same scale.

5. A virtual reality psychological disease cognitive experience method according to claim 4, characterized in that: The deep learning model in S23 uses a deep neural network.

6. A virtual reality psychological disease cognitive experience method according to claim 1, characterized in that: The automatic emotion recognition technology in S26 includes the following steps: S261: Capture facial images and video streams through cameras to analyze facial expressions, record sounds through microphones to analyze voice features, and collect physiological data such as heart rate, skin conductance, and skin temperature through wearable devices to assist in emotion recognition; S262: Clean, convert and standardize the collected data to ensure that the data quality meets the requirements of subsequent analysis; S263: Use computer vision technology to extract key feature points of facial expressions, extract acoustic features of pitch, intensity, and speech rate, and analyze physiological indicators such as heart rate variability and skin temperature; S264: Build a deep learning model to map the extracted features to specific emotion categories; S265: Based on the output of the model, the emotions are classified into several common emotional states, including happiness, sadness, anger, surprise, fear, disgust, and calmness; S266: Smoothing the output results of the model to remove noise and outliers.

7. A virtual reality psychological disease cognitive experience method according to claim 6, characterized in that: The S262 includes the following detailed steps: S2621: Remove invalid or erroneous data, confirm that all necessary file names and tags are correct, and detect and remove duplicate images and video clips; S2622: Unify the file format, convert all images into the same format, and unify the video encoding format, resolution, and frame rate parameters; S2623: Scale all images to the same size, ensure that all images are in the same color mode, and normalize pixel values; S2624: extract frames from the video at fixed intervals to form a series of static images. For videos containing fast motion, motion compensation is performed to reduce blur caused by camera movement or fast object movement. S2625: Increase data diversity and improve the generalization ability of the model through operations such as rotation, flipping, cropping, and brightness adjustment; S2626: Establish a data feedback mechanism to trace poorly performing models back to data-level problems and make targeted improvements.

8. A virtual reality psychological disease cognitive experience method according to claim 6, characterized in that: The deep learning model in S264 uses a convolutional neural network, the sliding window average algorithm is used in S266, and the facial expression extraction in S263 includes the following steps: S2631: Based on the collected information, the treatment scene is displayed through a head-mounted display, and the patient's emotions are judged based on the physiological changes shown by the patient; S2632: Switch colors and add items in the same treatment scene to collect whether the patient still maintains the corresponding emotions and determine whether the treatment scene has a stimulating effect; S2633: After determining that the treatment scene has a stimulating effect, the treatment scene is marked and used preferentially in subsequent treatments; S2634: After determining that the treatment scenario does not have a treatment effect, the treatment scenario is switched through the head mounted display according to the collected information until different emotions of the patient are recorded.

9. A virtual reality mental illness cognitive experience system, based on the application of a virtual reality mental illness cognitive experience method according to claim 8, characterized in that: include: Psychological illness patient information collection module: collects patient information data, including age, gender, occupation and educational background, past medical history, family medical history and drug use history, and collects patient symptom information, past treatment plans, treatment effects and patient feedback information through standardized psychological assessment tools; VR scene automatic recommendation module: uses big data analysis technology to automatically recommend the best VR scene and intervention strategy based on the patient's information data; Virtual reality scene module: Uses head-mounted displays to provide high-resolution, low-latency visual experiences, uses panoramic videos recorded by 360-degree cameras, and uses high-performance GPUs and advanced rendering technology to simulate realistic lighting and shadow effects; Multi-sensory feedback module: While using a head-mounted display, it generates corresponding sound effects in real time based on the user's movements and environmental changes, and uses tactile gloves to provide tactile feedback on the fingers; Automatic emotion recognition module: Uses automatic emotion recognition technology to collect patients' facial expressions, conversation language and physiological data in real time in virtual reality scenes to monitor patients' emotional changes; Emergency stop module: Set up an emergency stop mechanism to allow patients to immediately exit the VR experience if they feel uncomfortable; Condition tracking module: Establish a long-term tracking mechanism, regularly evaluate the patient's condition changes and treatment effects, collect and analyze patient feedback, and continuously improve treatment plans and technical means; Data encryption module: Implement strict data encryption and access control measures to protect patients' personal information and treatment data.

10. A virtual reality psychological disease cognitive experience method and system according to claim 9, characterized in that: The automatic emotion recognition module includes the following units: Emotional data collection unit: captures facial images and video streams through cameras for analyzing facial expressions, records sounds through microphones for analyzing voice features, and collects physiological data such as heart rate, skin conductance, and skin temperature through wearable devices; Emotional data processing unit: cleans, converts and standardizes the collected data to ensure that the data quality meets the requirements of subsequent analysis; Data feature extraction unit: Use computer vision technology to extract key feature points of facial expressions, extract acoustic features of pitch, sound intensity, and speech speed, and analyze physiological indicators such as heart rate variability and skin temperature; Deep learning model unit: build a deep learning model to map the extracted features to specific emotion categories; Emotion classification unit: classify emotions into several common emotional states, including happy, sad, angry, surprised, fearful, disgusted, and calm emotions; Result smoothing unit: Smooths the output results of the model to remove noise and outliers.

Images