Building design preference evaluation method and system based on multi-modal data

Abstract

The invention discloses an architectural design preference evaluation method and system based on multi-modal data, and the method comprises the steps: constructing a virtual reality architectural scene based on architectural design data, reinforcing the emotional response of a user through the design of an emotion amplifier, obtaining the neurophysiological data, emotion data and cognitive data of the user, and forming a multi-modal data set; and performing deep analysis on the building design preference according to the multi-modal data to obtain a building design evaluation result. According to the method provided by the embodiment of the invention, the virtual scene simulation technology and the neurophysiological data are combined, and a traditional method for obtaining the multi-modal data of the emotional response of the user in the architectural design is optimized in an immersive virtual reality man-machine interaction scene; the multi-modal data is utilized to perform cross validation on various types of emotion experience data to provide a more accurate preference evaluation result for building design, so that the building design can be optimized according to the preference evaluation result of the user, and a scientific basis is provided for creating high-quality building design.

Description

Technical Field

[0001] The present invention relates to the technical field of architectural design, and in particular to an architectural design preference evaluation method and system based on multimodal data. Background Art

[0002] Since architectural design is closely related to users' emotional responses, many aspects of architectural design, including shape, color, material and light, will have an impact on users' emotions. Therefore, during the architectural design stage, the architectural design plan will be presented to users, who will experience the plan emotionally and evaluate and optimize the plan based on the results of their emotional experience.

[0003] In the prior art, the usual design preference evaluation method relies on user interviews, questionnaires and scoring. However, the collected user feedback information is easily affected by emotional fluctuations, memory bias or environmental factors, and it is difficult to fully and objectively reflect the user's real emotional experience in three-dimensional space. Therefore, it is impossible to obtain accurate user emotional experience results, and further, it is impossible to accurately optimize the architectural design scheme based on the user emotional experience results.

[0004] Therefore, the prior art needs to be further improved. Summary of the invention

[0005] In view of the above-mentioned deficiencies in the prior art, the purpose of the present invention is to provide users with a method and system for evaluating architectural design preferences based on multimodal data, so as to overcome the defect that the emotional experience results fed back by users in the prior art are inaccurate, resulting in the inability to accurately optimize the architectural design based on the user's emotional experience results.

[0006] The technical solution adopted by the present invention to solve the technical problem is as follows: In a first aspect, the present application provides a method for evaluating architectural design preferences based on multimodal data, which includes: Pre-preparation stage: according to the execution steps of the preset virtual reality game in the preset emotion amplifier and the task type of the game, the preset virtual reality game is presented on the human-computer interaction interface; Data collection stage: presenting a pre-built virtual reality architectural scene based on architectural design data on a human-computer interaction interface, and obtaining user neurophysiological data, user emotional data, and user cognitive data in the virtual reality architectural scene; Data analysis stage: Integrate the user's neurophysiological data, user emotional data and user cognitive data to obtain multimodal data; analyze the user's emotional experience and cognitive response based on the multimodal data to determine the preference evaluation result corresponding to the architectural design data.

[0007] Optionally, the task types of the preset virtual reality game include: a stress task type and a relaxation task type; the stress task type corresponds to increasing an initial stress value in the user's emotional state, and the relaxation task type corresponds to reducing the stress value in the user's emotional state.

[0008] Optionally, the step of presenting a pre-built virtual reality architectural scene based on architectural design data on a human-computer interaction interface includes: Dynamic virtual scene presentation stage: simulate walking experience along a preset path in the virtual reality building scene, and establish connection with the motion interaction device to synchronously realize dynamic movement of the user's virtual position with the help of the motion interaction device; Static virtual scene presentation stage: presenting a local area of ​​the building in the virtual reality building scene, providing scene layout information with a fixed perspective, and outputting and displaying ERP cognitive test information in the process.

[0009] Optionally, the step of obtaining user neurophysiological data, user emotional data and user cognitive data in the virtual reality building scene includes: Obtain the user's EEG data, ECG data, and eye tracking data respectively to obtain the user's neurophysiological data and user emotional data; The ERP cognitive test information input by the user during the static virtual scene presentation phase is received, and the user cognitive data is obtained by combining the user's response records during the test process.

[0010] Optionally, before the step of integrating the user neurophysiological data, the user emotional data and the user cognitive data to obtain multimodal data, the method further includes: Obtain ERP emotional face recognition information input by users in virtual reality building scenes; The ERP emotional face recognition information is added to the user cognitive data to update the user cognitive data.

[0011] Optionally, the step of integrating the user neurophysiological data, the user emotional data and the user cognitive data to obtain multimodal data includes: Standardize user neurophysiological data, user emotional data, and user cognitive data; Extract characteristic information contained in the user's neurophysiological data, user's emotional data, and user's cognitive data after standardization; The extracted feature information is fused into multimodal data.

[0012] Optionally, after the step of integrating the user neurophysiological data, the user emotional data and the user cognitive data to obtain multimodal data, the step further includes: The integrated multimodal data are cross-validated using the preset cross-validation algorithm.

[0013] Optionally, the step of determining a preference evaluation result corresponding to the architectural design data according to the multimodal data includes: The multimodal data is input into the trained preference assessment model to obtain the preference assessment results output by the preference assessment model.

[0014] Optionally, the step of determining the preference evaluation result corresponding to the architectural design data according to the multimodal data further includes: Map the physiological indicator data contained in the multimodal data to the corresponding geographical location in the architectural design in the form of a heat map to form an emotional plane map; Alternatively, the physiological index data contained in the multimodal data is fused with the corresponding spatial features or visual setting diagram in the architectural design in the form of a heat map in different spaces to obtain an emotion and feature fusion map; Alternatively, the physiological index data contained in the multimodal data are integrated with the corresponding three-dimensional axis view in the architectural design in the form of a heat map in different spaces to obtain an emotional stress map; The emotion plane map, emotion and feature fusion map and emotion pressure map are output and displayed.

[0015] In a second aspect, this embodiment provides a building design preference evaluation system based on multimodal data, which includes: The emotion amplification module is used to present the preset virtual reality game on the human-computer interaction interface according to the execution steps of the preset virtual reality game and the task type of the game in the preset emotion amplifier; A data acquisition module, used to present a pre-built virtual reality building scene based on the building design data on a human-computer interaction interface, and to obtain user neurophysiological data, user emotional data, and user cognitive data in the virtual reality building scene; The data analysis module is used to integrate the user's neurophysiological data, user emotional data and user cognitive data to obtain multimodal data; analyze the user's emotional experience and cognitive response based on the multimodal data to determine the preference evaluation result corresponding to the architectural design data.

[0016] The present invention discloses a method and system for evaluating architectural design preferences based on multimodal data, including: a pre-preparation stage: presenting a preset virtual reality game on a human-computer interaction interface according to the execution steps of a preset virtual reality game in a preset emotion amplifier and the task type of the game; a data collection stage: presenting a pre-constructed virtual reality architectural scene on a human-computer interaction interface based on architectural design data, and obtaining user neurophysiological data, user emotional data, and user cognitive data under the virtual reality architectural scene; a data analysis stage: integrating the user neurophysiological data, user emotional data, and user cognitive data to obtain multimodal data; analyzing the user's emotional experience and cognitive response according to the multimodal data, and determining the preference evaluation result corresponding to the architectural design data. The method of this embodiment combines virtual scene simulation technology with neurophysiological data, obtains multimodal data of the user's emotional response in a virtual reality architectural scene in a human-computer interaction scene, and uses various types of emotional experience data contained in the multimodal data to provide a preference evaluation result for architectural design, so that the architectural design can be optimized according to the user's preference evaluation result, providing a scientific basis for creating high-quality architectural design. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 is a flowchart of the steps of a method for evaluating architectural design preferences based on multimodal data in an embodiment of the present invention; Figure 2 is a schematic diagram of a three-dimensional axis view in an embodiment of the present invention; Figure 3 It is a principle structure block diagram of the architectural design preference evaluation system based on multimodal data in an embodiment of the present invention. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0019] Emotional experience measurement based on physiological data is often used to quantify and evaluate a user's emotional response to a product or service. In the architectural design stage, it is also possible to quantify the user's emotional experience to optimize architectural design decisions, thereby obtaining a building that is more in line with the user's emotional health.

[0020] Architectural design not only affects users' spatial perception and behavior patterns, but also directly affects their emotions and psychological experience, which in turn affects their work efficiency and physical and mental health. Since users' emotional experience is highly subjective, instantaneous, and affected by complex interactions of environmental factors, traditional design preference evaluation methods rely on user interviews, questionnaires, and scoring, which are easily affected by individual emotional fluctuations, memory bias, and environmental factors. Therefore, it is difficult to fully and objectively reflect users' real emotional experience in three-dimensional space. As a result, this feedback method lacks objective data support, resulting in a lack of accuracy and efficiency in the optimization of design solutions.

[0021] In order to overcome the above problems, the present invention proposes a method and system for evaluating architectural design preferences based on multimodal data. The architectural design data is displayed in a virtual reality device, and the emotional response of the user is first enhanced by using an emotional amplifier design, and then the user's neurophysiological data, emotional data and cognitive data of the user's immersive experience in the architectural design scene are obtained, and the above data are integrated to obtain multimodal data; the preference evaluation results corresponding to the architectural design data are determined according to the user's emotional information contained in the multimodal data. The method of the present invention combines immersive virtual scene simulation technology with data image visualization methods. Before the building is built, the emotional response of the user in the virtual scene of architectural design is evaluated to provide preference evaluation results for the architectural design, and then the architectural design can be optimized according to the user's preference evaluation results for the architectural design, providing a scientific basis for improving the quality of architectural design.

[0022] The following is a further and more detailed description of a method and system for evaluating architectural design preferences based on multimodal data disclosed in this embodiment in conjunction with the accompanying drawings provided in this embodiment.

[0023] This application provides a method for evaluating architectural design preferences based on multimodal data, such as Figure 1 As shown, including: Step S1, pre-preparation stage: presenting the preset virtual reality game on the human-computer interaction interface according to the execution steps of the preset virtual reality game in the preset emotion amplifier and the task type of the game.

[0024] Since emotional experience is highly subjective, and each user has different emotional experiences, different preferences in the same architectural virtual space, and different physiological indicators, in order to enhance the significance of emotions and control individual differences, and achieve more accurate user preference evaluation results, this step performs preparatory work before obtaining user emotional feedback information, that is, using a preset emotional amplifier to strengthen the user's emotional response, so as to accurately capture and amplify the user's emotional fluctuations on various elements in the architectural design, thereby obtaining more sensitive and detailed emotional data.

[0025] Specifically, in this step, the virtual reality device controls the task progress of the game in the virtual reality device according to the execution steps of the preset virtual reality game and the task type of the game to adjust the user's emotional state, wherein the task type of the game corresponds to the type of preference assessment.

[0026] In one embodiment, the preparatory stage of this step is the processing stage of emotion regulation, which includes two types of games: one is stress games, and the other is emotion relief games. According to the different needs of the specific experimental scenes, before officially entering the virtual architectural space scene experience, select a suitable game as a user emotion amplifier to adjust the emotional state of the participants before entering the experimental stage to ensure that their stress state when entering the experiment is comparable. That is, the selection is based on the purpose of this architectural design preference assessment. The stress game is to increase the initial stress value to amplify the effect of environmental stimuli on stress relief. The purpose of the soothing game design is to reduce the initial stress value to amplify the increased response. Therefore, if the purpose of the study is to monitor whether a specific environmental stimulus will increase user stress, a soothing game is selected, and vice versa.

[0027] In one embodiment, the "emotional stress game" is set as a "quick choice challenge" task. In this virtual reality (VR) game, users need to make decisions quickly under stressful circumstances. The user is placed in a simulated busy office environment, with documents, task lists, and prompt cards piled on the desktop. The entire situation simulates a high-pressure work environment with rich details. The user's task is to select the correct object from four options as quickly as possible and drag it to the specified location within the specified time. As the game progresses, the system will gradually increase the complexity of the task and the time pressure. For example, the initial task is limited to a fixed time (for example: 20 seconds), which is gradually shortened to a preset time (for example: 5 seconds), further enhancing the sense of time urgency. In addition, the game introduces a series of interference elements, including background noise (such as office noise), flashing red warning lights and alarms, etc., which further increase the user's tension. Whenever the user makes an incorrect choice or times out, the system will immediately feedback a red "Error!" or "Timeout! Task failed" prompt, and accelerate the task rhythm, so that the user gradually enters a higher stress state. The design stimulates users' stress responses through intense task settings and realistic environmental interference, and records their physiological indicators (such as heart rate, eye movements, and brain waves) in real time.

[0028] "Emotional Soothing Game": To achieve emotional relaxation, the emotional soothing game is designed as a "natural exploration" scene. Users are placed in a quiet forest and can walk freely, accompanied by soothing natural sounds such as birdsong and running water. There is no task pressure or time limit in the scene. Users can explore at their own pace, which will gradually calm their mood. Users can choose to enter the "breathing guidance mode", and a gradually enlarged breathing ring will appear on the screen to guide users to breathe synchronously and help them relax their body and mind. As the breathing guidance progresses, the light of the scene will gradually become softer, soothing the user's emotions.

[0029] By amplifying the scene design of the game as described above, the significance of the impact on users' emotions is increased, and at the same time, the emotional level of users when entering the virtual reality scene corresponding to the architectural design can be adjusted, so that the emotional level of users after adjustment is in a similar state.

[0030] Step S2, data collection stage: presenting a pre-constructed virtual reality architectural scene based on architectural design data on a human-computer interaction interface, and obtaining user neurophysiological data, user emotional data, and user cognitive data in the virtual reality architectural scene.

[0031] In this step, the architectural design data to be evaluated is first obtained, and a corresponding three-dimensional architectural model is created based on the architectural design data. The created three-dimensional architectural model is then integrated into the virtual reality game engine platform to realize the visualization of the architectural design data.

[0032] Furthermore, in this step, VR technology can be used to simulate and present the architectural scene. Specifically, VR technology is a virtual reality device, which can be a VR helmet, VR glasses, or other devices such as a VR ring screen. Select appropriate 3D modeling software, and create a spatial 3D model in the 3D modeling software. The 3D model includes various elements of the building, such as walls, doors and windows, roofs, furniture, etc. Import the 3D model into the VR platform, and set the camera, light source or sound effect, etc. in the VR platform to create a realistic immersive virtual reality architectural environment.

[0033] In detail, in order to better obtain the emotional information of user feedback, the architectural design data includes: dynamic stage data and static stage data. Through the two-stage architectural design space scene display, more comprehensive user feedback information can be obtained.

[0034] The step of presenting a pre-built virtual reality architectural scene based on architectural design data on a human-computer interaction interface comprises: Step S21, dynamic virtual scene presentation stage: simulate walking experience along a preset path in the virtual reality building scene, and establish a connection with a motion interaction device to synchronously realize dynamic movement of the user's virtual position with the help of the motion interaction device.

[0035] In order to ensure the coherence and authenticity of the space in the dynamic virtual scene, a dynamic tour of a predetermined route is adopted in this step, and the user simulates the walking experience in the virtual scene. At the same time, the virtual reality device also establishes a connection with the motion interaction device, and realizes the dynamic movement of the user's virtual position in synchronization with the motion interaction device through data interaction. In one implementation method, the user wears a head-mounted virtual reality device on the one hand, simulates walking in the virtual reality building scene of the virtual reality device, and the user runs in place on the other hand with the help of a VR universal support platform. The position of the user running on the VR universal support platform is consistent with the virtual space in the virtual reality device, thereby achieving the user's experience of "walking" in the virtual space. This dynamic walking experience overcomes the lack of immersion in static browsing of architectural design scenes and further enhances the authenticity of the experience. In addition, in this step, multi-sensory simulation is combined, including vision (building scene), hearing (such as conversation sound, wind sound), and smell (if applicable, flower fragrance, grass fragrance, etc. can be added), and a three-dimensional game-based dynamic space scene is constructed in the virtual reality swimming engine, and interactive elements (such as furniture, doors, etc.) are designed to achieve the maximum immersive experience.

[0036] Step S22, static virtual scene presentation stage: presenting a local area of ​​the building in the virtual reality building scene, providing scene layout information of a fixed perspective, and outputting and displaying ERP cognitive test information in the process.

[0037] In the static virtual scene presentation stage, the user is fixed in a certain area of ​​the virtual reality architectural scene, and the virtual reality device provides scene layout information with a fixed perspective, so that the user can explore the static scene in the VR space and give the user a different visual experience. In this stage, the user is placed in a specific area (such as a seat in an open or closed area) for a static immersive experience of a single scene for a preset time period (such as a few minutes). At the same time, in the experience of the static virtual scene presentation stage, the user needs to perform specific ERP cognitive test tasks in the scene, which includes simple memory tasks, math problem solving, choice judgment, etc., to test the user's concentration and cognitive ability under different spatial designs. For example, a closed space may increase the user's sense of pressure, while an open space may make the user feel more relaxed. This design after the tour rather than during the tour is to minimize the additional cognitive load that may be caused by the space switching itself when the user goes from point A to point B. Through this design, the spatial design, visual range and related perceptual stimuli (such as light, color, noise, etc.) of different areas interact with each other, thereby jointly affecting the user's emotions. At the same time, through repeated ERP cognitive test tasks, it can be more clearly revealed how changes in spatial design regulate users' emotional responses. To reduce potential interaction effects, a sequential randomization design can be used in the experiment to ensure that each participant's experience in different environments is not biased by the order.

[0038] In order to solve the problem of emotional fluctuations caused by the complexity of space design and dynamic changes, this step combines the design of dynamic and static experience. In the dynamic virtual scene, through the VR universal support platform, users can walk freely in the space, thereby maximizing the simulation of the real space experience and reducing the lack of immersion in the traditional static browsing method. In the virtual environment, multi-sensory methods are adopted, including vision, hearing and smell, and the game engine is used to increase the interactive design of gamification to restore the real space experience to the greatest extent.

[0039] Furthermore, in the architectural space design corresponding to the dynamic virtual scene and the static virtual scene, multiple different space designs can be added to compare the impact of users on spatial emotions and cognitive task execution in different space designs. Only specific space elements are changed between different space designs to control the variables of space elements, so as to obtain more accurate emotional feedback results of users on different types of environments.

[0040] After the virtual reality building scene is constructed based on the building design data in the above steps, a variety of user feedback information in the virtual reality building scene is obtained, and the feedback information includes: user neurophysiological data, user emotional data and user cognitive data.

[0041] Specifically, this step includes: The user's EEG data, ECG data and eye tracking data are obtained respectively to obtain the user's neurophysiological data and user emotional data; the ERP cognitive test information input by the user is received to obtain the user's cognitive data.

[0042] In one embodiment, the user's neurophysiological data includes: brain wave data, electrocardiogram data, and eye tracking data.

[0043] Heart rate sensors are used to collect users' ECG data to assess their stress levels and emotional responses. For example, the heart rate variability index (HRV) sensitively reflects the activity of the autonomic nervous system (SDNN and RMSSD are analyzed in the HRV time domain, and low values ​​indicate increased stress). It is a good indicator of stress and relaxation. High HRV usually indicates relaxation, and low HRV indicates stress or fatigue. In space design, HRV data can help identify the unconscious impact of different space element designs on user emotions.

[0044] Eye tracking: With the help of the eye tracking function of VR headsets, the user's gaze path, gaze time, gaze frequency and other data are recorded. These data reflect the user's focus preference on specific areas in the virtual reality architectural environment, so it can provide a reliable user preference reference and help improve spatial layout and visual hierarchy.

[0045] Brainwave data is the user's electroencephalogram (EEG) recorded with the help of EEG technology. EEG is a method of examination that records the spontaneous and rhythmic electrical activity of brain cell groups through scalp electrodes. By observing the frequency range of the waveform in the EEG, the user's emotional changes can be identified.

[0046] Since EEG technology can monitor the user's brain activity in different architectural design spatial scenarios, it can identify neural responses associated with specific emotional states (such as relaxation, tension, or concentration). EEG can provide direct feedback on emotional effects through changes in specific frequency bands in brain activity, which helps to identify how spatial characteristics affect the user's emotional experience. Especially in high or low stimulation space design, EEG data can reveal the user's neural load and help optimize the design. In specific implementation, the user's emotional changes can be obtained by monitoring the fluctuations in the alpha (8-13) and beta (13-30) frequency bands. For example: an increase in alpha waves usually indicates a calm and relaxed mood; a decrease may indicate anxiety or tension. High beta waves indicate stress, anxiety, or overexcitement; weakened beta waves are associated with relaxation. RAB can also be calculated to analyze the positive and negative emotions (positive emotions: such as pleasure and relaxation (enhanced alpha waves and moderate beta waves). Negative emotions: such as stress and anxiety (enhanced beta waves, reduced alpha waves, imbalanced left and right brain activity). It is also possible to identify whether the user's emotions are positive or negative by monitoring the activity of brain regions, especially the asymmetry of left and right brain activity in the frontal and occipital lobes.

[0047] User cognitive data refers to the cognitive information input by users based on ERP cognitive test tasks and the user's response records during the test.

[0048] Since multiple aspects of user data are collected in this step, the incompleteness of evaluating users' evaluation of architectural design based on a single data is overcome. For example, if only eye tracking technology is used to evaluate users' preferences for a certain area. Although eye movement data can provide information about the area that users focus on, it cannot directly reveal the user's emotional state such as stress or pleasure; while EEG data can capture the user's brain activity state, but due to its low spatial resolution, it is difficult to directly associate it with a specific spatial location or element.

[0049] Step S3, data analysis phase: Integrate the user's neurophysiological data, user emotional data and user cognitive data to obtain multimodal data; analyze the user's emotional experience and cognitive response based on the multimodal data to determine the preference evaluation result corresponding to the architectural design data.

[0050] After obtaining the user neurophysiological data, user emotional data and user cognitive data in the above steps, since the user neurophysiological data is often high-dimensional, noisy raw data, and the user emotional data and user cognitive data are in different dimensions, these data need to be integrated to obtain multimodal data.

[0051] Specifically, the step of integrating the user neurophysiological data, the user emotional data and the user cognitive data to obtain integrated multimodal data specifically includes: The user's neurophysiological data, user's emotional data and user's cognitive data are standardized; the characteristic information contained in the standardized user's neurophysiological data, user's emotional data and user's cognitive data is extracted; and the extracted characteristic information is fused into multimodal data.

[0052] Since the EEG data, ECG data, and eye tracking data in neurophysiological data have different dimensions and value ranges, they need to be standardized. Before standardization, the data needs to be cleaned and outliers removed to ensure the integrity and accuracy of the data. The data is then converted to a unified dimension and value range. For dimensions with obvious differences, normalization can be used to eliminate the impact of the dimension on data analysis.

[0053] Furthermore, feature information contained in the user's neurophysiological data, the user's emotional data, and the user's cognitive data is extracted respectively, so as to fuse the extracted feature information to obtain multimodal data.

[0054] After the multimodal data is obtained, the user's preference evaluation result for the architectural design data is determined according to the multimodal data. In this step, in order to extract more accurate preference evaluation results from the multimodal data, the multimodal data can be input into a trained preference evaluation model to obtain the preference evaluation results output by the preference evaluation model.

[0055] The preference evaluation model is trained using historical multimodal data. The historical multimodal data is marked with corresponding preference identifiers. The preset network model learns the historical multimodal data to obtain a trained preference evaluation model.

[0056] Furthermore, in order to obtain more accurate user emotion feedback information, before the step of integrating the user neurophysiological data, the user emotion data and the user cognitive data to obtain multimodal data, the method further includes: The ERP emotional face recognition information input by the user in the virtual reality building scene is obtained; the ERP emotional face recognition information is added to the user cognitive data to update the user cognitive data.

[0057] In this step, participants were asked to perform an ERP emotional face recognition task to more intuitively test the neurophysiological basis of their emotional responses. In this task, participants will watch emotional faces (including positive and negative emotions) of 3D virtual characters with different dynamic expressions in a VR environment. Participants need to make emotional judgments on these faces and evaluate the emotions conveyed by their faces (such as happiness, anger, sadness, etc.). After each face presentation, participants press a button to select the option corresponding to the emotion expressed by the face to measure the impact of different spatial designs on the emotional recognition task. In particular, after experiencing different spatial designs, the differences in participants' responses to emotional stimuli help assess their emotional load and stress state. EEG can measure the overall electrical activity of the brain and provide continuous data on the brain's neural activity, but it cannot accurately capture the immediate response to specific emotional stimuli or cognitive tasks. In contrast, the ERP emotional face recognition task is a specific component of the EEG that can accurately reflect the brain's time-locked response to specific events or stimuli, and therefore has a higher temporal resolution. The ERP emotional face recognition task can capture specific brain waves related to emotions and cognitive tasks (such as P300, N400, etc.). These waveforms can reveal the timing and brain area characteristics of emotional responses, while EEG can only provide overall electrical activity patterns and cannot locate specific emotions or task responses. HRV reflects more emotional fluctuations and stress levels over a long period of time, and cannot go deep into the specific neural basis and time process of emotional stimuli like ERP. Adding ERP emotional face recognition helps to reveal the neural mechanism of spatial environment or task response to emotional response. Through ERP emotional face recognition, the specific response waveforms and activity patterns of related brain areas when the brain processes emotional stimuli can be identified, and how different spatial designs affect emotional regulation through neural pathways can be further analyzed. This is detailed neural basis data that EEG, HRV and questionnaires cannot provide.

[0058] Furthermore, since the present application provides a comprehensive user emotional experience evaluation method based on obtaining user emotional feedback information from different dimensions, in order to ensure the reliability of the present evaluation method, the present method also includes: inputting the ERP emotional face recognition information and multimodal data into a trained cross-validation model to obtain the verification result output by the cross-validation model.

[0059] By combining multimodal data (EEG, HRV, eye movement data and questionnaires), researchers can obtain information about users' emotions, cognition and physiological reactions from different dimensions, which can provide a comprehensive user experience evaluation framework. By cross-validating these data, the impact of self-report bias can be effectively reduced. Correlation analysis. For example, studies have shown that there is a clear correlation between P300 fluctuations in EEG and changes in emotional state, while HRV can reflect changes in stress levels. By combining these data, researchers can confirm whether the experimental design is consistent with the user's actual experience. Through deep learning algorithm analysis of multimodal data, the potential relationship between various physiological data can be more efficiently mined, providing basic support for future emotion and stress prediction models. This AI-driven analysis can help designers predict users' emotional reactions to different spaces, so that they can make more optimized decisions that meet emotional needs in the early stages of space design.

[0060] In addition, the combination of ERP experiments and multimodal data can deeply verify the effectiveness of the experimental method itself. ERP (event-related potential) experiments have high temporal resolution and can accurately capture the brain's immediate response to specific emotional stimuli. By analyzing these responses, it is possible to clearly identify how the spatial environment design affects the brain's emotional processing. Therefore, the ERP experiment of the emotional face recognition task after the virtual environment experience can not only verify the impact of the spatial design on emotions, but also detect whether the experimental method effectively captures the neural basis of emotional changes. At the same time, combining EEG, HRV, and eye movement data with ERP can provide more comprehensive physiological and cognitive response data, which helps to confirm the accuracy and consistency of the experimental method in measuring emotional and cognitive responses. If different physiological data (such as EEG and HRV) are consistent with the ERP response and consistent with the results of the questionnaire survey, this can effectively verify the ability of the experimental method to measure emotional and cognitive responses.

[0061] Specifically, the specific steps of cross-validating the above method are: cross-validating the integrated multimodal data using a preset cross-validation algorithm. The preset cross-validation algorithm is K-fold cross-validation, leave-one-out cross-validation, bootstrapped cross-validation, or stratified cross-validation.

[0062] In this step, the experimental method of emotion and multiple cognitive ability detection is combined with EEG technology to detect the impact of space design. EEG ERP (event-related potential) detection of emotion and cognition is introduced in the experience of static simulation scenes. Users stay in a specific area to perform cognitive task tests, which more objectively and sensitively reflects the emotional and cognitive impact of space design.

[0063] Furthermore, in order to better display the preference evaluation results, this embodiment provides four different display methods. Through different visualization methods, the user's emotional changes are intuitively presented. Figure 2 As shown, it is easy to quickly identify the impact of different spatial elements on user emotions.

[0064] Specifically, the step of determining the preference evaluation result corresponding to the architectural design data according to the multimodal data further includes: Map the physiological indicator data contained in the multimodal data to the corresponding geographical location in the architectural design in the form of a heat map to form an emotional plane map; Alternatively, the physiological indicator data contained in the multimodal data is fused with the corresponding spatial feature map or visual setting map in the architectural design in the form of a heat map in different spaces to obtain an emotion and feature fusion map; Alternatively, the physiological index data contained in the multimodal data are integrated with the corresponding three-dimensional axis view in the architectural design in the form of a heat map in different spaces to obtain an emotional stress map; The emotion plane map, emotion and feature fusion map and emotion pressure map are output and displayed.

[0065] The above emotional plane map maps the user's emotional response in the virtual space by geographical location, and combines it with the spatial layout map to form a visual "emotional map" in the form of a heat map. Different colors are used to represent the intensity of the user's emotions. For example, blue represents a relaxed state, while red represents a state of tension or high pressure.

[0066] The emotion and feature fusion map is a comprehensive emotion map that combines spatial feature maps (such as light intensity, color arrangement, noise distribution) or visual setting maps to form a multi-dimensional data superposition. For example, there may be a correlation between low light or high noise in a certain area and the user's emotional response. By superimposing emotion data with the heat map of these spatial features, designers can understand the direct impact of specific environmental factors on user emotions and help optimize the design. Such an emotion-spatial feature fusion map can give designers a more intuitive understanding of "emotion triggers", making future design decisions more scientific and data-driven.

[0067] The emotional stress map is an emotional map obtained by combining emotional preference information with a three-dimensional axis view. By integrating the three-dimensional axis view of the space, the geographical distribution of user emotions can be observed from different perspectives. The floor plan can show the distribution of emotions in a single floor or area, while the three-dimensional axis view can show the distribution of emotions in multiple floors or three-dimensional spaces. This combination allows designers to track the changes in users' emotions in a larger spatial dimension and more accurately analyze the effects of different floors or spatial structures on emotions.

[0068] When the method of this embodiment is implemented, a full range of immersive experience is achieved through a high-precision virtual reality helmet combined with a virtual reality universal motion support platform. Since the high-precision equipment can synchronously and accurately capture and record the detailed eye movement data of users in the virtual environment, including gaze points and gaze paths, it provides reliable visual data support for analyzing the impact of spatial design on emotions and preferences. The head display device uses dual OLED display screens, and the resolution of the glasses is set to 1440 x 1600 pixels, with a total resolution of 2880 x 1600 pixels, which can provide higher quality visual effects and reduce the common "screen window effect". In addition, setting the field of view FOV width to approximately 110 degrees and the refresh rate to 90Hz can ensure the smoothness of the picture and help reduce the dizziness of most users.

[0069] The virtual reality universal motion support platform supports users to walk and stand freely in multiple directions, simulating the mobile experience in real space, and is compatible with multiple postures (including sitting). The support platform can also capture the frequency and amplitude of the user's leg movements, further helping to analyze the user's behavioral responses in different environments. The characteristics of the platform can provide multi-dimensional support for data collection of VR scene task modules and interactive elements, allowing users to experience the environmental effects in different design scenes more realistically. Therefore, combining a virtual head display device with a virtual display universal motion support platform can achieve a more realistic experience effect.

[0070] When acquiring the user's EEG data, an electrode cap can be used. Since the electrode cap uses wet electrodes, these electrodes are made of high-quality silver / silver chloride (Ag / AgCl) material, and conductive gel or solution is required to ensure good conductive contact between the electrode and the skin to ensure signal quality. At the same time, the electrode cap supports high-impedance recording in an unshielded environment, and each electrode is equipped with an LED indicator to display the impedance status in real time, which is convenient for quick adjustment.

[0071] The portable heart rate recorder (HRV) is equipped with multiple sensors and can monitor a variety of physiological signals in real time. It is particularly suitable for emotion regulation, cognitive load and stress research. In the specific implementation, the sampling rate of the heart rate recorder is set to 32Hz, and it contains 4 sensor channels, which are used for electrocardiogram (ECG), photoplethysmography (PPG), galvanic skin response (GSR) and temperature monitoring. The electrocardiogram (ECG) sensor, one of the core functions of the device, can accurately measure heart rate and heart rate variability. Heart rate variability is a key indicator for evaluating the function of the autonomic nervous system and stress response. It can reflect the balance of the sympathetic and parasympathetic nerves, and thus help researchers analyze the physiological responses of individuals in different environments.

[0072] The multimodal data architecture disclosed in this embodiment is a method for evaluating preferences in aggregate. First, the user's emotions are adjusted through the design of an emotion amplifier, and then the multimodal data is synchronously collected and integrated in a three-dimensional space. In an immersive (audio-visual combined) virtual reality environment, EEG, ECG and eye movement data are synchronously collected and used to quantify the spatial emotional experience, thereby obtaining a more accurate architectural design evaluation result. Furthermore, this embodiment also proposes combining the user's neurophysiological data (EEG, heart rate variability HRV, eye tracking) with a spatial feature map (such as light intensity, color arrangement, noise distribution) or a visual setting map to form a comprehensive emotional map with multidimensional data superposition, so as to more intuitively display the user's preferences for each architectural space element, thereby better providing an optimization strategy for architectural design.

[0073] The present invention provides a method for evaluating architectural design preferences, and also provides an architectural design preference evaluation system based on multimodal data, such as Figure 3 As shown, including: The emotion amplification module 310 is used to present the preset virtual reality game on the human-computer interaction interface according to the execution steps of the preset virtual reality game and the task type of the game in the preset emotion amplifier; its function is as described in step S1.

[0074] The data acquisition module 320 is used to present a pre-built virtual reality architectural scene based on the architectural design data in the human-computer interaction interface, and obtain the user's neurophysiological data, user emotional data and user cognitive data in the virtual reality architectural scene; its function is as described in step S2.

[0075] The data analysis module 330 is used to integrate the user's neurophysiological data, user emotional data and user cognitive data to obtain multimodal data; analyze the user's emotional experience and cognitive response based on the multimodal data to determine the preference evaluation result corresponding to the architectural design data. Its function is as described in step S3.

[0076] The method and system provided by the present invention amplify the effects of different space designs on emotions through the experimental design of "emotion amplifier", making these differences more significant. At the same time, multimodal physiological detection methods are used to obtain emotional and physiological responses under different architectural space styles. And a multi-sensory VR experience is designed to simulate real space transitions and emotional changes, and improve the accuracy of emotional data in virtual environments and its applicability to real situations. Furthermore, in response to the complex problem of neurophysiological data processing, the present invention uses data visualization technology to convert complex physiological data into easy-to-understand user preference and emotional response information, thereby lowering the threshold for data analysis and application, and converting user experience data into practical design optimization basis.

[0077] Those skilled in the art will readily appreciate other embodiments of the invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses or adaptations of the present invention that follow the general principles of the present invention and include common knowledge or customary techniques in the art that are not disclosed in this disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present invention are indicated by the following claims.

[0078] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application.

Claims

1. A method for evaluating architectural design preferences based on multimodal data, characterized in that: include: Pre-preparation stage: according to the execution steps of the preset virtual reality game in the preset emotion amplifier and the task type of the game, the preset virtual reality game is presented on the human-computer interaction interface; Data collection stage: presenting a pre-built virtual reality architectural scene based on architectural design data on a human-computer interaction interface, and obtaining user neurophysiological data, user emotional data, and user cognitive data in the virtual reality architectural scene; Data analysis stage: integrating the user's neurophysiological data, user's emotional data and user's cognitive data to obtain multimodal data; The user's emotional experience and cognitive response are analyzed according to the multimodal data to determine a preference evaluation result corresponding to the architectural design data.

2. The architectural design preference evaluation method based on multimodal data according to claim 1, characterized in that: The task types of the preset virtual reality game include: a stress task type and a relaxation task type; the stress task type corresponds to increasing the initial stress value in the user's emotional state, and the relaxation task type corresponds to reducing the stress value in the user's emotional state.

3. The architectural design preference evaluation method based on multimodal data according to claim 1, characterized in that: The step of presenting a pre-built virtual reality architectural scene based on architectural design data on a human-computer interaction interface comprises: Dynamic virtual scene presentation stage: simulate walking experience along a preset path in the virtual reality building scene, and establish connection with the motion interaction device to synchronously realize dynamic movement of the user's virtual position with the help of the motion interaction device; Static virtual scene presentation stage: presenting a local area of ​​the building in the virtual reality building scene, providing scene layout information with a fixed perspective, and outputting and displaying ERP cognitive test information in the process.

4. The architectural design preference evaluation method based on multimodal data according to claim 1, 2 or 3, characterized in that: The steps of obtaining the user's neurophysiological data, user's emotional data and user's cognitive data in the virtual reality building scene include: Obtain the user's EEG data, ECG data, and eye tracking data respectively to obtain the user's neurophysiological data and user emotional data; The ERP cognitive test information input by the user during the static virtual scene presentation phase is received, and the user cognitive data is obtained by combining the user's response records during the test process.

5. The architectural design preference evaluation method based on multimodal data according to claim 4 is characterized in that: Before the step of integrating the user neurophysiological data, the user emotional data and the user cognitive data to obtain multimodal data, the method further includes: Obtain ERP emotional face recognition information input by users in virtual reality building scenes; The ERP emotional face recognition information is added to the user cognitive data to update the user cognitive data.

6. The architectural design preference evaluation method based on multimodal data according to claim 1, characterized in that: The step of integrating the user neurophysiological data, the user emotional data and the user cognitive data to obtain multimodal data includes: Standardize user neurophysiological data, user emotional data, and user cognitive data; Extract characteristic information contained in the user's neurophysiological data, user's emotional data, and user's cognitive data after standardization; The extracted feature information is fused into multimodal data.

7. The architectural design preference evaluation method based on multimodal data according to claim 1, characterized in that: After the step of integrating the user's neurophysiological data, the user's emotional data and the user's cognitive data to obtain multimodal data, the method further includes: The integrated multimodal data are cross-validated using the preset cross-validation algorithm.

8. The architectural design preference evaluation method based on multimodal data according to claim 1, characterized in that: The step of determining the preference evaluation result corresponding to the architectural design data according to the multimodal data comprises: The multimodal data is input into the trained preference assessment model to obtain the preference assessment results output by the preference assessment model.

9. The architectural design preference evaluation method based on multimodal data according to claim 8, characterized in that: The step of determining the preference evaluation result corresponding to the architectural design data according to the multimodal data further includes: Map the physiological indicator data contained in the multimodal data to the corresponding geographical location in the architectural design in the form of a heat map to form an emotional plane map; Alternatively, the physiological index data contained in the multimodal data is fused with the corresponding spatial features or visual setting diagram in the architectural design in the form of a heat map in different spaces to obtain an emotion and feature fusion map; Alternatively, the physiological index data contained in the multimodal data are integrated with the corresponding three-dimensional axis view in the architectural design in the form of a heat map in different spaces to obtain an emotional stress map; The emotion plane map, emotion and feature fusion map and emotion pressure map are output and displayed.

10. An architectural design preference evaluation system based on multimodal data, characterized in that: include: The emotion amplification module is used to present the preset virtual reality game on the human-computer interaction interface according to the execution steps of the preset virtual reality game and the task type of the game in the preset emotion amplifier; A data acquisition module, used to present a pre-built virtual reality building scene based on the building design data on a human-computer interaction interface, and to obtain user neurophysiological data, user emotional data, and user cognitive data in the virtual reality building scene; A data analysis module, used to integrate the user's neurophysiological data, user's emotional data and user's cognitive data to obtain multimodal data; The user's emotional experience and cognitive response are analyzed according to the multimodal data to determine a preference evaluation result corresponding to the architectural design data.

Images