Multi-dimensional user data acquisition and analysis method based on virtual reality scene

Abstract

The invention discloses a multi-dimensional user data acquisition and analysis method based on a virtual reality scene, and particularly relates to the field of user data acquisition and analysis, which is characterized in that a monitoring area is delimited by analyzing virtual crowd distribution in a VR scene, a virtual sentry is set, an intelligent monitoring scene is simulated, and the user data acquisition and analysis efficiency is improved. A virtual sentry can monitor a specific target according to a preset task, and at the same time, label and update a user portrait of a virtual crowd to refine user feature description, realize integration of real scene materials in a VR environment and construction of a dynamic virtual crowd, and can provide a user experience for the virtual sentry in a place or scene requiring behavior monitoring, target identification and user feature analysis. And powerful virtual data support is provided for related identification AI model training.

Description

Technical Field

[0001] The present invention relates to the technical field of user data collection and analysis, and more specifically, to a multi-dimensional user data collection and analysis method based on a virtual reality scene. Background Art

[0002] The combination of virtual reality (VR) technology and recognition-based artificial intelligence (AI) is becoming an important direction for cutting-edge research and practice. User recognition-based AI learns and processes a large amount of user data to perform target recognition (such as actions, movement trajectories, user portraits, etc.), while VR technology can create a highly simulated virtual environment, provide unique training and testing scenarios, and make up for the shortcomings of traditional data collection methods. Therefore, how to build a highly simulated VR scene and monitor simulated dynamic crowd behavior as virtual training data for user recognition-based AI has become the key.

[0003] In order to solve the above problems, a technical solution is provided. By analyzing the distribution of virtual crowds in VR scenes, monitoring areas are defined, and virtual sentinels are set up to simulate intelligent monitoring scenarios. Virtual sentinels can monitor specific targets according to preset tasks, and at the same time, the user portraits of virtual crowds are annotated and updated to refine user feature descriptions. Integrating real scene materials in VR environments and building dynamic virtual crowds can provide strong technical support for behavior monitoring, target identification, and user feature analysis. Summary of the invention

[0004] In order to overcome the above-mentioned defects of the prior art, an embodiment of the present invention provides a multi-dimensional user data collection and analysis method based on a virtual reality scene to solve the problems raised in the above-mentioned background technology.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] S1: Collect real-life scene materials, combine 3D modeling technology to decompose the real-life scene elements and integrate them into the VR scene;

[0007] S2: Extract pedestrian trajectory, movement and behavior pattern data from real scene materials, and simulate 3D virtual crowds in VR scenes based on the extracted data;

[0008] S3: Deconstruct the integrated VR scene, define the monitoring area based on the distribution of virtual people, and set virtual sentinels in the monitoring area;

[0009] S4: Based on the preset target user type to be monitored, all virtual sentinel viewpoints are tasked and user portraits of the virtual crowd are labeled;

[0010] S5: Perform line of sight analysis on the virtual sentinel’s viewpoint and perform secondary annotation on the user portraits of the concerned individuals in the virtual crowd;

[0011] S6: Extract the user features of all labeled virtual individuals and establish a virtual data sample library for target user identification.

[0012] In a preferred embodiment, in S1, collecting real scene materials, combining 3D modeling technology to decompose the real scene into elements and integrate them into the VR scene specifically includes:

[0013] Use a high-definition camera or drone to shoot the designated material collection site from multiple angles, including horizontal and vertical perspectives and local close-ups, to generate complete image data;

[0014] Use laser scanning equipment to comprehensively scan the collection site, collect depth data and generate accurate point clouds, import the point cloud data into the modeling software to convert it into a preliminary geometric model, fuse the photogrammetric image data and point cloud to generate a high-precision 3D model;

[0015] Perform denoising, color correction and texture stitching on image data, and crop images outside the area where material is collected to reduce data redundancy according to scene requirements;

[0016] The key elements of the main components of the 3D model are marked and separated into independent geometric modules. The number of faces of the model is reduced through polygon reduction technology to improve rendering performance while retaining the key details of the elements. The disassembled geometric modules are imported and rearranged according to the spatial layout of the actual scene of the venue to form a VR scene based on real-life materials.

[0017] In a preferred embodiment, in S2, extracting pedestrian trajectory, movement and behavior pattern data from real scene materials, and simulating 3D virtual crowds in the VR scene based on the extracted data specifically includes:

[0018] Select a real-world area with dense pedestrian flow that meets the requirements of a preset specific scenario, and capture and record pedestrian behavior patterns and movement trajectories;

[0019] The trajectory curve of pedestrians is extracted through motion detection algorithm and stored as time series data. The pedestrian movements in the video are analyzed, the movements are decomposed into joint angle changes, and the pedestrian behavior patterns are classified.

[0020] Based on the classification of behavior patterns, behavior triggering rules are established to simulate the autonomous behavior of virtual people;

[0021] The pedestrian trajectory curve time series data is converted into a mathematical model to describe the movement patterns of pedestrians in three-dimensional space. Using 3D modeling and animation technology, a 3D virtual crowd is generated based on the pedestrian behavior pattern and trajectory model, and the virtual crowd is integrated into the VR scene.

[0022] In a preferred embodiment, in S3, the integrated VR scene is deconstructed, and a monitoring area is defined based on the distribution of virtual people. Setting virtual sentinels in the monitoring area specifically includes:

[0023] Functionally deconstruct the VR scene and divide it into multiple custom logic layers, including the geometric layer, dynamic layer, and interactive layer. The geometric layer is used to define the spatial boundaries of the monitoring area within the scene, and the dynamic layer is used to track the distribution and movement trajectory of the virtual crowd.

[0024] According to the dynamic layer, the distribution density heat map of virtual crowds in various areas of the VR scene is counted in real time, and the areas with frequent crowd flow are screened and marked as virtual monitoring areas, and the spatial boundaries are delineated in the geometric layer;

[0025] Predefine static viewpoints and set static viewpoints at the spatial boundary of the virtual monitoring area according to the size of the virtual monitoring area. The specific method is as follows:

[0026] Get the maximum observable angle and the farthest observation distance set by the static viewpoint;

[0027] Determine whether the diameter of the circumscribed circle of the virtual monitoring area space boundary is greater than the farthest observation distance preset by the static viewpoint;

[0028] If yes, take the center of the circumscribed circle of the virtual monitoring area space boundary as the midpoint, divide the virtual monitoring area into a cross according to the horizontal axis and the vertical axis of the VR scene, and each small monitoring area after division is used as a virtual monitoring sub-area, and the intersection of the virtual monitoring sub-areas is the new space boundary;

[0029] Iteratively segment the virtual monitoring sub-areas until the diameter of the circumscribed circles of the spatial boundaries of all virtual monitoring sub-areas is greater than the preset farthest observation distance of the static viewpoint, and then the segmentation is terminated;

[0030] According to the maximum observable angle set by the static viewpoint, equidistant monitoring positions are set on the circumscribed circle of the virtual monitoring area space boundary, and these static viewpoints are marked as virtual sentinel viewpoints on the interaction layer.

[0031] In a preferred embodiment, in S4, based on the preset target user type to be monitored, tasks are issued to all virtual sentinel viewpoints, and user portraits of the virtual crowd are annotated, specifically including:

[0032] According to the target user type to be monitored, set scene tasks for the virtual sentinel viewpoints. Each virtual sentinel viewpoint corresponds to a VR device, which is then placed for test users to use.

[0033] The test user monitors the virtual crowd in the virtual monitoring area based on the scene tasks corresponding to the VR device, and annotates the user portrait at the same time.

[0034] In a preferred embodiment, in S5, performing line of sight analysis on the viewpoint of the virtual sentinel and secondary labeling of the user portrait of the concerned individual in the virtual crowd specifically includes:

[0035] Acquire and analyze the time-series sight data of the virtual sentinel viewpoints within the set monitoring time window in the VR scene. The time-series sight data includes the sight angle of each virtual sentinel viewpoint, the individual target of the virtual crowd that the sight is focused on, and the duration of the sight focusing on the individual target;

[0036] Filter out the data where the sight lines are not focused on individuals in the virtual crowd, and perform time series coincidence analysis on the remaining time series sight line data;

[0037] The maximum number of viewpoints and the gaze time of each virtual individual in the virtual crowd when being simultaneously gazed at by the virtual sentinels are counted. When there is a virtual individual that is simultaneously gazed at by more than half of the virtual sentinels and the average gaze time reaches the set duration, the user portrait of the virtual individual is re-annotated.

[0038] In a preferred embodiment, in S6, extracting user features of all labeled virtual individuals and establishing a virtual data sample library for target user identification includes:

[0039] Obtain virtual user portrait data, behavior patterns and movement trajectories of the labeled individuals in the VR scene, extract user portrait features, movement trajectory features and behavior features for feature integration;

[0040] The integrated features are stored in the training data sample library, the feature data corresponding to the virtual individuals whose user portraits have been annotated only once are marked as training set samples, and the feature data corresponding to the virtual individuals whose user portraits have been annotated once and twice are marked as validation set samples;

[0041] The training set samples and the validation set samples are used as a virtual data sample library for the target user recognition model.

[0042] The technical effects and advantages of the multi-dimensional user data collection and analysis method based on virtual reality scene of the present invention are as follows:

[0043] By combining real-life scene materials with 3D modeling technology, the real environment can be highly restored and integrated into the VR scene to achieve dynamic crowd simulation. Modern computer vision technology can extract pedestrian trajectories, movements, and behavior patterns from real scenes to provide real data support for virtual crowds. On this basis, the monitoring area can be delineated by analyzing the distribution of the crowd, and virtual sentinels can be set up to simulate intelligent monitoring scenarios. Virtual sentinels can monitor specific targets according to preset tasks, and at the same time annotate and update the user portraits of the virtual crowd to refine the user feature description. By extracting the user features of the annotated individuals, a high-quality target user recognition training data set can be generated to provide support for the training and verification of machine learning models. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 This is a schematic diagram of a multi-dimensional user data collection and analysis method based on a virtual reality scene in the present invention. DETAILED DESCRIPTION

[0045] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0046] Example 1

[0047] Figure 1 The present invention provides a multi-dimensional user data collection and analysis method based on a virtual reality scene, which comprises the following steps:

[0048] S1: Collect real-life scene materials, combine 3D modeling technology to decompose the real-life scene elements and integrate them into the VR scene;

[0049] S2: Extract pedestrian trajectory, movement and behavior pattern data from real scene materials, and simulate 3D virtual crowds in VR scenes based on the extracted data;

[0050] S3: Deconstruct the integrated VR scene, define the monitoring area based on the distribution of virtual people, and set virtual sentinels in the monitoring area;

[0051] S4: Based on the preset target user type to be monitored, all virtual sentinel viewpoints are tasked and user portraits of the virtual crowd are labeled;

[0052] S5: Perform line of sight analysis on the virtual sentinel’s viewpoint and perform secondary annotation on the user portraits of the concerned individuals in the virtual crowd;

[0053] S6: Extract the user features of all labeled virtual individuals and establish a virtual data sample library for target user identification.

[0054] In S1, real-life scene materials are collected, and 3D modeling technology is used to disassemble the real-life scene elements and integrate them into the VR scene. High-precision restoration of real-life scenes is an important part of VR technology application. By collecting high-resolution images and video materials and combining them with 3D modeling tools (such as Blender, Unity or Unreal Engine), complex real-life scenes can be transformed into highly realistic VR environments. The elements included in the scene (such as buildings, roads, and dynamic crowds) need to be disassembled and integrated in layers to ensure the authenticity and interactivity of the VR scene.

[0055] At material collection sites such as city squares, stations, shopping malls, etc., use high-definition cameras with a resolution of no less than 4K and drones equipped with a stabilization system to shoot the sites in all directions and from multiple angles. This includes horizontal perspective shooting, circling the site along the horizontal plane to obtain a panoramic image; vertical perspective shooting, using drones to obtain the overall layout of the site from a high-altitude bird's-eye view; and local close-ups of key areas to capture high-precision information such as architectural details and textures to generate complete image data.

[0056] Use high-precision laser scanners (such as LiDAR) to conduct a comprehensive three-dimensional scan of the collection site. Through multiple measurements, the depth data of the site is obtained and accurate point cloud data is generated. During the scanning process, measurements are taken from different positions and angles to ensure that the three-dimensional structure of the site is covered without blind spots. The point cloud data is imported into the modeling software and converted into a preliminary geometric model. The image data and point cloud of the photogrammetry are fused to generate a high-precision three-dimensional model, in which the point cloud data provides accurate spatial position and builds the basic framework of the model.

[0057] In the modeling software, the imported image data is denoised to eliminate noise and impurities generated during the shooting process. Color correction is performed to unify the hue and brightness under different lighting conditions to ensure the consistency of the model texture. The texture of the model is stitched to eliminate splicing marks and ensure the continuity and integrity of the texture. According to the scene requirements, the image and model parts outside the material collection area are cropped to reduce data redundancy and improve the efficiency of model processing and rendering.

[0058] Label the key elements of the main components of the 3D model, separate the key elements into independent geometric modules, and use polygon reduction technology to reduce the number of faces of the model to improve rendering performance while retaining the key details of the elements;

[0059] Import the processed geometric modules into VR development engines (such as Unity, Unreal Engine). These modules serve as the basic building blocks of the scene, providing a real physical basis for the virtual environment. Rearrange the spatial layout of the real scene to form a VR scene based on real-life materials.

[0060] In S2, pedestrian trajectory, movement and behavior pattern data are extracted from real-life scene materials, and 3D virtual crowds in VR scenes are simulated based on the extracted data. Modern computer vision technology can automatically identify and record pedestrian movement trajectories, behavior characteristics and interaction patterns. These data can be further used to construct and simulate dynamic crowds in VR scenes.

[0061] Select a real-world area with dense pedestrian flow that meets the requirements of a preset specific scenario, and capture and record pedestrian behavior patterns and movement trajectories;

[0062] Use motion detection algorithms (such as optical flow method) to extract pedestrian movement trajectories from videos and represent them as time series data: r(t) = {x(t), y(t), z(t)}, where x(t), y(t), z(t) are the horizontal, vertical and height coordinates at time t, respectively, and r(t) is the time series movement trajectory. The trajectory curve is smoothed by Kalman filtering to correct trajectory fluctuations caused by environmental interference.

[0063] Analyze the pedestrian movements in the video, decompose the movements into joint angle changes, and classify the pedestrian behavior patterns. The joint angle calculation method is: θ ab is the angle between joints a and b, P a and P b are the position vectors of joints a and b respectively.

[0064] Based on the classification of behavior patterns, behavior triggering rules are established to simulate the autonomous behavior of virtual people. The behavior pattern classification can be based on trajectory and action characteristics, and the clustering algorithm is used to classify the behavior patterns. The following are several typical simple behavior patterns:

[0065] Free walking: The joint angles change regularly and the trajectory is relatively straight.

[0066] Stay and observe: The joint angle does not move, the movement speed is close to zero, and the position changes frequently.

[0067] Aggregation behavior: multiple trajectories converge in the same area.

[0068] Rapid movement: abnormally increased speed and frequent changes in joint angles, usually corresponding to avoidance or emergency behavior.

[0069] The pedestrian trajectory curve time series data is converted into a continuous function R(t), R(t)=r0+∫v(τ)dτ, which is used to describe the movement law of pedestrians in three-dimensional space. r0 and v(τ) are the initial position of the pedestrian and the instantaneous rate and direction when the time integral variable τ is used, respectively. Using 3D modeling and animation technology, a 3D virtual crowd is generated based on the pedestrian's behavior pattern and trajectory model. The virtual crowd is imported into a VR development engine (such as UnrealEngi ne), the initial position of the characters is assigned according to the spatial layout of the real place, and the dynamic path is set according to the trajectory mathematical model to simulate the flow trend of the crowd.

[0070] In S3, the integrated VR scene is deconstructed, and the monitoring area is delineated based on the distribution of virtual people. Virtual sentinels are set up in the monitoring area. As "observers" of intelligent monitoring, virtual sentinels can monitor specific groups of people according to the set tasks and viewpoints, simulating the role of the monitoring system in real scenes.

[0071] Functionally deconstruct the VR scene and divide it into multiple custom logic layers, including the geometric layer, dynamic layer, and interactive layer. The geometric layer is used to define the spatial boundaries of the monitoring area within the scene, and the dynamic layer is used to track the distribution and movement trajectory of the virtual crowd.

[0072] According to the dynamic layer, the distribution density heat map of virtual crowds in various areas of the VR scene is counted in real time, and the heat map is generated according to the following formula: ρ is the regional population density, N and A represent the virtual number of people in the region and the regional area defined by the geometric layer, respectively.

[0073] Screen areas with frequent crowd flow, mark them as virtual monitoring areas, and define spatial boundaries in the geometric layer;

[0074] Predefine static viewpoints and set static viewpoints at the spatial boundary of the virtual monitoring area according to the size of the virtual monitoring area. The specific method is as follows:

[0075] Get the maximum observable angle and the farthest observation distance set by the static viewpoint;

[0076] Determine whether the diameter of the circumscribed circle of the virtual monitoring area space boundary is greater than the farthest observation distance preset by the static viewpoint;

[0077] If so, take the center of the circumscribed circle of the virtual monitoring area space boundary as the midpoint, and divide the virtual monitoring area into a cross according to the horizontal and vertical axis directions of the VR scene. Each small monitoring area after division is used as a virtual monitoring sub-area, and the junction of the virtual monitoring sub-areas is the new spatial boundary.

[0078] The virtual monitoring sub-areas are segmented iteratively until the diameter of the circumscribed circle of the spatial boundaries of all virtual monitoring sub-areas is greater than the preset farthest observation distance of the static viewpoint, and then the segmentation is terminated.

[0079] According to the maximum observable angle set by the static viewpoint, equidistant monitoring positions are set on the circumscribed circle of the virtual monitoring area space boundary, and these static viewpoints are marked as virtual sentinel viewpoints on the interaction layer.

[0080] Based on the fact that static viewpoints are distributed at equidistant angles on the circumscribed circle of the spatial boundary, assuming that the maximum observable angle of the static viewpoint is set to 120 degrees based on the normal human realization range, then the static viewpoints required to fully cover the virtual monitoring area are at least 360 degrees / 120 degrees = 3, which also means that the static viewpoints should be set at three equally divided points on the circumscribed circle of the spatial boundary.

[0081] In S4, based on the preset target user type to be monitored, tasks are issued to all virtual sentinel viewpoints, and user portraits of the virtual crowd are labeled at the same time.

[0082] According to the target user type to be monitored, scene tasks are set for the virtual sentinel viewpoint. Each virtual sentinel viewpoint corresponds to a VR device, and the VR device is provided to the test user for use. Assuming that the target user type to be monitored is an elderly person with suspicious behavior, the test user will continuously monitor and identify virtual individuals that meet the characteristics in the virtual crowd based on the monitoring area within his or her own perspective, and then dynamically mark them.

[0083] The "test user" here can be a real person or other AI models that have been trained for special personnel identification. The test user monitors the virtual crowd in the virtual monitoring area based on the scene tasks corresponding to the VR device, and annotates the user portrait at the same time. The user portrait mainly includes the basic feature data of the virtual individual in the VR scene, such as age, appearance proportion, body shape, gait and other basic data.

[0084] In S5, the line of sight analysis is performed on the virtual sentinel’s viewpoint, and the user portraits of the concerned individuals in the virtual crowd are secondary labeled.

[0085] Acquire and analyze the time-series sight data of the virtual sentinel viewpoints within the set monitoring time window in the VR scene. The time-series sight data includes the sight angle of each virtual sentinel viewpoint, the individual target of the virtual crowd that the sight is focused on, and the duration of the sight focusing on the individual target;

[0086] The data whose sight lines are not focused on individuals in the virtual crowd are filtered out, and the remaining time-series sight line data are subjected to time-series overlap analysis (the data of all virtual sentinel viewpoints within the set monitoring time window are combined with the monitored virtual individuals). The maximum number of viewpoints and the gaze time when each virtual individual in the virtual crowd is simultaneously gazed at by the virtual sentinel viewpoints are counted. When there is a virtual individual that is simultaneously gazed at by more than half of the virtual sentinel viewpoints and the average gaze time reaches the set duration, the user portrait of the virtual individual is re-annotated.

[0087] Count the number of virtual target individual IDs numbered i that the virtual sentinel simultaneously watches at each time point in the set monitoring time window i The number of virtual sentinel viewpoints N sentinel(k,IDi) , its calculation expression is: Where M is the number of all virtual sentinel viewpoints, k is the number of the time point in the set monitoring time window, and δ ij ( k ) =1 indicates that the jth sentinel is watching the target individual ID at the kth time point i , otherwise δ ij ( k ) =0.

[0088] When the maximum number of virtual sentinel viewpoints N max When the number of all virtual sentinel viewpoints reaches half (i.e. N max ≥[M / 2]), which meets the attention requirement of secondary annotation.

[0089] In S6, the user features of all labeled virtual individuals are extracted to establish a virtual data sample library for target user identification. By extracting features of the virtual population of interest in the VR scene, structured data samples can be generated. Combined with the labeled user portrait information, these data samples can be used for training and verification of machine learning models to further improve the recognition accuracy of target users.

[0090] Obtain virtual user portrait data, behavior patterns and movement trajectories of the labeled individuals in the VR scene, extract user portrait features, movement trajectory features and behavior features for feature integration;

[0091] All labeled virtual individual data are obtained from the user portrait system of the VR scene, which is divided into primary and secondary annotation. The primary annotation is for virtual individuals with obvious features, which are used for targeted training of user recognition models. The secondary annotation is for individuals with less obvious features. If they are not labeled after being reviewed by multiple sentinels, they can be judged as imperfect or suspicious data. This type of data can test the robustness and generalization ability of the model to avoid overfitting of the model to obvious features.

[0092] The integrated features are stored in the training data sample library, the feature data corresponding to the virtual individuals whose user portraits have been annotated only once are marked as training set samples, and the feature data corresponding to the virtual individuals whose user portraits have been annotated once and twice are marked as validation set samples.

[0093] The training set samples and the validation set samples are used as the virtual data sample library of the target user identification model. Through feature integration and optimization, the data is divided into training set and validation set to ensure that the sample library covers the diversity and representativeness of the target users. Finally, the sample library is used to train and verify the target user identification model, providing strong technical support for user management and behavior analysis in a virtual environment.

[0094] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters and thresholds in the formula are set by technicians in this field according to actual conditions.

[0095] The above embodiments can be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0096] Those of ordinary skill in the art will appreciate that the modules and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0097] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and modules described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0098] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0099] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, and may be located in one place or distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0100] In addition, each functional module in each embodiment of the present application may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0101] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0102] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0103] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A multi-dimensional user data collection and analysis method based on virtual reality scene, characterized in that: The steps include: S1: Collect real-life scene materials, combine 3D modeling technology to decompose the real-life scene elements and integrate them into the VR scene; S2: Extract pedestrian trajectory, movement and behavior pattern data from real scene materials, and simulate 3D virtual crowds in VR scenes based on the extracted data; S3: Deconstruct the integrated VR scene, define the monitoring area based on the distribution of virtual people, and set virtual sentinels in the monitoring area; S4: Based on the preset target user type to be monitored, tasks are issued to all virtual sentinel viewpoints, and user portraits of the virtual crowd are annotated; S5: Perform line of sight analysis on the virtual sentinel’s viewpoint and perform secondary annotation on the user portraits of the concerned individuals in the virtual crowd; S6: Extract the user features of all labeled virtual individuals and establish a virtual data sample library for target user identification.

2. The multi-dimensional user data collection and analysis method based on virtual reality scene according to claim 1 is characterized in that: In S1, real-life scene materials are collected, and 3D modeling technology is used to decompose the real-life scene into elements and integrate them into the VR scene. Specifically, the following steps are taken: Use a high-definition camera or drone to shoot the designated material collection site from multiple angles, including horizontal and vertical perspectives and local close-ups, to generate complete image data; Use laser scanning equipment to comprehensively scan the collection site, collect depth data and generate accurate point clouds, import the point cloud data into the modeling software to convert it into a preliminary geometric model, fuse the photogrammetric image data and point cloud to generate a high-precision 3D model; Perform denoising, color correction and texture stitching on image data, and crop images outside the area where material is collected to reduce data redundancy according to scene requirements; Label the key elements of the main components of the 3D model, separate the key elements into independent geometric modules, and use polygon reduction technology to reduce the number of faces of the model to improve rendering performance while retaining the key details of the elements; Import the disassembled geometric modules and rearrange them according to the spatial layout of the real scene of the venue to form a VR scene based on real-life materials.

3. The multi-dimensional user data collection and analysis method based on virtual reality scene according to claim 2 is characterized in that: In S2, pedestrian trajectory, movement and behavior pattern data are extracted from real scene materials, and 3D virtual crowds in VR scenes are simulated based on the extracted data, including: Select a real-world area with dense pedestrian flow that meets the requirements of a preset specific scenario, and capture and record pedestrian behavior patterns and movement trajectories; The trajectory curve of pedestrians is extracted through motion detection algorithm and stored as time series data. The pedestrian movements in the video are analyzed, the movements are decomposed into joint angle changes, and the pedestrian behavior patterns are classified. Based on the classification of behavior patterns, behavior triggering rules are established to simulate the autonomous behavior of virtual people; The pedestrian trajectory curve time series data is converted into a mathematical model to describe the movement patterns of pedestrians in three-dimensional space. Using 3D modeling and animation technology, a 3D virtual crowd is generated based on the pedestrian behavior pattern and trajectory model, and the virtual crowd is integrated into the VR scene.

4. The multi-dimensional user data collection and analysis method based on virtual reality scene according to claim 3 is characterized in that: In S3, the integrated VR scene is deconstructed, and the monitoring area is delineated based on the distribution of virtual people. The virtual sentinels are set in the monitoring area, including: Functionally deconstruct the VR scene and divide it into multiple custom logic layers, including the geometric layer, dynamic layer, and interactive layer. The geometric layer is used to define the spatial boundaries of the monitoring area within the scene, and the dynamic layer is used to track the distribution and movement trajectory of the virtual crowd. According to the dynamic layer, the distribution density heat map of virtual crowds in various areas of the VR scene is counted in real time, and the areas with frequent crowd flow are screened and marked as virtual monitoring areas, and the spatial boundaries are delineated in the geometric layer; Predefine static viewpoints and set static viewpoints at the spatial boundary of the virtual monitoring area according to the size of the virtual monitoring area. The specific method is as follows: Get the maximum observable angle and the farthest observation distance set by the static viewpoint; Determine whether the diameter of the circumscribed circle of the virtual monitoring area space boundary is greater than the farthest observation distance preset by the static viewpoint; If yes, take the center of the circumscribed circle of the virtual monitoring area space boundary as the midpoint, divide the virtual monitoring area into a cross according to the horizontal axis and the vertical axis of the VR scene, and each small monitoring area after division is used as a virtual monitoring sub-area, and the intersection of the virtual monitoring sub-areas is the new space boundary; Iteratively segment the virtual monitoring sub-areas until the diameter of the circumscribed circles of the spatial boundaries of all virtual monitoring sub-areas is greater than the preset farthest observation distance of the static viewpoint, and then the segmentation is terminated; According to the maximum observable angle set by the static viewpoint, equidistant monitoring positions are set on the circumscribed circle of the virtual monitoring area space boundary, and these static viewpoints are marked as virtual sentinel viewpoints on the interaction layer.

5. The multi-dimensional user data collection and analysis method based on virtual reality scene according to claim 4 is characterized in that: In S4, based on the preset target user type to be monitored, tasks are issued to all virtual sentinel viewpoints, and user portraits of the virtual crowd are annotated, including: According to the target user type to be monitored, set scene tasks for the virtual sentinel viewpoints. Each virtual sentinel viewpoint corresponds to a VR device, which is then placed for test users to use. The test user monitors the virtual crowd in the virtual monitoring area based on the scene tasks corresponding to the VR device, and annotates the user portrait at the same time.

6. The multi-dimensional user data collection and analysis method based on virtual reality scene according to claim 5 is characterized in that: In S5, the line of sight analysis is performed on the virtual sentinel’s viewpoint, and the user portrait of the concerned individual in the virtual crowd is secondary labeled, which specifically includes: Acquire and analyze the time-series sight data of the virtual sentinel viewpoints within the set monitoring time window in the VR scene. The time-series sight data includes the sight angle of each virtual sentinel viewpoint, the individual target of the virtual crowd that the sight is focused on, and the duration of the sight focusing on the individual target; Filter out the data where the sight lines are not focused on individuals in the virtual crowd, and perform time series coincidence analysis on the remaining time series sight line data; The maximum number of viewpoints and the gaze time of each virtual individual in the virtual crowd when being simultaneously gazed at by the virtual sentinels are counted. When there is a virtual individual that is simultaneously gazed at by more than half of the virtual sentinels and the average gaze time reaches the set duration, the user portrait of the virtual individual is re-annotated.

7. The multi-dimensional user data collection and analysis method based on virtual reality scene according to claim 6 is characterized in that: In S6, the user features of all labeled virtual individuals are extracted to establish a virtual data sample library for target user identification, including: Obtain virtual user portrait data, behavior patterns and movement trajectories of the labeled individuals in the VR scene, extract user portrait features, movement trajectory features and behavior features for feature integration; The integrated features are stored in the training data sample library, the feature data corresponding to the virtual individuals whose user portraits have been annotated only once are marked as training set samples, and the feature data corresponding to the virtual individuals whose user portraits have been annotated once and twice are marked as validation set samples; The training set samples and the validation set samples are used as a virtual data sample library for the target user recognition model.

Images