An experimental method based on crowd disaster video data and virtual scene reconstruction

Abstract

The application discloses an experimental method based on crowd disaster video data and virtual scene reconstruction, comprising: frame-by-frame analysis of obtained disaster monitoring video and division into close-range and long-range areas, extraction of two-dimensional coordinate positions of identifiable pedestrian individuals by target detection technology, obtaining of overall density distribution data by crowd density estimation technology, and frame-by-frame extraction of pixel motion speed and direction by a particle image velocimetry tool. A three-dimensional virtual scene model corresponding to a real accident site is established, crowd position and motion information obtained by processing is mapped into three-dimensional data matched with the virtual scene by a coordinate projection tool, a unique identity is given and cross-frame association and trajectory optimization processing are carried out according to time sequence and spatial proximity, a three-dimensional pedestrian continuous trajectory file for virtual reality simulation is generated, dynamic position and posture changes of virtual pedestrian entities are driven, and credible reproduction of a crowd disaster situation and human-in-the-loop experimental evaluation are realized.

Description

Technical Field

[0001] This invention belongs to the fields of virtual reality technology and video image analysis technology, specifically relating to an experimental method for reconstructing virtual scenes based on crowd disaster video data. Background Technology

[0002] With the acceleration of urbanization, the safety management of densely populated areas has become a critical issue that urgently needs to be addressed in today's society. Therefore, in order to effectively prevent and respond to similar disasters, it is imperative to use technological means to realistically reproduce and deeply analyze the disaster occurrence process, thereby revealing its causal mechanisms, identifying key risk factors, and providing scientific basis and technical support for population organization and management, emergency evacuation decisions, and disaster emergency response.

[0003] Currently, research and response technologies for population-related disasters mainly focus on two major areas: one is population dynamic simulation technology based on mathematical models, and the other is static scene reconstruction technology based on 3D modeling.

[0004] Crowd dynamics simulation technology based on mathematical models typically reproduces the movement process of crowds through mathematical abstraction and simulation. A typical example is the Hughes macroscopic simulation model, which simplifies the crowd movement scene into a surface region and a set of points, constructing a system of partial differential equations through assumptions such as pedestrian flow continuity and potential energy field assumptions, thereby simulating the dynamic behavior of crowds. This type of model can provide a certain degree of theoretical prediction for specific scenarios, playing a particularly important role in understanding the mechanisms of certain disaster scenarios. However, these models are often based on a series of idealized assumptions, such as the assumption of regularity in crowd behavior and uniform flow, which leads to a certain gap between the simulation results and actual disaster scenarios. Due to the lack of data support from actual disaster events, the realism and reliability of the simulation results are often limited, making it difficult to accurately reflect complex and unpredictable disaster situations.

[0005] On the other hand, with the rapid development of 3D modeling and virtual reality (VR) technologies, disaster scene environment reconstruction technology based on 3D modeling is gradually becoming a reality. By leveraging mature 3D modeling technology, the environment in which a disaster occurs can be quickly reconstructed at a lower cost. Combined with a microscopic pedestrian flow simulation model, this can be immersively reproduced through a virtual reality simulation engine. In existing technology, CN117971036A discloses a virtual reality-based pre-simulation method and system, including: constructing a virtual scene based on the actual scene; obtaining the specific coordinates of personnel during their actions in the actual scene to obtain their 3D trajectory and visual information, and fusing the 3D trajectory and visual information to obtain multi-dimensional data on personnel behavior; using virtual humans to reproduce the personnel's movement trajectory in the virtual scene using the multi-dimensional data on personnel behavior; and simultaneously displaying the personnel's movement trajectory and visual information when the virtual human arrives at the event location. This method provides a more visually realistic reconstruction of the disaster environment, making the scene of crowd movement more intuitive and concrete. However, existing disaster scene reproduction based on 3D modeling still relies on hypothetical scenarios and simulation parameters, lacking the driving force of real disaster video data. Therefore, although the scenarios and environments are relatively realistic, there are still deviations from the actual situation in terms of pedestrian movement behavior, speed changes, and behavioral decisions.

[0006] Current disaster reconstruction technologies suffer from several significant limitations, one of the biggest challenges being the lack of technical means to realistically reproduce the movement and state characteristics of crowds in disaster scenarios. Traditional techniques based on hypothetical models and static 3D scene reconstruction, while providing a visual representation of disaster situations, cannot realistically simulate the dynamic behavior of crowds in actual scenarios and their interaction with the environment. Therefore, traditional methods often fail to fully reflect the true movement patterns and behavioral responses of crowds during disaster events, leading to uncertainties in disaster prevention, emergency evacuation strategies, and behavioral response predictions.

[0007] Therefore, it is necessary to propose a comprehensive technical method that integrates disaster video analysis, real-time extraction of crowd trajectories, 3D scene reconstruction, and virtual reality immersive simulation. Summary of the Invention

[0008] The purpose of this invention is to overcome the shortcomings of the existing technology and provide an experimental method for reconstructing disaster video data and virtual scenes based on crowds.

[0009] The objective of this invention can be achieved through the following technical solutions:

[0010] This invention provides an experimental method for reconstructing disaster scenes based on crowd disaster video data, comprising the following steps:

[0011] Acquire video data of crowd disasters, analyze the acquired video data frame by frame, and divide each frame into near-field and far-field areas;

[0012] For the foreground area, a target detection algorithm is used to extract the two-dimensional image coordinates of identifiable individuals to obtain a coordinate dataset;

[0013] For distant areas, a temporal two-dimensional density matrix is ​​obtained using crowd density estimation techniques;

[0014] The video data of the disaster crowd is input into the particle image velocimetry processing tool, and the velocity vector of pixel motion in the video is extracted frame by frame based on pixel unit to obtain two-dimensional velocity information data.

[0015] A three-dimensional virtual scene model corresponding to the real accident site was established based on the environmental map, image features and geographic information of the disaster video data.

[0016] The coordinate dataset, temporal two-dimensional density matrix, and two-dimensional velocity information data are mapped into three-dimensional world coordinates and three-dimensional velocity vectors that match the three-dimensional virtual scene model through coordinate projection tools, and then matched and fused based on the spatial position correspondence to form multi-frame three-dimensional crowd state data.

[0017] A unique identifier is assigned to each frame of 3D crowd status data, and cross-frame association and trajectory optimization are performed according to the correspondence between time series and spatial proximity to generate a 3D pedestrian continuous trajectory data file.

[0018] The three-dimensional virtual scene model is loaded into the immersive virtual reality simulation platform, and the generated three-dimensional pedestrian continuous trajectory data file is called to drive the dynamic position and posture changes of the virtual pedestrian entity, so as to realize the dynamic reproduction of the crowd disaster situation and the evaluation of human loop experiment.

[0019] Furthermore, the process of performing frame-by-frame analysis on the acquired disaster videos of crowds, dividing each frame into near-field and far-field regions, specifically includes:

[0020] For each frame of the video, head or human body feature detection technology is used to determine the effective range in the near-field area that can be stably identified and tracked, and the effective range is used as the boundary between the near and far scenes; the head or human body feature detection technology is a target detection algorithm based on deep learning, which is used to locate the feature points of individual pedestrians in the two-dimensional image pixel coordinate system and output the near-field individual position information for tracking.

[0021] Based on the aforementioned dividing line, the remaining image after removing the foreground area is determined as the background area.

[0022] Furthermore, for the near-field region, a target detection algorithm is used to extract the two-dimensional image coordinates of identifiable individuals to obtain a coordinate dataset, specifically including:

[0023] For each frame of the defined close-up area, a deep learning object detection algorithm based on the YOLO series is used to locate the two-dimensional image coordinates of identifiable individuals. This yields the position of the head or other human feature targets of identifiable individuals in the close-up area within the two-dimensional pixel coordinate system of the video frame. The object detection uses the video frame as the basic processing unit, independently performing recognition and localization processing on each frame, and directly outputting the set of two-dimensional pixel coordinates corresponding to the current frame, forming a coordinate dataset containing only the pixel position information of identifiable individuals frame by frame. The identifiable individuals include head-centered targets, human body outline targets, and visual targets of individual pedestrians that are not completely obscured in disaster situations, which can represent the position of a single pedestrian in the video footage.

[0024] Furthermore, for the distant area, the temporal two-dimensional density matrix is ​​obtained using crowd density estimation techniques, specifically including:

[0025] After removing the foreground area, the background area is then... The pixels are segmented into grid cells based on the smallest unit side length. Then, statistical processing is performed on the pixels of each grid cell using region analysis and feature clustering algorithms, including:

[0026] The probability of crowd distribution in each grid cell is calculated using a heatmap probability estimation method, and a temporal two-dimensional density matrix of the distant area is formed. The temporal two-dimensional density matrix includes the probability value of crowd distribution corresponding to each grid cell in each frame. The probability value of crowd distribution is used to represent the relative density of crowd in the grid cell in the corresponding frame. The heatmap probability estimation method is used to convert the pixel statistics within the grid cell into probability values ​​to reflect the spatial distribution characteristics of crowd in the distant area.

[0027] Furthermore, the step of calculating the population distribution probability of each grid cell using the heatmap probability estimation method and forming a temporal two-dimensional density matrix of the distant area specifically includes:

[0028] The grid units after dividing the distant view area are numbered as follows: The corresponding pixel set is ;

[0029] For each grid cell Count the number of pixels in the data that are identified as part of the crowd. ;

[0030] Based on the number of crowd pixels in each grid cell The probability of population distribution in each grid cell is calculated using the following formula:

[0031]

[0032] in, Represents grid cells The probability of population distribution; , These represent the number of rows and columns of the grid cells after the distant view area has been divided;

[0033] The probability of crowd distribution is repeatedly calculated for each frame in the video data of crowd disasters, generating a two-dimensional probability matrix that changes over time. The temporal two-dimensional density matrix constituting the distant view area ,in, Indicates the first Crowd distribution probability data in frames; This indicates the total number of video frames.

[0034] Furthermore, the step of inputting the video data of the disaster involving crowds into a particle image velocimetry processing tool, and extracting the velocity vectors of pixel motion in the video frame by frame based on pixel units to obtain two-dimensional velocity information data, specifically includes:

[0035] The video data of the disaster involving crowds is input into a particle image velocimetry processing tool. For each frame, the particle image velocimetry method is used to calculate the velocity vector of each pixel unit based on the pixel changes between adjacent frames. ,in, , These represent the horizontal velocity component and the vertical velocity component, respectively.

[0036] For each frame of the video, calculate the velocity vector of each pixel, and form a two-dimensional matrix from the velocity vectors of each pixel to obtain two-dimensional velocity information data. .

[0037] Furthermore, the step of establishing a three-dimensional virtual scene model corresponding to the actual accident scene based on the environmental map, image features, and geographic information of the disaster video data specifically includes:

[0038] Based on the environmental maps, image features, and geographic information in the disaster video data, the simulation boundary is determined. Furthermore, a 3D virtual scene model corresponding to the real accident scene was constructed based on virtual scene modeling tools; the 3D virtual scene model includes terrain, buildings, obstacles, and vegetation;

[0039] The camera viewport position and rotation settings in the 3D virtual scene model are kept consistent with the actual camera viewpoint in the crowd disaster video data.

[0040] Furthermore, the coordinate dataset, temporal two-dimensional density matrix, and two-dimensional velocity information data are mapped to three-dimensional world coordinates and three-dimensional velocity vectors that match the three-dimensional virtual scene model using coordinate projection tools, and then matched and fused based on spatial position correspondence to form multi-frame three-dimensional crowd state data, specifically including:

[0041] Coordinate dataset obtained from object detection algorithm in near field area By using coordinate projection tools, each two-dimensional image coordinate position is mapped to the corresponding three-dimensional world coordinates;

[0042] The temporal two-dimensional density matrix obtained from the density estimation algorithm in the distant region By using a 3D coordinate mapping tool, the 2D coordinates of the matrix mesh are mapped to the corresponding 3D scene coordinates, thus obtaining the 3D coordinates of each mesh cell. And based on the actual area occupied Based on the maximum number of people that can be generated per square meter Calculate the number of people generated in each grid cell. The formula is:

[0043]

[0044] in, Represents grid cells The number of people generated within; The maximum number of people that can be accommodated per square meter is set based on the actual physical space. , These represent the number of rows and columns of the grid cells after the distant view area has been divided; Represents grid cells The probability of population distribution; For grid cells The actual land area occupied; This is the magnification factor; Indicates the rounding operation;

[0045] Traverse each grid cell, calculate the number of people generated in each grid cell, and randomly distribute the corresponding 3D coordinates of the crowd within the grid cell. ;

[0046] The three-dimensional coordinates of the foreground area and the background area are merged to form a multi-frame set of three-dimensional coordinates of the crowd. ,in, Indicates the first A set of three-dimensional coordinate vectors of the crowd in a frame; Indicates the total number of video frames;

[0047] Two-dimensional velocity information data obtained through particle image velocimetry The velocity vector in the image is converted into a three-dimensional velocity vector using a three-dimensional coordinate mapping tool. ;

[0048] Matching the set of 3D coordinate vectors of the crowd in the same frame with the 3D velocity vector, for Each coordinate point in any frame By matching the data with a set of 3D velocity vectors in the same frame and assigning a corresponding velocity vector to each coordinate point using a spatial nearest neighbor approach, 3D crowd state data containing velocity information is generated. :

[0049]

[0050] in, Indicates the first The set of three-dimensional velocity vectors of a frame; for It satisfies the following formula:

[0051]

[0052] in, Indicates the first The first frame A three-dimensional velocity vector that can identify an individual; Indicates the first In the frame, the closest velocity vector point is selected. The three-dimensional velocity vector; Indicates the first The first frame j The three-dimensional position coordinates corresponding to each velocity vector point; Indicates the first The first frame The three-dimensional coordinates of an identifiable individual; This represents the Euclidean distance.

[0053] Furthermore, the process of assigning unique identifiers to multi-frame 3D crowd state data and performing cross-frame association and trajectory optimization processing according to the temporal sequence and spatial proximity correspondence to generate a 3D pedestrian continuous trajectory data file specifically includes:

[0054] A collection of multi-frame 3D crowd state data The first frame As the starting frame, obtain the 3D coordinates of all individuals in the starting frame. Assume the starting frame contains... Individuals, creating a collection A set of trajectories R ;

[0055] Read the next frame Data, traversing the trajectory set REach trajectory in the image is based on the three-dimensional coordinates of the previous frame. and velocity vector The formula for predicting the coordinates of the next frame is:

[0056]

[0057] in, Indicates the predicted first The first frame The three-dimensional coordinates of each individual; Indicates the first The first frame The three-dimensional coordinates of each individual; Indicates the first The first frame The three-dimensional velocity vector of each individual; This refers to the inter-frame time interval. Indicates the first The total number of individuals in the frame, the total number of individuals being generated by the number of people. Sure;

[0058] For the first frame The three-dimensional coordinates in the data are used to find the distance to the predicted position for each trajectory. The nearest 3D coordinate point is selected as the matching object, and the following matching conditions are met:

[0059]

[0060]

[0061] in, Indicates the Euclidean distance threshold; Indicates the threshold for the difference in velocity direction;

[0062] Add the successfully matched coordinates to the corresponding trajectory. If the first... t There is a set of unmatched coordinate points in the frame. The quantity is Then a new trajectory is created and added to the trajectory set. R Update the total number of tracks:

[0063]

[0064] Repeat the above steps until all frames of 3D crowd state data have been processed. Data from all frames in the dataset is used to generate a complete trajectory set. R ;

[0065] For the set of trajectories RFor each trajectory, a cubic spline interpolation algorithm is used for curve fitting and smoothing. The smoothed trajectory is then split into frames, and timestamps and unique pedestrian ID information are added to generate a three-dimensional continuous pedestrian trajectory data file that can be called by a virtual reality simulation platform.

[0066] Furthermore, the process of loading the three-dimensional virtual scene model into the immersive virtual reality simulation platform and calling the generated three-dimensional pedestrian continuous trajectory data file to drive the dynamic position and posture changes of the virtual pedestrian entities, thereby realizing the dynamic reproduction of the crowd disaster scenario and the evaluation of human-in-loop experiments, specifically includes:

[0067] The three-dimensional virtual scene model is loaded and established in the immersive virtual reality simulation platform. The three-dimensional pedestrian continuous trajectory data file is read. According to the frame number and timestamp in the trajectory data, a three-dimensional spatial node corresponding to its identity identifier is created for each virtual pedestrian entity.

[0068] Based on the recorded information in the 3D pedestrian continuous trajectory data file, in the first... t Frame Timing Driven i The three-dimensional position of each virtual pedestrian entity changes, and the attitude rotation quaternion of the virtual pedestrian entity is calculated using the velocity and direction information attached to the trajectory data.

[0069] By traversing all frames of data in the three-dimensional pedestrian continuous trajectory data file, the three-dimensional position and posture of all virtual pedestrian entities are continuously driven to realize the reproduction of the crowd movement state under multi-frame time sequence, and obtain the three-dimensional dynamic simulation process of the crowd before and after the disaster.

[0070] A human-loop experiment module is established in the immersive virtual reality simulation platform. Subjects are connected to a virtual reality interactive terminal, and immersive experience and behavioral response tests are conducted based on the three-dimensional crowd state reproduced by the trajectory.

[0071] The human loop experiment module records the subject's viewpoint coordinates, head posture, operation input, and physiological feedback data to form a human loop assessment dataset. The assessment dataset is used to perform time series alignment with the trajectory data file to complete the experimental assessment of the population disaster scenario under the corresponding frame.

[0072] Using the aforementioned human loop assessment dataset, the emergency evacuation strategy was validated and tested, including loading and comparing different evacuation guidance paths to obtain a set of evacuation strategy assessment results.

[0073] Compared with the prior art, the present invention has the following advantages:

[0074] (1) In the prior art, although crowd dynamic simulation technology based on mathematical models can theoretically simulate the process of crowd movement, it usually relies on assumptions and simplified models, and cannot accurately reproduce the complex crowd behavior in actual disaster scenarios. The application of these models is often based on a series of idealized assumptions, such as the assumption of uniformity and regularity of crowd behavior, resulting in low fit with real disaster situations and low authenticity and reliability of simulation results. This invention, through the analysis of real disaster video data, combined with accurate three-dimensional scene reconstruction and virtual reality simulation technology, uses real disaster data to drive crowd behavior simulation, avoiding the limitations of hypothetical models. Through this technical solution, this invention can achieve more realistic and reliable crowd dynamic reconstruction and simulation, significantly improving the authenticity of simulation results, and thus providing a more scientific basis for disaster emergency decision-making and evacuation strategy optimization.

[0075] (2) Traditional disaster scene reconstruction techniques based on 3D modeling, while providing relatively realistic visual effects in environmental reproduction, still rely on hypothetical scenarios and unrealistic simulation parameters, failing to accurately simulate crowd behavior during disasters. Due to the lack of real crowd movement data, the movement trajectories of virtual pedestrians often deviate significantly from reality, affecting the accurate assessment of disaster scenarios. This invention achieves a realistic reproduction of the disaster environment and crowd movement by combining disaster video analysis and 3D crowd trajectories extracted from video data. By combining crowd state data, 3D scene models, and virtual reality technology, this invention can dynamically display the movement trajectories and state characteristics of crowds before and after a disaster, providing a more realistic and accurate reconstruction of disaster scenarios, ensuring that the simulation of crowd behavior is more consistent with the actual disaster scenario.

[0076] (3) Existing simulation systems based on 3D modeling typically rely on artificially set hypothetical scenarios and limited parameter adjustments, making it impossible to dynamically adjust or reconstruct the environment based on actual disaster events. Such technologies cannot fully adapt to changes in different scenarios, such as real-time feedback on environmental changes and crowd behavior responses during a disaster. To address this issue, this invention introduces a combination of particle image velocimetry technology and multi-frame video data. By extracting the 3D coordinates and velocity information of crowds in the video frame by frame, realistic 3D continuous pedestrian trajectory data is formed. This technical feature enables the virtual reality simulation platform to dynamically drive the movement of virtual pedestrian entities based on real video data, accurately reproducing changes in crowd behavior in disaster scenarios, thereby providing a highly flexible and real-time adjustable disaster scene reconstruction system.

[0077] (4) Existing technologies, while including crowd recognition techniques based on target detection algorithms, are typically limited to processing near-field areas and cannot effectively handle dense crowds and complex scenes in distant areas of disaster scenarios. Especially in distant areas, it is difficult to accurately estimate crowd density changes, leading to gaps in crowd behavior simulation. This invention employs crowd density estimation technology, combined with pixel distribution in distant areas of the video, to accurately extract density distribution data. Furthermore, by mapping this data to three-dimensional scene coordinates, density information is combined with crowd trajectory data. This technical feature overcomes the shortcomings of existing technologies in processing distant area data, enabling this invention to accurately reproduce the dynamic behavior of dense crowds in complex disaster scenarios, thus improving the accuracy and realism of scene reconstruction. Attached Figure Description

[0078] Figure 1 This is a flowchart of the virtual scene reconstruction and experimental method according to an embodiment of the present invention;

[0079] Figure 2 This is a flowchart of step S6 in an embodiment of the present invention;

[0080] Figure 3 This is a schematic diagram illustrating the connection and communication relationships of various devices in the virtual reality platform of this invention. Detailed Implementation

[0081] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0082] Example 1

[0083] This embodiment provides an experimental method based on crowd disaster video data and virtual scene reconstruction. Through static scene modeling and dynamic crowd movement data extraction, a real disaster scene is reconstructed in a virtual simulation environment, forming an immersive crowd disaster experimental system that can be used for human loop testing. This method can realistically reflect the movement trajectory, behavioral characteristics, and risk evolution process of pedestrians before and after a disaster, providing support for pedestrian perception assessment, causal analysis, and emergency management decision support. Figure 1 As shown, the specific steps are as follows:

[0084] Step S1: Acquire crowd disaster video data, analyze the acquired crowd disaster video frame by frame, and divide each frame into near-field and far-field areas;

[0085] Given the stringent requirements for image clarity in crowd recognition and data extraction, and the poor performance of existing crowd disaster videos when directly extracting trajectories using object tracking methods, this invention proposes a method for identifying crowds in both close-up and long-range videos: using the effective recognition range of the close-up crowd recognition algorithm as a dividing line, a long-range crowd extraction strategy is employed in areas not covered by the close-up algorithm. This is further refined into the following steps S2-S4:

[0086] Step S2: For the near-field area, use an object detection algorithm to extract the two-dimensional image coordinates of identifiable individuals to obtain a coordinate dataset;

[0087] Near-field crowd coordinate data extraction: Based on the YOLOv7 object detection algorithm, the target video file is read and facial feature points are identified frame by frame. The positions of the detected near-field pedestrian faces in each frame are extracted, and the corresponding screen coordinates (including pixel values ​​in both the x and y directions) are output, forming a two-dimensional screen coordinate data file of the face, denoted as... .

[0088] Step S3: For the distant area, use crowd density estimation techniques to obtain a temporal two-dimensional density matrix;

[0089] Coordinate extraction of distant crowds: To address the issues of small individual target scale and severe occlusion in distant areas, region analysis and feature clustering algorithms are employed. After removing near-field recognition areas, the remaining distant image is processed using... Pixels are segmented using the smallest unit side length to construct a matrix. Based on a heatmap probability estimation algorithm, a probability density matrix of the distant crowd distribution is obtained. The row and column information of the heatmap probability density is statistically analyzed to form a time-series matrix density data file, denoted as [file name missing]. .

[0090] Step S4: Input the video data of the disaster crowd into the particle image velocimetry processing tool, extract the velocity vector of pixel motion in the video frame by frame based on pixel unit, and obtain two-dimensional velocity information data;

[0091] Crowd velocity and direction data extraction: The surveillance video was imported into PIVlab, and the velocity magnitude and direction of each pixel were extracted frame by frame using particle image velocimetry. The final result was a matrix-formatted data file, denoted as [data file name missing]. This matrix contains the video pixel coordinates, motion angle, and motion intensity (speed) of the current region.

[0092] Step S5: Based on the environmental map, image features, and geographic information of the disaster video data, establish a 3D virtual scene model corresponding to the actual accident site, specifically including:

[0093] Determine simulation boundaries The system utilizes Unreal Engine to construct a simulated scene, incorporating environmental elements such as terrain, buildings, obstacles, and vegetation. The physical properties, lighting conditions, and material characteristics of the scene are adjusted to ensure the generated spatial environment meets the realism requirements for simulating crowd behavior and recreating disaster scenarios. The camera viewport position and rotation in the virtual scene are kept consistent with the actual camera viewpoint in the source video.

[0094] Step S6: Map the coordinate dataset, temporal 2D density matrix, and 2D velocity information data into 3D world coordinates and 3D velocity vectors that match the 3D virtual scene model using a coordinate projection tool. Then, perform matching and fusion based on spatial position correspondence to form multi-frame 3D crowd state data, such as... Figure 2 As shown, it specifically includes:

[0095] In Unreal Engine, near-field data The extracted two-dimensional coordinates of the task are mapped to three-dimensional spatial coordinates using the tool "DeprojectScreenPositionToWorld".

[0096] For the set of prospective density matrices Take one frame and denote the size of the matrix grid as... First, the 2D coordinates of the vertices of the matrix mesh are mapped to the 3D scene using a 3D coordinate mapping tool to obtain the mapped space of the matrix mesh, and then the actual area occupied by this region is calculated. Set the maximum number of people that can be generated per square meter, PMAX, and calculate the number of people using the following formula. Okay, number Number of people in the column cell :

[0097]

[0098] in, Represents grid cells The number of people generated within; The maximum number of people that can be accommodated per square meter is set based on the actual physical space. , These represent the number of rows and columns of the grid cells after the distant view area has been divided; Represents grid cells The probability of population distribution; For grid cells The actual land area occupied; This is the magnification factor; Indicates the rounding operation;

[0099] Similarly, the number of individuals generated in each region is calculated, and the corresponding 3D coordinates of the crowd are randomly distributed within that interval. Finally, the 3D coordinates of the foreground and background views within the same frame are merged to form multi-frame crowd 3D coordinate data, denoted as... :

[0100]

[0101] in, Representing the The three-dimensional coordinates (vector set) of the crowd in the frame.

[0102] Next, the velocity and direction matrices obtained from the image are... In Unreal Engine, this is converted into 3D position and motion vector information using a 3D coordinate mapping tool, denoted as a series of velocity vector point pairs with position, and denoted as... :

[0103]

[0104] In the same frame and The data is matched to provide for For any frame and each coordinate point, find The nearest velocity and orientation data point in the frame is matched to generate 3D crowd state data containing velocity data. :

[0105]

[0106] in, Indicates the first The set of three-dimensional velocity vectors of a frame; for It satisfies the following formula:

[0107]

[0108] in, Indicates the first The first frame A three-dimensional velocity vector of an identifiable individual; Indicates the first In the frame, the closest velocity vector point is selected. The three-dimensional velocity vector; Indicates the first The first frame j The three-dimensional position coordinates corresponding to each velocity vector point; Indicates the first The first frame The three-dimensional coordinates of an identifiable individual; This represents the Euclidean distance.

[0109] Step S7: Assign unique identifiers to multi-frame 3D crowd state data, and perform cross-frame association and trajectory optimization processing according to the temporal sequence and spatial proximity correspondence to generate a 3D pedestrian continuous trajectory data file, specifically including:

[0110] Based on the frame-by-frame coordinate and velocity data compiled above, a trajectory set data containing pedestrian IDs is obtained through a matching algorithm. First, using... Starting with the first frame, obtain the coordinates of all pedestrians in that frame. Assuming the first frame file contains If there are 1 coordinate point, then create a collection of 1 coordinate point. A collection of pedestrian trajectories Read the next frame of data and iterate through it. Predict the location of the pedestrian in the next frame:

[0111]

[0112] in, Indicates the predicted first The first frame The three-dimensional coordinates of each individual; Indicates the first The first frame The three-dimensional coordinates of each individual; Indicates the first The first frame The three-dimensional velocity vector of each individual; This refers to the inter-frame time interval. Indicates the first The total number of individuals in a frame, based on the number of generators. Sure;

[0113] In the set The next frame The system searches for the nearest point in the data as a match for the pedestrian's trajectory. It's important to note that if the nearest point meets either a distance threshold or a direction difference threshold, then no matching pedestrian is considered to exist. Finally, the... The target points matched in the trajectory set are added to the trajectory set. .

[0114] At the end of the traversal, if There exists a set of coordinates of points that have not been matched with a preceding point, denoted as . The number of pedestrians is This indicates the existence of new pedestrian tracks. Add the population trajectory collection:

[0115]

[0116] Similarly, process the loop. The trajectory of the frame until the set is complete. Matching all frames in the dataset. Using a spline algorithm... The trajectories of each pedestrian are subjected to curve fitting and smoothing to ensure that they conform to the laws of human movement and the real physical environment.

[0117] Finally, the trajectory is split by frame number, and timestamps and ID information are added to generate a pedestrian trajectory data file T that can be used for virtual reality simulation.

[0118] Step S8: Load the three-dimensional virtual scene model into the immersive virtual reality simulation platform, and call the generated three-dimensional pedestrian continuous trajectory data file to drive the dynamic position and posture changes of the virtual pedestrian entity, so as to realize the dynamic reproduction of the crowd disaster situation and the evaluation of the human loop experiment.

[0119] like Figure 3 The diagram illustrates the connections and communication relationships between devices within the virtual reality platform. A virtual simulation application runs on computer 1, utilizing the Unreal Engine to construct a disaster scenario simulation environment, including basic environmental elements such as terrain, buildings, obstacles, and vegetation. The physical properties, lighting conditions, and material characteristics of the scene are configured according to the simulation requirements to ensure it conforms to the real spatial characteristics of a disaster. Two types of models are pre-set in the simulation scene: a pedestrian target model for interactive testing, and a background crowd model generated based on data.

[0120] Participants establish an interactive connection with computer 1 by wearing VR glasses (device 3) and controllers (device 4). Specifically:

[0121] Computer 1 connects to the 3D glasses and the controller via Wi-Fi communication protocol;

[0122] Pedestrians wear 3D glasses and hold a controller, which can control the walking, turning, and hazard avoidance behaviors of the pedestrian target model in the virtual simulation system.

[0123] The 3D glasses and controllers use positioning tracking and posture capture to collect the participant's movement position and interaction commands in real time, and map them onto the corresponding pedestrian target model in the three-dimensional scene;

[0124] The background crowd model then moves automatically along the actual path based on the previously generated T-trajectory data.

[0125] Simultaneously, computer 1 establishes a data synchronization channel with computer 2 via the TCP / IP communication protocol. Computer 2, acting as a server host, receives and stores the motion data of the pedestrian target model in real time, specifically including:

[0126] The trajectory points of the pedestrian target model in three-dimensional space;

[0127] The movement state of the background crowd at the corresponding time;

[0128] Timestamp information.

[0129] By using a human-centered loop experiment, participants are guided to immerse themselves in disaster scenarios, and pedestrian behavior data is collected. This data, together with background simulated crowd data, constitutes human factors data in a disaster scenario, which can be used to conduct behavioral response tests, sensory assessments, and verification of emergency evacuation strategies.

[0130] Example 2:

[0131] This embodiment provides a virtual scene reconstruction and experimental system based on crowd disaster video data, including:

[0132] The video acquisition module is used to access and store video data of crowd disasters, decode the video stream according to a predetermined frame rate, and output time-series two-dimensional video frame images.

[0133] The region division module is used to analyze and process each frame of video, determine the effective range that can be stably identified based on the detection results of human head or human body features, and use the boundary of this range as the dividing line between the near-field region and the far-field region to divide each frame into the near-field region and the far-field region.

[0134] The close-up analysis module is used to apply deep learning object detection algorithms based on the YOLO series to the close-up area, extracting the position information of identifiable individuals in a two-dimensional pixel coordinate system frame by frame, forming a coordinate dataset containing only the pixel position information of identifiable individuals frame by frame. ;

[0135] The distant view analysis module uses a heatmap probability estimation algorithm to estimate crowd density in the distant view area. It divides the distant view area into grids with m pixels as the smallest unit side length, counts the number of crowd pixels in each grid unit, and calculates the crowd distribution probability matrix to form a temporal two-dimensional density matrix. Simultaneously, based on the actual area occupied by each grid unit, the maximum number of people that can be accommodated per square meter, and the magnification factor, the number of people generated in each grid unit is calculated using a rounding formula, and the corresponding three-dimensional coordinates of the crowd are randomly distributed in the three-dimensional scene coordinates.

[0136] The particle image velocimetry module is used to perform particle image velocimetry processing on the pixel changes between adjacent frames of the crowd disaster video, extracting the two-dimensional velocity vectors of pixel motion frame by frame and forming a two-dimensional velocity information matrix. ;

[0137] The scene modeling module is used to construct a 3D virtual scene model corresponding to the real accident scene in Unreal Engine based on the environmental map, image features and geographic information in the video, and to keep the virtual camera viewport position and rotation settings consistent with the actual camera viewpoint, so as to obtain the 3D environment boundary Γ and scene geometric information that can be used for simulation.

[0138] The 3D mapping and fusion module is used to map the coordinate dataset and the coordinates of the crowd generated from the distant view into 3D world coordinates using coordinate projection tools. It also converts the 2D velocity information matrix into 3D position and motion vector point pairs within Unreal Engine using 3D coordinate mapping tools. Furthermore, it merges the 3D coordinates of the foreground and background views within the same frame to form multi-frame crowd 3D coordinate data, and based on the Euclidean nearest neighbor principle, it... E and Matching and fusion are performed to assign a corresponding three-dimensional velocity vector to each individual coordinate, forming a frame-by-frame three-dimensional crowd state data set.

[0139] The trajectory association and optimization module is used to assign a unique pedestrian ID to the frame-by-frame 3D crowd state data, perform cross-frame prediction matching and association processing according to time series, spatial proximity threshold, and directional difference threshold, create new trajectories for unmatched coordinate points and update the trajectory set R; and perform curve fitting and smoothing optimization for each trajectory using a cubic spline interpolation algorithm to generate a 3D continuous pedestrian trajectory data file T with additional timestamps and unique pedestrian ID information.

[0140] The virtual reality simulation module is used to load the three-dimensional virtual scene model and the trajectory data file T into the immersive virtual simulation platform, generate virtual pedestrian entities, and drive the three-dimensional position and posture rotation quaternion changes of each virtual pedestrian entity according to the timestamp; at the same time, a human-loop experiment submodule is established, which is connected to the subject's virtual reality interactive terminal to record the subject's viewpoint coordinates, head posture, operation input and physiological feedback data, forming a human-loop assessment dataset, which is used to perform time series alignment with the trajectory data to complete behavioral response testing, somatosensory assessment and emergency evacuation strategy verification.

[0141] The modules interact with each other and coordinate timing through a data bus and Unreal Engine interface to jointly realize the functions of virtual scene reconstruction, dynamic reproduction and experimental evaluation based on real disaster video.

[0142] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0143] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. An experimental method for reconstructing disaster scenes based on crowd disaster video data, characterized in that, Includes the following steps: Acquire video data of crowd disasters, analyze the acquired video data frame by frame, and divide each frame into near-field and far-field areas; For the foreground area, a target detection algorithm is used to extract the two-dimensional image coordinates of identifiable individuals to obtain a coordinate dataset; For distant areas, a temporal two-dimensional density matrix is ​​obtained using crowd density estimation techniques; The video data of the disaster crowd is input into the particle image velocimetry processing tool, and the velocity vector of pixel motion in the video is extracted frame by frame based on pixel unit to obtain two-dimensional velocity information data. A three-dimensional virtual scene model corresponding to the real accident site was established based on the environmental map, image features and geographic information of the disaster video data. The coordinate dataset, temporal two-dimensional density matrix, and two-dimensional velocity information data are mapped into three-dimensional world coordinates and three-dimensional velocity vectors that match the three-dimensional virtual scene model through coordinate projection tools, and then matched and fused based on the spatial position correspondence to form multi-frame three-dimensional crowd state data. A unique identifier is assigned to each frame of 3D crowd status data, and cross-frame association and trajectory optimization are performed according to the correspondence between time series and spatial proximity to generate a 3D pedestrian continuous trajectory data file. The three-dimensional virtual scene model is loaded into the immersive virtual reality simulation platform, and the generated three-dimensional pedestrian continuous trajectory data file is called to drive the dynamic position and posture changes of the virtual pedestrian entity, so as to realize the dynamic reproduction of the crowd disaster situation and the evaluation of human loop experiment.

2. The experimental method for reconstructing a disaster scene based on crowd disaster video data according to claim 1, characterized in that, The process of performing frame-by-frame analysis on the acquired disaster videos involving crowds, dividing each frame into foreground and background regions, specifically includes: For each frame of the video, head or human body feature detection technology is used to determine the effective range in the near-field area that can be stably identified and tracked, and the effective range is used as the boundary between the near and far scenes; the head or human body feature detection technology is a target detection algorithm based on deep learning, which is used to locate the feature points of individual pedestrians in the two-dimensional image pixel coordinate system and output the near-field individual position information for tracking. Based on the aforementioned dividing line, the remaining image after removing the foreground area is determined as the background area.

3. The experimental method for reconstructing disaster scenes based on crowd disaster video data according to claim 1, characterized in that, For the near-field region, a target detection algorithm is used to extract the two-dimensional image coordinates of identifiable individuals to obtain a coordinate dataset, specifically including: For each frame of the defined close-up area, a deep learning object detection algorithm based on the YOLO series is used to locate the two-dimensional image coordinates of identifiable individuals. This yields the position of the head or other human feature targets of identifiable individuals in the close-up area within the two-dimensional pixel coordinate system of the video frame. The object detection uses the video frame as the basic processing unit, independently performing recognition and localization processing on each frame, and directly outputting the set of two-dimensional pixel coordinates corresponding to the current frame, forming a coordinate dataset containing only the pixel position information of identifiable individuals frame by frame. The identifiable individuals include head-centered targets, human body outline targets, and visual targets of individual pedestrians that are not completely obscured in disaster situations, which can represent the position of a single pedestrian in the video footage.

4. The experimental method for reconstructing a disaster scene based on crowd disaster video data according to claim 1, characterized in that, For the distant area, a temporal two-dimensional density matrix is ​​obtained using crowd density estimation techniques, specifically including: After removing the foreground area, the background area is then... The pixels are segmented into grid cells based on the smallest unit side length. Then, statistical processing is performed on the pixels of each grid cell using region analysis and feature clustering algorithms, including: The probability of crowd distribution in each grid cell is calculated using a heatmap probability estimation method, and a temporal two-dimensional density matrix of the distant area is formed. The temporal two-dimensional density matrix includes the probability value of crowd distribution corresponding to each grid cell in each frame. The probability value of crowd distribution is used to represent the relative density of crowd in the grid cell in the corresponding frame. The heatmap probability estimation method is used to convert the pixel statistics within the grid cell into probability values ​​to reflect the spatial distribution characteristics of crowd in the distant area.

5. The experimental method for reconstructing a disaster scene based on crowd disaster video data according to claim 4, characterized in that, The method of calculating the population distribution probability of each grid cell using heatmap probability estimation and forming a temporal two-dimensional density matrix of the distant area specifically includes: The grid units after dividing the distant view area are numbered as follows: The corresponding pixel set is ; For each grid cell Count the number of pixels in the data that are identified as part of the crowd. ; Based on the number of crowd pixels in each grid cell The probability of population distribution in each grid cell is calculated using the following formula: in, Represents grid cells The probability of population distribution; , These represent the number of rows and columns of the grid cells after the distant view area has been divided; The probability of crowd distribution is repeatedly calculated for each frame in the video data of crowd disasters, generating a two-dimensional probability matrix that changes over time. The temporal two-dimensional density matrix constituting the distant view area ,in, Indicates the first Crowd distribution probability data in frames; This indicates the total number of video frames.

6. The experimental method for reconstructing a disaster scene based on crowd disaster video data according to claim 1, characterized in that, The process of inputting disaster video data into a particle image velocimetry processing tool, extracting velocity vectors of pixel motion in the video frame by frame based on pixel units, and obtaining two-dimensional velocity information data specifically includes: The video data of the disaster involving crowds is input into a particle image velocimetry processing tool. For each frame, the particle image velocimetry method is used to calculate the velocity vector of each pixel unit based on the pixel changes between adjacent frames. ,in, , These represent the horizontal velocity component and the vertical velocity component, respectively. For each frame of the video, calculate the velocity vector of each pixel, and form a two-dimensional matrix from the velocity vectors of each pixel to obtain two-dimensional velocity information data. .

7. The experimental method for reconstructing a disaster scene based on crowd disaster video data according to claim 1, characterized in that, The process of establishing a 3D virtual scene model corresponding to the actual accident scene based on environmental maps, image features, and geographic information from disaster video data specifically includes: Based on the environmental maps, image features, and geographic information in the disaster video data, the simulation boundary is determined. Furthermore, a 3D virtual scene model corresponding to the real accident scene was constructed based on virtual scene modeling tools; the 3D virtual scene model includes terrain, buildings, obstacles, and vegetation; The camera viewport position and rotation settings in the 3D virtual scene model are kept consistent with the actual camera viewpoint in the crowd disaster video data.

8. The experimental method for reconstructing a disaster scene based on crowd disaster video data according to claim 1, characterized in that, The process involves mapping the coordinate dataset, temporal two-dimensional density matrix, and two-dimensional velocity information data into three-dimensional world coordinates and three-dimensional velocity vectors that match the three-dimensional virtual scene model using a coordinate projection tool, and then performing matching and fusion based on spatial position correspondence to form multi-frame three-dimensional crowd state data. Specifically, this includes: Coordinate dataset obtained from object detection algorithm in near field area By using coordinate projection tools, each two-dimensional image coordinate position is mapped to the corresponding three-dimensional world coordinates; The temporal two-dimensional density matrix obtained from the density estimation algorithm in the distant region By using a 3D coordinate mapping tool, the 2D coordinates of the matrix mesh are mapped to the corresponding 3D scene coordinates, thus obtaining the 3D coordinates of each mesh cell. And based on the actual area occupied Based on the maximum number of people that can be generated per square meter Calculate the number of people generated in each grid cell. The formula is: in, Represents grid cells The number of people generated within; The maximum number of people that can be accommodated per square meter is set based on the actual physical space. , These represent the number of rows and columns of the grid cells after the distant view area has been divided; Represents grid cells The probability of population distribution; For grid cells The actual land area occupied; This is the magnification factor; Indicates the rounding operation; Traverse each grid cell, calculate the number of people generated in each grid cell, and randomly distribute the corresponding 3D coordinates of the crowd within the grid cell. ; The three-dimensional coordinates of the foreground area and the background area are merged to form a multi-frame set of three-dimensional coordinates of the crowd. ,in, Indicates the first A set of three-dimensional coordinate vectors of the crowd in a frame; Indicates the total number of video frames; Two-dimensional velocity information data obtained through particle image velocimetry The velocity vector in the image is converted into a three-dimensional velocity vector using a three-dimensional coordinate mapping tool. ; Matching the set of 3D coordinate vectors of the crowd in the same frame with the 3D velocity vector, for Each coordinate point in any frame By matching the data with a set of 3D velocity vectors in the same frame and assigning a corresponding velocity vector to each coordinate point using a spatial nearest neighbor approach, 3D crowd state data containing velocity information is generated. : in, Indicates the first The set of three-dimensional velocity vectors of a frame; for It satisfies the following formula: in, Indicates the first The first frame A three-dimensional velocity vector that can identify an individual; Indicates the first In the frame, the closest velocity vector point is selected. The three-dimensional velocity vector; Indicates the first The first frame j The three-dimensional position coordinates corresponding to each velocity vector point; Indicates the first The first frame The three-dimensional coordinates of an identifiable individual; This represents the Euclidean distance.

9. The experimental method for reconstructing a disaster scene based on crowd disaster video data according to claim 1, characterized in that, The process of assigning unique identifiers to multi-frame 3D crowd state data and performing cross-frame association and trajectory optimization according to the correspondence between time series and spatial proximity to generate a 3D pedestrian continuous trajectory data file specifically includes: A collection of multi-frame 3D crowd state data The first frame As the starting frame, obtain the 3D coordinates of all individuals in the starting frame. Assume the starting frame contains... Individuals, creating a collection A set of trajectories R ; Read the next frame Data, traversing the trajectory set R Each trajectory in the image is based on the three-dimensional coordinates of the previous frame. and velocity vector The formula for predicting the coordinates of the next frame is: in, Indicates the predicted first The first frame The three-dimensional coordinates of each individual; Indicates the first The first frame The three-dimensional coordinates of each individual; Indicates the first The first frame The three-dimensional velocity vector of each individual; This refers to the inter-frame time interval. Indicates the first The total number of individuals in the frame, the total number of individuals being generated by the number of people. Sure; For the frame The three-dimensional coordinates in the data are used to find the distance to the predicted position for each trajectory. The nearest 3D coordinate point is selected as the matching object, and the following matching conditions are met: in, Indicates the Euclidean distance threshold; Indicates the threshold for the difference in velocity direction; Add the successfully matched coordinates to the corresponding trajectory. If the first... t There is a set of unmatched coordinate points in the frame. The quantity is Then a new trajectory is created and added to the trajectory set. R Update the total number of tracks: Repeat the above steps until all frames of 3D crowd state data have been processed. Data from all frames in the dataset is used to generate a complete trajectory set. R ; For the set of trajectories R For each trajectory, a cubic spline interpolation algorithm is used for curve fitting and smoothing. The smoothed trajectory is then split into frames, and timestamps and unique pedestrian ID information are added to generate a three-dimensional continuous pedestrian trajectory data file that can be called by a virtual reality simulation platform.

10. The experimental method for reconstructing a disaster scene based on crowd disaster video data according to claim 1, characterized in that, The process of loading the 3D virtual scene model into the immersive virtual reality simulation platform and calling the generated 3D pedestrian continuous trajectory data file to drive the dynamic position and posture changes of the virtual pedestrian entities, thereby realizing the dynamic reproduction of the crowd disaster scenario and the evaluation of human-in-loop experiments, specifically includes: The three-dimensional virtual scene model is loaded and established in the immersive virtual reality simulation platform. The three-dimensional pedestrian continuous trajectory data file is read. According to the frame number and timestamp in the trajectory data, a three-dimensional spatial node corresponding to its identity identifier is created for each virtual pedestrian entity. Based on the recorded information in the 3D pedestrian continuous trajectory data file, in the first... t Frame Timing Driven i The three-dimensional position of each virtual pedestrian entity changes, and the attitude rotation quaternion of the virtual pedestrian entity is calculated using the velocity and direction information attached to the trajectory data. By traversing all frames of data in the three-dimensional pedestrian continuous trajectory data file, the three-dimensional position and posture of all virtual pedestrian entities are continuously driven to realize the reproduction of the crowd movement state under multi-frame time sequence, and obtain the three-dimensional dynamic simulation process of the crowd before and after the disaster. A human-loop experiment module is established in the immersive virtual reality simulation platform. Subjects are connected to a virtual reality interactive terminal, and immersive experience and behavioral response tests are conducted based on the three-dimensional crowd state reproduced by the trajectory. The human loop experiment module records the subject's viewpoint coordinates, head posture, operation input, and physiological feedback data to form a human loop assessment dataset. The assessment dataset is used to perform time series alignment with the trajectory data file to complete the experimental assessment of the population disaster scenario under the corresponding frame. Using the aforementioned human loop assessment dataset, the emergency evacuation strategy was validated and tested, including loading and comparing different evacuation guidance paths to obtain a set of evacuation strategy assessment results.

Images