Pedestrian behavior analysis method and apparatus, and electronic device

Abstract

The invention relates to a pedestrian behavior analysis method and apparatus, and an electronic device. The method comprises the steps of displaying a start interface when a behavior analysis request for a target pedestrian is received; displaying a setting interface under the condition that a trigger operation for the first setting control is detected; under the condition that a construction instruction for the target virtual scene is received, constructing the target virtual scene based on the construction instruction and a preset virtual scene; displaying the constructed target virtual scene, so that the target pedestrian performs each target behavior related to the behavior analysis task based on the target virtual scene seen from a virtual reality VR display; and under the condition that an analysis instruction for the target pedestrian is received, performing behavior analysis based on the obtained action data and environment interaction information under each target behavior to obtain an analysis result for the behavior analysis task. According to the invention, high-trueness and high-controllability virtual environment simulation, action data acquisition and human and virtual environment interaction analysis can be carried out.

Description

Technical Field

[0001] The present disclosure relates to the field of virtual reality technology, and in particular, to a method and apparatus for analyzing pedestrian behavior, and an electronic device. Background Art

[0002] Due to considerations such as safety and experimental feasibility, most of the current analyses of pedestrian behavior are limited to motion and gait analysis under laboratory or clinical conditions, and cannot meet the needs of studying pedestrian behavior in daily travel scenarios and under the influence of multiple travel factors. In addition, limited by modeling and rendering, the virtual reality scenarios currently available for pedestrian behavior analysis are usually relatively simple or distorted, which will have a certain impact on pedestrian behavior. Therefore, the existing solutions are difficult to provide a virtual scene with high fidelity and high manipulability and obtain action data and environmental interaction information therein, thus hindering the analysis of pedestrian behavior.

[0003] Content of the Disclosed Embodiments

[0004] In view of this, the present disclosure provides a method and apparatus for analyzing pedestrian behavior, and an electronic device, which can simulate a virtual environment with high fidelity and high manipulability, collect action data, and analyze the interaction between a person and the virtual environment.

[0005] According to one aspect of the present disclosure, a method for analyzing pedestrian behavior is provided, including: when a behavior analysis request for a target pedestrian is received, presenting a start interface, where the start interface presents a first setting control, and wherein the behavior analysis request is used to indicate a behavior analysis task to be processed and a target virtual scenario required for processing the behavior analysis task; when a triggering operation on the first setting control is detected, presenting a setting interface, where the setting interface presents an overall scene map, and the overall scene map is used to indicate the distribution of roads and buildings in a preset virtual scenario; when a construction instruction for the target virtual scenario is received, constructing the target virtual scenario based on the construction instruction and the preset virtual scenario, the construction instruction indicating first setting information and / or second setting information, the first setting information including weather and time, and the second setting information including the respective numbers and movement routes of non-target pedestrians and vehicles at each target location, wherein each of the target locations is determined according to a detected selection operation on a road location in the overall scene map; presenting the constructed target virtual scenario such that the target pedestrian performs various target behaviors related to the behavior analysis task based on the target virtual scenario seen from a virtual reality (VR) display; when an analysis instruction for the target pedestrian is received, performing behavior analysis based on the obtained action data and environmental interaction information under each of the target behaviors to obtain an analysis result for the behavior analysis task, wherein the environmental interaction information includes at least one of weather, time, the location of non-target pedestrians, and the location of vehicles when the corresponding target behavior occurs.

[0006] In a possible implementation manner, the start interface further presents a first start control; and wherein the method further includes: when it is detected that the first start control is triggered, constructing the target virtual scenario based on the preset virtual scenario and preset default setting information, the default setting information including the numbers and movement routes of non-target pedestrians, the numbers and movement routes of vehicles, weather, and time.

[0007] In a possible implementation, the setting interface further displays a first input box and a second start control. The method further includes: according to the detected input operation on the first input box, determining the determined weather and time as the first setting information; in response to a selection operation to determine a target position where the road position of the scene overview map is selected, displaying an input prompt for the target position, and a second input box is displayed in the input prompt; according to the detected input operation on the second input box, determining the respective quantities and movement routes of the determined non-target pedestrians and vehicles as the second setting information, so as to construct the target virtual scene based on the preset virtual scene, the first setting information, and the second setting information when the second start control is triggered.

[0008] In a possible implementation, a second setting control is displayed on the simulation interface corresponding to the target virtual scene; wherein, the method further includes: during the process of the target pedestrian performing the target behavior, when a trigger operation on the second setting control is detected, displaying a third input box and a third start control; according to the detected input operation on the third input box, determining at least one of the quantity of the non-target pedestrians, the quantity of the vehicles, the weather, and the time as the latest setting information; when it is detected that the third start control is triggered, refreshing the target virtual scene based on the latest setting information, so that the target pedestrian continues to perform the target behavior related to the behavior analysis task based on the refreshed target virtual scene seen from the VR display.

[0009] In a possible implementation, the method further includes the construction process of the preset virtual scene, and this process includes the following steps: obtaining multiple real images under the same real scene, the shooting angles of different real images are different, and each of the multiple real images indicates part or all of the environmental elements in the real scene, and the environmental elements include at least one of roads, buildings, trees, street lamps, zebra crossings, and manhole covers; determining the basic structure of the real scene based on the multiple real images; performing perspective correction on each of the real images to obtain a corresponding first image, and removing occlusions in each of the first images to obtain a corresponding second image, and determining the texture maps corresponding to the environmental elements based on all the second images; constructing the preset virtual scene according to the basic structure and the multiple texture maps.

[0010] In a possible implementation, the method further includes: during the process of the target pedestrian performing the target behavior, when it is detected that the position of the target pedestrian in the real environment exceeds a preset safety range, displaying a safety prompt.

[0011] In a possible implementation, the method further includes: when a reproduction instruction for the behavior analysis task is detected, determining an animation based on the environmental interaction information and action data under all target behaviors, where the animation is used to describe each of the target actions and action trajectories performed by the target pedestrian in the target virtual scene.

[0012] In a possible implementation, behavior analysis is performed based on the obtained action data and environmental interaction information under each of the target behaviors, and an analysis result for the behavior analysis task is obtained, including: performing a predetermined process based on the environmental interaction information and action data under all target behaviors to obtain a result of the corresponding process, where the process includes at least one of collision detection, attention recognition, and environmental factor comparison; analyzing the behavior of the target pedestrian according to the results of all processes to obtain the analysis result.

[0013] According to another aspect of the present disclosure, there is provided an analysis device for pedestrian behavior, including: a first display module, configured to display a start interface when a behavior analysis request for a target pedestrian is received, where the start interface displays a first setting control, and where the behavior analysis request is used to indicate a behavior analysis task to be processed and a target virtual scene required for processing the behavior analysis task; a second display module, configured to display a setting interface when a trigger operation for the first setting control is detected, where the setting interface displays an overall scene diagram, and the overall scene diagram is used to indicate the distribution of roads and buildings in a preset virtual scene; a scene construction module, configured to construct the target virtual scene based on the construction instruction and the preset virtual scene when a construction instruction for the target virtual scene is received, where the construction instruction indicates first setting information and / or second setting information, the first setting information includes weather and time, and the second setting information includes the respective numbers and action routes of non-target pedestrians and vehicles at each target location, and where each of the target locations is determined according to a selection operation detected for a road location in the overall scene diagram; a scene display module, configured to display the constructed target virtual scene, so that the target pedestrian performs each target behavior related to the behavior analysis task based on the target virtual scene seen from a virtual reality (VR) display; a behavior analysis module, configured to perform behavior analysis based on the obtained action data and environmental interaction information under each of the target behaviors when an analysis instruction for the target pedestrian is received, and obtain an analysis result for the behavior analysis task, where the environmental interaction information includes at least one of weather, time, the location of non-target pedestrians, and the location of vehicles when the corresponding target behavior occurs.

[0014] In a possible implementation, the start interface further displays a first start control; wherein, the device further includes a first construction module, configured to: when detecting that the first start control is triggered, construct the target virtual scene based on the preset virtual scene and the preset default setting information, where the default setting information includes the number and movement routes of the non-target pedestrians, the number and movement routes of the vehicles, the weather, and the time.

[0015] In a possible implementation, the setting interface further displays a first input box and a second start control, and the device further includes a first determination module, configured to: according to the detected input operation on the first input box, determine the determined weather and time as the first setting information; in response to a selection operation to determine the target position where the road position of the scene overview map is selected, display an input prompt for the target position, where a second input box is displayed in the input prompt; according to the detected input operation on the second input box, determine the determined number and movement routes of the non-target pedestrians and the vehicles respectively as the second setting information, so as to construct the target virtual scene based on the preset virtual scene, the first setting information, and the second setting information when the second start control is triggered.

[0016] In a possible implementation, a second setting control is displayed on the simulation interface corresponding to the target virtual scene; wherein, the device further includes a second determination module, configured to: during the process of the target pedestrian performing the target behavior, when detecting a trigger operation on the second setting control, display a third input box and a third start control; according to the detected input operation on the third input box, determine at least one of the number of non-target pedestrians, the number of vehicles, the weather, and the time as the latest setting information; when detecting that the third start control is triggered, refresh the target virtual scene based on the latest setting information, so that the target pedestrian continues to perform the target behavior related to the behavior analysis task based on the refreshed target virtual scene seen from the VR display.

[0017] In a possible implementation, the device further includes a second construction module for performing the construction process of the preset virtual scene, and the process includes: obtaining multiple real images under the same real scene, where the shooting angles of different real images are different, and each of the multiple real images indicates part or all of the environmental elements in the real scene, and the environmental elements include at least one of roads, buildings, trees, street lamps, zebra crossings, and manhole covers; determining the basic structure of the real scene based on the multiple real images; performing perspective correction on each of the real images to obtain a corresponding first image, removing occlusions in each of the first images to obtain a corresponding second image, and determining texture maps corresponding to each of the environmental elements based on all the second images; and constructing the preset virtual scene according to the basic structure and the multiple texture maps.

[0018] In a possible implementation, the device further includes a prompt display module for: during the process of the target pedestrian performing the target behavior, when it is detected that the position of the target pedestrian in the real environment exceeds a preset safety range, displaying a safety prompt.

[0019] In a possible implementation, the device further includes an animation determination module for: when a reproduction instruction for the behavior analysis task is detected, determining an animation based on the environmental interaction information and action data under all target behaviors, and the animation is used to describe each of the target actions and action trajectories of the target pedestrian in the target virtual scene.

[0020] In a possible implementation, behavior analysis is performed based on the obtained action data and environmental interaction information under each of the target behaviors to obtain an analysis result for the behavior analysis task, including: performing a predetermined process based on the environmental interaction information and action data under all target behaviors to obtain a result of the corresponding process, and the process includes at least one of collision detection, attention recognition, and environmental factor comparison; and analyzing the behavior of the target pedestrian according to all the results of the process to obtain the analysis result.

[0021] According to another aspect of the present disclosure, an electronic device is provided, including: a processor; a memory for storing processor-executable instructions; wherein, the processor is configured to implement the above method when executing the instructions stored in the memory.

[0022] According to another aspect of the present disclosure, a non-volatile computer-readable storage medium is provided, on which computer program instructions are stored, and wherein, the computer program instructions implement the above method when executed by a processor.

[0023] According to another aspect of the present disclosure, there is provided a computer program product, including computer-readable code or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above method.

[0024] Other features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The accompanying drawings, which are included in and constitute a part of this specification, illustrate exemplary embodiments, features, and aspects of the present disclosure and are used to explain the principles of the present disclosure together with the specification.

[0026] Figure 1 A flowchart showing a method for analyzing pedestrian behavior according to an embodiment of the present disclosure.

[0027] Figure 2 A schematic diagram showing a geometric structure constructed according to an embodiment of the present disclosure.

[0028] Figure 3 A schematic diagram showing a process of making a texture map according to an embodiment of the present disclosure.

[0029] Figure 4 A schematic diagram showing a three-dimensional model obtained after segmenting and stretching a plane according to an embodiment of the present disclosure.

[0030] Figure 5 A schematic diagram showing a bungalow and a zebra crossing made using a stencil printing function according to an embodiment of the present disclosure.

[0031] Figure 6 A schematic diagram showing a result of making a long-distance view of a virtual scene according to an embodiment of the present disclosure.

[0032] Figure 7 A schematic diagram showing a scene dust effect according to an embodiment of the present disclosure.

[0033] Figure 8 A schematic diagram showing a final effect of a character model according to an embodiment of the present disclosure.

[0034] Figure 9 A schematic diagram showing parameter settings of a BlendTree animation system according to an embodiment of the present disclosure.

[0035] Figure 10 A schematic diagram showing a collision body for vehicle anti-penetration collision according to an embodiment of the present disclosure.

[0036] Figures 11 to 14 A schematic diagram showing an interface displayed during a virtual environment control process according to an embodiment of the present disclosure.

[0037] Figure 15 A schematic diagram showing the weather effect according to an embodiment of the present disclosure.

[0038] Figure 16 A schematic diagram showing the construction result of a traffic system according to an embodiment of the present disclosure.

[0039] Figure 17 A schematic diagram showing the data reproduction demonstration result according to an embodiment of the present disclosure.

[0040] Figure 18 A schematic diagram showing the analysis method of pedestrian behavior according to an embodiment of the present disclosure.

[0041] Figure 19 A schematic diagram showing the experimental process according to an embodiment of the present disclosure.

[0042] Figure 20 A block diagram showing the analysis device of pedestrian behavior according to an embodiment of the present disclosure. Detailed implementation manners

[0043] Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the drawings denote elements having the same or similar functions. Although various aspects of the embodiments are shown in the drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0044] The word "exemplary" used herein means "serving as an example, embodiment or illustration". Any embodiment described herein as "exemplary" is not necessarily to be construed as superior to or better than other embodiments.

[0045] In addition, for a better illustration of the present disclosure, numerous specific details are given in the following detailed implementation manners. Those skilled in the art should understand that the present disclosure can be implemented without some specific details. In some instances, methods, means, elements and circuits well known to those skilled in the art are not described in detail so as to highlight the gist of the present disclosure.

[0046] To facilitate the understanding of those skilled in the art of the technical solutions provided by the embodiments of the present disclosure, the technical environment for implementing the technical solutions will be described below first.

[0047] Walking is the main mode of travel for the elderly in my country. Studying the gait and travel behavior of the elderly will help us understand their health status, fall and disease risks, promote travel safety and aging-friendly travel environment for the elderly, thereby improving their social participation and quality of life, and providing a basis for functional maintenance and aging-friendly improvement for the elderly. However, due to safety and experimental feasibility considerations, the current analysis of the motor function and gait of the elderly is mostly limited to laboratory or clinical conditions, which cannot meet the needs of functional and behavioral research in the daily travel scenarios of the elderly and under the influence of multiple travel influencing factors.

[0048] Virtual reality (VR) technology can provide a possible solution to the above problems. Virtual reality is a technology that uses head displays, handles, locators and other supporting equipment to build a three-dimensional environment and provide multi-sensory feedback and realistic virtual reality experience by integrating modern computer graphics. With the development of modern graphics rendering technology and hardware, virtual reality, with its unique advantages and characteristics, combined with the Unity3D engine and three-dimensional motion capture technology, can build experimental conditions that are difficult to achieve in reality. It has been widely used in many fields such as sports rehabilitation, cognition, and behavioral research. However, there is currently no software that can provide a highly realistic and highly controllable environment, and combine with motion capture systems to record experiments, reproduce scenes, and analyze and evaluate the travel behavior of the elderly under multiple influencing factors.

[0049] In order to solve the above technical problems, an embodiment of the present disclosure provides a method for analyzing pedestrian behavior. Figure 1 FIG. 1 is a flow chart showing a method for analyzing pedestrian behavior according to an embodiment of the present disclosure. Figure 1 As shown, the pedestrian behavior analysis method may include the following steps S101 to S105.

[0050] Step S101: upon receiving a behavior analysis request for a target pedestrian, display a start interface.

[0051] The target pedestrian is the subject. The start interface may display a first setting control. The behavior analysis request is used to indicate the behavior analysis task to be processed and the target virtual scene required to process the behavior analysis task. Among them, the content of the behavior analysis task can be flexibly set according to actual needs, and the embodiment of the present disclosure does not limit this.

[0052] Step S102: When a trigger operation on the first setting control is detected, a setting interface is displayed.

[0053] The setting interface may display a scene overview map, which is used to indicate the distribution of roads and buildings in a preset virtual scene.

[0054] Step S103: When a construction instruction for a target virtual scene is received, construct the target virtual scene based on the construction instruction and a preset virtual scene.

[0055] The construction instruction may indicate first setting information and / or second setting information. The first setting information includes weather and time. The second setting information includes the respective quantities and movement routes of non-target pedestrians and vehicles at each target location. Each target location is determined according to a selection operation detected for a road location in a scene overview map.

[0056] Step S104: Display the constructed target virtual scene so that target pedestrians perform various target behaviors related to a behavior analysis task based on the target virtual scene seen from a virtual reality (VR) display.

[0057] Step S105: When an analysis instruction for a target pedestrian is received, perform behavior analysis based on the acquired action data and environmental interaction information under various target behaviors to obtain an analysis result for the behavior analysis task.

[0058] The quantity and types of action data can be determined according to an actual behavior analysis task. The environmental interaction information may include at least one of weather, time, the location of non-target pedestrians, and the location of vehicles when the corresponding target behavior occurs.

[0059] In this way, through the above steps S101 to S105, a target virtual scene can be constructed so that target pedestrians can move freely in this scene, and the behavior patterns of pedestrians in the actual environment can be simulated more realistically. This method can capture the subtle behaviors and reactions of pedestrians more accurately. Moreover, by setting an interface to customize various parameters in the virtual scene, such as weather, time, the quantity and movement routes of non-target pedestrians and vehicles, this flexibility enables experimenters to quickly adjust the scene settings according to different behavior analysis tasks to meet different research needs. At the same time, using virtual reality technology for pedestrian behavior analysis can avoid large-scale and long-term on-site observations and experiments in the actual environment, thus saving a large amount of human and material resources.

[0060] The analysis method provided by the embodiments of the present disclosure can be applied to the personal computer (PC) side in a virtual reality system. The virtual reality system provided by the embodiments of the present disclosure may further include a motion capture system, a VR head-mounted display (head-mounted display), two supporting locators, and a handle. The constructed target virtual scene can be displayed to the experimenter through the PC side and to the target pedestrian through the VR head-mounted display. A corresponding application program is installed on the PC side to implement the above analysis method. This method involves the modeling of virtual scenes and their elements, virtual environment control, experimental calibration, calibration and prompt of the environmental safety range, experimental data recording, data reproduction, behavior analysis, and other contents. The analysis method provided by the embodiments of the present disclosure will be schematically described below with reference to the accompanying drawings.

[0061] Regarding the modeling of virtual scenes and their elements, the embodiments of the present disclosure provide various ways to perform high-fidelity scene modeling. Generally speaking, a preset virtual scene can be constructed through the following steps: obtaining multiple real images under the same real scene, with different shooting angles for different real images, and each of the multiple real images indicating part or all of the environmental elements in the real scene, where the environmental elements include at least one of roads, buildings, trees, street lights, zebra crossings, and manhole covers; determining the basic geometric structure of the real scene based on the multiple real images; performing perspective correction on each real image to obtain the corresponding first image, removing the occluders in each first image to obtain the corresponding second image, and determining the texture maps corresponding to each environmental element based on all the second images; and constructing the preset virtual scene according to the basic structure and multiple texture maps. The target virtual scene is constructed by combining some adjustable parameters on the basis of the preset virtual scene, and both are virtual scenes.

[0062] In some embodiments, the photo modeling method can be used, and the free perspective-assisted modeling plug-in (fSpy plug-in) and the free 3D modeling software (Blender) can be used for modeling. Figure 2 A schematic diagram showing the geometric structure constructed according to the embodiments of the present disclosure. The perspective relationship can be calibrated through a photo and imported into the 3D modeling software Blender for modeling of the reference perspective, so as to quickly construct the basic geometric structure of buildings, etc. in the photo subsequently. See Figure 2 . Then, real texture maps and textures can be made. Most of the texture maps in the embodiments of the present disclosure are derived from real images obtained by taking photos on the spot. When taking photos, select an angle with fewer occluders and basically covering the entire object to obtain the photo, and use image processing software to perform perspective correction and remove occluders. Figure 3 A schematic diagram showing the texture map production process according to the embodiments of the present disclosure. As shown in Figure 3As shown, the process of making the texture map of the roadside stump can be obtained. The texture maps of objects, i.e., environmental elements, in virtual scenes such as buildings, street surfaces, and manhole covers, can all be made using the same method. Figure 4 A schematic diagram of a three-dimensional model obtained after dividing and stretching a plane according to an embodiment of the present disclosure is shown. Then, the obtained real image is imported into Blender in the form of a plane, and the plane is appropriately divided and stretched to obtain the geometric structure of the object in the real image in reality, and the texture map of the real image is further adjusted. Refer to Figure 4 . After the above processing, the model effect presented in Blender is very close to the real object. In this way, the method of directly taking photos to obtain texture maps eliminates steps such as specifying materials for each part of the model, ambient light rendering, and baking texture maps in the traditional modeling process. Only one texture map is used for one object, greatly simplifying the entire modeling process and having a fast modeling speed.

[0063] In some embodiments, a modeling and texture map making method based on the stencil printing function of Blender can be used. Using the stencil printing function of Blender can make texture maps of any scale for objects while ensuring photo-realism. Stencil printing is a function provided by Blender that uses a photo as a brush, supporting the drawing of new textures on existing texture maps, and having higher flexibility compared to directly modeling using photos. Figure 5 A schematic diagram of a bungalow and a zebra crossing made using the stencil printing function according to an embodiment of the present disclosure is shown. For example, the bungalow and zebra crossing in the scene are all made using this function, and the production results can be referred to Figure 5 .

[0064] The embodiment of the present disclosure also provides a method for making the distant view of a virtual scene. Since the range of the made virtual scene is limited and the scene is too empty in the distance, some virtual scenes are more distorted. In normal experiments, the target pedestrians will not reach the distant area of the virtual scene, and the requirement for environmental accuracy is not high. Figure 6 A schematic diagram of the production result of the distant view of a virtual scene according to an embodiment of the present disclosure is shown. The embodiment of the present disclosure is based on the OSM (OpenStreetMap) plugin of Blender, obtains the building distribution near the reference street, exports it as a blank building block, and adds a custom texture map. The production result of the distant view can be referred to Figure 6 , and such a method for making the distant view makes the virtual scene more natural and real.

[0065] The embodiments of the present disclosure also provide a method for optimizing the details of a virtual scene. Through the above operations, after the virtual scene is generally made, there is still a sense of unnaturalness in the realism of the scene. After analysis, the unnaturalness mainly comes from factors such as the scene being too regular, the transition between objects being abrupt, and the high repetition degree of textures. The embodiments of the present disclosure further optimize this in the construction of the virtual scene, including adding chamfer effects to the edges of buildings and generating random stains on some repeated textures. In the real world, dust is likely to accumulate at positions such as curbs and corners of walls, while the virtual scene is too clean, so the scene appears abrupt and unnatural. Figure 7 A schematic diagram showing the dust effect of the scene according to the embodiments of the present disclosure. The embodiments of the present disclosure add a dust effect to the ground texture based on the ambient occlusion node function of Blender (refer to Figure 7 ), increasing the randomness of the scene and enhancing the realism of the virtual scene.

[0066] The embodiments of the present disclosure also provide the production of a character model in a virtual scene. This production is based on photos and mainly targets NPCs, that is, non-target pedestrians. Pedestrians are an important part of a real street scene. However, most of the existing open-source 3D character models are in a cartoon style and of Western ethnic groups, which do not conform to an actual Chinese street environment. Therefore, the embodiments of the present disclosure establish a Blender workflow for generating character modeling based on real-person photos, which can conveniently and quickly produce realistic Asian character modeling. The tools for producing the character model include a character head generation tool, Blender 3D modeling software, and the MB-Lab plugin. Figure 8 A schematic diagram showing the final effect of the character model according to the embodiments of the present disclosure. The character head model is generated by the modeling tool based on the photo. The obtained model has good wiring and reasonable UV texture distribution. The models of parts such as the eyeballs and oral cavity are independent of the face, facilitating subsequent bone binding and facial animation production. The character body model is generated by MB-Lab, and parameters such as the age, height, skin color, and obesity degree of the character model can be freely adjusted. After obtaining the character head and body modeling respectively, use Blender to edit the model, merge and connect the two, and modify the material of the skin part to add clothing to the character. The final effect is as Figure 8 shown. After completing the production of the character modeling, use the free animation library (Mixamo) to perform bone binding on the character and add animations such as standing, walking, turning left and right, and turning in place. Figure 9 A schematic diagram showing the parameter settings of the BlendTree animation system according to the embodiments of the present disclosure. By controlling the animation blending ratio through some parameters (see Figure 9 ), multiple animations can be blended to achieve a seamless animation system.

[0067] The embodiments of the present disclosure also provide methods for creating interactive elements such as vehicles in a virtual scene and setting artificial intelligence (AI). In a 3D engine, for the movement of most objects, the method of directly controlling the object's orientation and displacement per frame is generally adopted. However, for two-wheeled vehicles such as electric vehicles and bicycles, using this simple control method will cause effects such as tire drift and inconsistent forward direction and tire direction. Therefore, in the process of constructing a virtual scene in the embodiments of the present disclosure, based on the law of the movement of a real electric vehicle, it is simplified into a two-point and one-rod linkage problem, so that the movement of the virtual electric vehicle has the characteristics that the front wheel drives the rear wheel and the front wheel is consistent with the forward direction of the vehicle head in reality. At the same time, the embodiments of the present disclosure also add the binding of the driver's hand to the vehicle handlebar, so that when the electric vehicle turns, the hands of the virtual character move along with the rotation of the handlebar. In addition, in the process of constructing a virtual scene in the embodiments of the present disclosure, the animation of the electric vehicle driver is changed according to the speed of the electric vehicle. When the speed is reduced to 0, the parking animation is automatically played, and when starting from the parking state, the starting animation is played. Compared with simply controlling the movement of the model by rotating and displacing the whole model, these improvements greatly enhance the authenticity of the animation performance. In addition, in the embodiments of the present disclosure, to avoid the mutual penetration of multiple vehicles when driving on the same road, an AI for automatic avoidance is added to each electric vehicle module. Figure 10 A schematic diagram showing a collision body for preventing penetration and collision of a vehicle according to an embodiment of the present disclosure. This function is based on the collision body component provided by Unity, such as Figure 10 the capsule-shaped green line shown. When the vehicles approach each other so that they are in contact within the green line range, if the vehicles are traveling in the same direction, the vehicle at the rear decelerates to a stop. If they are traveling in opposite directions, one of the vehicles randomly decelerates. Until the collision bodies of the vehicles leave each other's range, the stopped vehicle resumes driving. After testing, this AI setting can preferably provide an interactive effect between multiple electric vehicles.

[0068] The preset virtual scene in the embodiments of the present disclosure can be a daily street scene, including environmental factors such as weather and time, and interactive elements such as non-target pedestrians and vehicles. In addition to the daily street scene, it can also be other types of scenes such as a railway waiting hall and a rural road. The target type of the required target virtual scene can be determined according to the behavior analysis request. In the case of determining the required target type from multiple optional types, the start interface corresponding to the target type is displayed. The adjustable parameters in the virtual scene include the number of pedestrians and vehicles, the traveling route, weather, time, etc.

[0069] Figures 11 to 14A schematic diagram showing an interface presented during the virtual environment control process according to an embodiment of the present disclosure. In some embodiments, the relevant settings regarding the virtual scene in the analysis method can be implemented through a virtual environment control system for simulating the virtual scene. The system may include a start interface (see Figure 11 ), a settings interface (see Figure 12 ), and a simulation interface (see Figure 13 ).

[0070] When a behavior analysis request for a target pedestrian is received, the start interface is presented. The start interface may also present a first start control. When it is detected that the first start control is triggered, a target virtual scene can be constructed based on a preset virtual scene and preset default setting information. The default setting information may include the number and movement routes of non-target pedestrians, the number and movement routes of vehicles, weather, and time. In actual operation, the experimenter can click the Figure 11 's Start button on the start interface to directly enter the simulation interface using the default setting information. The control function for the movement routes of non-target pedestrians and vehicles provided by the embodiments of the present disclosure can be developed based on the Unity component NavMeshAgent. After selecting the setting category (i.e., NPC or vehicle), continuously click on the corresponding positions of the map top view (i.e., the overall scene map) in sequence to set waypoints on the map. Waypoints of different categories are represented by different colors and there are number prompts for the movement order. After setting the waypoints and entering the simulation scene, pedestrians and vehicles will be generated at the corresponding waypoints and move in a loop according to the order of the waypoints until the final set location. In this way, the consistency of the experimental virtual environment for each target pedestrian and the controllability of environmental elements can be ensured.

[0071] When it is detected for a first setting control such as Figure 11In the case of the triggering operation of the "Setting" button, a setting interface is displayed for customizing the virtual scene. The setting interface also displays a first input box and a second activation control. According to the detected input operation on the first input box, the weather and time determined based on this input operation are determined as the first setting information; in response to the selection operation determining the target position where the road position of the scene overview map is selected, an input prompt for this target position is displayed, and a second input box is displayed in the input prompt; according to the detected input operation on the second input box, the respective quantities and movement routes of non-target pedestrians and vehicles determined based on this input operation are determined as the second setting information, so as to construct a target virtual scene based on the preset virtual scene, the first setting information, and the second setting information when the second activation control is triggered, and thus, in response to the construction instruction, display the target virtual scene on the PC and the VR headset, so that both the experimenter and the target pedestrians can see this scene in the simulation interface. At this time, the experimenter can still operate through the PC to adjust the quantities of pedestrians and vehicles, as well as the time and weather of the simulation environment in real time in the setting panel at the lower right corner of the simulation interface (see Figure 14 ). The simulation interface corresponding to the target virtual scene displays a second setting control. During the process of the target pedestrian performing the target behavior, in the case of detecting the triggering operation on the second setting control, a third input box and a third activation control are displayed; according to the detected input operation on the third input box, at least one of the determined quantities of non-target pedestrians, the quantity of vehicles, the weather, and the time is determined as the latest setting information; in the case of detecting that the third activation control is triggered, the target virtual scene is refreshed based on the latest setting information, so that the target pedestrian continues to perform the target behavior related to the behavior analysis task based on the refreshed target virtual scene seen from the VR display. In this way, during the process of the target pedestrian performing the target behavior, the environmental information such as NPCs and vehicles in the target virtual scene can be adjusted in real time as needed to observe the actions of the target pedestrian.

[0072] The control functions of environmental factors such as lighting and field of view visibility provided by the embodiments of the present disclosure can be added through Unity plugins. After the virtual scene starts running, the weather and time of the virtual scene can be controlled through the setting panel. The default weather of this system is sunny, and the settings for rain, snow, and foggy weather are additionally added. Figure 15 A schematic diagram showing the weather effect according to the embodiments of the present disclosure. Figure 15 On the left is the weather effect at noon on a sunny day, Figure 15 and on the right is the weather effect at 8 am on a snowy day. In addition to the changes in lighting and visibility, the embodiments of the present disclosure also provide corresponding sound effects for the weather, which can better restore the real weather effect.

[0073] The embodiments of the present disclosure also provide a construction method for a simple traffic system.Figure 16 A schematic diagram showing the construction result of a traffic system according to an embodiment of the present disclosure. In this embodiment, an intersection and pedestrian traffic lights are constructed as shown in Figure 16 the figure, and the traffic light change cycle is set to 60 seconds, and the colors of red, green, and blue are changed in sequence every 30 seconds. At the same time, the corresponding color lights are set to flash 5 seconds before the change. It is also possible to set that when a vehicle (such as an electric vehicle or a car) drives to the zebra crossing, if the pedestrian traffic light is red, the vehicle will automatically stop, and after the traffic light turns green, it will slowly accelerate to the normal driving speed.

[0074] Regarding the recording of experimental data, the embodiment of the present disclosure provides a function of data saving in order to quantitatively reproduce the influence of various environmental factors on the target pedestrian during the subsequent data analysis process. By clicking the Figure 13 recording button at the lower left corner of the simulation interface shown in the figure, based on the FixedUpdate function provided by the Unity engine, the position and orientation information of the target pedestrian, non-target pedestrians (NPCs), and vehicles in the virtual scene and the system time at the time of recording are recorded at a fixed frequency of 50 Hz, and are saved as Comma Separated Value (CSV) files in the form of three-dimensional vector arrays respectively. In addition, the embodiment of the present disclosure also provides a first-person video recording function of the target pedestrian during the experiment, that is, during the process of the target pedestrian performing the target behavior, the content in the target virtual scene seen by the target pedestrian is saved.

[0075] Regarding the experimental calibration of mixed reality, the embodiment of the present disclosure also provides a VR real-scene perspective calibration function, which is controlled by a handle. During the experiment, the experimenter can switch the field of view shown to the target pedestrian at any time through the handle buttons. When the PC detects a switching request sent by the handle, it switches the virtual scene and the real scene displayed in the VR headset, so that it is convenient to guide the position and actions of the target pedestrian. In addition, the embodiment of the present disclosure also adds a function of adjusting the stride and the viewing height. When the PC detects a stride adjustment request sent by the handle, it adjusts the stride size according to the stride amount corresponding to the request. When the PC detects a viewing height adjustment request sent by the handle, it adjusts the viewing height according to the height amount corresponding to the request. In this way, by combining the PC and the handle, the viewing height and the stride size can be quickly adjusted to match the actual height and movement of the target pedestrian.

[0076] Regarding the calibration and prompt of the environmental safety range, since the target pedestrian cannot see the real experimental environment when moving in the virtual scene, to reduce the safety risk, the embodiments of the present disclosure provide a function for calibrating and prompting the environmental safety range. In the experimental calibration stage, by pressing the trigger key on the handle and walking around the edge of the experimental site for one circle, the PC can automatically record the coordinate positions of the handle during this process to obtain the safety range. Thus, during the process of the target pedestrian performing the target behavior, when it is detected that the position of the target pedestrian in the real environment exceeds the preset safety range, a safety prompt is displayed. For example, when the target pedestrian approaches the edge of the experimental site during the experiment, a safety range prompt will automatically appear on the simulation interface corresponding to the target virtual scene to prevent the target pedestrian from colliding with objects such as the site wall during the experiment.

[0077] Regarding data reproduction, the embodiments of the present disclosure provide a function for reading and playing back CSV files to more intuitively check the CSV files of each experiment, where each experiment can be an experiment (i.e., a behavior analysis task) for the target pedestrian under different target virtual scenes. Figure 17 A schematic diagram showing the data reproduction demonstration result according to the embodiments of the present disclosure. The embodiments of the present disclosure also provide an additional interface for playing back the previously recorded CSV files. The scene corresponding to this interface is the top view map of the target virtual scene and icons representing environmental interaction elements (vehicles, NPCs, target pedestrian positions). Generally, the PC can automatically read all CSV files from the specified save path of the CSV files, and automatically create objects representing different interaction elements in a new scene according to the name and number of the CSV files, and at the same time draw the movement path and the orientation at that time of the target pedestrian during the recording period. The demonstration result can be referred to Figure 17 , which is convenient for data demonstration.

[0078] In addition to Figure 17Regarding the demonstrated results, embodiments of the present disclosure can also implement the data reproduction function in the form of an animation. When a reproduction instruction for the behavior analysis task is detected, an animation is determined based on the environmental interaction information and action data under all target behaviors. The animation is used to describe each target action and action trajectory of the target pedestrian in the target virtual scene. In some embodiments, based on the CSV file, three-dimensional motion capture data (i.e., action data) that has been punctuated and proofread can be imported to reproduce the limb movements and target behaviors of the target pedestrian in the target virtual scene. The action data obtained by using the motion capture system can be exported as an.fbx model file and then imported into the PC. The action data and environmental interaction information are automatically aligned, and the default character model is used to complete the detailed limb movements of the target pedestrian during the experiment recorded by the motion capture system, so as to display the motion trajectory, target actions, and information on the interaction between the target pedestrian and the target virtual environment in the form of an animation. This method can more intuitively display the more detailed action information of the target pedestrian during the experiment.

[0079] Regarding data analysis, embodiments of the present disclosure can perform data analysis based on the action data and environmental interaction information of the target pedestrian recorded during the experiment, such as the coordinates and orientation information of each object in the environment. Figure 18 A schematic diagram showing the analysis method of pedestrian behavior according to an embodiment of the present disclosure is as follows. Figure 18 As shown, data analysis can be performed from aspects such as collision event detection, target pedestrian attention recognition, and behavior comparison under different travel influencing factors (abbreviated as environmental factor comparison). In this way, the behavior analysis in step S105 above, based on the action data and environmental interaction information under all target behaviors, to obtain the analysis result for the behavior analysis task, can include: performing a predetermined process on the environmental interaction information and action data under all target behaviors to obtain the result of the corresponding process, and the process includes at least one of collision detection, attention recognition, and environmental factor comparison; analyzing the behavior of the target pedestrian based on the results of all processes to obtain the analysis result.

[0080] For the detection of collision events, this embodiment mainly detects the collision events between the target pedestrian and the vehicles among the environmental interaction objects. To accurately reproduce the collision events during the experiment, embodiments of the present disclosure can detect collision events based on the collision body components provided by Unity. When the target pedestrian comes into contact with the vehicle, the time and space coordinates of this event are recorded, and a red semi-transparent sphere is marked at the corresponding position in the data reproduction scene interface to prompt the relevant information of the collision event.

[0081] For the recognition of objects of interest, this embodiment mainly recognizes moving objects among environmental interaction objects. By monitoring the position and line-of-sight direction of the target pedestrian, the embodiments of the present disclosure can analyze and infer specific environmental interaction objects that the target pedestrian may be interested in. This function relies on the spatial position data of objects in the virtual scene and the head movement data of the target pedestrian, combines the theory of visual attention and related algorithms to determine the target objects that the target pedestrian may be interested in or focus on in the virtual scene, thereby providing a deeper dimension for the analysis of experimental results and behavior evaluation. This feature helps to simulate the attention distribution in the real world and provides important clues for the understanding and interpretation of the behavior of the target pedestrian.

[0082] For the behavior comparison under different travel influencing factors, in this embodiment, variables such as the type of virtual scene (such as urban streets, rural roads, etc.), environmental conditions (such as sunny weather, typhoon, heavy rain, morning, evening, etc.), and interaction objects (such as the traffic flow and the number of NPCs on a certain section of the road) are controlled and changed using a PC. According to the influencing effects and danger levels of these variables on the travel of the target pedestrian (such as the elderly), influence labels (such as positive influence or negative influence) and coefficients (such as the magnitude of the influence degree) of each variable are set to determine the influence of each variable on the target pedestrian, and an influence factor matrix is generated to determine the influence of different variable combinations on the target pedestrian. Under different travel events and the influence factor matrix, the changes in the movement trajectories and behavior decisions of the target pedestrian under each variable or multi-variable factors are compared. In addition, it is also possible to estimate the influence labels and coefficients of variables in a new virtual scene based on the influence labels and coefficients of variables in the existing target virtual scene.

[0083] The embodiments of the present disclosure also provide a function for exporting the curve graphs of the target pedestrian's position, orientation, and speed information. After completing the motion analysis task, the key information such as the position, orientation, and speed of the target pedestrian in the target virtual scene can be converted into curve graphs and exported, enabling researchers to more clearly analyze and evaluate the behavior performance of the target pedestrian in the experiment. The embodiments of the present disclosure can also mark information such as the time period when a collision event occurs on the curve graph, facilitating the intuitive analysis of the interaction between the target pedestrian and environmental elements (such as pedestrians, vehicles, etc.). These marks can clearly show the occurrence time period of the collision event, providing important event records and analysis bases for researchers. The embodiments of the present disclosure also support the top-down view function for the entire experimental process. This function enables researchers to observe the position movement, behavior path, etc. of the target pedestrian during the entire experimental process. Through the top-down view, the behavior trajectory of the target pedestrian in the virtual environment can be clearly displayed. This provides an important visual reference for evaluating the effectiveness of the experiment, the behavior patterns of the target pedestrian in a specific environment, etc. The embodiments of the present disclosure can also draw the time process graph of the movement and orientation information of the selected target pedestrian, the top-down view of the movement path of the target pedestrian, and the change curves of the head angular velocity and the moving speed component or absolute value with respect to time.

[0084] In some embodiments, the motion capture system used can have a three-dimensional motion capture system independent of the PC side. For this reason, the embodiments of the present disclosure also provide a joint recording function with the three-dimensional motion capture system. The PC side communicates with the host equipped with the three-dimensional motion capture system through a serial port, receiving or sending serial port signals to other recording systems. When a pre-set serial port signal is detected, the system timestamp of the current event occurrence is recorded. The three-dimensional motion capture system in the embodiments of the present disclosure can issue different serial port signals at the start and end of recording, enabling the PC side to implement the joint recording function with the three-dimensional motion capture system by monitoring this signal.

[0085] Figure 19 A schematic diagram showing the experimental process according to the embodiments of the present disclosure is as Figure 19As shown in the figure, to perform the behavior analysis task, the following steps can be taken: Take a photo in the actual scenario to be modeled; Use the fSpy software to obtain the spatial reference (i.e., three-dimensional geometric structure) of the scenario from the photo and obtain the object texture map; Use methods such as photo modeling, engraving printing, or scanning modeling in Blender to model the scenario, and import the modeling result into Unity; Generate a human model based on a real person photo and optimize it using Blender, use Mixamo to bind bones and animations; Build interaction elements in Unity, set the interaction logic of people, vehicles, etc., and the weather control of the environment; Write the interaction interface of Unity to build functions such as data recording and saving; Install a PC host, a motion capture system, and a VR device in the experimental site; Open the virtual scene control system developed in the embodiment of the present disclosure for executing the above method and perform system configuration; Calibrate the site range and guide the target pedestrian to perform action calibration; Enter the target pedestrian number or the session information of the experiment in the input box in the upper right corner of the simulation interface; Set the weather, time, environmental elements of the current scenario, and the saving location of the data recording file; Let the target pedestrian wear the VR device, click the recording button in the lower left corner of the simulation interface when the target pedestrian starts to perform the behavior analysis task, and let the motion capture system start collecting action data. If there are other systems such as the motion capture system, record the serial port signal and save the record; Click the recording button in the lower left corner of the virtual environment control system again to end the recording when the experiment is over; If it is necessary to conduct experiments on the same target pedestrian under different virtual scenarios, the scenario, scenario settings, etc. can be changed in the settings interface. After changing the relevant setting information of the virtual scenario, the target pedestrian can continue to perform the desired target action to obtain the corresponding data.

[0086] The above method can be used for the evaluation of pedestrian walking behavior, and can overcome the limitations of traditional pedestrian gait and motion analysis methods in terms of controlling scenario and environmental factors. Here, the pedestrians can be the elderly. The embodiments of the present disclosure use virtual reality technology to enable researchers and target pedestrians to conduct experiments in different spatial and environmental scenarios in an indoor environment to collect experimental data under different scenarios, so as to analyze pedestrian behavior and judge the risks of safe travel. At the same time, in a real-time controlled environment, real environments and extreme conditions rare in experiments can be constructed without being limited by safety and experimental feasibility. The three-dimensional motion capture system itself can, by optical or other means, restore the movement and behavior of people in space and under VR, so as to study the changes and differences in pedestrian walking behavior under the influence of multiple factors. According to the above method provided by the embodiments of the present disclosure, the target pedestrian can experience a real walking travel scenario in a highly realistic virtual scenario. By real-time recording the motion data of the target pedestrian and integrating it with the virtual scenario, researchers can comprehensively understand the walking travel behavior of the target pedestrian under different environmental conditions, provide new research tools and methods for the evaluation of pedestrian walking behavior under the influence of multiple factors, and provide a platform for improving the pedestrian's travel response ability and the interaction ability with the travel environment and conducting targeted improvement training.

[0087] The embodiments of the present disclosure also provide an analysis device for pedestrian behavior. The analysis device for pedestrian behavior may include the following first display module, second display module, scene construction module, scene display module, and behavior analysis module.

[0088] The first display module is configured to display a start interface when receiving a behavior analysis request for a target pedestrian, where the start interface displays a first setting control, and where the behavior analysis request is used to indicate a behavior analysis task to be processed and a target virtual scenario required for processing the behavior analysis task.

[0089] The second display module is configured to display a setting interface when detecting a trigger operation on the first setting control, where the setting interface displays an overall scene diagram, and the overall scene diagram is used to indicate the distribution of roads and buildings in a preset virtual scenario.

[0090] The scene construction module is configured to construct the target virtual scenario based on the construction instruction and the preset virtual scenario when receiving a construction instruction for the target virtual scenario, where the construction instruction indicates first setting information and / or second setting information, the first setting information includes weather and time, and the second setting information includes the respective numbers and movement routes of non-target pedestrians and vehicles at each target location, and where each of the target locations is determined according to a selection operation detected on the road location in the overall scene diagram.

[0091] A scene display module for displaying the constructed target virtual scene, enabling the target pedestrian to perform various target behaviors related to the behavior analysis task based on the target virtual scene seen from the virtual reality (VR) display.

[0092] A behavior analysis module for, when receiving an analysis instruction for the target pedestrian, performing behavior analysis based on the acquired action data and environmental interaction information under each of the target behaviors to obtain an analysis result for the behavior analysis task, where the environmental interaction information includes at least one of the weather, time, position of non-target pedestrians, and position of vehicles when the corresponding target behavior occurs.

[0093] In a possible implementation, the start interface further displays a first start control; wherein, the device further includes a first construction module for: when detecting that the first start control is triggered, constructing the target virtual scene based on the preset virtual scene and the preset default setting information, and the default setting information includes the number and movement routes of the non-target pedestrians, the number and movement routes of the vehicles, the weather, and the time.

[0094] In a possible implementation, the setting interface further displays a first input box and a second start control, and the device further includes a first determination module for: according to the detected input operation for the first input box, determining the determined weather and time as the first setting information; in response to a selection operation to determine the target position where the road position of the scene overview map is selected, displaying an input prompt for the target position, and the input prompt displays a second input box; according to the detected input operation for the second input box, determining the determined number and movement routes of the non-target pedestrians and the vehicles respectively as the second setting information, so as to construct the target virtual scene based on the preset virtual scene, the first setting information, and the second setting information when the second start control is triggered.

[0095] In a possible implementation, a second setting control is displayed on the simulation interface corresponding to the target virtual scene; wherein, the device further includes a second determination module, configured to: during the process of the target pedestrian performing the target behavior, when a trigger operation on the second setting control is detected, display a third input box and a third start control; according to the detected input operation on the third input box, determine at least one of the number of non-target pedestrians, the number of vehicles, the weather, and the time as the latest setting information; when it is detected that the third start control is triggered, refresh the target virtual scene based on the latest setting information, so that the target pedestrian continues to perform the target behavior related to the behavior analysis task based on the refreshed target virtual scene seen from the VR display.

[0096] In a possible implementation, the device further includes a second construction module, configured to perform the construction process of the preset virtual scene, and the process includes: obtaining multiple real images in the same real scene, where the shooting angles of different real images are different, and each of the multiple real images indicates part or all of the environmental elements in the real scene, and the environmental elements include at least one of roads, buildings, trees, street lights, zebra crossings, and manhole covers; determining the basic structure of the real scene based on the multiple real images; performing perspective correction on each of the real images to obtain a corresponding first image, removing occlusions in each of the first images to obtain a corresponding second image, and determining textures corresponding to each of the environmental elements based on all the second images; constructing the preset virtual scene according to the basic structure and the multiple textures.

[0097] In a possible implementation, the device further includes a prompt display module, configured to: during the process of the target pedestrian performing the target behavior, when it is detected that the position of the target pedestrian in the real environment exceeds a preset safety range, display a safety prompt.

[0098] In a possible implementation, the device further includes an animation determination module, configured to: when a reproduction instruction for the behavior analysis task is detected, determine an animation based on the environmental interaction information and action data under all target behaviors, and the animation is used to describe each of the target actions and action trajectories performed by the target pedestrian in the target virtual scene.

[0099] In a possible implementation, behavior analysis is performed based on the obtained action data and environment interaction information under each of the target behaviors to obtain an analysis result for the behavior analysis task, including: performing a predetermined process on the environment interaction information and action data under all the target behaviors to obtain a result of the corresponding process, where the process includes at least one of collision detection, attention recognition, and environmental factor comparison; analyzing the behavior of the target pedestrian based on the results of all the processes to obtain the analysis result.

[0100] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the methods described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments. For the sake of brevity, it will not be repeated here.

[0101] The embodiments of the present disclosure also propose a computer-readable storage medium, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the above methods are implemented. The computer-readable storage medium can be a volatile or non-volatile computer-readable storage medium.

[0102] The embodiments of the present disclosure also propose an electronic device, including: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to implement the above methods when executing the instructions stored in the memory.

[0103] The embodiments of the present disclosure also provide a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in the processor of an electronic device, the processor in the electronic device executes the above methods.

[0104] Figure 20 A block diagram of an analysis device for pedestrian behavior according to an embodiment of the present disclosure is shown. For example, device 1900 can be provided as a server or a terminal device. Referring to Figure 20 , device 1900 includes a processing component 1922, which further includes one or more processors, and memory resources represented by a memory 1932 for storing instructions executable by the processing component 1922, such as application programs. The application programs stored in the memory 1932 can include one or more modules each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute instructions to perform the above methods.

[0105] The apparatus 1900 may further include a power supply component 1926 configured to perform power management of the apparatus 1900, a wired or wireless network interface 1950 configured to connect the apparatus 1900 to a network, and an input / output interface 1958 (I / O interface). The apparatus 1900 may operate based on an operating system stored in the memory 1932, such as Windows Server TM , MacOS X TM , Unix TM , Linux TM , FreeBSD TM or the like.

[0106] In an exemplary embodiment, a non-transitory computer-readable storage medium is also provided, such as the memory 1932 including computer program instructions, and the computer program instructions can be executed by the processing component 1922 of the apparatus 1900 to complete the above method.

[0107] The present disclosure may be a system, a method, and / or a computer program product. The computer program product may include a computer-readable storage medium having thereon computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0108] A computer-readable storage medium may be a tangible device that can retain and store instructions for use by an instruction execution device. The computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disc (DVD), a memory stick, a floppy disk, a mechanically encoded device, such as a punched card or raised structures in grooves storing instructions thereon, and any suitable combination of the foregoing. The computer-readable storage medium used herein is not construed as being a transitory signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., an optical pulse through an optical fiber cable), or an electrical signal transmitted through a wire.

[0109] The computer-readable program instructions described herein can be downloaded to various computing / processing devices from a computer-readable storage medium or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in each computing / processing device.

[0110] The computer program instructions for performing the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state-setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages such as Smalltalk, C++, C#, etc., and conventional procedural programming languages such as the "C" language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, executed as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via an Internet service provider through the Internet). In some embodiments, by using the state information of the computer-readable program instructions to customize an electronic circuit, such as a programmable logic circuit, a field-programmable gate array (FPGA), or a programmable logic array (PLA), the electronic circuit can execute the computer-readable program instructions to implement various aspects of the present disclosure.

[0111] Aspects of the present disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0112] These computer-readable program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus to produce a machine such that the instructions, when executed by the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in one or more boxes of the flowchart and / or block diagram. These computer-readable program instructions may also be stored in a computer-readable storage medium that causes a computer, a programmable data processing apparatus, and / or other devices to function in a particular manner, such that the computer-readable medium storing the instructions comprises a manufacture including instructions for implementing various aspects of the functions / acts specified in one or more boxes of the flowchart and / or block diagram.

[0113] The computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device, such that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / acts specified in one or more boxes of the flowchart and / or block diagram.

[0114] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram may represent a module, a segment of code, or a portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions noted in the boxes may occur out of the order noted in the figures. For example, two consecutive boxes may in fact be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each box in the block diagrams and / or flowcharts, and combinations of boxes in the block diagrams and / or flowcharts, can be implemented by a dedicated hardware-based system that performs the specified functions or acts, or by a combination of dedicated hardware and computer instructions.

[0115] The embodiments of the present disclosure have been described above. The above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The choice of terms used herein is intended to best explain the principles of the embodiments, the practical application, or improvements made to the technology in the market, or to enable other ordinary skilled persons in the art to understand the embodiments disclosed herein.

Claims

1. A method for analyzing pedestrian behavior, characterized in that: include: In the case of receiving a behavior analysis request for a target pedestrian, displaying a start interface, wherein the start interface displays a first setting control, wherein the behavior analysis request is used to indicate a behavior analysis task to be processed and a target virtual scene required for processing the behavior analysis task; In the case where a trigger operation for the first setting control is detected, a setting interface is displayed, wherein the setting interface displays a scene overview map, and the scene overview map is used to indicate the distribution of roads and buildings in a preset virtual scene; In the case of receiving a construction instruction for the target virtual scene, constructing the target virtual scene based on the construction instruction and the preset virtual scene, the construction instruction indicating first setting information and / or second setting information, the first setting information including weather and time, the second setting information including the number and action routes of non-target pedestrians and vehicles at each target position, wherein each of the target positions is determined according to a detected selection operation for a road position in the scene overview map; Displaying the constructed target virtual scene so that the target pedestrian performs various target behaviors related to the behavior analysis task based on the target virtual scene viewed from a virtual reality (VR) display; When an analysis instruction for the target pedestrian is received, a behavior analysis is performed based on the acquired motion data and environmental interaction information under each of the target behaviors to obtain an analysis result for the behavior analysis task, wherein the environmental interaction information includes at least one of the weather, time, location of non-target pedestrians, and location of the vehicle when the corresponding target behavior occurs.

2. The method according to claim 1, characterized in that The start interface also displays a first start control; Wherein, the method further comprises: When it is detected that the first start control is triggered, the target virtual scene is constructed based on the preset virtual scene and preset default setting information, and the default setting information includes the number and movement routes of the non-target pedestrians, the number and movement routes of the vehicles, weather, and time.

3. The method according to claim 1, characterized in that The setting interface also displays a first input box and a second start control, and the method further includes: According to the detected input operation for the first input box, determining the determined weather and the time as the first setting information; In response to the selection operation determining that the road position of the scene overview map is a selected target position, an input prompt for the target position is displayed, wherein a second input box is displayed in the input prompt; Based on the detected input operation for the second input box, the number and movement routes of the non-target pedestrians and the vehicle are determined as the second setting information, so that the target virtual scene can be constructed based on the preset virtual scene, the first setting information, and the second setting information when the second start control is triggered.

4. The method according to claim 1, characterized in that: A second setting control is displayed on the simulation interface corresponding to the target virtual scene; The method further comprises: during the process of the target pedestrian performing the target behavior, When a trigger operation on the second setting control is detected, displaying a third input box and a third start control; According to the input operation detected for the third input box, at least one of the determined number of non-target pedestrians, the number of vehicles, the weather, and the time is determined as the latest setting information; When it is detected that the third start control is triggered, the target virtual scene is refreshed based on the latest setting information so that the target pedestrian continues to perform the target behavior related to the behavior analysis task based on the refreshed target virtual scene seen from the VR display.

5. The method according to any one of claims 1 to 4, characterized in that: The method further comprises a process of constructing the preset virtual scene, which comprises the following steps: Acquire multiple real images of the same real scene, where different real images are taken at different angles, and each of the multiple real images indicates part or all of each environmental element in the real scene, where the environmental element includes at least one of a road, a building, a tree, a street lamp, a zebra crossing, and a manhole cover; Determining a basic structure of the real scene based on the multiple real images; Performing perspective correction on each of the real images to obtain a corresponding first image, removing occluders in each of the first images to obtain a corresponding second image, and determining a map corresponding to each of the environmental elements based on all of the second images; The preset virtual scene is constructed according to the basic structure and the multiple maps.

6. The method according to any one of claims 1 to 4, characterized in that The method further includes: during the process in which the target pedestrian performs the target behavior, When it is detected that the position of the target pedestrian in the real environment exceeds a preset safety range, a safety prompt is displayed.

7. The method according to any one of claims 1 to 4, characterized in that The method further comprises: When a reproduction instruction for the behavior analysis task is detected, an animation is determined based on the environmental interaction information and action data under all target behaviors, and the animation is used to describe the target actions and action trajectories performed by the target pedestrian in the target virtual scene.

8. The method according to any one of claims 1 to 4, characterized in that Based on the acquired action data and environment interaction information under each of the target behaviors, behavior analysis is performed to obtain analysis results for the behavior analysis task, including: Performing predetermined processing based on the environmental interaction information and action data under all target behaviors to obtain corresponding processing results, wherein the processing includes at least one of collision detection, attention recognition, and environmental factor comparison; The behavior of the target pedestrian is analyzed according to all the processed results to obtain the analysis result.

9. A pedestrian behavior analysis device, characterized in that: include: A first display module is used to display a start interface when receiving a behavior analysis request for a target pedestrian, wherein the start interface displays a first setting control, wherein the behavior analysis request is used to indicate a behavior analysis task to be processed and a target virtual scene required for processing the behavior analysis task; A second display module is used to display a setting interface when a trigger operation for the first setting control is detected, wherein the setting interface displays a scene overview map, and the scene overview map is used to indicate the distribution of roads and buildings in a preset virtual scene; A scene construction module, configured to construct the target virtual scene based on the construction instruction and the preset virtual scene when receiving a construction instruction for the target virtual scene, wherein the construction instruction indicates first setting information and / or second setting information, wherein the first setting information includes weather and time, and the second setting information includes the number and action routes of non-target pedestrians and vehicles at each target position, wherein each target position is determined according to a selection operation detected for a road position in the scene overview map; A scene display module, used to display the constructed target virtual scene, so that the target pedestrian performs various target behaviors related to the behavior analysis task based on the target virtual scene seen from the virtual reality VR display; A behavior analysis module is used to perform behavior analysis based on the acquired motion data and environmental interaction information under each of the target behaviors upon receiving an analysis instruction for the target pedestrian, so as to obtain an analysis result for the behavior analysis task, wherein the environmental interaction information includes at least one of the weather, time, location of non-target pedestrians, and location of the vehicle when the corresponding target behavior occurs.

10. An electronic device, characterized in that: include: processor; A memory for storing processor-executable instructions; wherein the processor is configured to implement the method of any one of claims 1 to 8 when executing the instructions stored in the memory.

Images