A method and system for driving simulator ai pedestrian

Abstract

The present application relates to the technical field of driving simulation, in particular to a driving simulator AI pedestrian method and system. Including the following steps: determining the AI pedestrian scene of driving simulation, frame selection and division simulation area, generating AI pedestrians in the waiting area through AI birth algorithm, applying motion conversion based on deep learning to the AI pedestrian model, determining the walking characteristics of AI pedestrians, setting the route and speed of AI pedestrians, establishing traffic lights in the overlapping area of lane area and pedestrian area, setting the priority rules of AI pedestrian motion, and associating the walking characteristics of AI pedestrians with the traffic lights for driving simulation. The user can practice driving in the pedestrian area to avoid collision, which can cultivate more accurate driving feeling, and also increase the authenticity and interest of practice.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of driving simulation, in particular, to a driving simulator AI pedestrian method and system. BACKGROUND

[0002] The driving simulator cockpit is composed of a cockpit seat, a visual computer, a video screen (a 19-inch display), operation sensors, a data acquisition card, earphones and a microphone, etc. The cockpit contains the same operating components as a real vehicle, i.e. the "five major" control mechanisms: a steering wheel, a clutch, a foot brake, an accelerator and a hand brake. The real vehicle transmission includes a reverse gear, a first gear, a second gear, a third gear, a fourth gear, a fifth gear and a neutral gear (an automatic transmission only contains a forward gear, a reverse gear and a parking gear). The real vehicle operation switches include a left turn signal, a right turn signal, an emergency light, a horn, an ignition switch, a main switch, a seat belt, a door, a windshield wiper, a high beam, a low beam and a high-low beam alternation. The cockpit can be used for network training or single-machine training.

[0003] However, the current driving simulator scene is single, which leads to differences from actual driving, resulting in that the driver cannot get good practice, especially when simulating driving close to a sidewalk, the traffic rules and the avoidance of pedestrians on the sidewalk are the most important processes in actual driving. If the driver practices in an open AI scene, the authenticity is poor, and the cultivation of driving feeling is limited. In view of this, the present application provides a driving simulator AI pedestrian method and system. SUMMARY

[0004] The present application aims to provide a driving simulator AI pedestrian method and system to solve the problems in the background.

[0005] To solve the above technical problems, one of the purposes of the present application is to provide a driving simulator AI pedestrian method, which comprises the following steps:

[0006] S1, determining an AI pedestrian scene of driving simulation, framing and dividing a simulation area, the simulation area including a waiting area, a lane area, a pedestrian area and a background area;

[0007] S2, generating AI pedestrians in the waiting area through an AI birth algorithm, and obtaining motion conversion based on deep learning and applying it to an AI pedestrian model;

[0008] S3, determining the walking characteristics of AI pedestrians, and setting the route and speed of AI pedestrians;

[0009] S4, establishing a traffic light in the overlapping area of the lane area and the pedestrian area, setting the priority rules of AI pedestrian motion, and associating and relying the walking characteristics of AI pedestrians with the traffic light for driving simulation.

[0010] Through the user in the driving simulator simulation practice driving, when the simulation vehicle approaches the simulation area, the AI pedestrian scene is formed, the AI pedestrian simulates the actual walking characteristics of crossing the road, and meanwhile, the AI pedestrian is associated and dependent through the red and green lights, further in line with the actual situation, the user can practice driving in the pedestrian area to avoid collision, realize more accurate driving feeling cultivation, also increase the authenticity and interestingness of practice.

[0011] As a further improvement of the technical solution, the simulation area of S1 adopts 3D modeling technology, including the following steps:

[0012] Input scene pictures and extract model feature points;

[0013] Build a triangular mesh to generate a three-dimensional mesh model;

[0014] Texture mapping: obtain the texture coordinates of the model vertex in the two-dimensional texture space, which is a mapping process from a three-dimensional space point to a two-dimensional plane point;

[0015] Provide an interface to realize the editing function of the 3D model, and adjust the 3D model to show the 3D model of the simulation area.

[0016] As a further improvement of the technical solution, the AI birth algorithm in S2 includes the following steps:

[0017] Generate an AI pedestrian model based on 3D modeling technology;

[0018] Based on the human motion captured by the vision, the human motion is transferred to the AI pedestrian model, so that the AI pedestrian simulates the real pedestrian walking.

[0019] As a further improvement of the technical solution, the walking characteristics of S3 include walking area, walking route, walking speed, walking interval and walking logic, wherein:

[0020] The walking area is used to determine that the AI pedestrian walks along the pedestrian area;

[0021] The walking route is used to determine the number of walkable routes according to the width of the sidewalk;

[0022] The walking speed is used to set the walking speed and running speed according to different ages and genders;

[0023] The walking interval is used to set the interval between every two AI pedestrians through the interval random algorithm;

[0024] The behavior logic is used to adjust through the obstacle avoidance algorithm.

[0025] As a further improvement of the technical solution, the interval random algorithm includes the following steps:

[0026] The preset interval factor input keeps the interval between two AI pedestrians of the same age and gender unchanged, and in the case that the interval factor is unchanged, the interval between two AI pedestrians of different ages and genders is different, so as to realize the interval randomness.

[0027] As a further improvement of the technical solution, the obstacle avoidance algorithm comprises the following steps:

[0028] The function IsInTriangle() is written to determine the position relationship between the point and the triangle.

[0029] Taking the AI pedestrian M as an example, the obstacle (traveling vehicle) is represented by an irregular figure, which can be represented by a series of points and encapsulated into the Point class.

[0030] The AI pedestrian M can be represented by a series of triangles and encapsulated into the Triangle class, and the object collision detection is simplified to the problem of judging whether the point is inside or outside the triangle.

[0031] The coordinates of the AI pedestrian M are set as M(x, y), if the return value of IsInTriangle() of the AI pedestrian M is true, a collision occurs, and the coordinates of M are reset as M(x+1, y+1); if the return value is false, no collision occurs.

[0032] As a further improvement of the technical solution, the traffic light adopts a simulated traffic light device, including a red light, a green light and a yellow light, which is used to switch on the red light and the green light within a certain time, and the yellow light is used to light up at the critical value when the red light and the green light are switched.

[0033] As a further improvement of the technical solution, in the S4, the traffic light is used as a priority rule when driving simulation is performed, including the following events:

[0034] Event one, the AI pedestrian reaches the end of the pedestrian area, the traffic light displays a green light, and the AI pedestrian is controlled to walk along the route to the other end of the pedestrian area;

[0035] Event two, the AI pedestrian reaches the end of the pedestrian area, the traffic light displays a yellow light and a green light, and the AI pedestrian is controlled to stop in the waiting area.

[0036] The second object of the application is to provide a system for realizing the AI pedestrian of the driving simulator, which comprises the method of the AI pedestrian of the driving simulator, and comprises a simulation area division module, an AI pedestrian model generation module, a walking feature setting module and a traffic light association module.

[0037] The simulation area division module is used to determine the AI pedestrian scene of the driving simulation, and the simulation area is framed and divided.

[0038] The AI pedestrian model generation module is used for generating AI pedestrians in the waiting area through an AI birth algorithm, and motion conversion based on deep learning is applied to the AI pedestrian model;

[0039] The walking feature setting module is used for determining the walking features of the AI pedestrians, and setting the routes and speeds of the AI pedestrians, so as to generate different motion effects according to different people in reality, and improve the authenticity of the driving simulator.

[0040] The traffic light association module is used for establishing traffic lights in the overlapping area of the lane area and the pedestrian area, setting the priority rules of the AI pedestrian motion, and associating the walking features of the AI pedestrians with the traffic lights for driving simulation.

[0041] Compared with the prior art, the beneficial effects of the present application are as follows: when a user practices driving in the driving simulator, the simulated vehicle approaches the simulation area, and the AI pedestrian scene is formed, so that the AI pedestrian simulates the actual walking features to cross the road, and the AI pedestrian further conforms to the actual situation through the traffic light association dependency, so that the user can practice driving to avoid collision in the pedestrian area in a first-person perspective, the driving feeling is more accurate, and the authenticity and interestingness of the practice are improved. BRIEF DESCRIPTION OF DRAWINGS

[0042] Figure 1 It is the overall flowchart of embodiment 1;

[0043] Figure 2 It is the overall structure schematic diagram of embodiment 1;

[0044] Figure 3 It is the simulation area schematic diagram of embodiment 1;

[0045] Figure 4 It is the walking feature structure schematic diagram of embodiment 1. DETAILED DESCRIPTION

[0046] The technical solutions in the embodiments of the present application will be described clearly and completely in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the protection scope of the present application.

[0047] The driving simulator cockpit is composed of a cockpit seat, a visual computer, a screen (a 19-inch display), operation sensors, a data acquisition card, earphones and a microphone, etc. The cockpit contains the same operating components as a real vehicle, i.e., the "five major" control mechanisms: a steering wheel, a clutch, a foot brake, an accelerator and a hand brake; a real vehicle transmission: a reverse gear, a first gear, a second gear, a third gear, a fourth gear, a fifth gear and a neutral gear (an automatic transmission contains only a forward gear, a reverse gear and a parking gear); real vehicle operating switches: a left turn signal, a right turn signal, an emergency light, a horn, an ignition switch, a main power switch, a seat belt, a door, a windshield wiper, a high beam, a low beam and a high-low beam alternation; the cockpit can be used for both network training and single-machine training.

[0048] However, the current driving simulator scene is single, which leads to differences from actual driving, and causes the driver to not get good practice, especially when the simulation driving is close to the sidewalk, the traffic law of squatting on the sidewalk and avoiding pedestrians are the most important processes in actual driving. If the driver practices in an empty AI scene, the reality is poor, and the cultivation of driving feeling is limited.

[0049] As shown in Figures 1-4 One of the purposes of the present application is to provide a method for AI pedestrians of a driving simulator, which comprises the following steps:

[0050] S1, determining an AI pedestrian scene of driving simulation, and framing and dividing a simulation area, the simulation area comprising a waiting area, a lane area, a pedestrian area and a background area, wherein, as shown in Figure 3 The AI pedestrian scene of driving simulation is only for a group of simulation areas, so as to avoid mutual influence of multiple scenes and cause inaccurate simulation, wherein the waiting area is mostly an area where pedestrians wait, and can be arranged at both ends of the pedestrian area; the lane area is an area for driving simulation of a car, and can be a two-way lane; the pedestrian area (also called a zebra crossing) is an area where pedestrians can cross the lane; and the background area is a simulation driving background, such as roadside trees, street lamps and buildings, etc., so as to improve the picture fitting to the actual situation and improve the driving simulation environment.

[0051] The simulation area of S1 adopts a 3D modeling technology, which comprises the following steps:

[0052] Input a scene picture, and extract model feature points, which can be obtained by establishing the information of the width and height of x and y directions in a front picture, and the depth and height information (i.e., the concave-convex information of the feature points) of z and y directions in a side picture;

[0053] Build a triangular mesh, and generate a three-dimensional mesh model, which can be specifically built by means of a 3D model building platform such as 3Dmax;

[0054] Texture mapping: obtaining the texture coordinates of the model vertex in the two-dimensional texture space is a mapping process from a three-dimensional space point to a two-dimensional plane point, the classic texture mapping process is two-step texture mapping, this method first maps the texture to a simple three-dimensional surface, that is, the intermediate surface, such as a plane, a circular surface, a spherical surface or a cube, and then maps the deformed mapping result to the final surface;

[0055] An interface for implementing the editing function of the 3D model is provided, and the 3D model is adjusted to show the 3D model of the simulation area.

[0056] S2, generating an AI pedestrian in the waiting area through an AI birth algorithm, and obtaining motion conversion based on deep learning and applying it to the AI pedestrian model;

[0057] The AI birth algorithm in S2 includes the following steps:

[0058] Generating an AI pedestrian model based on 3D modeling technology;

[0059] Transferring the human motion captured by vision to the AI pedestrian model, so that the AI pedestrian simulates real pedestrians walking, through motion capture technology, i.e.

[0060] Infrared markers, obtaining the motion pattern of the human model in infrared photography data.

[0061] S3, determining the walking characteristics of the AI pedestrian, setting the route and speed of the AI pedestrian, which is conducive to meeting the different motion effects of different people,

[0062] Improving the authenticity of the driving simulator;

[0063] S4, establishing a traffic light in the overlapping area of the lane area and the pedestrian area, for setting the priority rule of the AI pedestrian motion, and associating the walking characteristics of the AI pedestrian with the traffic light for driving simulation.

[0064] The improvement of the embodiment is that when the user simulates driving in the driving simulator, the simulation vehicle approaches the simulation area to form an AI pedestrian scene, so that the AI pedestrian simulates the actual walking characteristics of crossing the road, and at the same time, the AI pedestrian passes through the traffic light association and dependence, further meeting the actual situation, so that the user can practice driving in the pedestrian area to avoid collision, realize more accurate cultivation of driving feeling, and increase the authenticity and interest of practice.

[0065] The traffic light adopts a simulated traffic light device, including a red light, a green light and a yellow light, which is used to switch on the red light and the green light within a certain time, and the yellow light is used to light up at the critical value when the red light and the green light are switched on. The principle of the simulated traffic light can be as the traffic light device in real life, the difference is that it is within a certain time, and the time can be set by the user to avoid long waiting time during driving simulation, causing fatigue. Among them, the critical value of the yellow light can be set to a time that is not enough to cross the lane, so as to avoid the AI pedestrian penetrating and stopping in the pedestrian area during the yellow light, affecting the driving simulation.

[0066] Specifically, in S4, when driving simulation is performed, the traffic light is used as a priority rule, including the following events:

[0067] Event one, the AI pedestrian reaches the end of the pedestrian area, the traffic light displays a green light, and the AI pedestrian is controlled to walk along the route to the other end of the pedestrian area;

[0068] Event two, the AI pedestrian reaches the end of the pedestrian area, the traffic light displays a yellow light and a green light, and the AI pedestrian is controlled to stop in the waiting area;

[0069] Event two, the AI pedestrian reaches the end of the pedestrian area, the traffic light displays a yellow light and a green light, and the AI pedestrian is controlled to stop in the waiting area;

[0070] In summary, the above implementation can restore the situation of pedestrians crossing the pedestrian crossing when the user performs driving simulation, develop driving sense, avoid dangerous accidents caused by directly running a red light, and improve the practice effect.

[0071] It is worth noting that the walking characteristics of S3 include walking area, walking route, walking speed, walking interval and walking logic, wherein:

[0072] The walking area is used to determine that the AI pedestrian walks along the pedestrian area;

[0073] The walking route is used to determine the number of walkable routes according to the width of the pedestrian crossing;

[0074] The walking speed is used to set the walking speed and running speed according to different ages,

[0075] and genders. Since different types of pedestrians have different movement characteristics, the speed of the elderly when crossing the pedestrian area is slow, and the speed of the young when crossing the pedestrian area is relatively fast. Therefore, in order to conform to the characteristics of actual pedestrians in AI pedestrians, the walking speed and running speed are set according to different ages and genders, which is more realistic and improves the accuracy of driving simulation;

[0076] The walking interval is used to set the interval between every two AI pedestrians by a random algorithm;

[0077] The behavior logic is used to adjust through an obstacle avoidance algorithm.

[0078] The behavior logic is used to adjust through an obstacle avoidance algorithm.

[0079] Specifically, in order to reflect the randomness of interpersonal interactions in reality and improve the realism of the simulation, the interval randomization algorithm includes the following steps:

[0080] The preset interval factor input ensures that the distance between two AI pedestrians of the same age and gender remains the same. Similarly, if two AI pedestrians of different ages and genders have different walking and running speeds, the distance between the two AI pedestrians will be different while the interval factor remains unchanged, thus achieving randomness in the interval. This is beneficial for conforming to real life and making the visuals of the driving simulation more realistic.

[0081] In real life, pedestrians will automatically avoid oncoming vehicles, but this avoidance process can also cause chain reactions, especially in multi-lane roads, affecting the driving of vehicles in other lanes. In order to simulate the emergency events caused by pedestrian avoidance, it is necessary to further disclose the obstacle avoidance algorithm. The obstacle avoidance algorithm includes the following steps:

[0082] Write a function IsInTriangle() to determine the positional relationship between a point and a triangle. A point P and triangle ABC in the same plane have only two possible positional relationships: the point is outside the triangle or inside the triangle (including the boundary). To determine if point P is inside triangle ABC, three conditions must be met simultaneously: point P and point C are on the same side of line AB; point P and point B are on the same side of line AC; and point P and point A are on the same side of line BC. The determination that two points are on the same side of a line can be described as follows: substitute the two points into the general equation of the line; if the two values ​​have the same sign, then the two points are on the same side of the line.

[0083] If the coordinates of point P are P(x) p y p Point C

[0084] Coordinates are C(x) c y c The general equation of a straight line is ax + by + c = 0. Then, when (ax...) p +by p +c)*(ax c +by c If (+c)≥0 holds, we can determine that point P and point C are on the same side of line AB. Similarly, according to the above general equation, we can determine whether point P and point B are on the same side of line AC, and whether point P and point A are on the same side of line BC. If all three conditions are met, we can determine that point P is inside A4BC; otherwise, point P is outside △ABC.

[0085] Taking AI pedestrian M as an example, irregular shapes are used to represent obstacles (driving vehicles). Obstacles can be represented by a series of points and encapsulated in the Point class.

[0086] AI pedestrian M can be represented by a series of triangles, encapsulated

[0087] In the Triangle class, object collision detection is simplified to the problem of determining whether a point is inside or outside a triangle.

[0088] Suppose the coordinates of the AI pedestrian M are M(x, y), if the return value of IsInTriangle() of the AI pedestrian M is true, collision occurs, and the coordinates of M are reset to M(x+1, y+1); if the return value is false, no collision occurs.

[0089] The second object of the present application is to provide a system for implementing a driving simulator AI pedestrian, comprising the driving simulator AI pedestrian method described above, comprising a simulation area division module, an AI pedestrian model generation module, a walking feature setting module and a traffic light association module.

[0090] The simulation area division module is used to determine the AI pedestrian scene of the driving simulation, and the simulation area is framed and divided.

[0091] The AI pedestrian model generation module is used to generate AI pedestrians in the waiting area through an AI birth algorithm, and motion conversion based on deep learning is applied to the AI pedestrian model.

[0092] The walking feature setting module is used to determine the walking features of the AI pedestrian, and the route and speed of the AI pedestrian are set, which is conducive to meeting the actual different motion effects of different people and improving the authenticity of the driving simulator.

[0093] The traffic light association module is used to establish a traffic light in the overlapping area of the lane area and the pedestrian area, to set the priority rules of the AI pedestrian motion, and to associate the walking features of the AI pedestrian with the traffic light for driving simulation.

[0094] The basic principles, main features and advantages of the present application are shown and described above. Those skilled in the art should understand that the present application is not limited by the above examples, and the above examples and descriptions in the specification are only preferred examples of the present application and are not intended to limit the present application. Without departing from the spirit and scope of the present application, various changes and improvements can be made to the present application, and these changes and improvements all fall within the scope of the claimed present application. The scope of protection of the present application is defined by the appended claims and their equivalents.

Claims

1. A method of driving simulator AI pedestrian, characterized in that: Comprise the following steps: S1, determine the AI pedestrian scene of driving simulation, frame division simulation area, simulation area includes waiting area, lane area, pedestrian area and background area, specifically: Input scene picture, extract model feature point; Build a triangular mesh, generate a three-dimensional mesh model; Texture mapping: get the texture coordinates of the model vertex in the two-dimensional texture space, which is a mapping process from three-dimensional space points to two-dimensional plane points; Provide an interface to realize the editing function of 3D model, adjust the 3D model and show the 3D model of simulation area; S2, generate AI pedestrians in the waiting area through AI birth algorithm, and get motion conversion based on deep learning and apply it to AI pedestrian model; S3, determine the walking characteristics of AI pedestrians, set the route and speed of AI pedestrians, and the walking characteristics include walking area, walking route, walking speed, walking interval and walking logic, wherein: Walking area is used to determine that AI pedestrians walk along the pedestrian area; Walking route is used to determine the number of walkable routes according to the width of the sidewalk; Walking speed is used to set walking speed and running speed according to different ages and genders; Walking interval is used to set the interval between every two AI pedestrians through interval random algorithm; Behavior logic is used to adjust through obstacle avoidance algorithm; S4, establish a traffic light in the overlapping area of lane area and pedestrian area, which is used to set the priority rule of AI pedestrian motion, and associate the walking characteristics of AI pedestrians with the traffic light for driving simulation, the interval random algorithm comprises the following steps: Preset interval factor input, keep the same distance between two AI pedestrians of the same age and gender, and if the walking speed and running speed of two AI pedestrians of different ages and genders are different, the interval between the two AI pedestrians is different under the condition that the interval factor is unchanged, realizing the randomness of interval; The obstacle avoidance algorithm comprises the following steps: Write function IsInTriangle( ) to determine the position relationship of point and triangle; Take AI pedestrian M as an example, use irregular figure to represent obstacle, which can be represented by a series of points and encapsulated into Point class; AI pedestrian M can be represented by a series of triangles and encapsulated into Triangle class, object collision detection is simplified to the problem of judging whether a point is inside or outside a triangle; If the return value of IsInTriangle( ) of AI pedestrian M is true, collision occurs, and the coordinates of M are reset to M(x+1, y+1); if the return value is false, no collision occurs.

2. The method of Claim 1, wherein: The AI birth algorithm in S2 comprises the following steps: Generate AI pedestrian model based on 3D modeling technology; Based on visual capture, human motion is transferred to AI pedestrian model, so that AI pedestrian simulates real pedestrian walking.

3. The method of Claim 1, wherein: The traffic light adopts a simulated traffic light device, including red light, green light and yellow light, which is used to switch on red light and green light for a certain time, and yellow light is used to light up at the critical value of red light and green light switching.

4. The method of Claim 3, wherein: In S4, the traffic light is used as the priority rule during driving simulation, including the following events: Event one, the AI pedestrian reaches the end of the pedestrian area, the traffic light shows green light, control the AI pedestrian to walk along the route to the other end of the pedestrian area; Event two, the AI pedestrian reaches the end of the pedestrian area, the traffic light shows yellow and green light, control the AI pedestrian to stop in the waiting area.

5. A system for implementing a driving simulator AI pedestrian comprising the method of any one of claims 1-4, characterized in that: It comprises a simulation area division module, an AI pedestrian model generation module, a walking feature setting module and a traffic light association module; The simulation area division module is used to determine the AI pedestrian scene of driving simulation, and frame division simulation area; The AI pedestrian model generation module is used to generate AI pedestrians in the waiting area through AI birth algorithm, and to obtain motion conversion based on deep learning and apply it to the AI pedestrian model; The walking feature setting module is used to determine the walking features of AI pedestrians, to set the route and speed of AI pedestrians, and to be conducive to meeting the actual different people to produce different motion effects, and to improve the authenticity of the driving simulator; The traffic light association module is used to establish traffic lights in the overlapping area of the lane area and the pedestrian area, to set the priority rules of AI pedestrian motion, and to associate and depend on the walking features of AI pedestrians and traffic lights for driving simulation.

Images