Training method and device of pedestrian model, electronic equipment and storage medium

Abstract

The present disclosure relates to a pedestrian model training method and device, electronic equipment and storage medium, and belongs to the technical field of autonomous driving. The method comprises: constructing a pedestrian model; obtaining training data of the pedestrian model according to map information of a simulation system and driving information of a target vehicle in the simulation system; wherein the target vehicle is a vehicle that is driving in the simulation system, and the probability of a traffic accident between the target vehicle and the pedestrian model is greater than a preset probability; using the training data to perform reinforcement learning training on the pedestrian model, so that the probability of a traffic accident between the trained pedestrian model and the target vehicle is greater than a preset threshold, and the preset threshold is greater than the preset probability. The present disclosure can simulate various unexpected situations in actual open road scenarios through the pedestrian model, and effectively test autonomous driving vehicles.

Description

Technical Field

[0001] This disclosure relates to the field of autonomous driving technology, and in particular to a method, apparatus, electronic device, and storage medium for training a pedestrian model. Background Technology

[0002] Autonomous vehicles (also known as driverless cars, computer-driven cars, or wheeled mobile robots) rely on the collaborative efforts of artificial intelligence, computer vision, radar, monitoring devices, and GPS to enable computers to operate motor vehicles automatically and safely. In real-world open road scenarios, autonomous vehicles must handle highly complex situations, including vehicles traveling in all directions, pedestrians along various paths, and other road users. Therefore, it is necessary to test various autonomous vehicles to confirm their ability to flexibly respond to unexpected events caused by different road users in real-world open road scenarios and to avoid traffic accidents.

[0003] However, existing technologies lack effective testing schemes to simulate pedestrian models in various unexpected situations on real open roads, in order to test various autonomous vehicles. Summary of the Invention

[0004] To overcome the problems existing in related technologies, this disclosure provides a method, apparatus, electronic device, and storage medium for training pedestrian models.

[0005] According to a first aspect of the present disclosure, a method for training a pedestrian model is provided, comprising: constructing a pedestrian model; obtaining training data for the pedestrian model based on map information of a simulation system and driving information of a target vehicle in the simulation system; wherein the target vehicle is a vehicle currently driving in the simulation system, and the probability of a traffic accident between the target vehicle and the pedestrian model is greater than a preset probability; the driving information includes the position and / or speed of the target vehicle; and using the training data to train the pedestrian model through reinforcement learning, such that the probability of a traffic accident between the trained pedestrian model and the target vehicle is greater than a preset threshold, the preset threshold being greater than the preset probability; wherein the input of the pedestrian model is a state representation of the map information and a state representation of the driving information, and the output of the pedestrian model is walking motion information.

[0006] Optionally, a set of samples in the training data includes state information, action information, and reward information; the state information includes the state representation of the map information of the simulation system and the state representation of the driving information of the target vehicle; or, the state information includes the state representation of the map information of the simulation system; the action information includes the walking action information output by the pedestrian model based on the input state information; the reward information includes the reward value generated by the pedestrian model according to the probability of a traffic accident between the pedestrian model and the target vehicle when the pedestrian model moves towards the target location according to the action information.

[0007] Optionally, the process of generating the reward value includes: obtaining a first value using a first neural network model based on historical movement information generated during multiple training rounds of the pedestrian model and movement information in the current round; wherein the first neural network model is a time-series based neural network model, and the first value is used to indicate the degree of influence of the movement information in the current round on the pedestrian model's involvement in a traffic accident with the target vehicle; obtaining a second value using a second neural network model based on border data in the map information; wherein the border data in the map information includes border data of the pedestrian model and / or border data of the target vehicle, the second neural network model is a convolutional neural network model, and the second value is used to indicate the positional information between the pedestrian model and the target vehicle; obtaining the reward value using a linear transformation algorithm based on the first value and the second value; wherein, compared to when the pedestrian model does not have a traffic accident with the target vehicle, the reward value obtained by the pedestrian model is greater when the pedestrian model has a traffic accident with the target vehicle.

[0008] Optionally, the process of generating the reward value includes: inputting the border data from the map information into a convolutional neural network model, wherein the convolutional neural network model determines whether a traffic accident has occurred between the pedestrian model and the target vehicle based on the border data from the map information; if a traffic accident has occurred between the pedestrian model and the target vehicle, the convolutional neural network model outputs a first reward value; if no traffic accident has occurred between the pedestrian model and the target vehicle, the convolutional neural network model outputs a second reward value; wherein the first reward value is greater than the second reward value, and the second reward value is determined based on the distance and relative position between the pedestrian model and the target vehicle.

[0009] Optionally, the process of generating the action information includes: inputting the state representation of the map information of the simulation system into a convolutional neural network model to obtain the location information in the simulation system where the pedestrian model can be placed; obtaining prior information based on the state representation of the driving information of the target vehicle, the prior information including the area information of the target vehicle along the driving direction; and obtaining the action information based on the location information where the pedestrian model can be placed and the prior information.

[0010] Optionally, after training the pedestrian model using the training data through reinforcement learning, the method further includes: acquiring map information of the simulation system and driving information of the test vehicle; wherein the test vehicle is driving in the simulation system under the control of an autonomous driving model; the driving information includes the position and / or speed of the test vehicle; obtaining the initial position of the trained pedestrian model in the simulation system based on the map information and the driving information; controlling the pedestrian model to move from the initial position to the target location; and determining that the test vehicle has failed the test if the pedestrian model has a traffic accident with the test vehicle during its movement.

[0011] According to a second aspect of the present disclosure, a training apparatus for a pedestrian model is provided, comprising: a model building module for building a pedestrian model; a data acquisition module for obtaining training data of the pedestrian model based on map information of a simulation system and driving information of a target vehicle in the simulation system; wherein the target vehicle is a vehicle currently driving in the simulation system, and the probability of a traffic accident between the target vehicle and the pedestrian model is greater than a preset probability; the driving information includes the position and / or speed of the target vehicle; and a model training module for using the training data to perform reinforcement learning training on the pedestrian model, such that the probability of a traffic accident between the trained pedestrian model and the target vehicle is greater than a preset threshold, the preset threshold being greater than the preset probability; wherein the input of the pedestrian model is a state representation of the map information and a state representation of the driving information, and the output of the pedestrian model is walking motion information.

[0012] According to a third aspect of the present disclosure, an electronic device is provided, including: a processor and a memory storing computer program instructions; the processor, when executing the computer program instructions, implements the pedestrian model training method provided in the first aspect of the present disclosure.

[0013] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided that stores computer program instructions thereon, which, when executed by a processor, implement the pedestrian model training method provided in the first aspect of the present disclosure.

[0014] The technical solutions provided in this disclosure may have the following beneficial effects:

[0015] In the embodiments provided in this disclosure, a pedestrian model is first constructed; then, training data for the pedestrian model is obtained based on the map information and the driving information of the target vehicle in the simulation system; finally, the training data is used to train the pedestrian model through reinforcement learning, so that the probability of the trained pedestrian model causing a traffic accident with the target vehicle is greater than a preset threshold. Since the trained pedestrian model has a high probability of causing a traffic accident with the target vehicle, when an autonomous vehicle is subsequently connected to the simulation system as the target vehicle for testing, the pedestrian model will make walking actions with a high probability of causing a traffic accident with the autonomous vehicle. Thus, various emergencies in real open road scenarios can be simulated through the pedestrian model, and the autonomous vehicle can be effectively tested.

[0016] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0018] Figure 1 This is an application scenario diagram illustrating a pedestrian model training method according to an exemplary embodiment. Figure 2 This is a flowchart illustrating a method for training a pedestrian model according to an exemplary embodiment.

[0019] Figure 3 This is a flowchart illustrating a method for obtaining a reward value for a pedestrian model according to an exemplary embodiment.

[0020] Figure 4 This is a flowchart illustrating yet another method for obtaining a reward value using a pedestrian model, according to an exemplary embodiment.

[0021] Figure 5 This is a flowchart illustrating a method for generating motion information from a pedestrian model according to an exemplary embodiment.

[0022] Figure 6 This is a flowchart illustrating a method for testing an autonomous vehicle according to an exemplary embodiment.

[0023] Figure 7 This is a schematic diagram illustrating a reinforcement learning process according to an exemplary embodiment.

[0024] Figure 8 This is a schematic diagram of a simulation system according to an exemplary embodiment.

[0025] Figure 9 This is a schematic diagram of a network structure for generating motion information from a pedestrian model, according to an exemplary embodiment.

[0026] Figure 10 This is a schematic diagram of a network structure for calculating the reward value of a pedestrian model, according to an exemplary embodiment.

[0027] Figure 11 This is a block diagram illustrating a training apparatus for a pedestrian model according to an exemplary embodiment.

[0028] Figure 12 This is a block diagram illustrating an electronic device 1200 according to an exemplary embodiment. Detailed Implementation

[0029] The exemplary embodiments will now be described in detail with reference to the accompanying drawings.

[0030] It should be noted that the relevant embodiments and accompanying drawings are only for describing and illustrating exemplary embodiments provided by this disclosure, and not all embodiments of this disclosure, nor should this disclosure be understood to be limited to the relevant exemplary embodiments.

[0031] It should be noted that the terms "first," "second," etc., used in this disclosure are only used to distinguish different steps, devices, or modules. These terms do not represent any specific technical meaning, nor do they indicate any order or interdependence between them.

[0032] It should be noted that the term "at least one" as used in this disclosure is illustrative rather than restrictive. Unless otherwise expressly indicated in the context, it should be understood as "one or more".

[0033] It should be noted that the term "and / or" used in this disclosure is used to describe the relationship between related objects, and generally indicates that there are at least three relationships. For example, A and / or B can at least indicate: the existence of A alone, the existence of both A and B, and the existence of B alone.

[0034] It should be noted that the various steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Unless otherwise specified, the scope of this disclosure is not limited by the order in which the steps are described in the relevant embodiments.

[0035] It should be noted that all actions involving the acquisition of signals, information, or data in this disclosure are carried out in compliance with the relevant data protection laws and policies of the country where the location is situated, and with authorization from the owner of the relevant device.

[0036] Figure 1 This is an application scenario diagram illustrating a pedestrian model training method according to an exemplary embodiment.

[0037] like Figure 1 As shown, in the application scenario, it can include server 110, terminal 120 and network 130.

[0038] In some embodiments, the server 110 and the terminal 120 can interact with each other via the network 130. For example, the server 110 can obtain information and / or data from the terminal 120 via the network 130, or it can send information and / or data to the terminal 120 via the network 130.

[0039] Terminal 120 is an electronic device equipped with a simulation system. For example... Figure 8 As shown, the simulation system can simulate real traffic scenarios including various traffic participants such as vehicles and pedestrians. In some embodiments, the terminal 120 can obtain training data from the simulation system (for example, it can collect one or more road window data containing the target vehicle in the simulation system as training data) and send the training data to the server 110 through the network 130.

[0040] Server 110 is used to train a pedestrian model based on training data. This pedestrian model is used to test autonomous vehicles from multiple manufacturers. Server 110 can be a single server or a group of servers. The server group can be centralized or distributed (e.g., server 110 can be a distributed system), and can be dedicated or simultaneously provided by other devices or systems. In some embodiments, server 110 can be regional or remote. In some embodiments, server 110 can be implemented on a cloud platform or provided virtually. By way of example only, the cloud platform can include private cloud, public cloud, hybrid cloud, community cloud, distributed cloud, internal cloud, multi-tiered cloud, etc., or any combination thereof.

[0041] In some embodiments, network 130 can be any one or more wireless networks. For example, network 130 may include a wireless local area network (WLAN), a metropolitan area network (MAN), or any combination thereof.

[0042] For ease of understanding, the technical solutions of this disclosure are described below in conjunction with the accompanying drawings and embodiments.

[0043] yes Figure 2 A flowchart illustrating a method for training a pedestrian model according to an exemplary embodiment is shown. Figure 2 As shown, the training method for the pedestrian model includes the following steps.

[0044] In step S210, a pedestrian model is constructed.

[0045] The pedestrian model is used to test whether autonomous vehicles can cope with various complex situations in real-world traffic scenarios. The pedestrian model provided in this disclosure includes, but is not limited to, human shapes, and may also be cats, dogs, or other animals, and is not limited to the description in this specification.

[0046] The pedestrian model in this embodiment is trained using reinforcement learning (RL). Reinforcement learning describes and solves the problem of an agent learning a strategy to maximize rewards or achieve specific goals during interactions with the environment. Reinforcement learning involves the agent learning through trial and error, using rewards obtained from interactions with the environment to guide behavior, with the goal of maximizing the agent's reward. Common reinforcement learning algorithms include Q-learning, policy gradient, and actor-critic.

[0047] Reinforcement learning primarily comprises five elements: agent, environment, state, action, and reward. The agent's input is its state, and its output is its action. The training process of reinforcement learning involves the agent interacting with the environment to obtain the action, state, and reward for each interaction. This set of actions, states, and rewards serves as training data for the agent in one training iteration. This process is repeated for subsequent training iterations until convergence is achieved.

[0048] In the embodiments provided in this disclosure, such as Figure 7 As shown, the pedestrian model is the agent in the reinforcement learning method, the simulation system is the environment, map information and the target vehicle's driving information serve as state information, and the pedestrian's walking actions serve as action information. In specific implementation, the pedestrian model's input consists of the state representation of the simulation system's map information and the state representation of the target vehicle's driving information; the pedestrian model's output is its walking action information.

[0049] Map information can include road structure information, location information of various traffic participants, location information of target vehicles, location information of static obstacles, and traffic rule information in the scene collected by the simulation system. Road structure information includes, but is not limited to: lane location information, sidewalk location information, and green belt location information. Traffic participants include, but are not limited to: pedestrians and vehicles. Static obstacles include, but are not limited to: trees, stopped vehicles, and stationary pedestrians. Traffic rule information includes, but is not limited to: traffic signs and traffic lights.

[0050] The simulation system contains multiple vehicles. A target vehicle can be selected from these vehicles that are currently in motion and have a certain probability of colliding with a pedestrian model (e.g., a collision between a person and a vehicle). In other words, the target vehicle is a vehicle currently in motion within the simulation system, and the probability of it colliding with the pedestrian model is greater than a preset probability, such as 0%, 50%, or 80%. For example, a vehicle turning west at an intersection has a certain probability of colliding with a pedestrian model traveling from south to north, and therefore can be selected as the target vehicle. The target vehicle's driving information includes various parameters and location information of the target vehicle within the scene collected by the simulation system, such as the target vehicle's position and speed.

[0051] The state representations of map information and driving information are data obtained after processing map information and driving information. In specific implementation, the processing methods for map information and driving information can be determined according to the definition of state in reinforcement learning algorithms, thus obtaining the state representations of map information and driving information. For example, the state representation of map information can be a multi-dimensional vector, using data from different dimensions to represent the different information contained in the map information.

[0052] In step S220, training data for the pedestrian model is obtained based on the map information of the simulation system and the driving information of the target vehicle in the simulation system; wherein, the driving information includes the position and / or speed of the target vehicle.

[0053] As can be seen from step S210 above, the target vehicle is a vehicle currently driving in the simulation system, and the probability of a traffic accident between the target vehicle and the pedestrian model is greater than a preset probability. In some embodiments, multiple scenes containing the target vehicle in the simulation system can be collected to obtain offline map information. For example, image data within the field of view corresponding to the target vehicle in the simulation system can be used as offline collected map information. In some embodiments, the position and corresponding speed of the target vehicle in the offline map information can be used as driving information.

[0054] In some embodiments, a pedestrian model can be integrated into a simulation system to obtain real-time map information and target vehicle driving information from the simulation system as training data. The embodiments provided in this disclosure, by placing a pedestrian model into a simulation system, can obtain complex and varied training data. The trained pedestrian model obtained from this training data is more suitable for testing autonomous vehicles and can output action information with a higher probability of causing a traffic accident with the tested vehicle.

[0055] In the specific implementation process, a set of samples in the training data can be obtained based on a set of map information and the driving information of the target vehicle; among them, the samples include: state information, action information, and reward information.

[0056] In some embodiments, the state information includes a state representation of the map information of the simulation system and a state representation of the driving information of the target vehicle. In some embodiments, the state information may only include a state representation of the map information of the simulation system, and is not limited to the description herein.

[0057] In some embodiments, the action information includes the walking action information output by the pedestrian model based on the input state information. That is, the state information is input into the pedestrian model, the walking action information output by the pedestrian model is obtained, and the walking action information is used as the action information. In some embodiments, the action information can be represented by multi-dimensional position information. For example, the action information of the pedestrian model can include the coordinate information of eight position points of the pedestrian model: up, down, left, right, upper left, lower left, upper right, and lower right. More detailed action information of the pedestrian model can be represented by multi-dimensional position information. For example, the angle formed between the plane formed by the four points of the pedestrian model (upper left, lower left, upper right, and lower right) and the north-south direction of the map coordinate system can be used to represent the turning action of the pedestrian model (e.g., turning 15° to the left). As another example, the distance traveled by the pedestrian model in the north-south direction (the difference between the currently output coordinate value and the previously output coordinate value) can be used to represent the coordinates of the straight line formed by the upper and lower coordinates of the pedestrian model along the north-south direction in the map coordinate system.

[0058] In practice, various reinforcement learning algorithms can be used to implement the network structure of the pedestrian model, without being limited by the description in this specification. For example, the ACKTR (Actor Critic using Kronecker-Factored Trust Region) algorithm can be used to implement the network structure of the pedestrian model. The ACKTR algorithm includes a policy network and a value network. A value network and a policy network containing convolutional layers and fully connected layers can be designed to construct the pedestrian model.

[0059] For an example of how a pedestrian model generates motion information based on state information, please refer to [link to example]. Figure 5 The relevant descriptions will not be repeated here.

[0060] In some embodiments, the reward information includes a reward value generated based on the probability of a traffic accident between the pedestrian model and the target vehicle when the pedestrian model moves towards the target location according to the action information. The higher the probability of a traffic accident, the higher the reward value obtained by the pedestrian model. In specific implementations, various reinforcement learning algorithms can be used to determine the reward value that the pedestrian model can obtain based on the action information output by the pedestrian model. For an embodiment of determining the reward value that the pedestrian model can obtain based on the action information output by the pedestrian model, please refer to [link to implementation details]. Figure 3 and Figure 4 The relevant descriptions will not be repeated here.

[0061] The embodiments provided in this disclosure determine the reward value of the pedestrian model by the probability of a traffic accident between the pedestrian model and the target vehicle. According to the reinforcement learning algorithm, the pedestrian model will select the action information that can obtain the maximum reward value (the reward value is obtained by using the reward function based on the reward value). Therefore, when the trained pedestrian model is used to test the autonomous vehicle in the simulation system, it can make a walking action with a high probability of causing a traffic accident with the test vehicle, thereby effectively testing the sensitivity of the autonomous vehicle.

[0062] In step S230, the pedestrian model is trained using the training data to perform reinforcement learning so that the probability of the trained pedestrian model having a traffic accident with the target vehicle is greater than a preset threshold.

[0063] In practice, the pedestrian model can be trained using multiple sets of samples from the training data, where each set of samples corresponds to a scenario in the simulation system. In practice, such as... Figure 7 As shown, the training process is as follows: First state information is input into the pedestrian model; based on the first action information output by the pedestrian model, the first reward value obtained by the pedestrian model is determined; the second state information of the next set of samples in the training data is used as input to the pedestrian model again, until the model convergence condition is met or the number of training iterations reaches a preset threshold (e.g., 100 times). The model convergence condition can be that the pedestrian model and the target vehicle have a traffic accident. In specific implementation, if the pedestrian model still does not meet the model convergence condition after the number of training iterations reaches the preset threshold, the training can be terminated. Other scenarios in the simulation system and / or other vehicles from the scenarios can be selected as target vehicles. Multiple sets of samples are obtained based on the reselected scenarios and / or target vehicles, and the pedestrian model is trained again until the probability of the trained pedestrian model having a traffic accident with the target vehicle is greater than the preset threshold. For example, if 99 out of 100 training iterations result in the pedestrian model having a traffic accident with the target vehicle, the training is considered complete, and the trained pedestrian model is obtained.

[0064] It should be noted that the target vehicle selected in step S220 may have a certain probability of colliding with the pedestrian model. Training the pedestrian model in step S230 can increase the probability of this collision. For example, if the target vehicle selected in step S220 has a 20% probability of colliding with the pedestrian model, after training in step S230, the probability increases to 90%. In other words, the preset threshold in step S230 is greater than the preset probability in step S220.

[0065] In the embodiments provided in this disclosure, training data for the pedestrian model is obtained based on map information and driving information of the target vehicle in the simulation system. The training data is then used to train the pedestrian model using reinforcement learning, so that the probability of the trained pedestrian model colliding with the target vehicle is greater than a preset threshold. Since the trained pedestrian model has a high probability of colliding with the target vehicle, when an autonomous vehicle is subsequently connected to the simulation system as the target vehicle for testing, the pedestrian model will make walking actions with a high probability of colliding with the autonomous vehicle, thereby effectively testing the sensitivity of the autonomous vehicle.

[0066] Figure 3 This is a flowchart illustrating a method for obtaining a reward value using a pedestrian model according to an exemplary embodiment. Figure 3 As shown, the method includes the following steps.

[0067] In step S310, the first value is obtained by using the first neural network model based on the historical movement information and the current movement information generated by the pedestrian model during multiple training rounds.

[0068] The first type of neural network model can be a time-series based neural network model, including but not limited to: RNN (Recurrent Neural Network) and LSTM (Long Short-Term Memory) models. An RNN provides an output for each time step, combining the input with the current model state. Its purpose is to characterize the relationship between the current output and previous inputs in a sequence. RNNs excel at solving time-series problems. For a sequence of data, data from different time steps can be sequentially fed into the input layer of an RNN. The RNN's output can be a prediction of the next time step in the sequence, or the result of processing the information at the current time step (e.g., speech recognition results). LSTM is an improved model based on RNN. LSTM can not only solve the problem of long-range dependencies that RNN cannot handle, but also solve common problems in neural networks such as gradient explosion or gradient vanishing, making it very effective in processing sequence data.

[0069] In some embodiments, the first neural network model can be an LSTM model, such as... Figure 10 As shown, historical movement information generated during multiple training rounds (e.g., 10 rounds) and the current round's movement information can be used as input data and sequentially fed into the LSTM model. The LSTM model outputs a first value. The outcome of a traffic accident between the pedestrian model and the target vehicle is related to a series of movement information output by the pedestrian model. As the pedestrian model moves from its initial position towards the target location, it can perform actions such as moving forward, choosing a path, and turning based on the movement information, gradually bringing its position closer and closer to the target vehicle's position, increasing the probability of a traffic accident. After multiple training sessions, the LSTM model can determine the impact of the current round's movement information on the outcome of a traffic accident between the pedestrian model and the target vehicle based on historical and current round movement information, and represent the magnitude of this impact numerically, i.e., the first value output by the LSTM model. In other words, in this embodiment, the first value is used to indicate the degree of influence of the current round's movement information on the traffic accident between the pedestrian model and the target vehicle.

[0070] In step S320, the second value is obtained using the second neural network model based on the border data in the map information.

[0071] The border data in the map information includes border data for pedestrian models and / or border data for target vehicles. In some embodiments, the border data can be a two-dimensional bounding box, which includes the position coordinates of the top-left corner, bottom-left corner, top-right corner, and bottom-right corner of the target object (e.g., pedestrian model, target vehicle, etc.).

[0072] The second neural network model can be a convolutional neural network model, such as... Figure 10 As shown, the bounding box data can be input into the convolutional neural network model, which then outputs a second value. After multiple training iterations, the convolutional neural network model can determine the distance and relative position between the pedestrian model and the target vehicle based on the bounding box data in the map information, and quantify this information using the second value. In other words, in this embodiment, the second value is used to indicate the positional information between the pedestrian model and the target vehicle, such as distance and relative position.

[0073] In step S330, the reward value is obtained using a linear transformation algorithm based on the first value and the second value.

[0074] A linear transformation is a transformation or mapping that preserves linear relationships. In some embodiments, the linear transformation algorithm can be in the form of a linear equation in two variables. As an example only, the linear transformation algorithm can be as shown in the following formula (1).

[0075] y = ax1 + bx2 (1)

[0076] In formula (1), x1 is the first value, x2 is the second value, a and b are coefficients obtained through training, and y is the reward value.

[0077] In some embodiments, the linear transformation algorithm can also be a fully connected layer, with the first and second values ​​as inputs to the fully connected layer, and the fully connected layer outputting a reward value. The weight coefficients of the fully connected layer (i.e., a and b in the above formula (1)) can be obtained through training. Figure 10 As shown, the first value output by the LSTM and the second value output by the convolutional neural network model are used as inputs to the fully connected layer, which outputs a reward value.

[0078] In some embodiments, when a pedestrian model is involved in a traffic accident with the target vehicle, a linear transformation algorithm is used to obtain an accident occurrence reward value; when the pedestrian model is not involved in a traffic accident with the target vehicle, a linear transformation algorithm is used to obtain an accident non-occurrence reward value; the accident occurrence reward value is greater than the accident non-occurrence reward value, that is, compared to when the pedestrian model is not involved in a traffic accident with the target vehicle, the pedestrian model obtains a greater reward value when the pedestrian model is involved in a traffic accident with the target vehicle.

[0079] In the embodiments provided in this disclosure, based on the historical movement information and the current movement information generated during multiple training rounds of the pedestrian model, a first neural network model is used to obtain a first value that can accurately represent the impact of the current movement information on the outcome of a traffic accident between the pedestrian model and the target vehicle. Based on the bounding box data in the map information, a second neural network model is used to obtain a second value that reflects information such as the distance and relative position between the pedestrian model and the target vehicle. Based on the first and second values, a linear transformation algorithm is used to obtain a reward value that combines the first and second values. Therefore, this embodiment can achieve the following training objective: the greater the probability of a traffic accident between the pedestrian model and the target vehicle, the greater the reward value obtained.

[0080] Figure 4 This is a flowchart illustrating yet another method for obtaining a reward value using a pedestrian model, according to an exemplary embodiment. For example... Figure 4 As shown, the method includes the following steps.

[0081] Step S410: Input the border data from the map information into the convolutional neural network model.

[0082] In this embodiment, the reward value is determined based on whether the pedestrian model has been involved in a traffic accident with the target vehicle. Regarding the border data in the map information and the processing of the border data by the convolutional neural network model, please refer to the relevant description in step S320, which will not be repeated here.

[0083] Step S420: In the event of a traffic accident between the pedestrian model and the target vehicle, the first reward value is output through the convolutional neural network model.

[0084] The convolutional neural network model can determine whether a pedestrian model and a target vehicle have been involved in a traffic accident by using bounding box data in map information. If so, it outputs a first reward value, which can be a value greater than 0.

[0085] As an example only, if the distance between the pedestrian model and the target vehicle is less than a preset distance threshold (e.g., 5 cm), it can be determined that a traffic accident is about to occur or has already occurred between the pedestrian model and the target vehicle, and the first reward value can be set to a larger value.

[0086] Step S430: If the pedestrian model does not have a traffic accident with the target vehicle, output a second reward value through the convolutional neural network model; wherein the first reward value is greater than the second reward value.

[0087] For the method of determining the reward value based on the distance and relative position between the pedestrian model and the target vehicle, please refer to the description of the method for obtaining the second value in step S320, which will not be repeated here.

[0088] In practice, the second reward value can be determined based on the distance and relative position between the pedestrian model and the target vehicle. As an example, if the distance between the pedestrian model and the target vehicle is greater than a preset distance threshold (e.g., 5 cm), it can be determined that no traffic accident has occurred between the pedestrian model and the target vehicle, and the second reward value can be set to a smaller value.

[0089] Optionally, the first reward value and the second reward value can be pre-set values, with the first reward value being greater than the second reward value. Furthermore, a pre-established correspondence is established between the first reward value and the situation where a traffic accident has occurred, and between the second reward value and the situation where no traffic accident has occurred. Therefore, when processing the border data in the map information using the convolutional neural network model, if the convolutional neural network model determines that a pedestrian model has been involved in a traffic accident with a target vehicle, the convolutional neural network model outputs the first reward value; if the convolutional neural network model determines that the pedestrian model has not been involved in a traffic accident with a target vehicle, the convolutional neural network model outputs the second reward value.

[0090] In the embodiments provided in this disclosure, border data from map information is input into a convolutional neural network model. If a pedestrian model is involved in a traffic accident with a target vehicle, the convolutional neural network model outputs a first reward value; if the pedestrian model is not involved in a traffic accident with a target vehicle, the convolutional neural network model outputs a second reward value. Since the reward value is generated based on whether the pedestrian model is involved in a traffic accident with a target vehicle in this embodiment, the pedestrian model can learn a strategy for causing a traffic accident with a target vehicle in a relatively short time.

[0091] Figure 5 This is a flowchart illustrating a method for generating motion information from a pedestrian model according to an exemplary embodiment. Figure 5 As shown, the method includes the following steps.

[0092] Step S510: Input the state representation of the map information of the simulation system into the convolutional neural network model to obtain the location information of the pedestrian model that can be placed in the simulation system.

[0093] like Figure 9 As shown, the state representation of map information can be input into a convolutional neural network model, and the convolutional neural network model outputs location information.

[0094] After training, the convolutional neural network model can determine the location information where pedestrian models can be placed in the simulation system based on map information. For example, the simulation system includes green belts, sidewalks, and vehicle roads. There may be pedestrians and parked vehicles on the sidewalks. The convolutional neural network model can use the location where there are no other objects on the sidewalk as the location information where pedestrian models can be placed.

[0095] In practical implementation, the location information for placing pedestrian models can include one or more locations where pedestrian models can be placed, and each location corresponds to one or more areas of a preset size. For example, the sidewalk can be divided into multiple areas according to its length and width, and one location for placing a pedestrian model corresponds to one or more areas. Therefore, the location information for placing pedestrian models can include part or all of the sidewalk area.

[0096] It should be understood that Figure 4 The convolutional neural network model used in the illustrated embodiments and Figure 5 The convolutional neural network models used in the illustrated embodiments are trained separately; they can be two independent convolutional neural network models.

[0097] Step S520: Obtain prior information based on the state representation of the target vehicle's driving information; wherein, the prior information includes the area information of the target vehicle along the driving direction.

[0098] like Figure 9 As shown, the state representation of driving information can be input into the prior information calculation model to obtain prior information.

[0099] Prior information can be location information where a traffic accident is highly probable, including information about the area along the target vehicle's direction of travel. This area can be a fan-shaped region formed by the front of the target vehicle along its possible direction of travel. When a pedestrian model is within this region, the probability of a traffic accident with the target vehicle is high. In practice, the size of the fan-shaped region can be determined based on the target vehicle's speed. For example, the location where a traffic accident between the pedestrian model and the target vehicle might occur can be calculated based on the target vehicle's speed and position, and the pedestrian model's speed and position, and the size of the fan-shaped region can then be determined based on this location.

[0100] Step S530: Obtain action information based on the location information and prior information of the placeable pedestrian model.

[0101] In the specific implementation process, the possible movement information of the pedestrian model can be determined based on the location information of the pedestrian model obtained in step S510. For example, if there is pedestrian A to the right of the pedestrian model, the pedestrian model can move forward or turn left. As another example, if there is a temporarily parked vehicle B in front of the pedestrian model and a green belt to the left, the pedestrian model can move forward to the right or turn right.

[0102] In the specific implementation process, the movement of the pedestrian model with the highest probability of causing a traffic accident with the target vehicle can be determined based on the location information of the placeable pedestrian model and prior information, and this can be used as the movement information of the pedestrian model. For example, if the right side of the pedestrian model is a placeable location for the pedestrian model, and it is also the location where the target vehicle is about to turn left (prior information), then "turn right" (for example, it can be represented by the coordinate information of the 8 position points of the pedestrian model, see the relevant description in step S220 for details) can be used as the movement movement of the pedestrian model. As another example, if the front of the pedestrian model is a placeable location for the pedestrian model, and it is also the position in front of the front of the target vehicle going straight (prior information), then "go straight" can be used as the movement movement of the pedestrian model.

[0103] In the embodiments provided in this disclosure, the state representation of the map information of the simulation system is input into a convolutional neural network model to obtain the location information of where pedestrian models can be placed in the simulation system; prior information is obtained based on the state representation of the target vehicle's driving information; and action information is obtained based on the location information and the prior information. Since the location information can determine the location in the simulation system where pedestrians can be placed, and the prior information can determine the location where there is a high probability of a traffic accident with the target vehicle, in this embodiment, the movement action information of pedestrian models that have a high probability of a traffic accident with the target vehicle can be determined based on the location information and the prior information.

[0104] Figure 6 This is a flowchart illustrating a method for testing an autonomous vehicle according to an exemplary embodiment. Figure 6 As shown, the method includes the following steps.

[0105] Step S610: Obtain map information of the simulation system and driving information of the test vehicle; wherein, the test vehicle drives in the simulation system under the control of the autonomous driving model; the driving information includes the position and / or speed of the test vehicle.

[0106] For further information on map and driving information, please refer to the relevant description in step S310, which will not be repeated here.

[0107] Step S620: Based on the map information and driving information, obtain the initial position of the trained pedestrian model in the simulation system.

[0108] The pedestrian model is used to test the vehicle under test. For details on the training process of the pedestrian model, please refer to the above. Figures 2 to 5The relevant descriptions in the embodiments will not be repeated here. In specific implementation, the location information where pedestrian models can be placed in the simulation system can be obtained based on map information; prior information can be obtained based on driving information, where the prior information includes the area information along the driving direction of the test vehicle. For further explanation regarding the location information and prior information of where pedestrian models can be placed, please refer to [link to relevant documentation]. Figure 5 The relevant descriptions in the illustrated embodiments will not be repeated here.

[0109] In the specific implementation process, a location that is also within the area information along the driving direction of the test vehicle can be selected from the location information where the pedestrian model can be placed, and used as the initial position of the pedestrian model in the simulation system. If multiple locations exist that simultaneously belong to both the location information where the pedestrian model can be placed and the area information along the driving direction of the test vehicle, then a location can be randomly selected from these multiple locations as the initial position of the pedestrian model, or the location closest to the test vehicle can be selected from these multiple locations as the initial position of the pedestrian model. This embodiment does not limit the specific implementation process.

[0110] Step S630: Control the pedestrian model to move from the initial position to the target location. If the pedestrian model has a traffic accident with the test vehicle during the movement, determine that the test vehicle has failed.

[0111] The target location is determined autonomously by the trained pedestrian model. (Based on the above...) Figures 2 to 5 The trained pedestrian model in this embodiment can autonomously determine its movement information based on the location information of the placeable pedestrian model in the simulation system (obtained by the state representation of the convolutional neural network model based on the map information of the simulation system) and the area information of the test vehicle along the direction of travel. This movement information includes, but is not limited to, at least one of the following: target location, movement type, movement direction, movement speed, etc. After determining the movement information, the trained pedestrian model moves from its initial position towards the target location according to that movement information.

[0112] Traffic accidents include, but are not limited to: collisions between pedestrian models and the test vehicle, and pedestrian models being run over by the test vehicle.

[0113] In practical implementation, if the test vehicle is confirmed to be successful in all predetermined number of tests (e.g., 100 tests), meaning that the pedestrian model does not cause a traffic accident with the test vehicle during its movement, it can be concluded that the sensitivity of the autonomous driving model used to control the test vehicle is high enough to pass the autonomous driving sensitivity and safety tests. Alternatively, if the test vehicle is confirmed to be successful in most cases (e.g., 90% of the predetermined number of tests, 90 tests) and fails in a small number of cases (e.g., 10% of the predetermined number of tests, 10 tests), it can be concluded that the sensitivity of the autonomous driving model used to control the test vehicle is high enough to pass the autonomous driving sensitivity and safety tests.

[0114] In the embodiments provided in this disclosure, map information of the simulation system and driving information of the test vehicle are obtained; based on the map information and driving information, the initial position of the trained pedestrian model in the simulation system is obtained, so that the pedestrian model can be placed in a position where there is a high probability of a traffic accident with the test vehicle based on the initial position; the pedestrian model is controlled to move from the initial position to the target location, and if a traffic accident occurs between the pedestrian model and the test vehicle during the movement, the test vehicle is determined to have failed. This helps to effectively test the sensitivity and safety of the autonomous driving model controlling the test vehicle.

[0115] Exemplary device

[0116] Figure 11 This is a block diagram illustrating a pedestrian model training apparatus according to an exemplary embodiment. (Refer to...) Figure 11 The device 1100 includes: a model building module 1110, a data acquisition module 1120, and a model training module 1130.

[0117] Model building module 1110 is used to build pedestrian models.

[0118] The data acquisition module 1120 is used to obtain training data for the pedestrian model based on the map information of the simulation system and the driving information of the target vehicle in the simulation system; wherein, the target vehicle is a vehicle that is currently driving in the simulation system, and the probability of the target vehicle and the pedestrian model having a traffic accident is greater than a preset probability; the driving information includes the position and / or speed of the target vehicle.

[0119] The model training module 1130 is used to train the pedestrian model using the training data to perform reinforcement learning, so that the probability of the trained pedestrian model having a traffic accident with the target vehicle is greater than a preset threshold, and the preset threshold is greater than the preset probability; wherein, the input of the pedestrian model is the state representation of the map information and the state representation of the driving information, and the output of the pedestrian model is the walking action information.

[0120] Optionally, a set of samples in the training data includes state information, action information, and reward information; the state information includes the state representation of the map information of the simulation system and the state representation of the driving information of the target vehicle; or, the state information includes the state representation of the map information of the simulation system; the action information includes the walking action information output by the pedestrian model based on the input state information; the reward information includes the reward value generated by the pedestrian model according to the probability of a traffic accident between the pedestrian model and the target vehicle when the pedestrian model moves towards the target location according to the action information.

[0121] Optionally, the device 1100 is further configured to: obtain a first value using a first neural network model based on historical movement information generated during multiple training rounds of the pedestrian model and movement information of the current round; wherein the first neural network model is a time-series based neural network model, and the first value is used to indicate the degree of influence of the movement information of the current round on the pedestrian model's involvement in a traffic accident with the target vehicle; obtain a second value using a second neural network model based on border data in the map information; wherein the border data in the map information includes border data of the pedestrian model and / or border data of the target vehicle, the second neural network model is a convolutional neural network model, and the second value is used to indicate the positional information between the pedestrian model and the target vehicle; obtain the reward value using a linear transformation algorithm based on the first value and the second value; wherein, compared to when the pedestrian model does not have a traffic accident with the target vehicle, the reward value obtained by the pedestrian model is greater when the pedestrian model has a traffic accident with the target vehicle.

[0122] Optionally, the device 1100 is further configured to: input the border data from the map information into a convolutional neural network model, wherein the convolutional neural network model determines whether a traffic accident has occurred between the pedestrian model and the target vehicle based on the border data from the map information; if a traffic accident has occurred between the pedestrian model and the target vehicle, output a first reward value through the convolutional neural network model; if a traffic accident has not occurred between the pedestrian model and the target vehicle, output a second reward value through the convolutional neural network model; wherein the first reward value is greater than the second reward value, and the second reward value is determined based on the distance and relative position between the pedestrian model and the target vehicle.

[0123] Optionally, the device 1100 is further configured to: input the state representation of the map information of the simulation system into a convolutional neural network model to obtain the location information in the simulation system where the pedestrian model can be placed; obtain prior information based on the state representation of the driving information of the target vehicle, the prior information including the area information of the target vehicle along the driving direction; and obtain the action information based on the location information where the pedestrian model can be placed and the prior information.

[0124] Optionally, the device 1100 is further configured to: acquire map information of the simulation system and driving information of the test vehicle; wherein the test vehicle drives in the simulation system under the control of an autonomous driving model; the driving information includes the position and / or speed of the test vehicle; obtain the initial position of the trained pedestrian model in the simulation system based on the map information and the driving information; control the pedestrian model to move from the initial position to the target location; and determine that the test vehicle has failed if the pedestrian model has a traffic accident with the test vehicle during its movement.

[0125] The specific processing and technical effects of each embodiment of the above-mentioned pedestrian model training device can be found in the relevant descriptions in the corresponding method embodiments, and will not be repeated here.

[0126] Exemplary electronic devices

[0127] Figure 12 This is a block diagram illustrating an electronic device 1200 according to an exemplary embodiment. The electronic device 1200 may be a computer device, a laptop computer, a server, a vehicle controller, an in-vehicle terminal, an in-vehicle computer, or other types of electronic devices.

[0128] Reference Figure 12The electronic device 1200 may include at least one processor 1210 and a memory 1220. The processor 1210 can execute instructions stored in the memory 1220. The processor 1210 is communicatively connected to the memory 1220 via a data bus. In addition to the memory 1220, the processor 1210 can also be communicatively connected to an input device 1230, an output device 1240, and a communication device 1250 via the data bus.

[0129] Processor 1210 can be any conventional processor. Processors may include central processing units (CPUs), graphics processing units (GPUs), field-programmable gate arrays (FPGAs), systems on chips (SoCs), application-specific integrated circuits (ASICs), or combinations thereof.

[0130] The memory 1220 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0131] In this embodiment of the present disclosure, the memory 1220 stores executable instructions, and the processor 1210 can read the executable instructions from the memory 1220 and execute the instructions to implement all or part of the steps of the pedestrian model training method in the exemplary embodiment described above.

[0132] Exemplary computer-readable storage media

[0133] In addition to the methods and apparatus described above, exemplary embodiments of this disclosure also include a computer program product or a computer-readable storage medium storing the computer program product. The computer product includes computer program instructions that can be executed by a processor to perform all or part of the steps described in the exemplary embodiments above.

[0134] Computer program products can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this application. These programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages, and scripting languages ​​(e.g., Python). The program code can be executed entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0135] Computer-readable storage media may take the form of any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may, for example, include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of readable storage media include: static random access memory (SRAM) having one or more electrically connected wires, electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk, or any suitable combination thereof.

[0136] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of this disclosure. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope.

Claims

1. A method for training a pedestrian model, characterized in that, The method comprises: constructing a pedestrian model; obtaining training data of the pedestrian model according to map information of a simulation system and driving information of a target vehicle in the simulation system; wherein the target vehicle is a vehicle that is driving in the simulation system, and a probability of the target vehicle having a traffic accident with the pedestrian model is greater than a preset probability; the driving information comprises a position and / or a speed of the target vehicle; training the pedestrian model by reinforcement learning using the training data, so that a probability of the trained pedestrian model having a traffic accident with the target vehicle is greater than a preset threshold, the preset threshold being greater than the preset probability; wherein an input of the pedestrian model is a state representation of the map information and a state representation of the driving information, and an output of the pedestrian model is travel action information; wherein a set of samples in the training data comprises state information, action information, and reward information; the state information comprises a state representation of the map information of the simulation system and a state representation of the driving information of the target vehicle; or the state information comprises a state representation of the map information of the simulation system; the action information comprises travel action information output by the pedestrian model based on the input state information; the reward information comprises a reward value generated by the pedestrian model according to a probability of the pedestrian model having a traffic accident with the target vehicle when the pedestrian model travels to a target location according to the action information; the reward value generation process comprises: obtaining a first value using a first neural network model according to historical travel action information generated by the pedestrian model in multiple rounds of training and travel action information of a current round; wherein the first neural network model is a time-based neural network model, and the first value is used to indicate an influence degree of the travel action information of the current round on the pedestrian model having a traffic accident with the target vehicle; obtaining a second value using a second neural network model according to bounding box data in the map information; wherein the bounding box data in the map information comprises bounding box data of the pedestrian model and / or bounding box data of the target vehicle, the second neural network model is a convolutional neural network model, and the second value is used to indicate position information between the pedestrian model and the target vehicle; obtaining the reward value using a linear transformation algorithm according to the first value and the second value; wherein the reward value obtained by the pedestrian model is greater in the case of the pedestrian model having a traffic accident with the target vehicle than in the case of the pedestrian model not having a traffic accident with the target vehicle; or, inputting the bounding box data in the map information into a convolutional neural network model, the convolutional neural network model determining whether the pedestrian model has a traffic accident with the target vehicle through the bounding box data in the map information; outputting a first reward value through the convolutional neural network model in the case of the pedestrian model having a traffic accident with the target vehicle; In a case where the pedestrian model does not have a traffic accident with the target vehicle, a second reward value is output by the convolutional neural network model; wherein the first reward value is greater than the second reward value, and the second reward value is determined according to a distance and a relative position between the pedestrian model and the target vehicle. 2.The training method of the pedestrian model according to claim 1, characterized in that, The generation process of the action information includes: inputting state representation of map information of the simulation system into a convolutional neural network model to obtain position information of a position where the pedestrian model can be placed in the simulation system; obtaining prior information according to state representation of driving information of the target vehicle, the prior information including area information of the target vehicle along a driving direction; obtaining the action information according to the position information where the pedestrian model can be placed and the prior information.

3. The training method of a pedestrian model according to claim 1, wherein, After the training of the pedestrian model by reinforcement learning using the training data, the method further includes: obtaining map information of the simulation system and driving information of a tested vehicle; wherein the tested vehicle drives in the simulation system under control of an automatic driving model; the driving information includes position and / or speed of the tested vehicle; obtaining an initial position of the trained pedestrian model in the simulation system according to the map information and the driving information; controlling the pedestrian model to travel from the initial position to a target location, and determining that the tested vehicle fails in the test in a case where the pedestrian model has a traffic accident with the tested vehicle during the travel.

4. A device for training a pedestrian model, characterized by, includes: a model construction module configured to construct a pedestrian model; a data acquisition module configured to obtain training data of the pedestrian model according to map information of a simulation system and driving information of a target vehicle in the simulation system; wherein the target vehicle is a vehicle driving in the simulation system, and a probability of the target vehicle having a traffic accident with the pedestrian model is greater than a preset probability; the driving information includes position and / or speed of the target vehicle; a model training module configured to train the pedestrian model by reinforcement learning using the training data, so that a probability of the trained pedestrian model having a traffic accident with the target vehicle is greater than a preset threshold value, the preset threshold value being greater than the preset probability; wherein input of the pedestrian model is state representation of the map information and state representation of the driving information, and output of the pedestrian model is travel action information; wherein a group of samples in the training data includes state information, action information, and reward information; the state information includes state representation of the map information of the simulation system and state representation of the driving information of the target vehicle; or the state information includes state representation of the map information of the simulation system; the action information includes travel action information output by the pedestrian model based on the input state information; the reward information includes a reward value generated according to a probability of the pedestrian model having a traffic accident with the target vehicle when the pedestrian model travels to a target location according to the action information; the generation process of the reward value includes: According to historical travel action information generated in a plurality of rounds of training processes of the pedestrian model and travel action information of a current round, a first value is obtained by using a first neural network model; wherein the first neural network model is a time series-based neural network model, and the first value is used to indicate an influence degree of the travel action information of the current round on a traffic accident between the pedestrian model and the target vehicle; According to the bounding box data in the map information, a second value is obtained by using a second neural network model; wherein the bounding box data in the map information includes bounding box data of the pedestrian model and / or bounding box data of the target vehicle, the second neural network model is a convolutional neural network model, and the second value is used to indicate position information between the pedestrian model and the target vehicle; According to the first value and the second value, a reward value is obtained by using a linear transformation algorithm; wherein the reward value obtained by the pedestrian model is larger in the case of a traffic accident between the pedestrian model and the target vehicle than in the case of no traffic accident between the pedestrian model and the target vehicle; Or, The bounding box data in the map information is input into a convolutional neural network model, and the convolutional neural network model determines whether a traffic accident occurs between the pedestrian model and the target vehicle through the bounding box data in the map information; In the case of a traffic accident between the pedestrian model and the target vehicle, a first reward value is output by the convolutional neural network model; In the case of no traffic accident between the pedestrian model and the target vehicle, a second reward value is output by the convolutional neural network model; wherein the first reward value is greater than the second reward value, and the second reward value is determined according to a distance and a relative position between the pedestrian model and the target vehicle.

5. An electronic device, comprising: Comprise: A processor and a memory storing computer program instructions; The processor executes the computer program instructions to implement the training method of the pedestrian model in any one of claims 1-3.

6. A computer-readable storage medium having stored thereon computer program instructions, wherein, The computer program instructions are executed by the processor to implement the training method of the pedestrian model in any one of claims 1-3.

Images