Information processing device, information processing method, and information processing program

Abstract

To provide an information processing device which is utilized for estimating an action of a pedestrian on a road, an information processing method and an information processing program.SOLUTION: An information processing device comprises: an identification unit for identifying one mobile visually recognized by a target person of simulation among a plurality of mobiles to be attention targets of the target person; a first individual estimation unit for estimating an action that the target person takes in accordance with the mobile identified by the identification unit; a second individual estimation unit for estimating an action that the target person takes in accordance with the other mobile different from the mobile identified by the identification unit; and an integral estimation unit for estimating actions that the target person takes in accordance with all the plurality of mobiles based on the action of the target person estimated by the first individual estimation unit and the action of the target person estimated by the second individual estimation unit.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to an information processing device, an information processing method, and an information processing program. [Background technology]

[0002] Conventionally, there have been devices that perform traffic simulations according to road conditions. The simulation devices, for example, reproduce vehicle operations on a computer to perform simulations of traffic flow and congestion. Patent document 1 describes a simulation device that virtually drives a vehicle model along a road, and when the vehicle model recognizes an object of attention that it should pay attention to, derives a selectable action from among a number of predetermined action rules, and operates the vehicle model based on that action. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2008-171358 A Summary of the Invention [Problem to be solved by the invention]

[0004] Not only vehicles but also pedestrians exist on roads. In recent years, it has become desirable to simulate the behavior of pedestrians moving on roads and predict road conditions. When multiple vehicles are moving on a road, a simulation is performed on the behavior of pedestrians depending on the movement conditions of the multiple vehicles. However, it is desirable to perform a simulation close to that of real people by considering the influence of vehicles that are not visible to pedestrians on pedestrians by taking into account whether or not pedestrians recognize vehicles.

[0005] The present disclosure provides an information processing device, an information processing method, and an information processing program used to estimate the behavior of traffic participants on a road. [Means for solving the problem]

[0006] An information processing device of one embodiment includes an identification unit that identifies one moving body that is visually recognized by the subject of a simulation among multiple moving bodies that are the focus of the subject's attention, a first individual estimation unit that estimates a behavior that the subject will take in response to the moving body identified by the identification unit, a second individual estimation unit that estimates a behavior that the subject will take in response to other moving bodies different from the moving body identified by the identification unit, and an integrated estimation unit that estimates a behavior that the subject will take in response to all of the multiple moving bodies based on the behavior of the subject estimated by the first individual estimation unit and the behavior of the subject estimated by the second individual estimation unit. Effect of the Invention

[0007] According to one aspect, by taking into account whether or not the subject of the simulation can see a moving object, it is possible to take into account the effect on the subject of a moving object that is not seen by the subject, thereby performing a simulation that is closer to that of a real person. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating an information processing device according to an embodiment. [Diagram 2] FIG. 1 is a block diagram illustrating an information processing device according to an embodiment. [Diagram 3] FIG. 13 is a diagram for explaining an example of a forgetting function. [Figure 4] FIG. 13 is a diagram for explaining an example of a simulation situation. [Diagram 5] FIG. 11 is a diagram for explaining an example of a judgment model. [Figure 6] 10 is a diagram for explaining an example of coefficients used in an integrated estimation unit (control unit). FIG. [Figure 7] 1 is a flowchart illustrating an information processing method according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] An embodiment will be described below.

[0010] [Overview of information processing device 100] First, an overview of an information processing device 100 according to an embodiment will be described. FIG. 1 is a diagram illustrating an information processing device 100 according to an embodiment.

[0011] The information processing device 100 may be configured, for example, as a judgment and estimation device that estimates the judgment of a pedestrian walking on a road. Also, the information processing device 100 may be configured, for example, as a judgment and estimation model providing device that provides a judgment and estimation model of the pedestrian.

[0012] Generally, pedestrians on a road may make decisions about passing depending on the situation of moving objects such as cars and motorcycles. For example, when a pedestrian crosses a road, he or she often decides whether or not to cross the road depending on the position and speed of the moving object. In this case, when multiple moving objects are traveling, it may be difficult for a pedestrian to see two or more objects at the same time. As illustrated in FIG. 1, when moving bodies 302 approach pedestrian 301 from both the right and left directions, it is difficult for pedestrian 301 to see both bodies at the same time, and it is likely that pedestrian 301 will alternate between looking at one moving body 302 and the other moving body 302 to confirm that it is safe before crossing the road.

[0013] The information processing device 100, for example, identifies one of a plurality of moving bodies 302 visually recognized by a pedestrian 301 who is a subject of a simulation. The information processing device 100 estimates the behavior of the pedestrian 301 in response to the identified moving body 302 (for example, behavior of whether or not to cross a road, etc.) (first individual estimation). In addition, the information processing device 100 estimates the behavior of the pedestrian 301 in response to another moving body 302 different from the identified moving body 302 as described above (for example, behavior of whether or not to cross a road, etc.) (second individual estimation). The information processing device 100 integrates the first individual estimation and the second individual estimation to estimate the behavior of the subject in response to all of the plurality of moving bodies 302 (for example, behavior of whether or not to cross a road, etc.) (integrated estimation). The information processing device 100 prepares a plurality of types of models according to the relationship between the pedestrian 301 and one moving body 302 as the integrated estimation, and can handle complex situations by combining them. In this case, the information processing device 100 may estimate the above-mentioned behavior by estimating the decision of the pedestrian 301 (target person) as to whether or not to perform the behavior. The information processing device 100 may be a computer such as a server, a desktop, a laptop, or a tablet.

[0014] [Details of the information processing device 100] Next, the information processing device 100 according to an embodiment will be described in detail. FIG. 2 is a block diagram illustrating the information processing device 100 according to an embodiment.

[0015] The information processing device 100 includes a communication unit 121, a storage unit 122, a display unit 123, and a control unit 110. The communication unit 121, the storage unit 122, and the display unit 123 may be an embodiment of an "output unit". The control unit 110 includes, for example, an identification unit 111, a first individual estimation unit 112, a second individual estimation unit 113, an integrated estimation unit 114, and an output control unit 115. The control unit 110 may be configured, for example, by an arithmetic processing device of the information processing device 100. The control unit 110 (for example, an arithmetic processing device, etc.) may realize the functions of each unit (for example, the identification unit 111, the first individual estimation unit 112, the second individual estimation unit 113, the integrated estimation unit 114, and the output control unit 115, etc.) by, for example, appropriately reading and executing various programs, etc. stored in the storage unit 122, etc. Here, the first individual estimation unit 112 and the second individual estimation unit 113 may be realized as different functions, or may be realized as a single function, i.e., an "individual estimation unit" that combines the first individual estimation unit 112 and the second individual estimation unit 113.

[0016] The communication unit 121 is capable of transmitting and receiving various information to and from, for example, a device (external device) outside the information processing device 100. The external device may be, for example, a server 200 (see FIG. 1) and a user terminal (not shown). The user terminal may be a terminal used by a user of the information processing device 100.

[0017] The storage unit 122 may store, for example, various information and programs. Examples of the storage unit 122 may be a memory, a solid state drive, a hard disk drive, or the like.

[0018] The display unit 123 is capable of displaying, for example, various characters, symbols, images, and the like.

[0019] The identification unit 111 identifies one moving body that is visually recognized by the subject of the simulation among multiple moving bodies that are the focus of the subject. The identification unit 111 may identify a vehicle as a moving body that is a focus of attention when estimating the behavior of the subject as to whether or not the subject will cross a road when the vehicle travels on the road. The identification unit 111 identifies one moving body in the situation in which the simulation is performed, that is, the subject (e.g., a pedestrian, etc.) and the number and running conditions (e.g., positions, running speeds, accelerations, etc. relative to the subject) of moving bodies (e.g., vehicles and two-wheeled vehicles, etc.) traveling around the subject. In this case, when there are multiple moving bodies (moving bodies of interest), the identification unit 111 identifies one of the multiple moving bodies. The one moving body identified by the identification unit 111 is identified as a moving body visually recognized by the subject.

[0020] The first individual estimation unit 112 estimates the behavior of the subject according to the moving object identified by the identification unit 111. That is, the first individual estimation unit 112 estimates the behavior (judgment) of the subject (pedestrian) whether or not to cross the road according to the running condition (moving condition) of the moving object identified by the identification unit 111. It is considered that in a situation where the subject visually recognizes a moving object, the subject judges whether or not to cross the road according to the position of the moving object (for example, the position relative to the subject, etc.), the running speed of the moving object, the acceleration of the moving object, etc., and takes the behavior according to the judgment result.

[0021] In this case, the first individual estimation unit 112 may estimate the behavior of the subject (the subject's judgment) based on the learned model and the traveling situation (moving state) of the moving body identified by the identification unit 111. The above-mentioned learned model is generated, for example, by learning the traveling situation (moving state) of the moving body and the behavior of the subject (the subject's judgment) taken by the subject according to the moving state of the moving body. The learned model may be generated, for example, by the control unit 110 of the information processing device 100 (for example, a learning unit, etc. (not shown)), or may be generated by a learning device (for example, the server 200) outside the information processing device 100.

[0022] For example, when the learned model is generated outside the information processing device 100, the first individual estimation unit 112 may acquire the learned model via the communication unit 121, or may acquire the learned model using an external memory or the like (not shown). The control unit 110 (for example, a learning unit, etc.) and the learning device may, for example, reproduce the movement status (driving status) of a vehicle in a vehicle simulation, and perform learning based on acquiring pending judgments of whether or not a person will cross a road and whether or not to cross depending on the movement status (driving status) of the vehicle, thereby generating a learned model.

[0023] That is, the first individual estimation unit 112 may use a trained model generated by learning the vehicle position and running speed as the moving state (running condition) of one moving body, and the target's behavior (judgment) of whether the target crosses the road, does not cross the road, and is undecided whether to cross the road according to the moving state (running state) of the vehicle as the target's behavior. Also, the first individual estimation unit 112 may use a plurality of trained models generated for the relationship between one target (pedestrian) and one moving body. Therefore, the first individual estimation unit 112 may calculate, for example, the probability that the target crosses the road, that the target does not cross the road, and that the target is undecided whether to cross the road as the target's behavior (judgment).

[0024] The second individual estimation unit 113 estimates the behavior of the subject in response to other moving bodies different from the moving body identified by the identification unit 111. When there are multiple other moving bodies, the second individual estimation unit 113 may estimate the behavior of the subject in response to each of the multiple other moving bodies. The other moving bodies are, for example, moving bodies that are not currently visually recognized by the subject (pedestrian). The second individual estimation unit 113 estimates the behavior (judgment) of the subject (pedestrian) as to whether or not to cross a road in response to the running situation (movement situation) of the other moving bodies not identified by the identification unit 111. The second individual estimation unit 113 may estimate the behavior (judgment) of the subject (pedestrian) in response to the other moving bodies by using a learned model, similar to the above-mentioned first individual estimation unit 112.

[0025] That is, the second individual estimation unit 113 may estimate the behavior (judgment of the subject) taken by the subject based on the learned model and the moving state (running state) of another moving body not identified by the identification unit 111. The learned model may be similar to the learned model used by the first individual estimation unit 112, and is generated by learning the moving state (running state) of the moving body (other moving body) and the behavior (judgment) taken by the subject according to the moving state (running state) of the moving body (other moving body). The learned model may be generated, for example, by the control unit 110 (for example, a learning unit, etc. (not shown)) and a learning device, etc. (for example, the server 200, etc.).

[0026] In other words, the second individual estimation unit 113 may use a trained model generated by learning the vehicle position and running speed as the moving state (running state) of the moving body (other moving body) and the behavior of the target person, that is, whether the target person crosses the road, does not cross the road, and is undecided whether to cross the road, depending on the moving state of the vehicle, as the behavior taken by the target person. The second individual estimation unit 113 may also use a plurality of trained models generated for the relationship between one target person (pedestrian) and one moving body. Therefore, the second individual estimation unit 113 may calculate, for example, the probability that the target person crosses the road, that the target person does not cross the road, and that the target person is undecided whether to cross the road, as the behavior (judgment) of the target person.

[0027] In the above-mentioned case, the second individual estimation unit 113 may estimate the behavior of the subject in response to the other moving object by using the time when the subject last paid attention to the other moving object as a reference, decreasing the probability of the behavior of the subject in response to a forgetting function that indicates that the subject forgets to pay attention to the other moving object, and increasing the probability that the behavior of the subject is undecided. The forgetting function is, for example, a function that reduces (reduces the probability) the weight of the behavior of the subject (the subject's judgment) in response to the elapsed time since the subject last viewed the moving object, and increases (increases the probability) the weight of the behavior of the subject being undecided. In other words, the forgetting function is, for example, a function that takes over the judgment when the subject last viewed the moving object, and decreases the probability of crossing the road and the probability of not crossing the road as time passes, and increases the probability of undecided whether to cross the road or not, as time passes. The second individual estimation unit 113 estimates the behavior of the subject by, for example, continuing the judgment of when the subject last saw the vehicle, reducing the probability of the judgment of when the subject last saw the vehicle based on a forgetting function, and increasing the probability that the behavior of the subject is undecided.

[0028] FIG. 3 is a diagram for explaining an example of the forgetting function.

[0029] As shown in FIG. 3 as an example, the forgetting function may be a function that maintains the weighting (probability) (i.e., the probability of judgment) for a certain time (for example, 1 second) from when the subject last saw the moving object, and then decreases the weighting (probability) (i.e., the probability of judgment). The above-mentioned certain time is not limited to 1 second, and may be various arbitrary times such as 0.5 seconds, 1.5 seconds, 2 seconds, 3 seconds, 4 seconds, 5 seconds, 8 seconds, and 10 seconds. The rate of decrease of the weighting (probability) may be, for example, 5% decrease every 0.1 seconds, or may be a certain rate such as 1%, 2%, 3%, 8%, or 10% every 0.1 seconds. The rate of decrease is not limited to the above example, and various rates can be set. The rate of decrease of the weighting (probability) is not limited to a monotonous decrease, but may be gradually decreased in multiple steps, may be decreased exponentially, or may be decreased inversely proportional to time. Although not shown, in addition to the example of decrease shown in FIG. 3, the forgetting function also includes the case of increase in the undetermined probability described above. Note that the second individual estimation unit 113 may not set a forgetting function depending on the embodiment, and is not limited to the above-mentioned example. That is, the second individual estimation unit 113 may maintain the weighting (probability) when the subject (pedestrian) last saw the moving object while the subject does not visually recognize the moving object.

[0030] The integrated estimation unit 114 estimates the behavior of the subject to be taken in response to all of the multiple moving objects, based on the behavior (judgment) of the subject to be estimated by the first individual estimation unit 112 and the behavior of the subject to be estimated by the second individual estimation unit 113. As an example, the integrated estimation unit 114 may multiply the probability of the behavior (judgment) of the subject to be estimated by the first individual estimation unit 112 and the probability of the behavior (judgment) of the subject to be estimated by the second individual estimation unit 113, and estimate the behavior (judgment) of the subject to be most likely as the behavior (judgment) of the subject to be taken in response to all of the multiple moving objects.

[0031] In this case, the integrated estimation unit 114 may multiply a value (probability) indicating the subject's behavior estimated by the first individual estimation unit 112 and the second individual estimation unit 113 by a coefficient for integration processing that is set in advance according to the subject's behavior, and integrate the multiplication results to estimate the behavior taken by the subject according to all of the multiple moving objects. As an example, the coefficient may be a value such that, for example, when the highest probability estimated by the first individual estimation unit 112 is "crossing the road" and the highest probability estimated by the second individual estimation unit 113 is "crossing the road," the integrated result (accumulated result) is the highest value for "crossing the road." Similarly, the coefficient may be a value such that, for example, when the highest probability estimated by the first individual estimation unit 112 is "not crossing the road" (or "undetermined") and the highest probability estimated by the second individual estimation unit 113 is "not crossing the road" (or "undetermined"), the integrated result (accumulated result) is the highest value for "not crossing the road" (or "undetermined"). The coefficient may be a value that makes the overall estimation result estimated by the integrated estimation unit 114 similar to the intention observed (observation based on observation information) by using the result of judgment (observation information) of whether or not a pedestrian will cross the road depending on the vehicle driving situation reproduced in the vehicle simulation. This coefficient (value) may be set based on experiments or the like conducted in advance.

[0032] Here, a specific example will be described. FIG. 4 is a diagram for explaining an example of a simulation situation. FIG. 5 is a diagram for explaining an example of a determination model. FIG. 6 is a diagram for explaining an example of coefficients used in the integrated estimation unit 114 (control unit 110).

[0033] Here, as shown in an example in Figure 4, we will explain the case where a subject (pedestrian) crosses an intersection at a crossroads without traffic lights, where a straight-moving vehicle S is traveling from the right side of Figure 4 and a right-turning vehicle R is traveling from the bottom of Figure 4.

[0034] The information processing device 100 (control unit 110) assumes that a pedestrian cannot observe all vehicles at once, and generates a model (gaze switching model) in which the pedestrian alternates his / her gaze to see vehicles, and judges whether or not to cross the road or whether it is undecided depending on the result. The gaze switching model can be realized by, for example, a dynamic Bayesian network. The dynamic Bayesian network may be, for example, a model used to describe the time evolution of probability such as a Bayesian network.

[0035] The information processing device 100 (control unit 110) constructs a model for judging whether a pedestrian should cross the road, depending on a straight-moving vehicle S coming from the left side of the pedestrian and a right-turning vehicle R coming from in front of the pedestrian as the pedestrian's judgment targets. In this case, the information processing device 100 (control unit 110) breaks down the overall judgment (judgment on two vehicles S and R) into individual judgments by identifying the vehicle that the pedestrian pays attention to. For example, the information processing device 100 (control unit 110) estimates, as the individual judgments, the probability of G (Go) crossing the road, W (Wait) not crossing the road, and U (Undecided) as to whether or not to cross the road. The information processing device 100 (control unit 110) may use, for example, a model of individual judgment using a logistic regression model, or may use a model of individual judgment using another model.

[0036] Here, the two vehicles shown in Fig. 4 are S and R, the label of the vehicle of interest is expressed by the following formula (1), the vehicle of interest is defined by the following formula (2), and the relative state quantity with respect to vehicle i is expressed as X t iLet the pedestrian's individual intention toward vehicle i be expressed by the following formula (3), and the pedestrian's intention toward the whole vehicle be expressed by the following formula (4). The judgment model is as shown in FIG. 5, for example, and the probability of each estimated pedestrian's whole intention (G / W / U) is calculated by the following formula (5). Here, G is the probability that the pedestrian will cross the road, W is the probability that the pedestrian will not cross the road, and U is the probability that the pedestrian has not yet decided whether to cross the road or not. The judgment model shown in FIG. 5, for example, estimates an individual judgment by an individual judgment model while the pedestrian is looking at a vehicle, and estimates an individual judgment by continuing the judgment when the pedestrian last saw the vehicle while the pedestrian is not looking at a vehicle (in this case, a forgetting function is considered), and then estimates the whole judgment by integrating the two individual judgments.

[0037]

number

[0038]

number

[0039]

number

[0040]

number

[0041]

number

[0042] The first term on the right-hand side of equation (5) represents the integration of individual judgments, the second term on the right-hand side represents the individual judgment for straight-moving vehicle S, and the third term on the right-hand side represents the individual judgment for right-turning vehicle R. Here, the overall probability of judgment at time t is calculated from the following formula (6) based on formula (5).

[0043]

number

[0044] Here, the second term on the right-hand side of equation (6) indicates the probability of whether or not a pedestrian will cross at time t-1, and the first term on the right-hand side is an update equation for the probability of switching between models like the one below (see, for example, equations (7)-(9) etc.) based on whether the pedestrian is able to see vehicle i.

[0045] In the first term on the right side of formula (6), the first individual judgment when the pedestrian visually recognizes the vehicle may be constructed by a logistic regression model. In this case, the first individual judgment is estimated based on a one-to-one observation experiment between the pedestrian and a straight-moving vehicle S, and a one-to-one observation experiment between the pedestrian and a right-turning vehicle R. The first term on the right-hand side of equation (6) becomes equation (7) below when the pedestrian is visually recognizing vehicle i.

[0046]

number

[0047] In equation (7), the probability G that a pedestrian will cross the road and the probability W that a pedestrian will not cross the road are expressed as equation (8), and the probability U that a pedestrian is undecided whether to cross the road or not (other probabilities) is expressed as equation (9). Here, i is expressed in equation (1) above, and φ t are explanatory variables, η s is the regression coefficient.

[0048]

number

[0049]

number

[0050] On the other hand, the first term on the right side of equation (6) uses a forgetting function, an example of which is shown in Figure 3, and as the time passes without visual recognition of vehicle i, the probability at the last visual recognition (the probability calculated using equations (7)-(9) above) is reduced according to the forgetting function. Of course, the forgetting function also includes the case where the undetermined probability increases.

[0051] When estimating the behavior (pedestrian's judgment) of the pedestrian in response to the straight-moving vehicle S and the right-turning vehicle R by integrating the individual judgments, the information processing device 100 (control unit 110) multiplies the individual judgments by a coefficient, an example of which is shown in FIG. 6. The coefficient may be, for example, a weighting coefficient for the probability of each state according to the state in which the probability of each individual judgment (first individual judgment and second individual judgment) is highest. The information processing device 100 (control unit 110) may multiply the respective probabilities of G crossing the road, W not crossing the road, and U undecided whether to cross the road, obtained in each individual judgment, by a weighting coefficient for integration, an example of which is shown in FIG. 6, and the highest probability obtained as a result of the multiplication may be the behavior (pedestrian's overall judgment) of the pedestrian in response to the straight-moving vehicle S and the right-turning vehicle R. The coefficients (weighting coefficients) are not limited to the values ​​exemplified in FIG. 6, and may be various values ​​that are set appropriately.

[0052] The output control unit 115 may provide the result of estimation by the integrated estimation unit 114 (the action taken by the subject) to the simulation device. Furthermore, the output control unit 115 may control the output unit to output the result (the action taken by the subject) estimated by the integrated estimation unit 114. The output unit may be, for example, the communication unit 121, the storage unit 122, the display unit 123, etc., as described above. That is, the output control unit 115 may control the communication unit 121 to transmit the result (information on the behavior of the subject) estimated by the integrated estimation unit 114 to the outside (external device). The external device may be, for example, the server 200 and a user terminal. The user terminal may be, for example, a terminal used by a user of the information processing device 100. The output control unit 115 may, for example, control the storage unit 122 to store the result of estimation by the integrated estimation unit 114 (information on the behavior of the subject). The output control unit 115 may control the display unit 123 to display the result estimated by the integrated estimation unit 114 (the action taken by the subject), for example.

[0053] [Information processing method] An information processing method according to an embodiment will be described. FIG. 7 is a flowchart illustrating an information processing method according to an embodiment.

[0054] In step ST101, the identification unit 111 acquires target information. The target information may be, for example, information about the situation of the simulation. That is, as an example, the target information may be information about the situation of a road, the position of a pedestrian on the road, and the movement situation (traveling situation) of a moving body (vehicle) traveling on the road. The identification unit 111 may acquire the target information from, for example, a simulation device that performs a simulation, or may acquire the target information based on an input operation of the target information by a user to the information processing device 100.

[0055] In step ST102, the identification unit 111 identifies one of the multiple moving bodies visually recognized by the subject of the simulation based on the target information of step ST101. The identification unit 111 may identify, as the moving body, a vehicle as a target of attention when estimating the behavior of the subject as to whether or not the subject crosses a road when the vehicle travels on a road. In this case, the identification unit 111 may further identify the position of the identified moving body (vehicle) relative to the subject.

[0056] In step ST103, the first individual estimation unit 112 estimates the behavior of the subject according to the moving body (vehicle) identified in step ST102. In this case, the first individual estimation unit 112 may use an individual judgment model for the moving body (vehicle) identified in step ST102. That is, the first individual estimation unit 112 may acquire a trained model according to the moving body (vehicle) identified in step ST102, and estimate the behavior of the subject based on the trained model and the moving state (running state) of the moving body (vehicle) identified in step ST102. In the example shown in Figure 4, a learned model for a straight-moving vehicle S and a learned model for a right-turning vehicle R are prepared in advance, and for example, when a subject (pedestrian) is viewing the straight-moving vehicle S, the first individual estimation unit 112 may estimate whether the subject (pedestrian) will cross the road or not, or whether it is undecided, in accordance with the straight-moving vehicle S, based on the learned model for the straight-moving vehicle S and the driving conditions of the straight-moving vehicle S.

[0057] In step ST104, the second individual estimation unit 113 estimates the behavior of the subject according to the other moving body (other vehicle) not identified in step ST103. In this case, the second individual estimation unit 113 may use an individual judgment model for the other moving body (other vehicle) not identified in step ST102, similar to the above-mentioned first individual estimation unit 112. That is, the second individual estimation unit 113 may acquire a trained model corresponding to the other moving body (other vehicle) not identified in step ST102, and estimate the behavior of the subject based on the trained model and the moving state (running state) of the other moving body (other vehicle) not identified in step ST102. In an example of step ST103 using Figure 4, for example, when a subject (pedestrian) does not see the right-turning vehicle R, the second individual estimation unit 113 may estimate whether the subject (pedestrian) will cross the road or not, or whether it is undecided, based on a learned model corresponding to the right-turning vehicle R and the driving conditions of the right-turning vehicle R.

[0058] In this case, the second individual estimation unit 113 may estimate the behavior that the subject will take in response to other moving bodies (other vehicles) by reducing the probability that the behavior is taken by the subject in response to the other moving bodies (other vehicles) in accordance with the forgetting function, and by increasing the probability that the behavior that the subject will take is undetermined. That is, in the above-mentioned example using Figure 4, for example, by reducing a value corresponding to a forgetting function from the probability when the subject (pedestrian) last saw the right-turning vehicle R, and increasing the probability that the action to be taken by the subject is undecided, it is possible to estimate whether the subject (pedestrian) will cross the road in response to the right-turning vehicle R, and whether the action is undecided.

[0059] In addition, the first individual estimation unit 112 and the second individual estimation unit 113 may alternate between the target moving objects, i.e., the moving objects identified in step ST102 (as an example, by alternating at predetermined time intervals), depending on a situation in which the subject (pedestrian) alternately views multiple moving objects, and estimate the behavior of the subject (pedestrian) in response to each moving object.

[0060] In step ST105, the integrated estimation unit 114 estimates the behavior of the subject in response to all of the multiple moving objects, based on the behavior (judgment) of the subject estimated in step ST103 and the behavior of the subject estimated in step ST104. In this case, the integrated estimation unit 114 may multiply (accumulate) the probability calculated in step ST103, the probability calculated in step ST104, and a weighting coefficient used when integrating the first individual judgment in step ST103 and the second individual judgment in step ST104, and estimate the state with the highest probability as the behavior (overall behavior) taken by the subject.

[0061] Each unit of the information processing device 100 described above may be realized as a function of a processing device of a computer, etc. That is, the identification unit 111, the first individual estimation unit 112, the second individual estimation unit 113 (individual estimation unit), the integrated estimation unit 114, and the output control unit 115 (control unit 110) of the information processing device 100 may be realized as a identification function, a first individual estimation function, a second individual estimation function (individual estimation function), an integrated estimation function, and an output control function (control function) by a processing device of a computer, etc. The information processing program can cause a computer to realize each of the above-mentioned functions. The information processing program may be recorded in a non-transitory computer-readable recording medium, such as a memory, a solid-state drive, a hard disk drive, or an optical disk. As described above, each unit of the information processing device 100 may be realized by an arithmetic processing device of a computer or the like. The arithmetic processing device or the like is configured by, for example, an integrated circuit or the like. Therefore, each unit of the information processing device 100 may be realized as a circuit constituting the arithmetic processing device or the like. That is, the identification unit 111, the first individual estimation unit 112, the second individual estimation unit 113 (individual estimation unit), the integrated estimation unit 114, and the output control unit 115 (control unit 110) of the information processing device 100 may be realized as a identification circuit, a first individual estimation circuit, a second individual estimation circuit (individual estimation circuit), an integrated estimation circuit, and an output control circuit (control circuit) constituting the arithmetic processing device of a computer or the like. The communication unit 121, the storage unit 122, and the display unit 123 (output unit) of the information processing device 100 may be realized as a communication function, a storage function, and a display function (output function) including the functions of an arithmetic processing device, etc. The communication unit 121, the storage unit 122, and the display unit 123 (output unit) of the information processing device 100 may be realized as a communication circuit, a storage circuit, and a display circuit (output circuit) by being configured, for example, by an integrated circuit, etc. The communication unit 121, the storage unit 122, and the display unit 123 (output unit) of the information processing device 100 may be realized as a communication device, a storage device, and a display device (output device) by being configured, for example, by being configured by a plurality of devices.

[0062] The information processing device 100 can combine one or any two or more of the above-mentioned units. In this disclosure, the term "information" is used, but the term "information" can be replaced with "data" and the term "data" can be replaced with "information."

[0063] [Aspects and Effects of the Present Embodiment] Next, one aspect of this embodiment and the effects of each aspect will be described. Note that this embodiment is not limited to each aspect described below, and may be realized by appropriately combining each of the above-mentioned parts. Also, the effects described below are examples, and the effects of each aspect are not limited to those described below.

[0064] (Aspect 1) An information processing device of one embodiment includes an identification unit that identifies one moving body that is visually recognized by the subject of a simulation among multiple moving bodies that are the focus of the subject's attention, a first individual estimation unit that estimates a behavior that the subject will take in response to the moving body identified by the identification unit, a second individual estimation unit that estimates a behavior that the subject will take in response to other moving bodies different from the moving body identified by the identification unit, and an integrated estimation unit that estimates a behavior that the subject will take in response to all of the multiple moving bodies based on the behavior of the subject estimated by the first individual estimation unit and the behavior of the subject estimated by the second individual estimation unit. As a result, the information processing device estimates the behavior of the subject based on a one-to-one relationship between the subject and the moving body in each of the first individual estimation unit and the second individual estimation unit, and therefore the judgment model for individual estimation can be simplified compared to a judgment model set according to the overall situation of the simulation. That is, by integrating multiple individual judgment models throughout the entire simulation, the information processing device can simplify the simulation compared to a case where one judgment model is used throughout the entire simulation. Furthermore, by integrating multiple individual judgment models, the information processing device can respond to more simulation situations. The information processing device separately uses moving objects that the subject visually recognizes and moving objects that the subject does not visually recognize, and therefore can perform a simulation that corresponds to the actual situation of traffic participants with high accuracy. In addition, the information processing device performs individual judgments on moving objects that the subject recognizes but does not visually recognize, and therefore can perform a simulation that takes into account the impact of moving objects that are not visually recognized on the subject. In other words, the information processing device can construct a judgment model that is closer to the judgment of an actual human being by considering the presence or absence of visual recognition according to the visual behavior model of the subject, and can perform a simulation that corresponds to reality.

[0065] (Aspect 2) In one embodiment of the information processing device, the second individual estimation unit may estimate an action that the subject will take in response to another moving body by using the time that the subject last paid attention to the other moving body as a reference, reducing the probability that the action that the subject will take in response to the other moving body is determined in accordance with a forgetting function that represents the subject forgetting to pay attention to the other moving body, and increasing the probability that the action that the subject will take is undetermined. This allows the information processing device to perform a simulation that matches the actual situation of the traffic participants, even when there are multiple moving objects of interest and one of them is not visible to the traffic participants. That is, the information processing device performs a judgment for each moving object as seen from the traffic participants, while using a forgetting function to express the fading of information within the traffic participants about moving objects that are not visible to the traffic participants, thereby obtaining a realistic judgment result as an integrated result.

[0066] (Aspect 3) In one embodiment of an information processing device, each of the first individual estimation unit and the second individual estimation unit may estimate the behavior taken by the subject based on a learned model generated by learning the movement state of the moving body and the behavior taken by the subject in accordance with the movement state of the moving body, and the movement state of the moving body identified by the identification unit. This allows the information processing device to estimate the behavior of the subject based on a one-to-one relationship between the subject and the moving object. The information processing device can also make judgments according to multiple individual situations by using a trained model that is appropriately selected according to the overall situation of the simulation. In other words, the information processing device can break down the overall situation of the simulation into multiple individual judgment models and estimate the behavior of the subject in each judgment model.

[0067] (Aspect 4) In one aspect of the information processing device, the identification unit may identify a vehicle as a moving object, as an object of interest when estimating the behavior of a subject as to whether or not to cross a road, when the vehicle is traveling on a road. This enables the information processing device to estimate whether the subject person will cross the road or not, depending on the driving conditions of the vehicle.

[0068] (Aspect 5) In one embodiment of the information processing device, each of the first individual estimation unit and the second individual estimation unit may utilize a learned model generated by learning the vehicle position and driving speed as the moving state of the moving body, and the behavior taken by the subject, such as whether the subject crosses the road, whether or not to cross the road, and whether or not to cross the road is undecided, depending on the moving state of the vehicle. This enables the information processing device to calculate the probability of an individual judgment as to whether or not the subject person will cross the road, and whether or not it is undecided, depending on the driving conditions of the vehicle.

[0069] (Aspect 6) In one embodiment of the information processing device, the integrated estimation unit may multiply a value indicating the subject's behavior estimated by the first individual estimation unit and the second individual estimation unit by a coefficient for integration processing that is set in advance according to the subject's behavior, and integrate the results of the multiplication to estimate the behavior taken by the subject according to all of the multiple moving objects. This allows the information processing device to integrate multiple individual judgments and estimate an overall judgment of the subject person according to multiple moving objects.

[0070] (Aspect 7) In one embodiment of an information processing method, a computer executes the following steps: an identification step of identifying one moving body that is visually recognized by the subject out of multiple moving bodies that are the focus of the subject's attention in a simulation; a first individual estimation step of estimating a behavior that the subject will take in response to the moving body identified in the identification step; a second individual estimation step of estimating a behavior that the subject will take in response to other moving bodies different from the moving body identified in the identification step; and an integrated estimation step of estimating a behavior that the subject will take in response to all of the multiple moving bodies based on the behavior of the subject estimated in the first individual estimation step and the behavior of the subject estimated in the second individual estimation step. As a result, the information processing method can achieve the same effects as the information processing device according to the above-described aspect.

[0071] (Aspect 8) An information processing program of one embodiment causes a computer to realize a identification function that identifies one moving object that is visually recognized by the subject out of multiple moving objects that are the focus of the subject's attention in a simulation, a first individual estimation function that estimates a behavior that the subject will take in response to the moving object identified by the identification function, a second individual estimation function that estimates a behavior that the subject will take in response to other moving objects different from the moving object identified by the identification function, and an integrated estimation function that estimates a behavior that the subject will take in response to all of the multiple moving objects based on the behavior of the subject estimated by the first individual estimation function and the behavior of the subject estimated by the second individual estimation function. As a result, the information processing program can achieve the same effects as the information processing device according to the above-described aspect. [Explanation of symbols]

[0072] 100 Information processing device 110 Control section 111 Specific section 112 First Individual Estimation Unit 113 Second Individual Estimation Division 114 Integrated Estimation Unit 115 Output control section 121 Communications Department 122 Memory Department 123 Display Department

Claims

1. The information processing device has position information of the vehicle relative to the pedestrian, An identification unit that identifies one vehicle that is visually recognized by a pedestrian among a plurality of vehicles that are targets of attention of a pedestrian in a simulation; a first individual estimation unit that estimates an action to be taken by the pedestrian according to a moving state including at least a position of the vehicle with respect to the pedestrian identified by the identification unit and information regarding a speed based on an amount of change in the position, the action being one of an action of the pedestrian crossing a road, an action of the pedestrian not crossing the road, and an action of the pedestrian not deciding whether to cross the road or not; a second individual estimation unit that estimates a behavior of the pedestrian based on a moving state including information on at least a position of another vehicle other than the vehicle identified by the identification unit with respect to the pedestrian and a speed based on an amount of change in the position; an integrated estimation unit that estimates a behavior of a pedestrian in response to all of a plurality of vehicles, based on the behavior of the pedestrian estimated by the first individual estimation unit and the behavior of the pedestrian estimated by the second individual estimation unit; An information processing device comprising:

2. The second individual estimation unit estimates the action that the pedestrian will take in response to another vehicle by using the time when the pedestrian last paid attention to the other vehicle as a reference, reducing the probability of the judgment at the time when the pedestrian last paid attention in accordance with a forgetting function that indicates that the pedestrian forgets to pay attention to the other vehicle, and increasing the probability of the judgment that the action that the pedestrian will take is undecided. The information processing device according to claim 1 .

3. Each of the first individual estimation unit and the second individual estimation unit estimates an action to be taken by a pedestrian based on a trained model generated by learning at least a position of a vehicle relative to a pedestrian, a moving state including information on a speed based on an amount of change in the position, and an action to be taken by a pedestrian according to the moving state of the vehicle, and the moving state of the vehicle identified by the identification unit, and the second individual estimation unit performs estimation for another vehicle different from the vehicle identified by the identification unit.

3. The information processing device according to claim 1 or 2.

4. The identification unit identifies a vehicle as a target of attention when estimating a pedestrian's behavior as to whether or not the pedestrian will cross a road when the vehicle is traveling on the road.

4. The information processing device according to claim 1.

5. Each of the first individual estimation unit and the second individual estimation unit utilizes a learned model generated by learning the position and traveling speed of the vehicle as the moving state of the vehicle, and the judgment of whether the pedestrian crosses the road, does not cross the road, or is undecided whether to cross the road, depending on the moving state of the vehicle, as the action taken by the pedestrian. The information processing device according to claim 4.

6. The integrated estimation unit multiplies a value indicating the pedestrian's behavior estimated by the first individual estimation unit and the second individual estimation unit by a coefficient for integration processing that is set in advance according to the pedestrian's behavior, and estimates the behavior of the pedestrian in accordance with all of the multiple vehicles by integrating a result of the multiplication. The information processing device according to any one of claims 1 to 5.

7. A computer that holds vehicle position information for pedestrians, A step of identifying one vehicle that is visible to a pedestrian among a plurality of vehicles that are the focus of attention of the pedestrian in the simulation; a first individual estimation step of estimating an action to be taken by the pedestrian according to a moving state including information on at least a position of the vehicle relative to the pedestrian identified by the identification step and a speed based on an amount of change in the position, the action being one of an action of the pedestrian crossing a road, an action of the pedestrian not crossing the road, and an action of the pedestrian not deciding whether to cross the road or not; a second individual estimation step of estimating a behavior of the pedestrian based on a moving state including information on at least a position of another vehicle other than the vehicle identified by the identification step with respect to the pedestrian and a speed based on an amount of change in the position; an integrated estimation step of estimating a behavior of a pedestrian in response to all of a plurality of vehicles, based on the behavior of the pedestrian estimated in the first individual estimation step and the behavior of the pedestrian estimated in the second individual estimation step; An information processing method for performing the above.

8. A computer that holds vehicle position information for pedestrians, An identification function that identifies one vehicle that is visible to a pedestrian among a plurality of vehicles that are the focus of attention of the pedestrian in the simulation; a first individual estimation function for estimating an action to be taken by the pedestrian according to a moving state including information on at least a position of the vehicle relative to the pedestrian identified by the identification function and a speed based on an amount of change in the position, the action being one of the actions of the pedestrian crossing a road, the action of the pedestrian not crossing the road, and the action of the pedestrian not deciding whether to cross the road or not; a second individual estimation function that estimates a behavior of the pedestrian based on a moving state including information on a position of a vehicle other than the vehicle identified by the identification function with respect to the pedestrian and a speed based on an amount of change in the position; an integrated estimation function that estimates a behavior of a pedestrian in response to all of a plurality of vehicles, based on the pedestrian behavior estimated by the first individual estimation function and the pedestrian behavior estimated by the second individual estimation function; An information processing program that realizes this.

Images