Information output method, information output device, and program

Abstract

To appropriately identify and output an event that can occur with a mobile body.SOLUTION: A model is acquired that outputs first event information indicating an event that occurred with a first mobile body on the basis of first mobile body information as information related to the first mobile body input to the model. Second mobile body information as information related to a second mobile body is acquired (S202). Third mobile body information obtained by modifying the acquired second mobile body information is generated (S203). Information which is output by input of the acquired second mobile body information to the model and indicates an event that occurred with the second mobile body is acquired as second event information (S204). Information indicating an event which is output by input of the generated third mobile body information to the model is acquired as third event information (S205). Difference information indicating a difference between the second event information and the third event information is generated (S206) and output (S207).SELECTED DRAWING: Figure 13

Description

Technical Field

[0001] The present invention relates to an information output method, an information output device, and a program.

Background Art

[0002] Conventionally, a method for predicting an event occurring within a road intersection has been disclosed (see Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, there is a problem that an event may not be predicted appropriately.

[0005] The present invention provides an information output method and the like that appropriately identify and output an event that may occur in a moving object.

Means for Solving the Problems

[0006] An information output method according to an aspect of the present invention is a model that outputs first event information indicating an event that has occurred in a first moving object based on first moving object information, which is information about the first moving object input to the model. The method acquires second moving object information, which is information about a second moving object, generates third moving object information obtained by modifying the acquired second moving object information, and acquires, as second event information, information indicating an event that has occurred in the second moving object and is output by inputting the acquired second moving object information into the model. The method also acquires, as third event information, information indicating an event output by inputting the generated third moving object information into the model, and generates and outputs difference information indicating a difference between the second event information and the third event information.

[0007] Note that these general or specific aspects may be implemented in a system, apparatus, integrated circuit, computer program, or recording medium such as a computer-readable CD-ROM, or may be implemented in any combination of a system, apparatus, integrated circuit, computer program, and recording medium.

Advantages of the Invention

[0008] The information output device of the present invention can appropriately identify and output events that may occur in a moving object.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Embodiments for Carrying Out the Invention

[0010] (Knowledge underlying the present invention) The present inventor has found that the following problems occur with respect to the technique of predicting events described in the "Background Art" section.

[0011] Based on the information acquired by a sensor of a moving object (e.g., a vehicle), it is possible to detect an event occurring in or around the moving object or to predict in advance an event that is likely to occur in or around the moving object. When an event is detected or predicted, it is assumed that the moving object will avoid the event or that measures will be taken in advance to improve the moving object or the travel route so that the event does not occur.

[0012] The above-mentioned moving object includes not only a moving object driven by a driver on board, but also a moving object that autonomously travels without a driver on board, or a moving object that is controlled via a communication line by a driver existing at a remote location rather than inside the vehicle and travels.

[0013] However, there may be cases where the information acquired by the sensor contains errors or there are deficiencies in the information acquired by the sensor. Thus, when there are errors or deficiencies in the information acquired by the sensor, there is a problem that an event cannot be appropriately predicted, in other words, the prediction of the event may fail.

[0014] The present invention provides an information output method and the like that can appropriately predict events occurring in a moving body. More specifically, the present invention provides an information output method and the like that identify and output in advance an event that may fail to be detected during actual movement among events that may occur in a moving body.

[0015] In order to solve such problems, an information output method according to an aspect of the present invention acquires a model that outputs first event information indicating an event that occurred in a first moving body based on first moving body information, which is information regarding the first moving body input to the model, acquires second moving body information, which is information regarding a second moving body, generates third moving body information obtained by modifying the acquired second moving body information, acquires information indicating an event that occurred in the second moving body, which is output by inputting the acquired second moving body information into the model, as second event information, acquires information indicating an event that is output by inputting the generated third moving body information into the model as third event information, and generates and outputs difference information indicating a difference between the second event information and the third event information.

[0016] According to the above aspect, it is possible to output difference information indicating an event that may fail to be detected during actual movement among events that may occur in a moving body. The output difference information indicates the change in event information caused by modifying the second moving body information. Further, the modification made to the second moving body information may correspond to an error or omission included in information acquired by a sensor that may occur during actual movement. Therefore, it can be said that the output difference information indicates an event that may not be detected due to an error or omission in information that can be acquired by a sensor during actual movement. Thus, the above information output method can appropriately identify and output events that may occur in a moving body by outputting difference information indicating an event that may fail to be detected during actual movement among events that occur in the moving body. can be appropriately identified and output.

[0017] For example, the second moving body information includes information generated from sensor values generated by sensing with a sensor provided in the second moving body during the movement of the second moving body. When generating the third moving body information, the third moving body information may be generated by applying the change corresponding to the failure of sensing by the sensor provided in the second moving body to the second moving body information.

[0018] According to the above aspect, since the change applied to the second moving body information is a change corresponding to the failure of sensing by the sensor provided in the second moving body, the output difference information is difference information indicating an event that may fail to be detected during actual movement. Therefore, the above information output method can more appropriately identify and output an event that may occur in the moving body.

[0019] For example, further, the frequency of failure of sensing by the sensor provided in the second moving body is measured, and when generating the third moving body information, the third moving body information may be generated by applying the change with the measured frequency.

[0020] According to the above aspect, since the change is applied to the second moving body information based on the frequency of failure of sensing by the sensor provided in the second moving body, the output difference information is difference information indicating an event that may fail to be detected during actual movement, considering the actual failure frequency. Therefore, the above information output method can more appropriately identify and output an event that may occur in the moving body.

[0021] For example, when generating the difference information, difference information indicating an event excluding the event indicated in the third event information among the events indicated in the second event information may be generated.

[0022] According to the above aspect, by excluding the event indicated in the third event information from the events indicated in the second event information, the difference information can be easily generated. Therefore, the above information output method can more easily and appropriately identify and output an event that may occur in the moving body.

[0023] For example, the first moving body information may further include information generated from sensor values generated by sensing of sensors installed on the moving path of the first moving body, and the second moving body information may further include information generated from sensor values generated by sensing of sensors installed on the moving path of the second moving body.

[0024] According to the above aspect, in addition to the sensors provided in the moving body, sensor values generated by sensing of sensors installed on the moving path are also used to generate moving body information, so that the generated moving body information becomes more appropriate information. As a result, the appropriateness of the event information and the difference information is also improved. Therefore, the above information output method can more appropriately identify and output events that may occur in the moving body.

[0025] For example, the first moving body information includes the position information or speed information of the first moving body, the presence or absence of obstacles around the first moving body, the position information or speed information of the obstacles existing around the first moving body, the number of passengers on the first moving body, the amount of the flow of people existing around the first moving body, or the amount of the flow of vehicles existing around the first moving body, and the second moving body information includes the position information or speed information of the second moving body, the presence or absence of obstacles around the second moving body, the position information or speed information of the obstacles existing around the second moving body, the number of passengers on the second moving body, the amount of the flow of people existing around the second moving body, or the amount of the flow of vehicles existing around the second moving body.

[0026] According to the above aspect, difference information is generated using, as the moving body information, the position information or speed information of the moving body, the presence or absence of obstacles around the moving body, or the position information or speed information of the obstacles existing around the moving body. Therefore, the above information output method can more easily and appropriately identify and output events that may occur in the moving body.

[0027] For example, the event may include a person or an object jumping out in the moving direction of the moving body which is the first moving body or the second moving body, a person or an object being involved when the moving direction of the moving body is changed, a collision when the moving body merges with another moving body, an interruption of another moving body in the moving direction of the moving body, a deviation of the moving path of the moving body, a collision caused by the simultaneous start of a vehicle and a pedestrian in a crosswalk area without a traffic signal, or an intentional obstruction around the vehicle.

[0028] According to the above aspect, based on the moving body information, by means of a model, a person or an object jumping out in the moving direction of the moving body, a person or an object being involved when the moving direction of the moving body is changed, a collision when the moving body merges with another moving body, an interruption of another moving body in the moving direction of the moving body, or a deviation of the moving path of the moving body is detected as an event. Therefore, the above information output method can more easily and appropriately identify and output an event that may occur to the moving body.

[0029] For example, the difference information may include an image showing the position on the map where the event indicated by the difference information occurred.

[0030] According to the above aspect, the position where an event that may not be detected during actual movement occurred is shown and presented on the map. Therefore, a person who views the above map can easily grasp the above event on the map, and it is assumed that measures will be taken to make the event less likely to actually occur. Therefore, the above information output method can lead to measures to reduce the detection of the event while appropriately identifying and outputting an event that may occur to the moving body.

[0031] For example, the difference information may include the degree to which each of a plurality of factors of the event indicated by the difference information affected the occurrence of the event.

[0032] According to the above aspect, the degree to which each of a plurality of factors of an event has affected the occurrence of the event is presented. Therefore, it is assumed that a person who visually recognizes the above degree will take measures to make the event less likely to actually occur, or measures to make it less likely to cause a failure in detecting the event. Thus, the above information output method can lead to measures to reduce the detection of the event while appropriately identifying and outputting an event that can occur in a moving object.

[0033] For example, the model may be a neural network model.

[0034] According to the above aspect, using a neural network model, it is possible to more easily and appropriately predict an event that occurs in a moving object.

[0035] Also, an information output device according to an aspect of the present invention includes an acquisition unit, a modification unit, a detection unit, and a generation unit. The detection unit is a model that outputs first event information indicating an event that has occurred in a first moving object based on first moving object information that is information about the first moving object input to the model. The acquisition unit acquires second moving object information that is information about a second moving object. The modification unit generates third moving object information obtained by modifying the second moving object information acquired by the acquisition unit. The detection unit acquires, as second event information, information indicating an event that has occurred in the second moving object and is output by inputting the second moving object information acquired by the acquisition unit into the model. The detection unit acquires, as third event information, information indicating an event output by inputting the third moving object information generated by the modification unit into the model. The generation unit is an information output device that generates and outputs difference information indicating the difference between the second event information and the third event information.

[0036] According to the above aspect, the same effect as the above information output method is achieved.

[0037] Also, a program according to an aspect of the present invention is a program that causes a computer to execute the above information output method.

[0038] According to the above aspect, the same effects as those of the above information output method are achieved.

[0039] These general or specific aspects may be implemented in a system, device, integrated circuit, computer program, or recording medium such as a computer-readable CD-ROM, or may be implemented by any combination of a system, device, integrated circuit, computer program, or recording medium.

[0040] Hereinafter, embodiments will be specifically described with reference to the drawings.

[0041] Note that all of the embodiments described below show general or specific examples. The numerical values, shapes, materials, components, arrangement positions and connection forms of components, steps, order of steps, etc. shown in the following embodiments are merely examples and are not intended to limit the present invention. In addition, among the components in the following embodiments, components not described in the independent claims indicating the most general concept are described as optional components.

[0042] (Embodiment) In the present embodiment, a specifying method and a specifying device for appropriately specifying and outputting an event that can occur in a moving body will be described. The specifying method and the specifying device are also referred to as an information output method and an information output device, respectively.

[0043] Hereinafter, (1) a detection system that detects an event occurring in a vehicle, and (2) a specifying system that specifies in advance an event that the detection by the detection system may fail when the vehicle moves will be described.

[0044] Note that the event means various events related to the vehicle, and more specifically, may mean an event that hinders the progress of the vehicle, or an event that hinders the safe progress of the vehicle.

[0045] (1) Detection System FIG. 1 is an explanatory diagram schematically showing the configuration of the detection system 5 in the present embodiment. The detection system 5 is a system that detects events occurring in the vehicle 7.

[0046] As shown in FIG. 1, the detection system 5 includes a presentation device 30 and a detection device 50, which are communicably connected via a network N. The presentation device 30 and the detection device 50 are communicably connected to the vehicle 7 via the network N. Note that the detection system 5 may further include the vehicle 7.

[0047] The vehicle 7 is an example of a moving object that is the target for which events are detected by the detection system 5. The vehicle 7 generates information regarding the vehicle 7 (also referred to as vehicle information) from sensor values generated by sensing with sensors, and transmits the generated vehicle information to the detection device 50. The vehicle information is an example of moving object information.

[0048] The detection device 50 is a device that detects events occurring in the vehicle 7. The detection device 50 acquires vehicle information from the vehicle 7, detects an event occurring in the vehicle 7 using a detection model, and transmits event information indicating the event to the presentation device 30.

[0049] The presentation device 30 is a device that presents events occurring in the vehicle 7. The presentation device 30 acquires the event information transmitted by the detection device 50 and presents the acquired event information. The presentation of the event information is performed by displaying an image indicating the presentation information on a display screen, or outputting a voice indicating the presentation information by a speaker, etc. The event information to be presented is assumed to be viewed or listened to by the monitor P. The monitor P who views or visually recognizes the event information is assumed to control the vehicle 7 according to the content of the event indicated by the event information. In particular, when the event is an event that hinders the safe progress of the vehicle 7, it is assumed that the monitor P controls the vehicle 7 so as to avoid the event.

[0050] FIG. 2 is an explanatory diagram schematically showing the functional configuration of the vehicle 7 in the present embodiment.

[0051] As shown in FIG. 2, vehicle 7 includes, as functional units, sensors 51A and 51B, a processing unit 52, and a providing unit 53. The functional units included in vehicle 7 can be realized by a processor (for example, a CPU (Central Processing Unit), not shown) included in vehicle 7 executing a predetermined program using a memory (not shown).

[0052] Sensor 51A is a sensor that performs sensing on vehicle 7 or the surroundings of vehicle 7. When sensor 51A performs sensing, it generates a sensor value indicating the result of the sensing and provides it to processing unit 52. Sensor 51A includes a camera, LiDAR (Light Detection and Ranging), IMU (Inertial Measurement Unit), or a GPS receiver. The sensor value includes at least an image captured by a camera (for example, an RGB image), a point cloud acquired by LiDAR, acceleration information and angular velocity information acquired by IMU, or GPS data obtained by a GPS receiver. Sensor 51A repeatedly acquires sensor values at appropriate intervals (for example, about 0.1 second to several seconds). The interval at which sensor 51A acquires sensor values may be determined according to the content of the sensing. Also, the interval at which sensor 51A acquires sensor values may be fixed or may vary.

[0053] Sensor 51B is the same type of sensor as sensor 51A and operates independently of sensor 51A. Although sensor 51B is assumed to be a sensor that senses information different from the information sensed by sensor 51A, it may also be a sensor that senses the same information as the information sensed by sensor 51A.

[0054] For example, sensor 51A is a GPS receiver and sensor 51B is a camera.

[0055] Note that in FIG. 2, two sensors, 51A and 51B, are shown as sensors, but there may be any number of sensors as long as there is one or more.

[0056] The processing unit 52 is a functional unit that obtains information about the vehicle 7 (vehicle information) by processing the sensor values provided by the sensors 51A and 51B. The vehicle information includes, for example, the position information or speed information of the vehicle 7, the presence or absence of obstacles around the vehicle 7, the position information or speed information of the obstacles existing around the vehicle 7, the number of passengers in the vehicle 7, the amount of the flow of people existing around the vehicle 7 (also referred to as the pedestrian flow), or the amount of the flow of vehicles existing around the vehicle 7 (also referred to as the vehicle flow). Note that the processing unit 52 may obtain vehicle information by processing not only the sensor values provided by the sensors 51A and the like but also the information obtained from an external device.

[0057] The method by which the processing unit 52 calculates vehicle information based on sensor values can be performed by a known technique. For example, when the sensor 51A is a GPS receiver, the processing unit 52 processes a signal (corresponding to the sensor value) received by the GPS receiver from a GPS satellite to obtain position information indicating the position of the GPS receiver, in other words, the vehicle 7. Further, for example, when a rental contract is made to ride in the vehicle 7, the processing unit 52 refers to the content of the rental contract obtained from an external device to obtain the number of passengers in the vehicle 7.

[0058] The providing unit 53 is a functional unit that provides the vehicle information calculated by the processing unit 52 to the detection device 50. The providing unit 53 transmits the vehicle information to the detection device 50 through a communication interface (not shown) connected to the network N.

[0059] FIG. 3 is an explanatory diagram schematically showing the functional configuration of the detection device 50 in the present embodiment.

[0060] As shown in FIG. 3, the detection device 50 includes, as functional units, an acquisition unit 501, a detection unit 502, and an output unit 503. The functional units included in the detection device 50 can be realized by a processor (for example, a CPU, not shown) included in the detection device 50 executing a predetermined program using a memory (not shown).

[0061] The acquisition unit 501 is a functional unit that acquires vehicle information from the vehicle 7. The acquisition unit 501 receives the vehicle information transmitted by the vehicle 7 (providing unit 53) through a communication interface (not shown) connected to the network N. The acquisition unit 501 provides the acquired vehicle information to the detection unit 502.

[0062] Note that the acquisition unit 501 may acquire the vehicle information of the vehicle 7 from a facility different from the vehicle 7. Specifically, the acquisition unit 501 may acquire vehicle information generated from sensor values obtained by a sensor (not shown) installed on the travel route of the vehicle 7 by sensing the vehicle 7 or its surroundings. The sensor installed on the travel route of the vehicle 7 is, for example, a camera or the like installed on a traffic signal or a street lamp connected to the network N. In this case, the acquisition unit 501 acquires the image generated by the camera through the network N.

[0063] The detection unit 502 is a functional unit that acquires event information indicating an event that has occurred in the vehicle 7 based on the vehicle information acquired by the acquisition unit 501. The detection unit 502 has a model 505 that outputs event information based on the vehicle information, and acquires the event information using the model 505.

[0064] The detection unit 502 provides the acquired event information to the presentation device 30. Specifically, the detection unit 502 transmits the acquired event information to the presentation device 30 through a communication interface (not shown) connected to the network N. Here, the event that has occurred in the vehicle 7 includes, for example, at least one of a person or an object jumping out in the moving direction of the vehicle 7, a person or an object being involved when the traveling direction of the vehicle 7 changes, a collision when the vehicle 7 merges with another vehicle, an intrusion of another moving body in the moving direction of the vehicle 7, a deviation of the travel route of the vehicle 7, a collision due to the start of simultaneous travel between the vehicle 7 and a pedestrian in a crosswalk area without a traffic signal, and an intentional obstruction around the vehicle 7.

[0065] The model 505 is a detection model that detects an event that occurs in a vehicle. Specifically, when vehicle information is input, the model 505 outputs event information indicating an event that occurs in the vehicle related to the input vehicle information, based on the input vehicle information. The model 505 is, for example, a learning model constructed in advance by machine learning. In advance machine learning, for example, a sufficient number (tens of thousands to hundreds of thousands) of pairs of vehicle information when various events occur in the vehicle and event information indicating the events are input to a learning machine as training data. Then, the model is configured to detect an event that occurs in the vehicle based on the characteristics of the vehicle information included in the training data, so that the event information included in the training data can be output. Learning progresses by adjusting the internal parameters included in the model. The model is, for example, a neural network model.

[0066] The output unit 503 is a functional unit that provides the event information acquired by the detection unit 502 to the presentation device 30. The output unit 503 transmits the event information to the presentation device 30 via a communication interface (not shown) connected to the network N.

[0067] FIG. 4 is a flow diagram showing the processing of the detection system 5 in this embodiment.

[0068] In step S101, the sensors 51A and the like of the vehicle 7 perform sensing of the vehicle 7 or the surroundings of the vehicle 7 to obtain sensor values.

[0069] In step S102, the processing unit 52 of the vehicle 7 processes the sensor value acquired in step S101 to acquire vehicle information.

[0070] In step S103, the detection unit 502 of the detection device 50 acquires event information using the vehicle information acquired in step S102 and the model 505.

[0071] In step S104, the output unit 503 outputs the event information acquired in step S103 to the presentation device 30. It is assumed that the output presentation information is visually recognized by the observer P in an appropriate format.

[0072] In this way, the detection system 5 detects an event occurring in the vehicle 7. It is assumed that the monitor P knows the event by visually recognizing the information detected and presented by the detection system 5.

[0073] (2) Specific system FIG. 5 is an explanatory diagram schematically showing the configuration of the specific system 1 in the present embodiment. The specific system 1 is a system that pre-identifies an event that may fail to be detected when the vehicle 7 moves.

[0074] As shown in FIG. 5, the specific system 1 includes a presentation device 20 and a specific device 10, which are communicably connected via a network N. The presentation device 20 and the specific device 10 are communicably connected to the vehicle 3 via the network N. Note that the specific system 1 may further include the vehicle 3.

[0075] The vehicle 3 is an example of a moving body used for identifying an event that the detection system 5 may fail to detect when the vehicle 7 moves. The functions of the vehicle 3 are the same as those of the vehicle 7 in the detection system 5. Note that the vehicle 3 may be the same vehicle as the vehicle 7 or a different vehicle.

[0076] The vehicle 3 generates information about the vehicle 3 (also referred to as vehicle information) from sensor values generated by sensing with sensors, and transmits the generated vehicle information to the detection device 50.

[0077] The specific device 10 is a device that pre-identifies an event that the detection system 5 may fail to detect when the vehicle 7 moves. The specific device 10 acquires vehicle information from the vehicle 3, and uses a detection model to identify an event that may fail to be detected when the vehicle 7 moves from the acquired vehicle information and the vehicle information obtained by modifying the acquired vehicle information. Further, the specific device 10 transmits specific information for identifying the above event to the presentation device 20.

[0078] The prompting device 20 is a device that prompts events that may fail to be detected when the vehicle 7 moves. The prompting device 20 acquires the event information transmitted by the specifying device 10 and prompts the acquired event information. The prompting of the event information is performed by displaying an image indicating the prompting information on the display screen, or by outputting a voice indicating the prompting information through the speaker. The event information to be prompted is assumed to be viewed or listened to by the monitor Q. It is assumed that the monitor Q who has viewed or visually recognized the event information will take measures to avoid the event or to prevent the occurrence of the event.

[0079] For example, when the above event is that a person jumps out in the traveling direction of the vehicle, the measures to make the event less likely to actually occur include the installation of facilities that prevent people from entering the road (for example, guardrails or signboards with cautionary information). Also, the measures to make it less likely to fail to detect the event include adding sensors or adjusting the installation positions of the sensors.

[0080] FIG. 6 is an explanatory diagram schematically showing the functional configuration of the specifying device 10 in the present embodiment.

[0081] As shown in FIG. 6, the specifying device 10 includes, as functional parts, an acquisition part 101, a change part 102, a detection part 103, and a generation part 104. The functional parts included in the specifying device 10 can be realized by a processor (for example, a CPU (Central Processing Unit), not shown) included in the specifying device 10 executing a predetermined program using a memory (not shown).

[0082] The acquisition part 101 is a functional part that acquires vehicle information. Specifically, the acquisition part 101 acquires vehicle information (corresponding to the second moving body information) regarding the movement of the vehicle 3. The acquisition part 101 provides the acquired vehicle information to the change part 102 and the detection part 103. The vehicle information thus provided is also referred to as vehicle information 111. The vehicle information 111 is information generated by the processing part 52 from sensor values generated by sensors 51A etc. provided in the vehicle 3 through sensing, and is information regarding the vehicle 3.

[0083] Note that the acquisition unit 101 may acquire the vehicle information 111 of the vehicle 3 from a facility different from the vehicle 3. Specifically, the acquisition unit 101 may acquire the vehicle information 111 generated from the sensor values obtained by a sensor (not shown) installed on the travel path of the vehicle 3 through sensing of the vehicle 3 or its surroundings.

[0084] The change unit 102 is a functional unit that makes changes to the vehicle information 111. Specifically, the change unit 102 acquires the vehicle information 111 from the acquisition unit 101 and generates vehicle information 112 (corresponding to the third moving body information) obtained by making changes to the acquired vehicle information 111. The change unit 102 provides the generated vehicle information 112 to the detection unit 103. Specifically, the change unit 102 makes changes to the latitude or longitude included in the position information as the vehicle information 111, or changes the time corresponding to the vehicle information 111.

[0085] When generating the vehicle information 112, the change unit 102 generates the vehicle information 112 by making changes to the vehicle information 111 corresponding to the failure of sensing by the sensor 51A or the like provided in the vehicle 3.

[0086] Note that when generating the vehicle information 112, the change unit 102 may generate the vehicle information 112 by making the above changes at the frequency of actual failure of sensing by the sensor 51A or the like provided in the vehicle 3. The frequency of actual failure of sensing by the sensor 51A or the like provided in the vehicle 3 is derived, for example, as follows.

[0087] That is, the change unit 102 acquires vehicle information 111 (also referred to as vehicle information A) of the vehicle 3 generated using the sensing results of sensors installed at appropriate positions where it is considered that sensing will not fail, with a sufficient number of sensors where it is considered that sensing will not fail. The change unit 102 acquires vehicle information 111 (also referred to as vehicle information A) of the vehicle 3 generated using the sensing results of sensors installed at appropriate positions where it is considered that sensing will not fail, with a sufficient number of sensors where it is considered that sensing will not fail.

[0088] Further, the modification unit 102 acquires vehicle information 111 (also referred to as vehicle information B) of the vehicle 3 generated using the result of sensing by sensors 51A etc. provided in the vehicle 3. Since the vehicle information B is obtained from sensor values obtained by fewer sensors than the sensors involved in the acquisition of the vehicle information A, it is assumed that the number of entries included in the vehicle information B is less than the number of entries included in the vehicle information A.

[0089] Then, the modification unit 102 compares the vehicle information A and the vehicle information B to identify the difference between the vehicle information B and the vehicle information A. The frequency at which the difference between the vehicle information B and the vehicle information A thus identified occurs corresponds to the frequency at which sensing by sensors 51A etc. provided in the vehicle 3 actually fails. The sufficient number of the above sensors that are considered not to fail in sensing may include sensors installed in other vehicles or on the travel path in addition to the sensor 51A provided in the vehicle 3.

[0090] The detection unit 103 is a functional unit that outputs event information using the model 106 based on the input vehicle information 111. The detection unit 103 has acquired the model 106 in advance. The model 106 is a model that outputs event information (corresponding to first event information) indicating an event that has occurred in the vehicle based on the vehicle information 111 (corresponding to first moving body information) of the vehicle input to the model 106, and is the same model as the model 505.

[0091] The detection unit 103 acquires the vehicle information 111 provided by the acquisition unit 101, and acquires event information 121 (corresponding to second event information) output by inputting the acquired vehicle information 111 to the model 106. The detection unit 103 provides the acquired event information 121 to the generation unit 104.

[0092] Further, when the detection unit 103 acquires the vehicle information 112 provided by the modification unit 102, it acquires event information 122 (corresponding to third event information) output by inputting the acquired vehicle information 112 to the model 106. The detection unit 103 provides the acquired event information 122 to the generation unit 104.

[0093] The generation unit 104 is a functional unit that generates and outputs difference information indicating the difference between the event information 121 and the event information 122. The generation unit 104 transmits the difference information to the presentation device 20 through a communication interface (not shown) connected to the network N.

[0094] When generating the difference information, the generation unit 104 first integrates the event information 121 and the event information 122. When integrating, if the same event that occurred at the same time and in the same place is included in each of the event information 121 and the event information 122, integrated event information is generated so that these events are regarded as one event. At this time, the above "same time" is not limited to being exactly the same time, and two times having a predetermined time difference (for example, a time difference within about several seconds) may be regarded as the same time. In other words, two times having the above predetermined time difference may be regarded as the same. Similarly, the above "same place" is not limited to being exactly the same place, and two points having a predetermined distance (for example, a distance within about several meters) may be regarded as the same place. In other words, two points having the above predetermined distance may be regarded as the same. Note that the latitude corresponding to 1 meter is, for example, about 10 -5 degrees near the equator, and the latitude corresponding to 1 meter is, for example, about 10 -5 degrees.

[0095] Next, the generation unit 104 generates, as the difference information, information indicating the events among the events indicated in the integrated event information excluding the events indicated in the event information 122. The difference information is the integrated event It can be obtained by subtracting the event indicated in the event information 122 from the event indicated in the image information. Note that the difference information is information indicating an event excluding the event indicated in the event information 122 among the events indicated in the event information 121 while regarding two times having the predetermined time difference as the same and regarding two locations having the predetermined distance as the same. That is, the difference information is information indicating an event that has become undetectable by the model 106 due to a change being made to the vehicle information 111 of the vehicle 3 among the events occurring in the vehicle 3. It can be said that the difference information is information indicating an event that may fail to be detected when the vehicle 7 moves.

[0096] The difference information can be expressed in various forms. The difference information may include, for example, an image showing on a map the position where the event indicated by the difference information occurred. Further, the difference information may include, for example, the degree to which each of a plurality of factors of the event indicated by the difference information has affected the event.

[0097] Hereinafter, the processing of the specific device 10 will be described in more detail.

[0098] FIG. 7 is an explanatory diagram showing an example of the vehicle information 111 acquired by the vehicle 3 and the event information 121 in the present embodiment.

[0099] The vehicle information 111 shown in FIG. 7 is position information including latitude and longitude indicating the position of the vehicle 3 at each time. The vehicle information 111 shown in FIG. 7 is, for example, position information generated by the GPS data acquired by the GPS receiver which is the sensor 51A provided in the vehicle 3 being processed by the processing unit 52. Note that the vehicle information 111 shown in FIG. 7 is obtained by extracting the vehicle information at the time when an event occurred in the vehicle 3 from the vehicle information repeatedly acquired by the sensor 51A provided in the vehicle 3.

[0100] The event information 121 of the vehicle 3 shown in FIG. 7 is information indicating the content of the event that occurred in the vehicle 3. The event information 121 of the vehicle 3 shown in FIG. 7 is the event information 121 output by inputting the vehicle information 111 shown in FIG. 7 to the detection unit 103 (model 106).

[0101] For example, the vehicle information in the first row shown in FIG. 7 (also referred to as vehicle information #1) indicates that at the time "2020-03-30 04:53:23.061953", the vehicle 3 is located at the position of "latitude 34.838968 degrees, longitude 135.673072 degrees".

[0102] Also, the event information in the first row shown in FIG. 7 (also referred to as event information #1) indicates that at the above time shown in vehicle information #1, an event of "jumping out" occurred in the vehicle 3, more specifically, an event of a person jumping out in the traveling direction of the vehicle 3.

[0103] FIG. 8 is an explanatory diagram showing an example of the vehicle information 112 changed by the changing unit 102 and the event information 122 in the present embodiment.

[0104] The vehicle information 112 shown in FIG. 8 is the vehicle information 112 changed by the changing unit 102 with respect to the vehicle information 111 shown in FIG. 7 (that is, the latitude and longitude indicating the position of the vehicle 3 at each time).

[0105] Specifically, the vehicle information 112 shown in FIG. 8 is such that among the vehicle information 111 shown in FIG. 7, (a) changes are made to delete vehicle information #2, #4, and #6, (b) changes are made to change the time information of vehicle information #1, #3, #5, and #7, (c) changes are made to change the latitude of vehicle information #5 and #7, and further, (d) changes are made to change the longitude of vehicle information #5.

[0106] Also, the event information 122 of the vehicle 3 shown in FIG. 8 is the event information 122 output by inputting the vehicle information 112 shown in FIG. 8 to the detection unit 103 (model 106).

[0107] For example, the vehicle information #1 shown in FIG. 8 indicates that at the time of "2020-03-30 04:53:24.061953", vehicle 3 is located at the position of "latitude 34.838968 degrees, longitude 135.673072 degrees".

[0108] In addition, the event information #1 shown in FIG. 8 indicates that an event of "jumping out" has occurred in vehicle 3, in other words, an event of a person jumping out in the traveling direction of vehicle 3 has occurred.

[0109] FIG. 9 is an explanatory diagram showing an example of integrated event information in the present embodiment.

[0110] The integrated event information shown in FIG. 9 is generated by integrating the event information shown in FIG. 7 and the event information shown in FIG. 8 by the generation unit 104.

[0111] When integrating, the generation unit 104 leaves the event information that exists only before the change among the before and after of the change as it is, and sets the column of "detection after change" to "No".

[0112] On the other hand, when integrating, for the vehicle information and event information that exist before and after the change, the generation unit 104 leaves the numerical value after the change and sets the column of "detection after change" to "Yes".

[0113] It can be said that the integrated event information is information indicating whether the event information detected from the vehicle information before the change is detected from the vehicle information after the change.

[0114] Note that, when integrating, the event information that exists only after the change among the before and after of the change is not included in the integrated event information. This is because the event information that exists only after the change has no particular meaning for the detection by the detection system 5.

[0115] FIG. 10 is an explanatory diagram showing an example of difference information in the embodiment.

[0116] The difference information shown in FIG. 10 indicates event information that existed before the change but does not exist after the change during integration (i.e., event information where the "detection after change" column in FIG. 9 is "No"). Specifically, the difference information #1, #2, and #3 in FIG. 10 is extracted from the integrated event information #2, #4, and #6 in FIG. 9, respectively.

[0117] The difference information shown in FIG. 10 corresponds to difference information indicating the difference between the event information indicating the events that occurred in vehicle 3 and the event information after the change by the change part 102. More specifically, it can be said that it indicates events among the events shown in the event information indicating the events that occurred in vehicle 3, excluding the events shown in the event information after the change by the change part 102.

[0118] FIG. 11 is an explanatory diagram showing a first example of the output information in the present embodiment.

[0119] The output information shown in FIG. 11 is an example (image 61) of an image showing on a map the position where the event indicated by the difference information occurred.

[0120] In image 61, a graphic 62 indicating the position where the event indicated by the difference information occurred is shown. The graphic 62 is displayed with a darker color for positions where more events occurred.

[0121] Image 61 is generated by the specific device 10, transmitted to the presentation device 20, and displayed by the presentation device 20. By visually recognizing image 61, the monitor Q can grasp the positions where many events occur as positions on the map.

[0122] The monitor Q who has grasped the positions where many events occur in this way is assumed to take measures to avoid, for example, the actual occurrence of the events indicated in the event information. Specifically, it is assumed that the monitor Q takes measures to install guardrails to prevent people or objects from jumping onto the road at positions where many events of people or objects jumping onto the road occur.

[0123] FIG. 12 is an explanatory diagram showing a second example of output information in the present embodiment.

[0124] The output information shown in FIG. 12 includes the degree to which each of a plurality of factors of an event indicated by difference information has affected the occurrence of the event.

[0125] Specifically, the output information shown in FIG. 12 indicates, as numerical values with a total of 1, the degree to which each of "the number of passengers", "the flow of people", and "the flow of vehicles", which are factors for the occurrence of the event, has affected the occurrence of the event.

[0126] The degree to which each factor has affected the occurrence of the event can be calculated, for example, by taking the correlation between an element that may be related to the occurrence or non-occurrence of the event and the occurrence or non-occurrence of the event. Also, the degree to which each factor has affected the occurrence of the event can be calculated, for example, as the standardized partial regression coefficient of the regression analysis result by an element that may be related to the occurrence or non-occurrence of the event.

[0127] For example, in FIG. 12, an example is shown in which the degree to which the factor of "the number of passengers" has affected the occurrence of the event is 0.4, the degree to which the factor of "the flow of people" has affected the occurrence of the event is 0.3, and the degree to which the factor of "the flow of vehicles" has affected the occurrence of the event is 0.3.

[0128] A person who visually recognizes the degree to which each factor has affected the occurrence of the event as shown above can take measures to make it difficult for the event to actually occur.

[0129] The processing of the specific system 1 configured as described above will be described.

[0130] FIG. 13 is a flowchart showing the processing of the specific system 1 in the embodiment.

[0131] In step S201, the sensors 51A etc. of the vehicle 3 acquire sensor values by performing sensing on the vehicle 3 or the surroundings of the vehicle 3.

[0132] In step S202, the processing unit 52 of the vehicle 3 obtains vehicle information 111 by processing the sensor values obtained in step S201. The acquisition unit 101 of the specific device 10 acquires the vehicle information 111 acquired by the processing unit 52.

[0133] In step S203, the modification unit 102 of the specific device 10 generates vehicle information 112 (see FIG. 6) by modifying the vehicle information (corresponding to vehicle information 111, see FIG. 6) acquired by the acquisition unit 101 in step S202.

[0134] In step S204, the detection unit 103 of the specific device 10 inputs the vehicle information 111 acquired by the acquisition unit 101 in step S202 into the model 106 to obtain event information 121 (see FIG. 6).

[0135] In step S205, the detection unit 103 of the specific device 10 inputs the vehicle information 112 (see FIG. 6) modified by the modification unit 102 in step S203 into the model 106 to obtain event information 122 (see FIG. 6).

[0136] In step S206, the generation unit 104 of the specific device 10 generates difference information indicating the difference between the event information 121 acquired by the detection unit 103 in step S204 and the event information 122 acquired by the detection unit 103 in step S205.

[0137] In step S207, the generation unit 104 of the specific device 10 outputs the difference information generated in step S206.

[0138] In this way, the specific system 1 can appropriately identify and output events that can occur in the moving body.

[0139] As described above, according to the specific method (i.e., information output method) according to this embodiment, among the events that can occur in the moving body, it is possible to output differential information indicating an event that may fail to be detected during actual movement. The output differential information indicates the change in the event information caused by changing the second moving body information. Also, the change made to the second moving body information may correspond to an error or omission included in the information acquired by a sensor that may occur during actual movement. Therefore, it can be said that the output differential information indicates an event that may not be detected due to an error or omission in the information that can be acquired by a sensor during actual movement. In this way, the above information output method can appropriately identify and output the events that can occur in the moving body by outputting differential information indicating an event that may fail to be detected during actual movement among the events that occur in the moving body.

[0140] Also, since the change made to the second moving body information corresponds to a change due to the failure of sensing by a sensor provided in the second moving body, the output differential information becomes differential information indicating an event that may fail to be detected during actual movement. Therefore, the above information output method can more appropriately identify and output the events that can occur in the moving body.

[0141] Also, since the second moving body information is changed according to the frequency of the failure of sensing by a sensor provided in the second moving body, the output differential information becomes differential information indicating an event that may fail to be detected during actual movement, taking into account the actual frequency of failure. Therefore, the above information output method can more appropriately identify and output the events that can occur in the moving body.

[0142] Also, by excluding the events indicated in the third event information from the events indicated in the second event information, it is possible to easily generate differential information. Therefore, the above information output method can more easily and appropriately identify and output the events that can occur in the moving body.

[0143] In addition to the sensors provided in the moving body, sensor values generated by sensors installed on the moving path through sensing are also used to generate moving body information, so that the generated moving body information becomes more appropriate information. As a result, the appropriateness of the event information and the difference information is also improved. Therefore, the above information output method can more appropriately identify and output events that may occur in the moving body.

[0144] In addition, as moving body information, difference information is generated using the position information or speed information of the moving body, the presence or absence of obstacles around the moving body, or the position information or speed information of obstacles existing around the moving body. Therefore, the above information output method can more easily and appropriately identify and output events that may occur in the moving body.

[0145] In addition, based on the moving body information, the model detects, as events, the jumping out of a person or an object in the moving direction of the moving body, the entrainment of a person or an object when the moving direction of the moving body changes, the collision when the moving body merges with another moving body, the interruption of another moving body in the moving direction of the moving body, or the deviation of the moving path of the moving body. Therefore, the above information output method can more easily and appropriately identify and output events that may occur in the moving body.

[0146] In addition, the position where an event that may not be detectable during actual movement occurs is shown and presented on the map. Therefore, a person who views the above map can easily grasp the above event on the map, and it is assumed that measures will be taken to make the event less likely to actually occur. Therefore, the above information output method can lead to measures to reduce the detection of the event while appropriately identifying and outputting events that may occur in the moving body.

[0147] In addition, the degree to which each of the multiple factors of the event has affected the occurrence of the event is presented. Therefore, a person who views the above degree is assumed to take measures to make the event less likely to actually occur, or measures to make the failure of event detection less likely to occur. Therefore, the above information output method can lead to measures to reduce the detection of the event while appropriately identifying and outputting events that may occur in the moving body.

[0148] In addition, by using a neural network model, it is possible to more easily and appropriately predict events occurring in a moving object.

[0149] In the above embodiment, each component may be configured by dedicated hardware or may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU or a processor reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory. Here, the software for realizing the specific device (that is, the information output device) of the above embodiment is the following program.

[0150] That is, this program causes a computer to obtain a model that outputs first event information indicating an event that occurred in the first moving object based on first moving object information, which is information regarding the first moving object input to the model, obtain second moving object information, which is information regarding the second moving object, generate third moving object information obtained by modifying the obtained second moving object information, obtain information indicating an event that occurred in the second moving object, which is output by inputting the obtained second moving object information into the model, as second event information, obtain information indicating an event that is output by inputting the generated third moving object information into the model, as third event information, and generate and output difference information indicating the difference between the second event information and the third event information.

[0151] As described above, the specific method and the like according to one or more aspects have been described based on the embodiment. However, the present invention is not limited to this embodiment. As long as the gist of the present invention is not deviated from, various modifications conceived by those skilled in the art applied to this embodiment or forms constructed by combining components in different embodiments may also be included within the scope of one or more aspects.

Industrial Applicability

[0152] The present invention can be used in a prediction device that predicts events that can occur in a vehicle.

Explanation of Signs

[0153] 1 Specific system 3, 7 Vehicle 5 Detection system 10 Specific device 20, 30 Presentation device 50 Detection device 51A, 51B Sensor 52 Processing unit 53 Providing unit 61 Image 62 Graphic 101, 501 Acquisition unit 102 Modifying unit 103, 502 Detection unit 104 Generation unit 106, 505 Model 111, 112 Vehicle information 121, 122 Event information 503 Output unit N Network P, Q Monitor

Claims

1. Obtain a model that outputs first event information indicating an event that occurred in the first moving body based on first moving body information, which is information regarding the first moving body input to the model, obtain second moving body information, which is information regarding the second moving body, generate third moving body information obtained by modifying the obtained second moving body information, obtain, as second event information, information indicating an event that occurred in the second moving body, which is output by inputting the obtained second moving body information into the model, obtain, as third event information, information indicating an event that is output by inputting the generated third moving body information into the model, generate and output difference information indicating a difference between the second event information and the third event information, the second moving body information includes information generated from sensor values generated by sensing with sensors provided in the second moving body during movement of the second moving body, when generating the third moving body information, generate the third moving body information by applying the modification corresponding to the failure of sensing by the sensors provided in the second moving body to the second moving body information, An information output method.

2. Further, measure the frequency of failure of sensing by the sensors provided in the second moving body, when generating the third moving body information, generate the third moving body information by applying the modification with the measured frequency, The information output method according to claim 1.

3. When generating the difference information, generate the difference information indicating an event excluding the event indicated in the third event information among the events indicated in the second event information, The information output method according to any one of claims 1 to 2.

4. The first moving body information further includes information generated from sensor values generated by sensing with sensors installed on the travel route of the first moving body, the second moving body information further includes information generated from sensor values generated by sensing with sensors installed on the travel route of the second moving body, The information output method according to claim 1.

5. The first moving body information is the position information or speed information of the first moving body, the presence or absence of obstacles around the first moving body, the position information or speed information of obstacles existing around the first moving body, the number of passengers in the first moving body, the amount of pedestrian flow existing around the first moving body, or the amount of vehicle flow existing around the first moving body, The second moving body information is the position information or speed information of the second moving body, the presence or absence of obstacles around the second moving body, the position information or speed information of the obstacles existing around the second moving body, the number of passengers on the second moving body, the amount of pedestrian flow existing around the second moving body, or the amount of vehicle flow existing around the second moving body The information output method according to any one of claims 1 to 4.

6. The event includes a person or object jumping out in the moving direction of a moving body which is the first moving body or the second moving body, a person or object being involved when the moving direction of the moving body changes, a collision when the moving body merges with another moving body, an interruption of another moving body in the moving direction of the moving body, a deviation of the moving path of the moving body, a collision caused by the simultaneous start of a vehicle and a pedestrian in a crosswalk area without a traffic signal, or an intentional obstruction around the vehicle. The information output method according to any one of claims 1 to 5.

7. The difference information includes an image showing the position on the map where the event indicated by the difference information occurred. The information output method according to any one of claims 1 to 6.

8. The difference information includes the degree to which each of a plurality of factors of the event indicated by the difference information affected the occurrence of the event. The information output method according to any one of claims 1 to 7.

9. The model is a neural network model. The information output method according to any one of claims 1 to 8.

10. It includes an acquisition unit, a modification unit, a detection unit, and a generation unit. The detection unit acquires a model which is a model that outputs first event information indicating an event that occurred to the first moving body based on first moving body information which is information about the first moving body input to the model. The acquisition unit acquires second moving body information which is information about the second moving body. The modification unit generates third moving body information obtained by modifying the second moving body information acquired by the acquisition unit. The detection unit acquires, as second event information, information indicating an event that occurred to the second moving body, which is output by inputting the second moving body information acquired by the acquisition unit into the model. The detection unit acquires, as third event information, information indicating an event output by inputting the third moving body information generated by the modification unit into the model. The generation unit generates and outputs difference information indicating the difference between the second event information and the third event information. The second moving body information includes information generated from sensor values generated by sensing with sensors provided in the second moving body when the second moving body moves. When generating the third moving body information, the changing unit generates the third moving body information by applying the change corresponding to the failure of sensing by the sensors provided in the second moving body to the second moving body information. Information output device. **Claim 11** A program for causing a computer to execute the information output method according to any one of Claims 1 to 9.

Images